Testing Strategies for Cloud Migration Success – DevOps Unbound EP 34

Hey everyone, welcome to another episode of devops. I'm down devops and down is a bi-weekly. Or is it semi-weekly?

I always forget Mitchell, but it's every other week show on. Various topics in devops and you know, we've been doing this now. She's I think we're over two years Mitchell on devops.

I'm bound and one of my favorite things every other week to do. But really my favorite thing is maybe once a month or every six weeks. We do a live version of this with a live audience and the audience drives the questions and the topics.

So if you're watching this, you know recorded, thank you. But you really want to check out the schedule and join us on our next live one as well. I should also mention that devops and bound is sponsored by our good friends at tricentes the leaders in continuous testing worldwide.

They're great sponsor to work with I've said it in the past on other shows and they've been with us since day one here on devops Unbound so many thanks to the good folks of tricentis. Let me introduce you to our panel today and then we'll jump into what our topic is. Starting from the top of my my screen is the one and only Mike Rothman who happens to be the GM of tech strong research business unit as well as Chief strategy officer here in Tech strong.

Hey, Mike welcome. Thank you. Alan.

Always great to be here. And again this I'm not sure if I've been in devops. I'm down before in any case.

I'm excited to be here for sure. We're excited to have you man. We're excited.

We're excited to have your text strong. We're also excited to have you here on devops. I'm down.

Stop. I want to introduce you to Jonathan Boswell Jonathan. You've been here before welcome I have thank you very much.

A little background maybe. Oh sure. I'm a product marketing director at tricentes.

I've been about four years previous to that sap and a bunch of other big tech companies. I've been about 25 years in in the business seeing all kinds of things and looking forward to seeing you know, what crazy things happened next. Absolutely.

Next is well, she needs no introduction. If you've watched any of our text dog TV shows over the years. She's the host of tech strong women.

It's a frequent contributor to our panel shows. She's also very involved. Well, I'm gonna let her talk about her involvements.

Tracy Reagan, Tracy welcome Thank you Alan. It's always fun to be on these panels. I actually love doing it.

And I agree. I love the live ones too. I love the chats.

Yes. I've been doing this kind of work for some time, you know full disclosure. I am a the CEO of deploy Hub.

I'm on the board of the open source security Foundation as well as technology advisory Council for the CD Foundation. I have done many migrations in my career, but I haven't had the pleasure of suffering through a full-on cloud migration for an Enterprise, even though we've done on-prem to Cloud migrations and deploy Hub. So this is an interesting topic and I'm looking forward to it.

Absolutely, we're gonna get in there in a second. But I do want to introduce our last panel member and I always save them for Less. He's my co-host.

It's been doing this with me since day one and he's our CTO as well as the principal research analyst that text strong research group. Mitchell Ashley. Hey Mitchell good to be here.

I'm wearing my jacket and honor of our first day snow on the ground here in Colorado. So it's not 85 Degrees like Boca where I was right here this week. So hey, I was just gonna throw in I have done actually a pretty large Cloud migration.

It wasn't for an inner price for us for a sass that did 92 million transactions a year going from our own Colo into AWS. So it was interesting. Let me tell you so and fun sort of you.

Let me just say that my big migration was from New York down here to Boca about 20 years ago and it is 85 degrees. But you know, we we've kind of let that out of the bag here on migration and that that is the topic of today's show, right the title is testing strategies for cloud migration success. And you know before we could get into testing.

Regarding Cloud migrations. I think we got to talk about Cloud migration and and Define that you know, there was a time, you know Mike's Mike's old partnered securoses our friend Rich mogul. We used to call them the redhead but there's not as much hair there as there used to be who might have taught who might have talked.

But anyway, right formally who invited talk Anyway You Know Rich kind of for me. Anyway coined the term of cloud washing. Right Cloud washing, and it was kind of like this shift and lift kind of thing.

Well, you just took what you were doing on Prem and voila. I I installed in in on top of the hypervisor and AWS now, I'm all about cloud. but that you know, we quickly found out that that's kind of a failed strategy that that Lifting shift doesn't work that cloud washing doesn't work.

some people used to say, well that's not Cloud native, but now Cloud natives a whole separate thing, but in any event right, there is There a Best Practices. I don't know if it's more science than art yet, but the best practices has evolved around Cloud migrations. Now it's gotten more complicated as we've had multi-cloud and hybrid clouds and and all of these things in different kinds of clouds, but nevertheless.

there is a there's a little bit of well, there's more than a little bit of Science and more than a little bit of Art. Cloud successful Cloud migrations Mitchell you mentioned you were involved in one Jonathan. I'm wondering your your involvement in any I mean I say p right they Yeah, sap definitely and with present this I mean what we do is we help people make their migrate into the big, you know, a lot of customers are moving.

They from ECC to esperan on the cloud or moving, you know, adopting Oracle Cloud as well moving there EBS to you know to the Oracle Cloud as well. So we see we see a lot of that we help a lot of customers with autonomigration. And there are some some patterns that emerge some challenges that are common that we can get into and some solutions also that work as well.

Absolutely, Mitch. If you had to say one lesson you learned during your Cloud migration with this SAS product. What would you say was the biggest lesson the biggest aha.

I'd say planning it it took a lot of preparation to do it and planning including included a lot of early testing kind of solving the engineering questions. Like how is this going to work? And can we do a b transaction flow?

How would our database sync work? Do we just push it all up? Do we do what you know, do do the Office 365 migration where first time we did it everybody was down for day and a half or do we like make it work?

You know, but it was that was unfortunately some other people helped us do the planning some people that have gone through before we didn't try it on our own. I think that was probably my the second maybe the first best advice was like don't go It Alone. Yeah, I want to clarify a couple things Alan right one is here.

You kind of intimated that lift and s*** doesn't work. That's not exactly right. Right.

So lift and Shift Works. It just doesn't work very well. Right.

So so you're you're sorry optimizing it's costing you way too much money. You're not, you know kind of again leveraging a lot of the capabilities that you have on a cloud platform. So, you know, I don't want to mislead anybody say that you can do that because some folks lose their data side or their Colo goes down or for whatever financial reasons and they have to lift and shift we can do it.

It's just some optimal right and and the thing to piggy back on on what Mitch had said about planning. Um, it's really about getting small right in order to get big so you really have to think about it one workload at a time. Right?

What am I trying to migrate? Yes you want and Jonathan's point you want to have design patterns you have, you know kind of structures you want to have a set of services that you can you know. As you are refactoring some applications, but this idea that you're going to do, you know 200 applications in the next six months.

I'll get maybe you'll get there but you got to start with one right and then you do two and then you get to half a dozen and then by the time you have a dozen you've got three or four different design patterns for what those things look like different processes for how you get there and then you can start stamping them out because you've built the tool chain you understand the migration process and then you can really start, you know, kind of moving much more aggressively but the first one's gonna be brutal. I mean absolutely brutal and to Mitch's point, if you're not doing a lot of planning and you don't understand what cloud native Services you're gonna use and you don't you know, and and you're not really engaging nowadays with service providers that's specialize in this migration. You're really suboptimizing because the reality is you're not the first moving in the cloud but to not take advantage of people that have done a lot.

It's already is, you know bordering on malpractice candidly. And I think every okay everybody wants to take it, you know eat an elephant one bite at a time. But some companies can't some companies have to do with that big lift and shift.

It really depends on how interconnected your applications are. So getting a plan together I'm obviously is the the first part of that strategy and in my experience one of some of the mistakes is not having all everybody with skin in the game. If you have to do a massive migration like that and a lot of companies just do just because of the way their software talks to each other especially like, you know, the financial industry.

I mean, you know, they have so many pieces that have to work together. It can't just take one application at a time. They have to take a few dozen at the same time.

You have to get the team really set up you have to have the developers acknowledging that this is happening. You have to have your devops teams ready for it. You have to have your QA your database your DBA.

Don't forget about that and your end users you need to have a group of end users that you are going to rely on to say you're this is you're going to be first We're Gonna Roll you guys out first when we do this massive shift. So each of those organizations have to have their own strategy. You have to break it out.

So that each one of those organizations understand what they have to do developers understand they have to be on call. They're not going to do a new release for a few weeks. Dba's have to be very devious left.

You know, it's like we can sneak this in right before the last minute because your little dog too. Okay, the devias this the schemas and the data is probably the most critical part and tooling around that, you know, a lot of companies. I think that they can do this with just manual intervention, but don't worry spend the money on the tooling even if it's just used for this one migration use it especially around data schemas and the data and how to track and compare do your metrics get your metrics together and compare that you don't have to do this much work if you're doing it one by at a time, but if you do have to have do that lift and shift you really have to spend the time to build out strategies for each team and really start thinking about the metrics and get your dbas involved in a big time serious way.

It's like Lessons Learned From many migrations cloud or not, right? Yeah. You know, someone said to me in the process at the beginning of the process that but you just a Class A cultural reference remember in The Blues Brothers Jake and I would driving the Crown Vic police cruiser is that if you're starting with a Crown Vic today, you'll have a crowned Vic in the cloud too.

Just remember it's still a Crown Vic when you get there, so it's not like it's magic and all sudden you you know, everything's good and golden and Rosie. It's still the app that you had before. So I I think one things to to think about is what is the compelling event that is causing you to go to the cloud.

Why are you doing it in my case? It was we have to we need to do a hardware refresh in 12 to 16 months and I was budgeting for that like, okay, we're gonna have the money to do this or do we want that cash for something else and go to a you know Services kind of payment model and that was our motivation. So it required getting everything up there.

So we had to kind of move the Crown Vic and then start working on it in the cloud. And do that preparation but it behaved very differently in the cloud than it did in our own off-site data center. I mean, these are servers and data stores stuff.

We didn't control and didn't know what to expect. So we had to really do a lot of testing up front. That Mitchell that take a little bit of an issue with that Crown Vic because what a lot of our customers are doing is if you're looking at your key modernization, for example, you're taking your old Crown Vic that you've been tinkering and working on for probably the last 20 years.

And essentially when you're moving to the cloud you're moving to S4 Honda you're training it in you're buying a whole new car a McLaren or whatever it is, you know, whatever the example you want to use and so it's not the same old workloads just in the cloud. You're actually doing a lot of redesigned potentially. Of your business processes and and you know everything to take advantage as we're talking out earlier of the advantage of the cloud.

You don't want it to do that lift and ship you don't want to put your Crown Vic and the cloud you want to you know, get in there and and in some cases have a whole new. If not, if not Greenfield something close to it, you know a whole new whole new implementation in the cloud whole new business way of doing business and the cloud and that brings me back to you the planning phase because you can talk about the you know, what's required to technically move, you know, your workloads to the cloud but from the business planning point of view, You know, what are all these customizations that we've made to our SAP systems anymore are those now some of them are a lot of them actually are going to be now standard in escrow Hana for example, so you don't want to bring that code over, you know, you have to do a lot of planning in order to figure out what business is going to look like in this and this Cloud. You know, how am I going to drive the new car?

You know when I'm used to drive in the Crown Vic, you know, that's what I did wrong. Okay that planning phase. Oh try to live your life life like Jake and Elwood.

That's the problem Oh and sentence speaking of God. So yeah, you know speaking. It's very funny speaking.

It is going south guys. I got to grab hold of this. So, you know speaking of cards though.

I remember being at an IBM think or whatever it was before it was called think conference. Probably in Las Vegas five six seven years ago, and they had the I think was the CIO of hurts. What the time at just opened up a huge?

facility Data Center In the on the west coast of Florida near Bonita Springs somewhere right Estero. Or esotero something like that. It's near Naples and Bonita Springs and they they made a decision to move to the cloud.

It was the most radical Cloud migration. I ever saw they migrated nothing. They just build Greenfield new applications in the cloud.

And as soon as they were up. They just shut the on-prem ones off. That CIO got fired here.

I'm sure that we did no, no kidding. He got fired within the year, but I I always admired his gumption or I don't want to use another word, you know, you know, right you're talking about running head in a will will party I'm gonna show my age will Palmer there from F Troop, right? I'm running right along into the into the charge here.

I mean you want to talk about Cloud right? Look he didn't worry about testing you didn't worry about what was my grading he just you turn going off to the other. So there there is, you know to my to your point about lifting being suboptimal.

There are other suboptimal kind of things, but you know, Here's the fact. for a long time the word on the street or the real, you know, the real word was yeah, there's stuff running in the cloud. How much of its Mission critical how much did companies like sap how much did large Enterprises really migrate to the cloud that they couldn't afford to go down.

Right, and maybe this is the reason why they didn't. right migrating to the cloud was You know man about managing risk. And and that and that you know that that's a true statement.

Have we overcome that now have has have best prep? You know, we went through this in devops, right? I'm one of the founders of devops Institute.

We all Tracy's worked with them, you know, we went through a period in devops Institute where we didn't want to use the word best practices. We called them evolving. practices because there really wasn't best practices yet.

Have we gotten to A yeah, yeah because they weren't fully evolved. Have we gotten to best practices and Cloud migration? And if so, what world is testing play in it?

I I think we have I think there are a good set of evolving practices. For example in some of the early migrations of some of the folks that I would talk to as they suffered through this. One other bigger problems with doing a proper application security audit because what happens is when you do these migrations you like to turn off a lot of security because it sure makes it a lot easier to get through it.

But then you forget to turn back on that security at the application Level at many of these different levels and I believe that we've learned the lesson and there are there's you know, there's a more a better understanding that that security that application security audit really has to happen before you can really turn off the others the other on premise. So I believe that there there has been and like I was talking about earlier this the schema and the data, I think there's some really evolving practices around. How to migrate move the schemas over there's tooling for it and we've gotten better at doing the metrics testing.

The better we have testing on the on-prem we can shift that over to the cloud and we can do those comparisons. So I think that the the practice we have evolved and there are really clear standard practices that companies understand they have to go through. Yeah, right overall from a conceptual point of view.

Not thinking of testing as the last thing you do, but think of it as one of the first things you need to plan for colleague of mine likes to talk about on migrations as equivalent to skydiving. You don't want to wait to your midair to make sure your parachutes gonna work. Right, you want to make sure that you plan for it that you understand what's going to go on and what we're actually encouraging people to do and building this into the essential or best practices for sap and Oracle these big ER Key met regions is to make sure that you're testing your your including testing you're planning for testing in those earliest planning stages.

Otherwise, if you wait to think about how we're going to test this you're gonna run into delays, you're gonna run it over runs and you know, you may fail like Splat at the end and nobody wants that but so, you know again the real, you know critical thing is to think about how you're going to be testing what needs to be tested. What's going to be different about your testing Tracy was saying you your Baseline for performance on your on-prem to make sure that you're not getting worse. There's all these things you need to start thinking about early as you earlier in the planning Pages planning stages then maybe you're used to and a typical release where you wait.

New testing at the end. It's pretty much flipped when you're talking about these Cloud migration. Yeah, I mean the other thing that factor in is that you have a lot of different components that you're thinking about especially as you move to the cloud.

Some of them will be native services from your cloud provider. Some of them will be third-party services that you integrate in that you'd access via apis and all of those linkages and integration points need to be exercised. Right, you know, not just from a functionality standpoint, right but also from a scalability standpoint because one of the advantages of refactoring one of the you know, kind of ideas behind Auto scaling and you know kind of orchestrating and automating, you know, all of these things is that we can have the infrastructure that our application needs at this scale that it needs at the time that it needs it right and that means we've got a test all those different scenarios.

So it's not just that hey is it up? Hey when we throw a boatload of traffic at it, does it feel over it's how does it scale up? How does IT contract?

How does it integrate all these other services in there? How do I make sure? I've got visibility across the entire environment, you know, one of the things Mitch and ionies Mitch has been very patient in schooling me right?

I'm an old SIM guy from the security world and he's using terms like tracing and I'm like what the hell you talking about? Right and he's like no. No, you know, it's like, you know, it's microservices.

You got to trace out there working together. And you know, I'm like dude, you're like talking to different language to me, right? Not that's just one database and and and all that but you you have to actually have that instrumentation as you're moving these workloads into the cloud or you're dead meat, right?

So that's just the testing side of it. The one thing. The other thing I want to add is the importance of taking a programmatic approach to this.

It's not the first one you're gonna do right your typical even medium size Enterprise as hundreds of applications that they're gonna want to reply form at some point. This thing has to be if you're not thinking about it as an ongoing thing that you're gonna do not wants not twice, but all hundred times. Right and what are the things that I need to put in place to ensure that the hundredth time basically worked by itself.

You're doing it wrong, right because this is going to be an ongoing thing that you have to deal with at all times. Yeah, just add onto that not only are the migrations gonna be going adding on but once you're into Cloud the updates are going to be ongoing one of the big challenges for Enterprises with you know, these Legacy Erp systems is that you know, if you're on Prem you can choose when you want to update which is how you update on the cloud. Now, you're on someone else's time schedule it typically quarterly Cleo Oracle sap, all your big ear providers are providing, you know, at least it's poorly so you got to have your testing processes in place to be able to keep up with that accelerated Pace, right?

So to get back you're saying have learning these best practices now, you know take transforming your testing capabilities to be able to keep up with this pace. Well, not only help you in your first migration for your hundredth migration, but all the, you know, all the work that comes afterwards that to keep those things up to date. I'm going to highlight something that John I'm sorry Alan, but I want to highlight something that Jonathan said earlier.

He said be watch for what your what testing is different. I think that is a it's a little tidbitty snuck in there. That's super critical and the two areas that you should be looking at where you're testing is different is your load and your pen testing.

When you when you shift to the cloud your load testing and your PIN testing are going to be very different from what it looked like on-prem. Just wanted to highlight that. no, I I think you're 100% right?

I think there are others obviously, you know, the idea with Cloud infrastructure is a service is You you're relying on the cloud provider for their infrastructure, right? So you're not. You know, you may not be used worried about sort of infrastructure tests though.

You want to make sure you have capacity, but that that's the nature of cloud but here's what I think what I'm hearing though today folks and that is Look, this is a hard enough problem to get your head around. And but we have we made a lot of progress now we start adding in things though like multi-cloud. Hybrid, I'm not that I'm not lifting the whole app.

I'm just putting my database up there or let me let me throw another one. We we mentioned a little bit. Let's go to a microservices architecture while we're at it.

What the heck? Right, my head's going to explode. All right, don't do Microsoft.

No, but that that's that's the real world. Right? These are real world things that we deal with and you know.

It's hard, you know, I'm a fan of microservices. I'm just saying don't do that at the same time. Yeah, you know, if you have you know, put everything in a container your monolithics and a container get it all working and then think about decoupling and think about it, you know domain strategy, but the other thing is especially when you thinking about these hybrid environments everybody has still as the Mainframe most most organizations we talk to still do stuff on the Mainframe.

So that's always going to be there and it's gonna be connected to your new Cloud environment. To have fun with it. Nobody's going to just put all this out in production day one.

This is going to be something that's initially built and it's big giant test environment. So have game days. Have game days break things don't be afraid of breaking things because the more you you practice the more confident your teams will be we talked about confidence.

I'm going to plug text on women. We just did a a interview with Norah Jones from jelly and she's into chaos engineering. She talked about how chaos engineering there was a adoption problem because of the confidence level, you know, you don't want to break things.

If you don't know if you can fix it. If you can get your team super confident in the in your test environment have some game days then when you go to do your your production migration, everybody's going to be much better prepared and understand that if something does break they understand and have the tools to fix it and get it up and running again. So you might protect your cios job if they do turn off the front because you're prepared your team's prepared and their confidence.

So game days are you know, I don't see people adding that as part of their testing strategy, but I think game days are a really good way of getting the team ready for it when you when you do make that switch. You know confidence that's just a to build up that that's such an it's such an important, you know trade and it's such an important thing to have and we it goes on what we're talking about before with micros talking about, you know, the first to the hundredth time, you're gonna be doing this a lot and going to be doing this, you know, you know testing forever. But especially if you're a big Enterprise company where maybe you're used to testing you're doing a lot of manual testing right the thing about your Cloud migration and going forward is that you're gonna need to be automated you gotta be doing perhaps a lot more Automation and you're testing then you've been currently on-prem.

And so how do you get confident that you start now start early start building so that as you're ready to go, you know, if you need new tools if you need new practices, you're getting that experience now so that when it's time to go you do have that confidence to know that, you know, you're testing will be successful. You will be addressing the risks that you're not trying to do too many new things all at once right you starting the master some of the skills and the techniques and the tools that you'll need to be successful in your Cloud migration and Beyond so starting that early it is as you step towards building that confidence. And one of the things that I would add to this is I always think of I think I mentioned engineering points earlier every every app every infrastructure every connect technology sort of has those things that we know that it's hypersensitive to login time.

It's very sensitive to latency and these kind of environments. It's very sensitive to whatever for our apps right and so you design you engineer the architecture application knowing that those are constraints that you're going to need to work with and for me. Those are the exact things that I want to test first up in the cloud like I need to move the whole lap and test all the whole app.

I want to go up and test. Okay is my identity management and authentication all that happen quickly because I may be doing machine to machine identity authentication. It's got to having very quickly and what kind of expect what do I which what how does it behave and what do I need to know about it when I move my app fully up there.

So those kind of design points of engineering points I think are the some of the most important things to test doesn't be first but at least in parallel with your testing functionality and performance and the other things kind of ultimately it performance doesn't matter if the critical things don't perform well because the rest of them won't anyway, And that that I'll get back to some. Oh, sorry. No, no, no, go ahead.

Now, I'll just I was just saying I'm wanted to you know, again bring it back around to a point. I think I made you know, when we first started, you know, the discussion right? There are people that know how to do this.

So, you know and there's the whole Pennywise pound foolish thing again, they're gonna be a bunch of shysters out there too. So, you know, you want to make sure that you're doing your homework about whatever service provider you have, you know coming in to do this, but they are folks. They're a playbooks.

There are you know kind of tooling kits they're all this stuff exists. And if you're sitting there, you know gonna learn in the school A Hard Knocks, right? You're an idiot.

I mean, it's I I know you're gonna pay any attacks. That's exactly what you're going to do. Oh, no, you see this.

Speaking this Jonathan. So look you are from tricentus. So I got to put you on the spot.

Does tricentis have a specific Cloud migration? Kind of testing recommend recommendation group of tests product line. What what where can we get help from tricentes here?

Yeah. Okay. Thanks.

Yeah, so we do have yeah and depending on you know, what you're migrating. We might have a different set of tools, right the idea is that not one tool is gonna fit all all use cases, but we do have and in continuous testing with our census Tosca, which lets you take your what your your current on premise and make sure that you use the same test cases as you moving forward into the cloud. That's the coldest automation.

So let's technical debt less technical maintenance required to create an upkeep those we've got performance testing tools and with your with sap, we have an impact analysis tool change intelligence will go live compare which is actually really helpful in S4 Honda migrations because you can tell you exactly what you have now what's changing and what you need to test so really We work closely with us with sap also on their roadmaps to for their rise programs and their s/400 Cloud migrations and we're building in the best practices that we've learned from our customers. That's it. He has learned and how to fit these tools and at the right times and in order to make sure you know, again that no one's paying that idiot tax.

We've seen enough migrations by now to know, you know what to do and whatnot to do and so building those into the actual methodology of disappear recommends is a project that's ongoing now so that we can help everybody have the most successful migration possible. com and look for that. Okay fair enough guys.

We're almost out of time here. First of all, thank thank you Tracy for joining us. It's always a pleasure to have you here, and thank you.

Well now. Jonathan, thank you. Mitchell Mike, thank you.

Special thanks to chase dentist has always responsoring devops Unbound. You know Mitch, I'm gonna if you don't mind, I'm gonna let you take the last word. But I think we had on a lot of great learnings.

You know, we said best practices probably learning is but as valuable as the best practices and leveraging that from a partner that can help you whether it's technology like a testing Cloud friendly testing as well as on-prem testing to someone that can help you estimate your costs, which what you think you're gonna what your environment's gonna cost. You moving to the cloud as you migrate in pieces or in whole so I think there's so much information and so much more expertise available now that I would leverage, you know stand on the shoulders of giants. Don't try to build a new Giant.

And I think you'll get along a lot farther faster. Great, I think all those Partners who claim to be gold and platinum partners of the various Cloud providers. There's a reason why they're golden platinum and and you know the whole idiotax thing is that It's a key takeaway here man.

I this is not something at this stage of where the market is. There's no reason to be doing this yourself. It's just no reason.

Anyway, thank you all thank you for watching. We'll see you soon on another devops on bound. And until then this Alan Shimmel for Textron TV.

Take care.