Bringing Data Integrity to DataOps – DevOps Unbound EP 29

Well, hey, thank you for joining us. We're we have another episode of devops Unbound. Have a great topic today.

We're going to be talking about data Ops and how do we manage the the plethora the mass amount of data that we both create through our applications? And we also create as part of our software development process devops and bound is sponsored by our wonderful friends at tricentis who work with us on some of the content for these shows though. It's not a product pitch by them by any means they're they're constantly helping us with kind of figure out topics that are interesting to people that are living in the world.

So creating software and testing and devops and data and all kinds of a great thing. So thank you to the team Lanier and our producer Jody for helping us put this panel together. So let's get started by the way out.

My friend Alan co-host alexandl is a way. He's just getting back into the country from some international travel. So I'm sure he'll join you sign up on a future episode so Jump right into it.

Let's start out by having our panel introduce themselves the vanity. Would you start us out? Please tell us a little bit about you kind of the kind of work that you do and you're welcome to mention who you're with if you choose to my name is Divani Lamas.

I'm the CEO of a startup called transposite. We are a process automation company focused on helping technical operations teams of all sorts from devops to SRE to it teams drive better customer experiences by automating collaborative workflows, like internet management help desk requests and developer experience. I've been in the data space most of my career previously ran product teams.

It's long and I'm very very excited to talk about a topic very close to my heart. There's a lot of data get created in those kind of products. So for sure wonderful experience.

How about Arun would you go next? Yeah, thanks Mitch. I'm Arun musandani and I am a startup narwal which is focused on data automation cloud, and we have been helping Industries in doing the data engineering data storage data automation as well as the in circle from the automation perspective on the application automation, which includes the devops data Ops and the ml offside.

And also we have a strong Cloud competencies which is from the cloud modernization and Cloud migration perspective being an industry for over 15 years private dominantly in the data analytics space and work with the likes of American Express Discover Mastercard visa. And this same as when he said this is a topic which is very close to our heart and happy to be here. Vast quantities of data being generated by those types of companies.

So you're very familiar with the topic. All right, Andrew if you want to jump in next love to have you introduce yourself. Sure.

My name is Andrew Hughes. I run QA devops and Service delivery teams at a company called trimetics. We are a clinical Engineering Services provider.

So we have technicians and hospitals all over the country repairing and maintaining medical equipment and my team's help help build and deliver software that supports our technicians as well as our customers and understanding what equipment they own what the cybersecurity posture of their connective medical devices looks like and we've been partnered with narwhalf for going on five years now, I think in our digital transformation Journey so they've been they've been a critical partner for us. Excellent. Well Curtis bringing home.

I introduce yourself here. All right. Hello, I'm Curtis Odell.

I'm the global director for tricenses data Integrity solution. I've been in data space about 35 years built a lot of risk systems across many different verticals. We are the test automation leader in the world and data is a subset of what we do.

We do the full stack. But yeah, it all comes or back around to devops. So this is right on top of the where we love to talk.

And that is what makes the world go around. That's for sure. No doubt.

Well, let's start out with this. I love to have this create kind of a working definition of data Ops like everything astric Ops, right? We've got a something Ops for everything, which is great.

I mean, I actually love that because it's a good indication of you know, hey, let's jump on the bandwag. We have kind of a part to play in this devops world of things too. We can go to Wikipedia and find a definition we go to any number of vendor companies who are selling in the space, but we don't have to have the perfect definition just kind of a working one so that we know kind of how we're defining the space.

I like to Wikipedia and it says that data Ops is set of practices process technology is a combines and integrated in process oriented respective on data with Automation and methods from agile software engineering to improve quality speak collaboration promoter culture of continuous Improvement. It's a longer sense and that's a long sense anyway. How does that strike you as a place to start or would you take exception to it?

Or maybe one amend or enhance that anybody can jump in? John Mitchell say that they definition you pick from Wikipedia as a quite a mouthful we can just make it very simple. It is just the data pipeline Automation and how we can just make it more robust.

We can make it more predictable and more useful essentially providing value to the organizations. Like simpler simple as usually hard but that's that was very good Okay jump in other folks. Go ahead.

One of the things that I that I've seen in a lot of the data Ops teams that I've worked with is it's really a you know, a lot of teams are looking at the success of devops and those practices and applying those practices that they've learned and have worked really well to lots of different other roles. And for me devops always comes back to culture and Team Dynamics and teen Behavior. So I think any conversation about data Ops also has to touch on how those teams are evolving the way they work within themselves from, you know, kind of top down waterfall style process to a much more collaborative integrated Dynamic engineering first kind of approach.

Great, I agree. I was just gonna add that. I always look at his devops as a larger and data Ops being kind of a subset of that and it's kind of said folks are familiar with devops.

It really is everything there, but just focused on beta. It seems like data Ops gets applied to the large world of data or more specifically to the world of data within devops were pipelines Etc. And probably a lot of the same principles apply and part of our sort of topic was dated Integrity of that.

Because I don't know about you but it's you know, what's the source of Truth as a is it truly, you know accurate to in today's definition of how we want to use that data has been modified for any number of reasons is it is something we could really use for enhancing our workflow pipelines, maybe even governance, you know for reporting, you know for compliance and other kind of reasons there's lots of uses for it. So I'd love for for somebody to start off with let's talk about workflow pipelines data pipelines in the devops process and What role does data Ops play in that because we know that we're using a lot of tools. We're doing a lot of work in processes that are creating a lot of data.

Well, I could say something to that. Basically the way we look at it or I look at it is any kind of tying in data Integrity to that as well as the data Ops is that Today we have a world where data is this increasing velocity and and all that's going on. So devops isn't created so that as you make change in the systems, you can easily continually and then continuous as the word often used for you can just integration continuous feel very continuous testing so that we can basically keep up with that rate of change and I think the data World from a standpoint is challenged because the majority of if you will the processes if they're not into that type of tool chained it allows for that continuous integration, maybe continuous delivery, but definitely continuous testing then everything kind of gets stagnant and you end up with a lot of problems and the governance issues and everything that come up because there's not coverage of what's going on.

You're not testing. I'm just gonna use that word the change that is happening in a timely fashion. So you can't keep up with what's going on.

So people then say, Well, forget it. Let's just put it in the dev data officer devops and just send it out into the world and we'll hope that our production monitoring and systems will get the problems. It does it may but it's too late.

We don't test in production and I think to me the whole idea behind data Ops is that now within an in a whole new way to look at the world is that you can now really test in these lower environments be able to run everything you need the whole idea behind devops work with operations to make quicker faster moves for the company and you're a verse To all that risk, you're not doing that. So to me data Ops in the way really boils down the risk and I mean we have customers I can give some stories on maybe a little later that have this is their main guiding principle for what they're trying to do because I can't go data Ops if I cannot make my processes work with that and at the end of the day if the data is not good. What was the purpose at all?

Maybe should I argue with that or agree? Yeah, interesting cut is I see that means most of the things that you said, right? That's how the things in real life just happens.

So from the data of perspective, there are six different components that we can just categorize right one is the you can see the collaboration of the code management. Another is the continuous integration. Third piece is the continuous delivery or the deployment aspect.

In this the collaboration and the integration the need to have the continuous testing and the continuous monitoring as well. The aspect of the data Integrity becomes important when the different teams they are not integrated. They are not collaborating and the data observability becomes an issue.

So this is a place where we have seen for example in one of a large payment processor where we were just working. They had different teams focusing on these different components. So if there was any data Integrity issue, they were only catching up when it was reflecting into the reports.

So right from the data source the issue as a generated, but you are catching up only in the reports when the damage has been done. So that is a type of problem that we see can be solved by this data of pipeline where you can have the data observability as a part of continuous monitoring and you can identify and catch up the issues right up front. If you can just identify this process streamline this process and then make it repeatable that can create a tremendous amount of synergies.

the spot on courtesy I think that's a healthy way to look at it the flow the pipeline across the that process and what you remind me of arune is. We've we've had for ages right The Silo problem data silos application team cultural silos data is an immune to that, you know as well and as we move through workflows, we move through Pipelines. How do you make sure the right data is effectively being used in subsequent steps or being able to look across all those steps to gain insight and intelligence because you know, as you mentioned you don't get to the end of the yeah, we release code, you know, five times this week, but we've now got this problem at five times if we would have known earlier in the process could have Changed and fixed the defect whatever it might be made a different decision based on that.

Divani I'm real Curious your experience can give in your background with this too. You know, I I think that data has gone from being a set of reports that might justify, you know, kind of hiring asks to our bosses to to Really, you know, data applications and every organization what really amazes me about most of our customers is that they are all building very heavy weight complex data applications that are driving core parts of their business. So there's a few things that I think about.

I think that you're spot on with the data silos. I think there's also a really big role to play for data Ops teams around making sure that the right data is being collected, you know, a lot of people are still using systems that Are very manual, they might have a good understanding of their log data. They might have good monitoring data, but they don't have as good of a sense of the human data around their organization what processes people are running what workflows and then there's you know, kind of that closed loop process about understanding whether the things that you're providing to your customers to your internal customers are actually meeting the mark and meeting the need so, you know again back to what we've seen from from devops.

Like I really think that we're in the middle of a Renaissance right now where it's not just about instrumentation. It's also about asking how can this be a real business Advantage for our organization? How can we build a culture of people you know, like many eyes make light work of identifying problems continuously improving their and then you know, like the thing that gets me really excited right now is just the number of companies that are asking how do we make sure that down to the collection point we have that full integrated Loop so that if we find that a report isn't getting us We need we have a really clean way to get that communication back to the data Ops Team so that they can go improve the instrumentation in the first place and get the data that we need and and that's really where you start seeing the leverage start happening for organizations in my experience.

Andrew I'm cute. I'm curious. Does this resonate is is data Ops sort of the data scientist applied to devops is that that analytics is it that insight and we were offset more than that because you're talking about the Integrity of data across the the pipeline but it is at about getting to the information getting to it in a way that you know is more than trapped in a report or maybe trapped in some storage medium that we can effectively use as part of the devops processes.

Yes. Yeah. Absolutely.

I think I apologize for the audio issues earlier. My my opening line was about how our our data team was actually our first early adopters for devops practices when we first started our devops transformation and so our data teams really led the charge on that front implementing ci/cd pipelines and linting and automated data quality checks and things like that into their own workflows. And I think the impact to the organization has not only been that that's really accelerated our devops adoption on our traditional software engineering teams, but it's also I don't know that if you were to say hey Andrew come lead a devops transformation and Company X.

I don't know that I would have thought before to start with data. But if you think about the way data is used in an organization and the blast radius and the potential to sort of offset bad practices or mitigate. Yeah, I think bringing visibility into the Integrity of your data across the pipeline.

In to divani's point, you know quickly surfacing those issues and getting them to the right team so they can see them for a data team has a different kind of impact in terms of scale across the organization than what you would have and more of a, you know, a siled application team. It's even honestly influenced I'd be curious. This is maybe out of scope for this conversation, but be curious for this group.

The rate at which are data Ops our data teams have adopted data Ops practices has influenced the way we even organize our teams. So previous to this transformation. Our teams were really siled by technology.

So we had our data warehousing team. We had a reporting team. We had our application teams.

Now, we've got cross-functional teams that have embedded data resources on them because of this transformation. So yeah, I think going back here initial question. I think where does this sit?

I would start with data off transformation frankly. You're not the only company that I've heard go through that. We actually have a number of customers.

I used to tell people we sell to devops in SRE teams and I had to change that to operations generally because what I kept finding was we have a number of companies that you know, many of them were even more modern companies but as they were really looking at their landscape, you know, the thing about your your core business and your core business of code is that there's often so much Legacy software sitting there. Whereas a lot of companies are newer to the data space. And so it just means that the the Technologies the systems like it's all new.

There's not that craft. There's not the politics. There's not all that other kind of stuff and so you can really say we're going to start the way that we would like everything else to go over time and create an example for the rest of the organization and then it's just such an immediate business impact right like because you're you're immediately affecting all of these different business groups that have those needs around data.

So I I have multiple examples in my head of Is that have gone through exactly that same transformation and I now tell people if you don't have data around what you're doing then even if you go and Implement a devops practice in your core engineering team, are you going to know it's like if a devops team transforms in a forest and no one says here it actually improved engineering quality. I don't know. So kind of counters is I have to outrun the Bear right the bear, but I just have to run you.

Census that know that's a very interesting piece. And when I see that this only gets more aggravated when we see the newer Technologies coming in earlier. You just mentioned right regarding the data science piece.

That's that's another Beast, right? We difference between the data science or you can see the machine learning code and the other courses that machine learning code is live and it is just keeps on changing with the data that it feeds on so it has to have a handle in a different way. So how we used to have a traditional code where we ship it into the production.

It is remains Statics and it is done. So the devops and the data of practices were different with machine learning you need to see that how the data drift comes into play how you can just manage the models which we're working earlier. Are they still valid or not and covid was a great example.

It was a data that was not seen by the machine learning models. So once that's covid happened the machine learning models, we're just behaving erratic because they have not seen this type of differences so that led to the high demand. Clients for the machine learning model operations, which is the ml Ops which has come in and again devops being the umbrella data off being this subset and then machine learning also become a further subset of it.

And that was an interesting piece which we have seen data Integrity became an issue and how we can just install those type of practices which can help these pipeline to be automated and more responsive to the changes that we are seeing in the environment. I've kind of a controversial opinion on that. I'm sure.

Have a lot of controversial opinions, you know, so I think that there's just some really really interesting stuff going on in the AI space around large language models and transformer-based models. And I'm sure you guys have all seen Dolly and you know all the work coming out of openai and Microsoft. It's like it's an arms race, right like the AI winter is done and we're like very much in the middle of very exciting time.

But what I found very interesting is that with the rise of these large models, you're going to see a lot of organizations that are going to move away from training super detailed models for themselves and really leveraging a lot of things out of the box that you know, kind of come pre-trained on on billions of data sets and are able to accelerate stuff and so in light of that the most important thing you can do is make sure that your data that you are using and that you are pushing into those models is clean that it is well understood that you understand any bias that might be in that data that might lead to a certain decision or not. Not and so what I've been seeing is a real rise in the machine learning engineer. And the Machine learning engineer as a path to Ops and I think again back to data Ops ml Ops all of these teams, right?

It's like how do we take these practices that we've like seen work very very well and apply them across our organizations. You know, I'm pretty sure we're gonna start seeing the same thing in finance anytime as well. Curtis your your nodding in a pretty big way there as well.

I'm not even I I'm gonna kind of agree and disagree too. Okay. So um, so I've been around a long time you can tell by the gray hair.

Um, so we work with a lot of the large Enterprises. So you've got systems of Records out there that are very old name trains and things like that, right? Those are not going away.

I know everybody's dream about the day when there's no mainframes. They're never going away. They're in fact getting bigger.

And so what what happens is is that you've got to look at it from the standpoint of and and what the wording that I would use is democratization of data, right? So data Ops is gonna allow you now to take these siled data and that's the mechanism that really drives it through the capabilities you get that out to all the people that need that data. So it's a critical piece that's often overlooked.

So we we're in the test Automation and like you were saying the full circle test. We do all that beautiful, but if it's not continuous, Not built into the data Ops processes. It's not going to get where it needs to go.

And I think that's what you were leading back to Mitch to and I think that you know on the machine learning side what we you know, again, we were talking about the data quality data Integrity of those of what we learned. So we had a major pharmaceutical one of the largest in the world. They were working on Innovation for vaccines and we're big in their vaccines group and they were feeding and so the a lot of the proprietary stuff in Innovation, I know what Davanni's saying as far as like all these things that you can have access to from a data and model but they're Innovation is built on their machine learning and AI they're building an internally.

Well the data was bad. So guess what? They didn't get artificial like I say, it's not artificial smart.

It could be artificial dumb intelligence. It's all based on the quality of the data. It reads from and their data was bad so they didn't have success and back in.

These are hundred. Some millions of dollars of innovation losses and this is proprietary to folks so it is very critical that as you try to and I understand, you know, people try to buy us out the bad data and such but that's gonna it depends that might work or it might not your best bet is to make sure you've got good data that feeds those models build a data Ops process to drive that that automation around that testing their case. They were pulling every medical thing that ever happened to them rat since 1974 every single so it was incredible amount of data, but nobody was checking that to make sure it's correct.

And so these machine learning and AI is going to try to it's always going to be biased in my opinion what you're trying to get to and you've got to trust your data. It's gonna take you on the right path. That's that's what our experience was.

it didn't Integrity is kind of everything sounds like I mean that's a very valid point right means it is always use case dependent means if your use case is proprietary, you need to have your internal models. If your use case is generic then you can definitely use the open source models as well and cut it the example that you shared I can share here that this is not the only case we have seen these type of issues across the different organization as well. Right and the in the bfsi areas.

They all want to have these models as they're a differentiate us and that's where the data Integrity becomes Paramount if you feed in bad data, it's just garbage and garbage out. So the same pieces coming. Yeah, but interesting the second piece which I have seen the trend which has made this data offs, very different is also the cloud modernization Journey earlier.

It was just all on Prince and as you said, right that means but now with the cloud data you have to have different consideration into place one consideration is how you can Leverage The Cloud capacity and it can just means Consuming large amount of data in a very short span of time second is how you can just also ensure that the data is secure. So those security Elements which then translates into your encryption of the data making it tokenization and then detorganizing it as well effectively managing it into the cloud cost. So these different considerations also needs to be played into the data pipeline that we create and that also becomes a part of the data offs on how we are going to integrate these pieces.

How are we going to ensure that the data offs can help in the cost optimization? It can also help in the security consideration into place. So these are the two major things that I have seen means the cloud modernization and the Machine learning aspects which are now making this data Ops evolve and making the you can say value of data Integrity of Paramount.

Maybe your thoughts are on what the entrances are coming for you. I'm curious everyone building off of what you said as is anyone have a really good example with it for themselves or one of their wonder your customers companies you've worked with of where we all want software to help us as a competitive advantage or to ability to deliver the right software at the right time. I think it really means the right software the right data right to be able to do that by taking a data Ops approach any good really good examples where you can see the business.

Thinks more strategically operates more effectively is more competitive some Dynamic that you know, the the sea level folks going to say that was worth it. Whatever they did back in those groups that do all this work that helped a lot. Yeah, we have multiple examples when I can share and then we can just build on it.

Right. So one is during the covid time when the different come companies or particularly. I'm talking about the vfsi area the banking Financial Services.

They had to have the Casual Reserve ratios at their end. Right they have to amount for the uncertainties which is there and then they have to build up the amount at their end so that they can adjust to the loan defaults which can happen and it was the time of March 2020 around that frame right where there was a lot of change Market has changed abruptly and everybody was anticipating that there will be large amount of loan default. So the companies or the large financial institutions they were struggling with it because they have to just report these type of casuals within a short span of time to regulatory bodies.

Now, what does that mean? All these cash reserves and the loan default are being predicted by the machine learning models. So it means Models have to be updated to the new data in a very short span of time.

Otherwise, they will have regulatory impact. They will have their balance sheet impact, right? So this struggle this just made sure that the cloud can be leveraged.

It also made sure that how your systems can be more automated how you can just feed in the right data so that you can just predict the right amount of impact that you are going to have on the loan default and essentially on your balance sheet. So that one example where we have seen even the CEO because it is going to impact their earning calls. So CEOs getting involved in the outcome of the data models which are going to be there and it comes to my mind where I have seen that not only one company but almost all the large companies they were impacted by this and they have to just address it.

Another examples are built on that one. Yeah, I I don't know that I have a specific example, but literally everything that we're building at trimetics right now is data products. I think davanni made this point earlier what we realized is.

Yes, we've got technicians in hospitals repairing maintaining medical equipment, but that that workflow generates lots of really valuable data. And so that's that's been the core of our investment over the last four or five years that we've been going through this digital transformation as a company everything that we're building is built on data. So we we literally built an informatics platforms essentially what how we're describing it that we use to present data in different ways to different different personas, you know of our customers that want to understand different aspects of their install base and I guess to use areas example of covid if you remember early Cove and one of the One of the big scrambles was for ventilators, you know appropriate ventilator equipment.

I had in my state a good friend of mine doesn't Consulting with the state and we had top leaders at the state reaching out to traumatics to help us help understand. What hospitals do we have in Indiana that may not be properly equipped and I think that was not necessarily a turning point. We had started this journey, but really just a firmed that our product Investments really need to be in and around our data.

Yeah, I have an interesting one. We were working with a an earlier Stage Company in the telemedicine space and at the beginning of covid their business went crazy as you guys can imagine and they had started a process around setting up a data Ops Team and building out a data Ops Team. But the team was very focused on their automation capabilities a lot of the stuff that Arun was talking about and this was a company that has you know, like very modern and they have a lot of machine learning models a lot of Automation and I think if you looked at it from a surface perspective, you would say they're kind of you know meeting all the best practices they've implemented everything you would expect what they were finding was that they were missing this human in the loop element, which a lot of organizations also find in their devops Journeys where they think we're going to automate all our cicd pipelines and then they forget that you know, those pipelines break all the time like these things require, you know, we're still not quite at the point with like General AI where we can go on vacation and let the system Do its own thing.

And so they ended up investing very deeply in you know, just some some workflows and human of the loop processes that would allow for better Interruption of problems when there was an issue like really quickly identifying it bringing all the right stakeholders into the room and one of the things that they that they said is if you don't know what the ultimate goal of the data is then you don't know if the data actually has Integrity or not because the outcome ultimately drives what integrity means and so being able to go in and do that ended up giving them some really fabulous data. It stops them from growing too early in certain areas that you would see little blips and they started being able to on, you know, kind of bring together lots of different sources of data and and, you know based on on, you know, all the conversations we had it was really transformational for their organization. What can I add the fourth wheel to this go for?

All right, so we talked about health care. We've talked about utilt Banks. I'll talk about utilities.

I'll give you the name Duke Energy the big giant Northeast provider. So the use cases is that with ESG and the and all these use cases. You should really note are multi-million dollar problems.

These are not small issues. These are huge things being solved in this case. They have to monitor for any type of leak of natural gas in their entire system at all times.

If they The Regulators find out first they get fined. They got to find it fix it before they do. So data offsets really about speed and velocity, right?

We got to be able to do this quickly. So their use cases is that they would overlay all their gas pump and monitoring data on top of it. All the gis data satellite imagery and then be able to tie that all together in real time so that they could spot gas leaks and get somebody out there get it fixed before we get in trouble.

That's really what it boils down to and big trouble. I think one of the if they did not have that Automation and those processes with data Ops driving the entire bringing all that different data together, it would never happen right and you know, they're great customer of ours, but the the thing I think was interesting they would they solve something going on outside the Mercedes plant there in North Carolina, and they Rolled the trucks and when they got there they found out they had left open the door on the warehouse and they used methane driving forklifts and they were actually catching that data real time as they left the door open. So I just want to tell you this is the kind of things that speed and velocity of these problems that can just the that you've got to be on top of and it and this is this is what this is here for data Ops to allow you to have that kind of response.

It's amazing when the integrity and the access to the data is to be able to do all those things right because that's happened to battle is like, where is this who's got it out of a get to it? We've solved all those problems your long waste down the road. Well, let's do this.

We have just a few minutes left. I wish you had another hour to tack on to this conversation because I'd love to hear some more stories as well as experiences. If there's been a takeaway, you know in you've had multiple.

I'm sure in your careers and working in data and data Ops that you would share with someone maybe his earlier in the process of adopting this and their organization or maybe themselves. What are what are the a takeaway that you would say boy if I need the new this or took me a while to figure this out. But once I kind of got that that really helped me accelerate our path my path whatever it is.

Arun you want to you want to start us out? Sure, so I say that one thing that we have learned from multiple implementations that we have done across in the different companies is going by a developed tool can definitely help you to gain that advantage and the second piece is that it requires a close coordination across the data. Testing as well as the applications teams to be together so that they can just have this data Ops and so you need to have collaboration across getting it all the stakeholders together and leverage the tools which are out there in the market.

It can give you a good start and it can just make sure that the data of values that you can expect they are derived. Great, Andrew jump in. Yeah, I think I'll just go back to my opening remark and say my one takeaway would be to not be afraid to start with data on your devops journey, you know don't feel like this is devops is software engineering capability because it's you know, like I said was tremendously impactful for us starting with our data teams.

Very cool curse. I'm saving you for last so davanni, you know, I I think that My biggest piece of advice would be don't let perfect be the enemy of done. I think very often people say.

Here's my like two-year roadmap and we're going to you know plan on all these pieces and those are the projects that always seem to just drag on and on and on and on and never really get to a great place every great transformation I've seen in this space has started with a small team with a great mandate and strong support from the executive team to do what it takes because this is a coordination heavy area. It does involve lots of different teams. Lots of different systems these days there are lots of you know off the shelf tools and more coming that can help with that Journey.

But even there, you know, I think before you've you know, It's a little bit against my company. But before you've gone and spent the money on all the technology, you know, like really look at what you've got in-house what expertise you've got in-house and use that as a starting point and it's okay to take baby steps along the way there because they do add up over time and reinforce themselves. That's great advice.

Okay, Curtis. Let you bring it home one more time. I think those are I don't think I could top those three.

It's perfect and really kind of the the idea is that what we look at the world is that executive sponsorship being able to get the people that really have a vision involved and often getting their focus because there's so many different things going on to make them understand how important this process is going to be to everyone in the organization. I will just give you a hint. This is the thing I use and I've learned is that if you focus on a problem that is millions of dollars, you can get people's attention.

Okay. So what I tell everyone is like, you know, there's lots of things out there you can do but when you find those kinds of problems and you go to CIO or a CEO and you say, you know, you've got 50 million dollars and know your customer finds at the bank and we can solve that you've got tens of mill. Of dollars, you've got hundreds of millions Innovation.

When you throw those kind of numbers out trust me it that you can get change made but you've got to really to me that is a helpful part don't have to have it but I will promise you if you can find those problems and solve them for companies they will work with you and they there it'll change Minds. Well to connect the dot August I'll wrap it up this way, you know finding those those be hag those meeting problems that really can make a difference for your organization. You don't have to solve the perfectly right you can to borrow um to borrow a phrase or co-opt it from the Martian Matt Damon, you can devops that you know, what out of it right go after it and do this incrementally and make progress and that's how you not only course, correct, but you help yourself learn along the way this is an awful lot of things you pick up by doing not just by planning so great conversation super excited about data Ops and I'm encouraged for all the data walks in the world then be part of this process too.

So my thanks to you Curtis and devaney and Andrew and ruin for joining us today. Thanks for sharing your expertise. By the way, your room congrats on the I think your partner of the year with if Trae Sanchez ever, right?

Nice definitely good work dude. So give us give other something to Aspire to congrats to all thanks to everybody this joints today. Hey, I want to point out that we have a devops and Bound Live Roundtable coming up on July 27th.

Now it's this kind of a format but with the audience engaged with us in chat and discussion and that that tracks to a lot of times we're the audience goes and with their questions and response and we adjust them morph and you talk about Devin devopsing, you know, what out of it. That's kind of what we do on those live round tables. com website the webinar section, July 27th, and I hope you will join us.

The topic is living at the intersection between testing and observability. There is an intersection. There is very interesting conversation.

I will have a good time. Thank you so much for joining us and we'll see you on the next episode of devops and bad. Thanks for hosting us.

Thank you.