Andi Mann & Steve Klugherz – Doing DevOps as a Digital Transformation Leader

This is texturing TV. So we're here today to talk about doing devops as a digital transformation leader. What does that mean?

So before I start I have to tell you kumu is a publicly traded company. This is our Safe Harbor statement. We may say things today about products or future releases or future performance, but you should always check our website and the SEC website for the latest news.

So with that I will start by introducing myself, so I'm on the Right hand side as you you're seeing it on the CTO at kumu. I have had 30 years experience across five different continents. I think it is leading technology teams in banking Insurance Logistics oil and gas and as a software vendor and as an industry analyst I've published a couple of times and you can always hit me on my Twitter handle at Andy man, and I'll hand off to Steve to introduce himself.

Hello, I'm Steve pflugerts. I've been in the industry about 20 years leading Global teams. My career has swung between giant insurance companies and small little startups.

I've been leading teams working in both the both public and setting up private clouds within the that large insurance company for about 10 years, but I've been automating operations even longer. I was actually one of the first I was one of the first teams to roll out no downtime deployments on websphere within the United Health Group. So I like to think that I was doing devops before I was really cool that But so what we're going to talk with you today doing devops as a digital transformation leader.

I'll explain a little bit about why we're transformation leader what that transformation really is. We'll talk a little bit about the different phases. We went through on our journey and we'll wrap it up with some lessons learned and maybe we'll even have time for Q&A.

So just a quick couple of slides about Kuma. I'm not trying to promote but I want you to know where we are and what we do. So we're a cloud-based Enterprise video platform.

So we provide the capability to create control deliver experience and analyze video at scale. So our platform is running on Google Cloud platform. It's integrated.

It's secure it's scalable. We provide specific use cases around things like internal Communications external Communications collaboration onboarding training education learning and obviously go to market. marketing for those videos So just in terms of our customers, this is sort of the this is the importance of our platform right?

We're working with some pretty significant businesses around the world. They rely on us to do live streaming and On Demand videos. So, you know, you look at these numbers.

I mean some of the examples maybe Toyota right during the pandemic Toyota was unable to bring all the dealers into Tokyo to do a product launch for their new tundra. So they used kumu video instead all the deals around the world watching including a 360-degree camera inside the cab. So the dealers could look around and see what that vehicle looked like right live real time.

This is a really Mission critical launch event for Toyota CVS use us to provide videos to educate their customers on how to use certain medications or certain medical procedures and so forth. Who else have we got up there American Express? So American Express the CEO gives Town Hall, I think once a month to around about 40,000 employees at a time.

So, you know the CEO of Amex needs this platform to work every time straight away. No problem. So for these businesses kumu is absolutely a mission critical platform, and we treat it that way.

The transition we're going through is a cloud transformation. We're moving from traditional on-premises installed software to delivering our service as a fully Cloud native SAS application multi-tenant the whole the whole nine yards. We're in that process.

We're not going to tell you we've got everything right and we finished yet, but we're still on our journey and we'll explain to you a little bit about how devops and our devops transformation has helped us on that Journey. So with that I'll hand over to Steve to talk about the actual transformation. We've been through.

All right. Well, I was brought into Kuma about 18 months ago. And when I came I assessed the team and how we were operating.

I saw that we really faced a number of problems and how how we were delivering code and how we were running that that code. We we had it was everything from how we arranged our people to how how we were constrained. We had a mismatch between work and people we were swinging around people to projects as things got behind.

We really arranged our structure in a very siled way. We allowed teams to build castles around themselves where it was a mystery what went in on inside that team but you made requests of that team to do work and hopefully work came out the the other end but what I also observed is that we had a very talented team, but they were constrained by the by the Paradigm we put in place for them to operate so we started thinking about how we're going to fix this. What are the things we need to do.

We want to put in place the system that allows us to release Master deliver faster value and really at its core liberate that engineering talent to deliver and to build something they're excited about and build it fast. So, where did we start we? We started to look we looking at how we arranged ourselves.

We like I said we are assigning we're assigning people to pieces of work and oftentimes people would be assigned to multiple pieces of work at a time at a time splitting their attention swinging how that that's distributed between features that they're working on based on how far behind it was who who was asking about it at a given time to the extent that we actually had things like scrum teams or squads. They they weren't cross-functional at all, you know, the back end team would hand off to the ux team the ux team would hand off to a QA team and the QA team would then ask operations team to do to do deployments. We you know, we set up these segregated duties and limited access to make sure that those responsibilities stayed within the given given teams because we didn't trust anybody to do to work outside of their scope and a lot of this comes from the legacy of where the software came from itself is 19 years old.

And so it started. You know when the idea of like a managed service organization was new and that's how you ran software. And that Legacy had came forward as we had migrated to the cloud.

Operationally we had a really done a lot of automation of that managed Services world. We you know people didn't do a lot of manual work, but that was a request-based system that was handing things off between between teams. We'd done what I referred to as an automated version of Conway's law that people had their scope and they had just automated their scope, but they never thought about the whole system.

We it really drove a couple bad behaviors. But the one the one thing that I really saw driving was what I called Mega releases. So you'd have a release piled up and take an entire quarter would have tons of features tons of bug fixes.

It was all rolled into one to one package and then you always had had the Colombo moment where you have already to release but some engineer would say, oh just one more thing. And they'd then you'd have to pause it put it in and go back through a QA cycle. So we where did we want to go?

We we wanted something a process that would allow us to do. To rapidly deliver small pieces of functionality from check-in all the way to production. But not wanting to be somebody that would let the ideal system get in the way of progress.

We figured we'd settle for something that would rapidly get a check-in into our QA or stage environment. We wanted to build ephemeral environments at this point. We had had we had multiple QA environments and they sat, you know at night they sat Idol, they oftentimes weren't used or or worse yet.

They were bottleneck because we had three things that needed to each have an environment, but we only had two of them around so we'd set them up and tear them down. It was expensive and it was inefficient. So we wanted to get something that resembled environments that were there when you needed them and then we're thrown away when you didn't essentially getting rid of the idea of multiple stages of non-production and just saying there's non production that goes away and there's production that sticks around And we wanted to do that by leveraging our Cloud the native Cloud capabilities that even at this point a lot of what we had been running has pulled forward that Legacy of how you wrote software up 10 years ago.

It was it was monolithic. We were a lot of how we're using. Our cloud provider was just as a provider of VMS that we then installed software on.

And we and we weren't leveraging a lot of the capabilities like Auto scaling and so forth and more important more than anything. We wanted to see how we could open up the access. So we wouldn't have bottlenecks on teams of special on Specialists.

So, how can we open that access? But put in place the guardrail so we could trust we could trust people to work outside the scope of their job so verify that they're doing it correctly. And we had to do that within a pretty tight series of constraints as Andy showed on the slide.

There's there we had big customers and it was mission critical. You can't screw up a product launch for a large Auto, you know a large automaker. You can't screw up a CEO town hall for one of the big a big bank.

So we had to come up with how are we going to do this that in a way that doesn't disrupt our customers our our sales and our business partners. aren't going to let us stop working on features to deliver this and Frankly, it was a big system to port and how do we and break up and and push in any sort of re-architecture that we needed to support our vision? and on the team side, you know we We had some limited knowledge and there's a saying you we could would have loved to hire a bunch of people that have done this before and made it easy, but you go to Art you go to war with the Army you have not the Army you want and so we went to we started working with our team.

Some of them worked in big companies had done parts of this before. Some of them had worked in small small companies and had experience doing parts of this. Some of them had deep hit knowledge of the product and the history of the company and some of them had experience in the cloud, but what it all boiled down to is we really needed to learn and we needed to make progress.

We started saying that we needed to make things 1% better every day. So we wanted to make sure that we weren't just trying to take giant swings that slow steady progress. Is actually a very agile way to approach your transformation a little bit better every day.

Well then. So I assume that means because we're talking devops here, right? So that's all about tools.

Right? Well you yeah, you think so, you know, we're we're Team engineers and so naturally everything everything's a tool problem. It's all you solve every problem you have with with tech.

So what did we do? We went out and we bought a tool to orchestrate all of the all sorts of cicd processes within it. We put one person in charge of it said go.

You know, he worked in our Ops Department. We said go crazy. We you know again because we're Engineers product management project management planning is not fun.

So we were able to hide from that by saying well, this is really internal improvement work. We're not going to put it on the road map. We're not really going to make a plan for it.

We'll just sort of do it in our spare time. Didn't really scope it didn't account for it. And as as you probably guessed we really made limited Headway there the you know, naturally everything else was a priority even running the old Legacy operations automation was a was a priority over this.

We didn't have a plan we knew we wanted automation, but we didn't really know what we wanted to do. And really focusing on the People level what ended up happening is the person working on it was was not happy doing it. That's what they were creating wasn't being used and they really felt like they were out on an automation Island there.

You know, they were sent out to do cicd's Siberia, you know, they're just out there alone. Building something that no one was using. so yeah, we pretty quickly learned that you know, this ain't a tools problem.

It's you know. Tools are easy there that there is something that you have to consider before you think about the tech that you're applying to applying to a problem that that our team our processes our structure. Our culture are much more impactful than the tech we use to address the problem.

And that we were probably scoping this too narrowly, you know thinking of it purely as an automation problem really doesn't transform your business. It just gives you a gives you your mess for less or maybe faster in our case. It just gave us our mess for more expensive.

Up, so we had to rethink what we were doing. So obviously I I have to walk back what I said before obviously devops is not tools. Devops is people right?

So it was just all about getting the right team. Yeah. Yeah, that's that's where we went next.

You know thinking deeply about it. We said well, obviously we didn't get this done because we just don't have enough people doing it. So we went out and brought on a bunch of people.

We just threw them at the problem. We hot we said we can't have this be in our Legacy operations team because they're too busy running the Legacy operation. Will spin up another team.

And we'll give it to them. We delivered more but it was delivered without input and it still wasn't really used very much. We really it's really driving back to the fact that we didn't change.

How our developers worked we didn't change how they were tasked to things. We didn't change what was expected of them. And we really never put in the time to think of holistically about what we wanted to accomplish and what the problem was that we were trying trying to solve.

So again this fell on its face again, and we had to go back and do and do some thinking and think about what we learned we learned. that that we need to have that the dedicated people can can help us. Some people say that a dedicated devops team is an anti-pattern but sometimes you need it just to get going and he told me once that walking on crutches is an antipatter too, but sometimes you need them.

more We needed to think about the entire delivery pipeline all the way from code to operations. We we had all our code in a in a monore repo and there's nothing wrong with mono repos. Some of my best friends use mono repos, but it didn't but it didn't work for us.

It didn't work for what we were trying trying to build and how we're trying to divide up the team and try to divide up the tasks. So we needed to get something that aligned with that we couldn't just focus on the D part of cicd. The delivery was was great and interesting, but if you don't have there's a reason the eye comes before the D.

If you don't get that part, right, all you're doing is automating the last step in a process and we hadn't really focused on that. So we spent some time going back and focusing on that. and we really needed to involve the team and have get everybody from the engineers to our Ops people on board with what exactly we were trying to accomplish accomplish.

And so we spent some time designing that Golden City on the hill where we want to get to but we probably will never actually get to it but and that's okay, but you got to have a Target that you're you're shooting towards. What's your ideal state? And you need that because if you don't have something you're moving towards you're just going to keep on moving towards what you already have.

And so you want to be very conscious of not. just rebuilding what you have if we hadn't done this, we would essentially just done a port of a bunch of jobs in Jenkins to a bunch of jobs in harness. And we the probably the biggest lesson we learned is we have a legacy monolith.

It's big it's complex. It's in the middle of everything. It's very risky to change it.

Yeah, you know and we have customers that aren't forgiving of that change. We learned that we couldn't do a big bang that that was probably not the right thing to attack. Okay, then so.

In devops tools are important but devops isn't tools people are important but devops is not a role or a team. So I guess the next step is just to make sure everyone knows what you're doing and make sure you're getting that executive support from people like me Evangel. Ize, and that's really what we we learned I said, like I said, we spend time we Define that vision and that Golden City on the on the hill.

We you know, it was difficult for engineers. We took a lot of time thinking about the people that we're going to do this at the end of the day people use tools to accomplish work tools. Don't do things on their own.

So we had to think about the the people we had to look at them as agents of change not just factories that turn out churn out code. And we had to win their hearts and Minds we have a very empowered developer group. And we had to you know, we had to convince them that this was something they should do.

It was that this was new work for a lot of them because they couldn't just hand it off to another team. So we had to we had to spend a lot of time. Convincing and selling the vision and and getting participation and buying in the vision.

So we had a lot of discussions a lot of debate a lot of talking about not just the what but the why we're doing something. And talking about how we divide all those steps up on the on the way. It's great to say it that I want to fly to go to London, but it's really nice to know that I'm going to connect through JFK first and so we started talking about all those connections connections on the way.

And we realized that we needed we really did need somebody that had been there before. We still had our dedicated team that was going to be focused for a while on building this. But we needed someone who had walked this path before so we brought in we brought in a some contractors that had done these conversions at other companies before and it was a great help.

He helped he helped us reestablish how we were dividing up our code in in our git repositories how we are how we were using the hooks to trigger jobs how we were going to put gating factors in to make up for places where we had shortcomings in say testing. We spent a lot of time talking about our application decomposition. We knew we weren't going to attack the monolith anymore because it was too big too risky.

But what are the decision decisions we're going to make to say when we do we attack something was what is appropriate to break off and then rebuild and then put into this new new process. And so we we settled on the Strangler pattern where when we have a council that that looks at whether a job is big enough and warrants the breakup of the breakup or the carve out of a capability and then any new capability we put into the new new deployment process and we set up a paralleling that we set up a breakdown strategy for a mono repo how What's the process that we decide when we're going to curl something out of there and set it up in its own process and have it live essentially as its own? And really we decided that we're going to focus on this on small wins.

We hadn't done a lot of this before. You know. Let's let's focus on getting getting hits.

Let's focus on the singles and the doubles before we worry about hitting home runs. And so we learned a lot we learned that we that small wins worked for us. We We learned that our Engineers are empowered and that they are risk-averse.

And that the small ones proves to them that something can work. They there they engineers at least my Engineers they are they are versed to Magic and they are over they don't believe in sales. There's there's nothing more than they like that that they like more than going to a vendor demo and just picking it apart.

So showing them the winds is very important. We learned that big really didn't work for us. Again.

It's way too high risk and more importantly. It's just way too long before you get value out of it. There's a you know, we were a public company.

We work on a quarterly basis. So we have quarterly Horizons. An Engineers have an even shorter attention span.

You need to get them value in a win quickly. and we thought about our people we learned a lot about our people we We learned that. That the way we had approached this and even bringing in a new tool is tearing down.

Somebody's Castle. It's a threat to the the order people can see it as a threat to their job. It's a threat to their ego people who are really good at writing those Jenkins jobs really good at going and manipulating settings and Linux kernels.

You're taking you're changing their job and it's scary to them and they they're going to be hard to bring along. So you have to show them the path. And and give them the opportunity to participate in the new in the new thing and put them alongside.

The people that are benefiting benefiting from the new new way of operating and try to bring them along but you have to prepare it for what happens if they don't want to come along. the last thing we learned is the Legacy is going to be here for a while. It's you're just gonna have to deal with it.

It doesn't magically go away and we're going to have it for a long time because it's it's big it's risky. It's going to be slow to change and there's a certain element of if it ain't broke. Don't touch it, especially when you have something that's old and maybe not have maybe isn't fully documented everywhere.

Maybe yeah. So what I'm hearing here is that the devops journey we've been on has not been a straight path, but tell us about how you were able to bend the curve really. It's getting the winds under our belt.

We we Showed the benefit to the engineers the first service. We did the first micro service we converted to the new the new way of operating. We had Engineers telling all the the other teams that they're getting bug fixes out to production in 30 minutes.

It used to take 12 to 24 hours to get a bug fix out to production that they're talking about how cool it was it was just driven off of the approval of a pull request instead of having to open a ticket through a work request system that And then write a bunch of documentation. So someone else could go do it go do the work. We we learned that we are we started getting more people involved instead of having this team just a centralized team.

We started transitioning them instead of just doing the work. We started transition transitioning them to being more of a consultancy group and education group and really focusing on the platform not necessarily building the pipelines for for a specific project. They'd build templates.

They'd build models they built how to use the the one of the keys was getting the development team involved and that was a facilitated by transferring that that cicd or the devops team into more of a facilitator and a Consulting role. and we we started to Expand what we're controlling we started pretty small. Like how do you get this one thing into a kubernetes cluster in gcp and then we moved to well, how do you set up the entire gcp cluster are the kubernetes cluster in gcp?

And then we started getting the people got excited about it. And they started asking questions. Like why can't I control my content delivery network with this too?

Why can't I? Why do when I set up a new environment do I have to then go manually create DNS records? Why don't I just put that in my my pipeline too that it'll set up and so people started getting wins and they start getting excited about it.

And as a leader, my job was really to magnify those winds back on and reflect them back on the team but more importantly to publicize those wins out to people like Andy our sales team our customer success team because this is a lot of work and it's not work that's resulting in a new feature or a widget that they can turn around and sell to a Toyota or CVS or a Barclays. So you have to Do a lot as a leader you have to do a lot of explaining about why. You're sacrificing a feature.

For this internal internal work and what I found is making that visible getting that transparency out. There is actually very well received people understand the story of going slow to go fast. And so you have to just continuously remind them about it and talk about the winds and make the work visible.

and along that line of making the work visible we We actually are actively putting this this work on the roadmap now. So it sits there right alongside building a new analytics capability or building, you know, integrating with this collaboration platform. It's right there alongside that we're going to extend ephemeral environments up to QA or we're going to create the ability to spin up a low testing environment.

It's it's right there right next to it on equal footing and we talk about the prioritization of that work against Feature work in the same meeting with the same groups of people. So everyone understands why it's important and why we're doing it and why we're sacrificing something else to do that work. So we learned that we were still learning but we learned a lot we learned monoliths are hard.

There's a lot of attitude especially in something that's been around for a while in your engineers. And your Ops Team to say just don't touch it. It's working.

So you need to go slow into that. We we needed to go a salon to that space. It might not be the best place to start to attack that monolith.

It might be too big a swing especially if you have a team that hasn't done it before. we learned that that success drives commitment in buy-in the The you need that bind because people are more important than the tools you're using and the culture you're establishing is more is even more important than the people that live that culture. We have to be trans we learn to be transparent because it drives interest it.

It drives by in from the people who are writing the checks to it to our team to do the work. And it shows the value about why we're delaying that value into the value that we can provide our customers into the future. And we learned what what a t-shaped engineer means does everyone's heard that term of t-shaped person.

We used to have I guess there'd be I or lowercase L-shaped people. We we originally went in saying well everyone's just gotta know how to do everything and we learned quickly that that wasn't realistic so we changed it to to our idea of a t-shaped person to having awareness and varying levels of ability in Surrounding skills a knowledge and an interest in what those those? Skill sets accomplish and how they do it.

and it's our idea that you don't we don't build a over the wall, you know an overall culture that it's the next guy's job. You need to know and be aware of how what you've done impacts the next guy down down the stream. Overall, we we learn some lessons.

We we achieved quite a bit that I'm actually really proud of the team. We have much faster deployments. One of the things we measure is our deployment velocity.

That's a pretty common devops metric and we have probably about 30 to 40% of the platform migrated to the new the new way of working over the last month. We've done we've done nearly as many deployments in the last month as we did in the entire entire quarter previous year. So that's pretty pretty solid Improvement.

We we have more reliable delivery. We one of the things sort of the side effects of this is we actually have we're meeting our commitments to our to the business. We used to set up a roadmap and was very common for most of that to slip into the next quarter.

We've had multiple quarters in row of actually getting those those commitments and that's really a result of not having all the unexpected delays. We have burndowns that burn down now just novel we we have a better from the people front. We have a better developer experience people like getting their fixes their their work out there fast.

The engineers are proud of what they do. They want it out there. They want it being used.

It drives them nuts to see that it's bottled up waiting for 10 other features to be delivered. And we actually have delivered our ephemeral environments for our Dev and our QA our QA levels. It's very cool.

I just got a report that we brought a team on in California and we had 10 10 developers set up their own dedicated development environments in a day, which is unheard of before you you would take a week or a week or more to get that set up the way we used to operate because all of it would have been individual requests to our operations team to do multiple steps along the way. And probably the best thing is we have shifted to having an environment where the developers are actually excited to work in it and they're interested in it. And we're we're really leaning hard into that and trying to accelerate accelerate that interest accelerate that transformation and capitalize on that interest.

We've got we've garnered yeah. Lest you think that we actually have it all figured out. There's a there's a lot we need to do the biggest things that we that this transformation is exposed is that we need to have much more rigorous rigorous testing.

We historically we had only we'd really only tested at a full system level, you know system level style regression testing maybe and then a little bit of feature testing that we'd roll into the next regression test. As we're dividing up into microservices. We really need to fill out those bottom levels of the test pyramid through our component testing and through our integration testing.

We need to as we're breaking this up. We need to really think more about how we're managing our contract and component life cycles and how the components talk to it. Each other mono repo is one of the things we put in place really because we didn't want to worry about breaking one thing by by changing a speck or a contract on another component.

So we just rolled it all into the same the same Repository. As we're breaking that out. We're finding places where We did change something and now it broke something and that leads us back to testing we need to have you know, not only better system testing.

But we also need to have better Post Release testing so we can roll that stuff back right away. And we need to convert continue to think about our people and convert the team that owns that Legacy automation platform. Like they they do good work and we need them around and we need to bring them over because they have a lot of knowledge and a lot of a lot of skill and as the demands on them decrease, we need to start bringing them in into the fold and and training them up on the new on the new platform.

so the lessons just in a quick order, you have to think about your people people are there's a reason that people are first in the people process and tools model that this is a this can be seen as a threat you're tearing down people's castles your your Impacting their jobs and what they do on a daily basis and you're telling them they have to change and sometimes that's not well received. I you we needed a dedicated team again. It's we may not we won't need it forever.

But we needed to get some wins out of the gate and this was part of getting our wins and that this is really a sales problem. Not a tech problem. We need to develop We need to go to our thought leaders Within.

Your engineering organization and get them on board get them involved with the vision get the buy-in and have them help you sell to the rest of the team. and you have to we need to have a culture of ownership and Trust because if you aren't doing things because you aren't opening up access because you say you that person doesn't know how to do the job. The answer isn't to lock it down the answers to provide training right the guardrails and so around it.

But you need to trust your people trust your people to do the work and trust your people to know what they know and know what they don't know. Really you have to be prepared that this is a big shift in how people are approaching their jobs. The reality is some people aren't going to be able to make that shift and you may lose people along the way and you just need to be prepared prepared for that that there are just they're people that won't come on the journey with you.

From a process side it the biggest learning was it doesn't come free. This isn't just something that continuous Improvement on your Ops teams scripts that they have been maintaining for 10 years. This is this is akin to writing a new feature in into your platform you need to plan for it.

It worked for us to start small. Because of the risk of going big the delay in going big and the need to generate Goodwill among our engineers. And that you have to think about the entire delivery process.

It's not just about automation. It goes all the way down to how you store your code. this tools need to go out and buy big expensive tools, but they can help.

We're a smaller company we decided we didn't want to have a team that was responsible for doing life cycles On devops Tools. So we bought it if we are, you know, if we were 3,000 person company, we may have had different story. We have to decide what your breakpoint is and where you want to put your energy and we decided that we would just buy the buy the maintenance and of all those tools the best tools though will not solve the solve the your problems of people and process you have to think you have to be forward thinking about that and I would suggest even thinking deeply and working on solving those before you go out and by a tool and again for us we were small we didn't want to run it ourselfs it was so long the long and the short of this is that what I learned through this journey is that one you're almost guaranteed to do it wrong out of the gate.

And that's okay. You should be okay with that. That's the heart of Agile development is do experiments fail fast.

Keep in mind and remind your teams that that 50% better is better than zero percent better. So we just keep moving. That you should experiment you should fail and above all that remember that this is a people change not a technology change tech tech is easy.

Tools are easy. It's the people and the mines and the culture that are hard. Excellent.

So thank you everyone. I think we probably have enough time for questions if there are any in the room. Way up the back.

Yes, speak up loud for me. Please hear me. We're very refreshing for your realistic description of the process.

That's the problem. In this possible presentation. Wow any Mission whatsoever of management involvement?

All right, so not just from you. But basically the national is completely uninvolved in this whole process. You have.

No knowledge. No no interest whatsoever. But what is going on here other than it's expensive.

So the so the the question just for the people on audio from all of this. There's no mention of management. So is management involved at all.

I assume they're not because they're not mentioned. Why don't you jump on this too? Well, you have to remember that that I am management.

But you know when we think I assume when you're talking about management you're talking about business. business management right, so the they're involved in they they do a great deal of establishing the requirements of what they want you they you know, when When we go and tell a car dealership, we want a car or an iPhone or whatever. We don't tell them how to make it.

We just tell them that we want it and we want it at this date and we want it at this price or an Apple's case. They tell us that's twice that price. So Management's involved and involved to that to that extent that they knew they'd set down requirements that we needed to get things out the door faster.

We needed to deliver value to our customers faster go forth and diagnose why we're not doing that and what we need to do. And then it goes down to the tech that from there. It really goes into the technology management where we do that diagnosis.

We figure out what our problems are. We figure out the steps. We need to take take to fix them.

We pick the tools to fix them. We talk about our culture about how we're you know, all that culture changes is to a great extent a reflection of my leaders and what they've done to push the those the values and the we are engineering philosophy pillars into into the teams. So there's a lot of work on that front that is is purely is purely management going back to the business management.

That's where the the making the work visible comes in because they've they've said what they want us to deliver. The their involvement from there is is really about receiving and understanding updates from us about how we're working towards that goal. They've established for us.

But no, I wouldn't expect a business someone on the business side to come in and say well you should really have a femoral Dev environments that you can set up in an hour with a push button that that that's really worth in my mind that's working outside of their scope. We say that we as a Technology Group would propose that as a solution to a problem that they're presenting to us and just from the from the safe sweet perspective as well making that work visual the evangelization of the work we're doing and why it's important to different partners of mine. So for my CEO the the opportunity to get things out for the CEO and CFO the opportunity to get more things out at lower cost was quite attractive for the head of Partners and revenue the opportunity to deliver new features really quickly for to get new customers on board or maybe to satisfy existing customers was really important for the chief customer success.

Office of the reliability stability of the platform whenever we put new features in by doing automated testing and making sure that as Steve said more and more reliability goes into the platform. So existing customers are happy that was part of my job to evangelize to my peers why this work that I refer to as engineering Innovation is actually really important to my peers in in the business. So you're absolutely right don't talk about a lot but there was a lot of evangelization especially making the work visible explaining why it was important to their function that we did this technical work.

Any one more question. Yes, I have. hearing to talk about The job lovely, the price is different.

positions and what that does in terms of years of so I'm just going to repeat that question for the recording just to understand a little bit more about the job leveling and how levels and titles impact the ability to go on that Journey. Um, we're still working our way through through that process to some extent but what we what was being referred to as when I came in to kumu there were job titles all over the place. He was a lead lead engineer a senior assisted men a cloud engineer a devops engineer at QA analyst at qatley.

There's just all over the place and it made it really hard to To build cross-functional teams because everybody saw their job slightly slightly differently. Essentially, we'd given job titles based on what they wanted when they came in on their resume and I had had the fortune of working under a really a really good leader at a previous job that she had walked me through that for her Engineers. She wanted three levels, you know, there's an engineer senior engineer principal engineer and you don't even talk about Specialties in that so that's what we rolled out that we set up various engineering levels.

And then you could have practices within that like QA devops front end back end, but your job would really be defined as a senior, you know and engineer senior principal and the way we Define that was centered around not really what you could do within your practice, but your scope of your sphere of influence. Are you An individual contributor that needs a little bit of guidance. Are you somebody that is exploring your industry, but mostly onto yourself.

Are you are you influencing within your team? Are you influencing within the business? Are you influencing within the industry?

Are you a thought leader? And it really didn't matter what tool or technology or what skill set you are applying that within your seniority level was based on on where you sat with that in terms of rolling it out. Yes, it has been Some people a lot of our job titles in in certain geographies, we're able to change because people saw them as similar and they understood the process certain geographies.

The title is very title is very prestigious. and we had to approach that more. in a more thoughtful way, especially as a small company where we didn't want to we didn't want to drive a good person out the door because he's no longer that Yet Chief technologist, and he's going to be a principal engineer now and and that he sees that as a step down even though there's really no functional change on the kuma side and what he what that person is going to do and accomplish in those geographies.

We try to handle it somewhat a little bit more through attrition as people leave we hire in under the new Under the new classification model and at some point when there's a critical mass will We may flip it. We may flip it over but one one last thing that just popped into my mind. So sorry for not being structured on this is we also separated out the idea of your human Capital Title and your business title.

So your business titles what you put on your email signature and you can have that be whatever you want. But internally when we're talking about engineers and we're saying we're going to do a talent assessment or a rack and stack we look at the human capital job titles not the fact that you wanted to be the lead University liaison in Dubai or something so well, thank you all I think that's all we have time for just before we wrap. This is my Shameless plug slide.

com/careers. Otherwise, you can connect with either of us here. Obviously Steve there is on Twitter and his email address my email address and I'm on Twitter as well.

No Gatekeepers. Please reach out to connect anytime. And with that I will just say thank you and enjoy the rest of the day.