How AI is Reshaping Engineering and Redefining Developer Satisfaction

Hey everyone. Welcome back here to Techstrong tv. My next guest is Amy Carillo Cotton.

I can't say the, the R is like a real person can, but Amy, forgive me. I did the best I could. Why don't you say it for me.

Uh, my name is Amy Car Cotton. There you go. I love it.

I love it. I wish I could say that. Amy is the director of Client transformation at a company called Uplevel.

Amy, first of all, let's talk about you and your role, you know, client transformation. What exactly does that mean? What have you done in your life to get here in that role?

Um, I hope it was all good, I'm sure, but give us a, give us a little background. Absolutely. First of all, thank you, Alan, for having me.

Very excited for this conversation today. Really like your respective on things. How I came to be director of client transformation, actually started in my kitchen growing up.

My dad is an engineer, a technologist, A CTO, and I thought everybody had a whiteboard in their kitchen. I just thought that was, turns out it's not. Um, and went on to have this liberal arts education, but gravitated back towards tech.

I've been in technology for 15 years. I started on the product side and I just saw engineering teams really struggle to do their best work because of systemic factors, things outside of their control. And that became frustrating after a decade or so.

And so I moved over to working in engineering effectiveness, engineering experience, developer experience, whatever you wanna call it, working to make systemic changes to make life better. And that's how I came to work at Uplevel. My work at Uplevel is taking the data and using it to make change, because it turns out that just having the data isn't enough.

Very cool. Very cool. I love it.

People who are familiar with Uplevel, give them, let's go a little deeper. Tell us about Uplevel. Absolutely.

Uplevel is an engineering effectiveness system. So it has some dashboards, takes in data, does analytics comes up with metrics, that's cool. But we are more than that, we're a system of transformation.

So we have what we call our method, which is a way to take that data and turn it into a system of ongoing improvement for your organization. And that's the part I run, I run the method part. Excellent.

Uh, website. Oh, yes, you can reach us. com.

You can, uh, follow myself or the CEO Joe, uh, Joe Levy on socials. Uh, we are often commenting about, uh, the state of engineering, the state of engineering measurement, and really, like both of us have a passion for helping leaders be effective in transforming their orgs. And that involves the data, but that also involves being part of the conversation around how engineering is perceived, and especially now in the age of ai, how to measure engineering.

I love it. Amy Uplevel recently did a survey and a report came out around kind of bottlenecks in, in software delivery and software in general. Um, you know, it's, no, I don't think it's a surprise to anyone on our, in our audience that we're generating more code with ai.

Everybody's using it. It seems 90%, maybe as high as 90% of developers using ai. Um, well, I don't want to steal your thought.

Is your report, why don't you give us some of the salient facts here and, and where, and you know, some of the conclusions you seek? Absolutely. So we've been researching AI and the impact on engineering for a while.

We have actually two reports. Our first report, we looked at the impact of AI on quality. We knew about this AI slot problem way from way back when we started to see that yes, engineers are creating more code, more merge request or pull requests, and at the same time creating more bugs.

And so we started to see that there were quality impacts, um, from AI from way back when. And then we followed up with, uh, a report on understanding how leaders measure ai, how they plan to address this skill gap and how they, uh, plan to deal with AI from a strategic point of view. Um, that was, um, our more recent report.

Very cool. Very cool. So, you know, it, it just coincidentally, the, the day we recorded this, you and I, our Textron gang this morning spoke about this issue, which is developer what's called developer satisfaction or developer happiness.

Developers like to develop Yeah, right. Software engineers like to develop software. Their role is changing.

Yeah. They're now becoming, and I think you call it in here, the AI quality guardians. Right?

So in other words, the AI is generating the code and, and it could be at the direction of the developer, don't get me wrong, or it could not be going forward, right. It becomes more autonomous. But these developers instead of actually writing code, are now sort of checking the code that gets generated by AI before they move it down the pipeline.

And that makes for very unhappy and dull developer, it seems. Um, they're not, they're not real happy with that role. They, they like to develop code, not check AI's development code development.

Um, and it creates, and that, you know, I'm a big Bolivia, I dunno if you know the book, the Goal, uh, yes. Right. Talking about the, the goal.

Oh, really? Okay. And then of course the, you know, the big DevOps book, the Phoenix Project by Gene Kim is based on the goal.

Yes. But really, you know, the, the, they're both about bottlenecks, right? They're both about theory of constraints and as you, you know, one bottleneck to the next.

So now it seems like we've moved to a era where, you know what, generating code's not such a big bottleneck anymore. We're generating more code than we know what to do with it seems. But checking that code, making sure the code works, it's safe, it's not buggy, it's, you know, it's quality.

That is the big bottleneck now, isn't it? Well, that's the bottleneck at the team level, but I think there's, there's actually something going on where there's other bottlenecks at the systemic level. Engineering leaders know that they actually have, uh, skeletons in the closet.

They have tech debt. They're dealing with, they have architectural complexities they're dealing with, they have things that are holding them back and will hold them back. Even if engineers become the best quality guardians they can.

So this is a, there's layers to this problem, certainly at the team level. Every engineer I talk to is super stoked on ai. I mean, I'm, we're seeing numbers higher than 90% right now.

Um, and they're really excited to use it. And also there is definitely a current of wait, but this is not what I wanted to do. I wanted to build, I wanted to make new things.

I wanted to generate code just like pop up on my IDE and just like bang away. And the more iterative conversations prompting AI and then the quality work on the backend, it changes the role for some people. And then, yeah, not everybody is loving that transition.

The engineers who I see navigating that, that, well, they already had an understanding that their job wasn't just code generation. That their job was about fit for purpose. Like not only making code, but making code that really meets the business need that is like scalable and beautiful and elegant, like no code smells kind of thing.

Those were the engineers who are like, oh, cool, this, this makes me better. And I don't mind the fact that I'm now spending more of my time thinking about is it fit for purpose? Is it scalable?

Is it beautiful? Is it elegant? Um, so yeah, but not every engineer's in that boat.

There are some engineers who kind of like didn't understand the assignment. They thought that their job was just co-generation. Yeah.

Well, At, At some organizations and in, and in some circumstances it is, It's, yeah, Right. I, I know people who like, they're just happy to just sit there and, and code. Um, but you know, the world, I think what people have to realize is the world's changing the world today.

It's AI that's changing a lot of it, but the world's always changing. And so, you know, I look at the, the arc of my career, right? And things I thought of and didn't think I would be doing over the course of it.

It's a crazy world out there. You know, you, you know, and, and the key to success is being nimble and versatile and embracing the new and, and understanding that what you did yesterday may not be good for what you want to do tomorrow or even later today. And you know, I I, I think there is, when we talk about transformation and you, you, you have to self transform as well, right?

As individuals, and you need to be upskilling or upleveling even if you will. Um, but let, let's go back now to the, to the systematic, the organizational level, right? How do we, and I've had this discussion with entrepreneurs, how do we keep our people happy, engaged, productive, not scared of AI embracing AI positive, right?

All of those good things, because I think there's a lot of people who, I mean, frankly have a lot of anxiety about this. Yes. I'm so glad you brought that up.

And I think actually as leaders, the first thing to do is to manage our own anxiety to understand. Yeah. I mean, I didn't, I did not think we would be talking about this today, but I'm actually working on an article about this with someone else who is really noticing that the anxiety narrative can bring out our worst tendencies, our tendencies to just like super hyper-focus instead of look holistically, our tendencies to really focus on individual success versus team outcomes.

And like that anxiety is a thing that leaders need to manage through this, through ai. Um, and we, we see this in the survey data as well. We saw that leaders know systemic outcomes are the thing to measure.

Most of them are measuring that outside of the AI context. And well, most of 'em are at least trying. But within the AI context, the salient message is everybody wants to measure individual productivity.

And it's like, wait, wait, we were just, you, you know, that, that's not the answer. But the answer, uh, that everybody goes to in this moment of anxiety is, again, this the hero narrative, the hero developer, the 10 X developer. That's what everybody goes to.

And so I think it's very clear that leaders need to manage their anxiety to remember what they already know, and then to look at that systemic view instead of letting the anxiety drive drive the boat. Got it. I love it.

Amy, where can people, if they wanna dig in here, where can they go take a look at these survey results and reports and kind of digest it on their own? com, and you can download the report there. Um, it has a lot of insight into how people are measuring what they think is important about ai, the risks and what people are doing to close the skills gap.

Absolutely. You know, I used to remember saying speed kills when it comes to just pushing software out. Um, but still, you know, when we, when we measured, uh, how much AI is helping us when we measure the, whether we're a high performing team or not, we still seem to focus on how fast we code and how much code we, we, we haven't, we never add the, the third thing in there, which is quality in my mind.

Right? And I think with, in these AI times or whatever you want to call the age of AI, quality has to be right there with speed and volume. Otherwise speed does kill.

Absolutely. You know, AI is an amplifier. It will amplify your lack of quality process.

You will Yes. A hundred miles right into a wall. Absolutely.

And the leaders that we interviewed, they have said that quality is the most important skill. Um, I think though there's a level beyond that, and I think in a couple years we're gonna move beyond that because I think AI is gonna cause us to rethink quality overall. Because when code is no longer the bottleneck, like we've talked about, then maybe it's a life of disposable code.

Maybe it's about how fast you can recover, how fast you can, uh, self-heal. Maybe it really goes beyond just bug rates into more of a sort of a recovery self-healing measurement, which is totally different. Resilience about quality Resiliency.

Yeah. Yeah. Maybe that would, I love it.

Be important. Hard to say. But right now, definitely the focus of the day is on quality to just avoid risk.

I mean, AI is out there, the genie is all the way outta the bottle, and organizations are struggling to, to catch up and to keep it, uh, in balance. Absolutely. Amy, we gotta wrap it up.

I want to thank you for coming on here. com is the site, right? Yep.

You go get this report and check it out there, Amy. Keep up the great work. Come back and keep us posted here on Text Trunk.

Okay. Thanks Alan. Great talking to you.

Great speaking to you. com here on Text Trunk tv. We're gonna take a break.

We'll be back in a moment.