GenAI in Software Development with Google Cloud’s Nathen Harvey

This is Textron tv. Hey, everyone. Welcome back here to techron tv.

You know, I, I love having my next guest on whenever I'm lucky enough to sign them up to come up here, or if we happen to be in the same conference at the same time, and I can pull 'em in for a, an interview. If you live in the world of DevOps or you have lived in the world of DevOps over the last dozen 15 years, you probably know him already. And if you haven't, you, you haven't really been in the world of DevOps, talking about Nathan Harvey.

Nathan is the Dora lead for Google Cloud these days, but as Nathan will tell you, he, him and DevOps have a very intertwined long road together. Nathan, welcome back to Tech Drunk tv. It's great to see you.

Hey, thanks so much, Alan. Really, really good to see you and happy to be here today. Fantastic.

So, Nathan, I didn't mean to embarrass you, but you know, for those people who maybe don't know the Nathan Harvey story, give him, give him a sense of, you know, just how involved you've been in this DevOps journey. Yeah, for sure. Uh, as you said, Alan, I've been kind of part of the DevOps community from the very beginning, uh, or at least nearly the very beginning.

At that time, I was, I was actually a practitioner working in, uh, web operations, migrating out of data centers into the cloud, et cetera. Uh, but then I spent about six years or so at Chef Software infrastructure is code leading up to community there. Obviously, chef is very, very, uh, tight with the DevOps community, was there at the birth of the DevOps community.

But about six and a half years ago, I joined Google Cloud, as you mentioned. And my journey at Google Cloud has been really incredible. I think that, uh, Google Cloud is certainly a, a developer's first cloud.

Uh, it is one of those cloud, uh, providers that really cares about the community. You can see that in all the open source contributions that we have from Kubernetes, uh, and, and, and so many other areas. Obviously, security's top of mind.

And, you know, shortly after I joined Google Cloud, Google Cloud acquired the company called dora. And if you're not familiar with Dora, uh, obviously I know you are Alan, but maybe not all of the listeners. Dora's a research program that's been running for over a decade now.

I'm, I'm privileged to lead up the Dora program here within Google Cloud. I get to work with great researchers, get to build on the legacy of some amazing researchers who, who started this program. Uh, but Dora's research is really about how do technology driven teams and organizations get better, uh, and what what should they get better at?

Well, I like to say Dora's all about how do you get better at getting better. It's, it's a journey of continuous improvement. We really want to drive, how do, how do we use technology to deliver value to our customers and value to our businesses?

Dora's center of Gravity is around software delivery performance, but we look at so many other things. Uh, so that's, that's a little little bit about me and a little background on Torah as well. Absolutely.

And we can't mention Dora, as you mentioned, that it was founded by, again, three other people who were giants in the DevOps world. That's right. Dr.

Nicole. Nicole sg. Yep.

Yep. J Humble and Jean Kim, my friend Jean Kim's, and that's it. We talk about Chef and, and all that.

I can't help but mention John Willis, another good friend, uh, Adam, Adam, Jake. com and I was new to DevOps, these were the people who, you know, I, I learned from and I looked up to, and I tried to just soak in everything, soak up everything they were doing. Yeah.

And, and Dora has had such a, for those of you out there who don't know, you know, we talk about things like they become a verb, or now Dora metrics, of course, you know what the DORA metrics are, right. And high performing IT teams. I don't know if anyone had heard of that before Dora Right.

Or they certainly made it more of a household name in the DevOps world. Um, and, and Google, you know, to be fair, sometimes, I don't have to tell you this, acquisitions happen, mergers happens, and sometimes you break a few eggs making omelets and, and, and there's a drop off. There's a brain drain, there's, you know, changing, uh, emphasis and so forth.

But Google Cloud has done a great job in maintaining the door of mission. Mm-hmm. Right?

Yeah. They're there hasn't, you know, if anything, it's even broadened it and strengthened it. So kudos to Google Cloud.

Kudos to you, Nathan, for doing it. Um, what year is this? This gotta be, what is it, about 10th year, a ninth year or Something?

Yeah, in, in, in 2024, we published the 10th annual Dora report. Yeah. So we're, we're, well, well past 10 years now, uh, which is pretty incredible.

And, and like you said, to Google's credit, like Dora is continuing to grow, and one of the things that I'm most proud of over the last few years is that we really started to sort of grow and foster a community around Dora, uh, which you can join at Dora Doc community, but it, like, like you said, as you were talking about DevOps there, and I think Dora follows this pattern as well. We get better when we learn from and collaborate with one another. Uh, this, we have, we have great research from Dora that gives us really excellent findings, but it's really about how do we take those findings and the, and the research and put it into practice.

And frankly, we do that together as a community where we can learn from each other, identify pitfalls, and maybe warn each other off from particular pitfalls. And, and this is, for me, the, the most rewarding part of leading up Dora, is seeing the community of practitioners and leaders and researchers that all come together to, like I said, put this research into practice. Excellent.

Very cool. So the latest report from Dora is out. Nathan, when did it come out?

Yeah, uh, it was just a few weeks ago now. There, it's, and I have it right here in hand, uh, so we can spend the next few minutes just reading right from it. Uh, it's the impact of generative AI in software development.

Look for over a decade, Dora has been evolving, right? And always trying to stay on, on the, on the top of mind of what's happening in the space. And obviously artificial intelligence is, is, is a big, big area of focus for so many technology teams and organizations right now.

In fact, in our research, you know, as you know, Alan, one of the ways that we gather much of our data is through surveys. And one of the questions that we asked in our most recent surveys was, to what extent is your organization prioritizing ai? And fascinating, about 89% of the respondents said, absolutely they're prioritizing AI across the board within their organizations.

And to me, that's a really interesting signal because that's opt down, right? That is from the executive level down that this is happening. Interestingly though, we also asked practitioners, you know, how much are you relying on AI in your daily work?

And over 76% of the practitioners that we spoke to are relying on AI for one or more tasks that they're doing daily. To me, this is what's really fascinating. You've got that top down, prioritize AI across the board, and you've got that grassroots.

We're really using this and starting to rely on it, that those two signals taken together to me are showing me that AI is really here to stay. It, it is going to fundamentally change how we do things. I kind of reflect back on previous journeys that we've had in this DevOps community, right?

Um, when you think about containerization as a big thing, right? Everyone was moving to containers, uh, and the practitioners were really driving that, but I don't think there were many boardroom conversations about what is our containerization strategy, but I can guarantee you that there are boardroom conversations happening this week around what is our AI strategy. And so it's that top down and bottoms up grassroots movement that I think is so special about what's happening with AI right now.

Absolutely. You know, Nathan, I I like to tell people things like even the cloud, like the move to the cloud around 2005, 2006, and that, that frame, the move to cloud native containerization, Kubernetes, all of that, 2014 maybe something around there, right? Right.

Um, these were, you know, seismic events in our tech world. People like you and me, my wife's family, she, they didn't really know as much, you know, they, they, it wasn't as big. I mean, to them, they're like, yeah, my pictures are in the cloud, and they point up there.

And that's as far as it went. You know, where ai, AI is one of those things like the internet itself, that it's just, you know, it has the potential to just change everything, change civilization, not just the tech world. And as in most things, it, you know, tech is sort leading the way in terms of these changes.

Um, I mean, just before I came into you, I, I got a notice that my chat GPT needed to be updated. I updated it and it wanted to show me what the updates did, and it, it, on its own said, let me write a, what do I think about Alan Shemel thing based upon all of the prompts I've given it over time and real time looking on the internet, right? And it wrote a thing about the maestro of tech media.

I'd love it. Like, why, God, where did I come up with this? But more importantly, we, we did a, a, a text on gang, uh, episode segment the other day about this new open source, uh, tech model, LLM, for just for coding.

That is just amazing. I mean, it's, it's heads and tails above what you're going to get out of, uh, a chat GPT or Gemini or, you know, clawed or these things. So, um, it is changing the way we do software, the way we do technology, the way we do business.

And I think every, you're right, every board has to be having the discussion of, yeah, what does AI mean to our business? Yeah, yeah. It, let's Talk important.

It's about, I mean, let's talk about some metrics, man, you know? Yeah. Look, when you, when you get the 70, 80% of companies that are doing things that's beyond crossing the chasm, that's, yeah, that's critical mass already, but give us some more, you know, what Dora is famous for.

Give us some metrics. Yeah. Uh, I mean, we will give you some more insights for sure.

I think that, you know, we're seeing AI having tremendous impact on that software de development lifecycle. And really, we, we care about metrics with Dora, and we care as much about what are the impacts of different technologies, different techniques, and so forth. We're seeing really good gains in things like flow and job satisfaction and productivity, even documentation quality, improving code complexity going down.

All of these are really strong signals that we're seeing as AI adoption increases, but it's not all sunshine and lollipops. Uh, we are also seeing, interestingly, that as you're improving or increasing the amount of AI adoption you have, we see software delivery performance, things like throughput and stability actually falling off. Teams are moving really slightly slower.

Yeah. Slightly slower throughput and significant drops in the stability of changes as you push them out. Now, we, we hypothesize that this is because, you know, if we, if we look across a system, there's always a bottleneck in any system, right?

And if you make an improvement to a system that's not focused on the bottleneck or the constraint, that improvement is, is basically an illusion or maybe even makes things worse. And, you know, as an industry, we've focused so much on generating code. Maybe today we're generating larger change sets, larger prs, and our systems are really finely tuned to handle smaller changes.

And think about this, Alan, when you have a large change and you need to wait for someone to review that change, that takes longer to review. We don't necessarily have all the context in that review. Perhaps we let bad code slip through our review process and we end up having to roll that back.

So I think there's potentially a lot of areas that we have to think about as we look across the entire development lifecycle and delivery lifecycle that where we put technology in front of our, our, our customers. And so that's really what this report digs into is, is some of those findings. And then what do we do and what are some of the other considerations that we have?

So for example, we're looking at, um, like how do we make sure that AI is amplifying developers' value? How do we help foster trust in the ai? How do we help developers trust the AI more?

And what strategies and metrics do we care about for organizations as they go on this AI adoption journey? You know, you need those guardrails to help you know that this is actually having the impacts that we want it to have. It's driving customer satisfaction, it's driving business value.

So, Nathan, how do we know we're not, you know, pinning the tail on the donkey and blaming AI for some of these shortfalls that may be due to other things, maybe due to increased pressure economically to get more out for less, uh, you know, more people taking shortcuts, more, you know, just a whole bunch of different things that may not necessarily be AI related. Yeah, that's a really good question, Alan, and, and thanks for asking that. So one of the ways that we come to those conclusions is we actually, you know, in our surveys, we ask, uh, I don't know, a hundred questions or so, to get really insight into all different aspects of this, to be very clear, a very complex organizational structure, right?

We're talking about people and process and technology. And what we do is we then model out how all of these things interact with one another. And so for some of those findings that I mentioned, like with software delivery performance falling off, the way that we arrive at that conclusion is we, we basically take our model and we say, let's adjust one parameter within this model, just one node, and that node will be AI adoption, let's crank it up by 25%, and let's see what are the impacts across the model.

And so in that way, that's, that's sort of how we model this out and get those statistics. But the, the reality is that if you work in an organization, it's a complex organization with emerging characteristics and, and complex behaviors, there's no way in your organization you can change just one thing. Or maybe there is, and I would love to see that, right?

There's always ripple effects and other things that are changing at the same time. So this is our finding that software delivery performance is fall, falling off, but it might not be your experience. In fact, one of the things that we're very clear on in our research is we want you to take our findings and, and replicate them within your own context as we go out and we research the world, but you work on a team in a specific organization serving a specific set of customers, your results are necessarily going to be different than a broad-based survey of as many technical practitioners as we can get, or as many technology driven teams as we can get in our survey.

So I encourage teams to replicate our findings, using our findings basically as a hypothesis. Look, we think that if we use AI over here, our documentation quality is gonna improve. We think that we have that hypothesis because Dora found that let's go prove it out within our own context.

I think that's a really important lesson to take away from Dora. Absolutely. Nathan, we can't talk about AI and its role on in DevOps and developers and software technology without also looking at its role in perhaps costing people their jobs.

I mean, for lack of a better word, I don't know if way to sugarcoat it. Um, you know, if you speak to many of the boardrooms and leaders that you're mentioning, a lot of them are telling, are saying that, Hey, we'd like to see AI take the role of junior or intern, you know, junior developers or interns. Well, if it does do that, how are we gonna get tomorrow's senior developers if AI's doing the role of the junior?

You know, they'll never get the seasoning they need. On the other hand, you know, I've spoken to many, many DevOps and developers who say, look, it's a tool. It's gonna make me better.

It's gonna make me 10 x. It's not going to, it'll never take my job away. Others say they're not so sure.

What is, what does our door report say about, you know, the job market given an AI world? Yeah, it's, it's a really good question. We haven't researched the job market in particular, but I, I think a lot of those sentiments that you just covered, I I, I'm seeing and feeling those across the job market right now.

And I do think that there are some, there are some studies out, you know, that are showing the impact of, of using generative AI specifically with developing code. But I think there's also, um, I think unfortunately it's, it's kind of pushing some mindset that a software engineer's job is to write code. I mean, yes, that is part of their job, but it's, it's actually a pretty small part of their job when, when, when research is done and is to be fair, it's not by Dora, it's by others.

Um, but I think the, the stats are something like, most software engineers only spend up to 30 at best, 40% of their time working in an editor actually writing code. So if you take something that you only do 30% of the time and you make it even 30% faster, that's still only a 9% increase or so, right? It's not, it's not eliminating the entirety of that job.

I think that AI is a really strong amplifier in some of our research lab studies. And otherwise, what we've seen is that, uh, let's take developers. If you're, if you're skilled in a language and you use AI for that language, you're going to be even more skilled in that language.

If you don't know that language and you use AI to help you, you, you'll be able to accomplish a task, but it won't necessarily be great, right? And so, I, I really think it is that amplifier. And then you raise a really important and critical question that I think, not just in software engineering, but in every, every profession we're sharing this concern.

If AI replaces all of the juniors, how do juniors become seniors? How do we bring up the next generation of senior engineers? And when I think about my own journey, uh, my engineering seniority comes from experience.

It's from running into those bugs being woken up at 2:00 AM to solve a production issue, and, and knowing the context of the application that I'm working on, being intimately familiar with that and, and the organization that I'm working in. So I know who, who I can call. I think it is really important that we look for ways of bringing up those juniors, making sure that they're exposed to all of the things that they need in order to become seniors.

There's a great book by Dr. Matt Bean called The Skill Code, which is research that he's done into exactly this topic. How, how is automation and artificial intelligence, how is it impacting how professionals develop their skills?

Excellent. Um, this is probably too new for, for this year's report, ag, agentic ai. Any, any findings around that or that'll maybe be next year?

Yeah, well, I suspect we'll see that later on this year, but we don't have, we, we don't really have a distinction between the, the use of generative AI and, and AG agentic AI in our survey yet. So we, we aren't really seeing any differences there. I think it's, it's really fascinating though, seeing all of the, the rapid development that's happening in this space, whether it's, you know, uh, the MCP, the C control protocol, or the, uh, agent to agent protocol that was announced last week at, at Google Cloud next.

There's lots of really fast movement that's happening here. And again, I think we're all just learning and experimenting with this, these new ways of working. And, and if you take away anything, I think the big thing to take away, whether you're a junior developer, a senior developer, or someone trying to get into this industry, I think the, the advice I have is to go use ai.

Just, just use it, uh, and start building up the muscles. Start getting a, a feel for where it's helping you, where it maybe needs some additional help along the way and so forth. But if you're not using it today, I, I, I do worry that you are falling behind.

This is something that's gonna be here to stay, and you need to get your hands on it as soon and, and as frequently as possible. I agree with you a hundred percent, man. Hey Nathan, for people who want to dig in, 'cause we've, we've barely scratched the surface here.

For people who want to dig in on the report, where can they go? dev, uh, pretty easy. URL to remember right on Dora Dev.

You'll find, uh, right on the homepage, you'll find the link to grab the, uh, impact of generative AI on software development. Uh, grab that report, uh, and read through it. We'd love to hear your feedback on it.

How, how do our findings fit within your context? How, how are you experiencing this new brave new world of generative ai? Absolutely.

Thank you Nathan. Nathan. I know Google next was just a week or two ago, but Yeah.

Where are you, where are you headed next? Yeah, I was just at Google next last week. Um, and I think my next conference will be DevOps stays Atlanta.

That's coming up at the end of April. So that's where I'll Be. Yes, they're, uh, they are, uh, roasting John there.

That's right. It'll be a, a little bit of a roast of John Willis and in fact, DevOps days. Atlanta this year has a slightly different name.

Uh, it's also known as Deming Days, Atlanta, right. John? The first Deming days Atlanta for John.

Yeah. Yeah. So we are sponsoring it here at Techstrong, but unfortunately it's the same week as RSA.

Oh, right. And I'm gonna be in San Francisco at the RSA conference, so I won't be there. I'm actually, we're gonna record a video for John though.

Um, Oh, that's great. Yep. But enjoy and give everyone hugs there, Nathan.

Man, it, it's, they gave, it's a good community. Chris, Chris Reer and John and folks down in Atlanta. Nathan, keep up the good work, man.

You do God's work. Thank you very much. If, if No, I'm sure people tell you that, but I'll tell you anyway.

You're doing good stuff, man. Thanks. Keep it up.

Keep us posted. Come back on here soon. Thanks, Alan.

I really appreciate you having me on. And, uh, thank you for spreading the good word on all things DevOps, uh, and let's keep this collaboration going. Talk to you soon.

Absolutely. Nathan Harvey, Dora, lead at Google Cloud here on Textron tv. We're gonna take a break.

We're gonna be back. We've got more show for you.