Generative AI and Developer Productivity – Doug Seven, Amazon Web Services

This is Textron tv. Hey guys, thanks for the throw. We're here with Doug seven, who's general manager for Amazon Code Whisperer, and we're gonna be talking about the impact generative AI is having on developers and DevOps teams and everybody else downstream from there.

Doug, welcome to the show. Hey, you, Mike. It's great to be here.

I think people have seen enough of these types of tools at this point, and we know that developers are using them pretty extensively. The question is, is this, you know, leading to better code, code being written faster, or is it kind of, you know, just elevating developers who may be not so good in the first place to be an average developers? What ultimately, what are we seeing in here and what do you think, uh, the impact is that ultimately?

Yeah, generative AI for software development is absolutely having an impact for sure. And if you look at the way these tools work, really the primary function of these tools is for productivity. So really time is the thing that we look at as one of the best benefits of how generative AI works.

The, if you, if you think about how these tools function, um, they write code for you. And that's something that can often take an awful lot of time, uh, particularly when you have lots of boilerplate code or, or things that are really common or road, uh, developers spend a lot of time unit test writing, things like that. Uh, they spend a lot of time on those things.

And so if generative AI can play a role in that, where it can accelerate the developer through those activities so that their time can be spent on things that require more, um, cognitive burden, things that, that, that are more novel, um, you know, figuring out new and interesting things. Um, that's great. And so what we're looking at primarily is what is the productivity boost that you can get from, from some generative AI tools?

Some examples of this are, you know, before we, uh, launched Code Whisper, we ran some internal, uh, comparisons. We had cohorts, uh, one cohort using Code Whisper and one cohort not using Code Whisper and asked them to complete the same tasks, giving them a goal to accomplish. Uh, the ta the, the cohort that was using Code Whisperer was able to finish in about 57% of the time that it took the cohort not using Code Whisperer.

So they had a huge, uh, productivity boost time, uh, that was saved in doing this. What was also interesting, you mentioned quality, um, not so much about quality of the code particularly, but, but quality of the outcome. Uh, we also found the group that was using Code Whisper was 27% more likely to be successful at completing the task.

Um, so not just getting it done faster, but actually completing that task and getting it done the way it needed to finish. Exactly. How is Code Whisperer constructed?

'cause some folks are using LLMs that are more general purpose per se, and as such, you we kind of run to the middle or sub average, I mean, is Code Whisperer kind of more developed specifically using examples of code that are known to be good and then we go from there? What's the difference in approaches? Well, we've trained Code Whisperer, you know, it is an l l M based solution, and we've trained the model on billions of lines of, uh, you know, public open source code as well as code from Amazon itself.

And so the, the idea here is not just give it the, give the model anything and everything, but, but make sure that in that data ingestion and preparation process, we're really looking at the quality of the code that's going into the model, uh, and then making sure also that we're, um, looking at the quality of code coming out of the model. So as we're making code suggestions that we're making accurate and good quality code suggestions, Is it your sense that developers are internalizing those suggestions and becoming better developers as a result? Or are they just kind of speeding through everything best?

I I think that's still to be determined. One of the things that we found, at least anecdotally from the feedback that we see from people using Code Whisper is the experience that they're having. So there's, there's a general feeling of, um, I don't know Joy is the right word here, but they enjoy the experience of, of writing code more when they have tools that are helping them be more successful and be more productive.

Um, we're also finding that people are reporting that these tools like Code Whispers are helping them in areas that they're less familiar. And so, you know, one of the, the scenarios that we were really trying to address was, if I'm writing code and I, I get to a point in the code writing process that I have to do something that maybe I have less experience with or less familiar familiarity with, um, the natural, uh, next step for a developers go search the internet, go, um, look on discussion forums, go look up, uh, things and find something that looks like what solves your problem. And then oftentimes it's a copy and paste like, okay, that looks like it solves my problem.

I'm gonna copy that. I'm gonna paste it in my code and I'll work through it. Um, that can be really time consuming and it can be very error prone because if you don't have all the right context, maybe that's not the right solution.

And so what we found is that at least we're hearing anecdotal reports for people who are using the, uh, code whisper that, uh, in areas that they're less familiar with, code Whisper is kind of guiding them. It's, it's giving them the suggestions that they can look at and understand and parse through and then move on to the next thing. So from that standpoint, you know, is it, is it making them better developers maybe?

Um, it's helping them learn and explore new areas and, and new things. They still have to understand code, they still have to be able to look at the code and make sure that it's correct for what they're trying to do. Um, but it's replacing that, you know, time spent searching the internet for something, uh, and giving them guidance along the way.

A lot of the focus today, of course, is on the building new applications, but we have a lot of app modernization efforts that people go through and trying to navigate your way through the code bases that drive a legacy app is a difficult chore. So do you think that this will play a significant role in encouraging people to modernize all the applications they've already deployed? Yeah, I think there's a really big opportunity here for what, um, generative AI tool tools can do in software development.

The idea that, um, you know, I could, uh, I could theoretically point, uh, a tool at some code and say, you know, tell me what this does and then translate it to something more modern is a really interesting proposition. It's not an area that we focus on yet today, uh, but it's an area that we're exploring and it's an area that, uh, customers have expressed an interest in. You know, I've got some legacy stuff, what do I do with it?

You know, I need to, I, I would prefer to modernize it. Um, it's easier to find developers today that, you know, know, know a modern language than maybe a legacy language. And I think these kinds of tools lend themselves to very well to those scenarios.

Uh, you know, the idea that I could go into an older code base today and I could use Code Whispers to do something like generate unit tests. So I could say, okay, write, write me, you know, tests around a certain set of functions and then use those tests in my modernization effort to make sure that the new code meets the, meets the functionality of the old code. Um, so some really interesting propositions there.

We talked about quality and of course, quality and security are, um, closely integrated with each other as it were. Will we see more secure applications because the quality of the code is better and there'll be fewer vulnerabilities and things to be exploited that are fairly common these days? I hope so.

And one of the things that, you know, we've done to, to work around that, there's, there's a challenge that occurs when you integrate, uh, humans and AI in almost any industry, in anything you do. Um, you know, uh, where's the, where's the boundary of responsibility? Uh, what is the human responsible for?

What is the AI responsible for? And when you get to, uh, topics like security, that becomes really sensitive because who's, who's, who bears the responsibility there. And in the end, the way I tend to look at this is we're playing an active role through Code Whisperer with our language model.

We're we're playing an active role in writing code with a developer. And so we should be responsible in understanding the security implications that could come from that, making sure that the code we generates really good code, making sure the code we generate is secure code, but also understanding that it is a hybrid. So we don't control everything that comes out of the model, um, in term or we don't control everything that goes into the code that the developer's writing, uh, the code coming outta the model.

Clearly that's our responsibility, but, um, that's being mixed with code that a human is writing or code that's been in existence for some period of time that if they're adding code to. And so we want to give tools to developers to make sure that they can understand if there's any security, um, concerns with that hybrid, uh, whether it's code that was written two years ago or a code that's being written today. So part of Code Whisper is, uh, with the integration of something called Code Guru Security, which is the ability to scan the code, both the code that was generated by Code Whisper and the code written by Human and even code that's been in the code base for years, uh, and look for common security vulnerabilities or look for places where some common best practices aren't being implemented and provide guidance to the developer on how to address that and how to fix that so that they do get the highest quality and most secure code that they can get.

A lot of the software engineers that are running DevOps are kind of looking across the table with a certain amount of envy when they see all these tools that the developers have, and they're also looking at it with a little, um, apprehension because they're like, wow, the amount of code that might be coming through that pipeline might increase considerably in the weeks and months ahead. So what is your sense of, well, we also need AI tools to help the software engineers to kind of cope with the advances being made by developers using AI tools. Yeah, I think that we're gonna find over the next few years that, you know, generative AI makes its way into the entire tooling pipeline for software development.

Every aspect of it, you know, starting with, um, the code writing was a natural place to start. It's a, it's a really good place to start in the terms of, you know, developers already experience, um, ML capabilities and other things with how Auto Complete works and some of this stuff, and this is just sort of an evolution of that, how Generat AI can move from Simple Auto complete to let me write code for you and let me, let me kind of be a partner in that code writing process. And I think as we continue to evolve that, you know, infrastructure's code is a natural evolution of what a software developer is doing when they're writing their application.

And that transitions nicely over to, uh, the DevOps space. But even being able to integrate, um, some of these language tools into what's happening at Runtime, I think is really interesting. We have tools like DevOps Guru where we, we have observability tools for runtime, and we can identify where things are happening and and notify you, but the idea that we could apply some of the same technology, uh, into that entire pipeline is really fascinating.

So I think over the next few years we'll see quite the evolution in that entire, uh, inner loop and outer loop of the developer and engineer experience. There's a lot of debate these days about how smart, smart can get, so, uh, will we always need developers to write code or will the machines take over most of it? And what will be the relationship between the proverbial man machine interface?

Well, Like I said before, this is, I'm in the business of time and productivity, so I'm not, uh, really trying to replace engineers. I'm just trying to make engineers more productive and more successful if I can help them take a two hour job and make it a one hour job. Uh, if I can reduce the time to market so that, uh, businesses can get their ideas turned into software and get that software into market faster than, than I think I'm doing a, a pretty good job.

And so I really look at this as time saving. Um, you know, in the same way you wouldn't mind looking at other tools and other industries and how they don't replace the person using the tool, but they make the person using the tool more effective and more productive. Um, and I think that's gonna be true for years to come.

This is really these, these tools and the application of generative AI isn't about replacing anyone. It's about making them more effective and more proficient. Right.

I'm gonna ask you to pull out your crystal ball. What comes next? What do you see coming down the road that people should be excited about?

Well, I think, you know, what we were just talking about is a really good place to start looking is if, if we're becoming very successful at having generative AI play a role in the code writing process of software development, the evolution of that as we look at the pipeline, you know, moving into infrastructure as code and moving into observability and operations and where we can apply generative AI into these places to make, um, you know, remove the toil across the entire lifecycle of an application, not just in the code writing process. So I think over the next few years, that's really where we're gonna see, uh, the application of generative AI for software development is work its way into the entire life cycle. Well, you've seen enough of people adopting it at this point.

Um, what's your best advice to them? I mean, do they just all dive in or is there, you know, a, a reasonable approach to get started to work your way up through this? Um, what's kind of the the modus a operandi?

Yeah, the good news is tools like Code Wisp are really easy to get started with within, within, you know, minutes, you're up and running and you're getting code suggestions, and you can start to see the productivity benefits. Um, I do find just in observing people using, uh, code Whispers that there's, there's the learning curve between getting started and getting really proficient. Um, you know, you're learning a new tool and you're adapting to a new tool.

And so some of it comes in how do you express yourself in a way that the generative AI can respond to you in the right way? Um, you know, I've watched people who approach these tools for the first time not quite understand how to interact with them and how to express themselves. So one of the use cases for Code Whisper is I can write a comment that describes the capability I want, and then Code Whisper will respond with the code that does what I asked for it.

And so there's a, there's a little bit of learning how to express yourself to get the best results. Uh, you know, in the, in the context of, of large language models, we talk about prompt engineering, where, you know, how do I, how do I provide the right prompt to the language model to get the right kind of response back out? Um, and that's sort of the same with tools like Code Whisper, there's just a little bit of a learning curve to go from using them and getting benefit to being really, uh, really getting a lot of benefit out of them.

Um, but that's just part of learning a new tool. So I think that's, that's fine. Right, folks, you heard it here.

It's not a question of whether we're not, we're gonna be using generative ai, but rather to what degree. Hey, Doug, thanks for being on the show. My pleasure, Mike.

Thank you Guys in the.