AWS’ Deepak Singh on the Impact of Generative AI

This is Textron tv. Hey guys, thanks for the throw. We're here with Deepak Singh, who's the Vice President of Next Generation Developer experience for AWS.

And while we're talking about all things generative ai Deepak, welcome the show, Matt. Thanks for having me. It's great to meet you again.

I think we've all seen by now the usage of, uh, gen AI to help write code and it's making developers more productive. But from your sense of what you're seeing so far, what's your assessment of it and where do you think we go from here? Yeah.

Um, so, you know, generative AI has now been around for, well, well, it's been around for longer than a year and a half, but I think for most people it's the last two year and a half, two years where they've started seriously looking at it. And it started off with things like image generation and, uh, more sort of consumer type applications and creativity. But, you know, over the last year, year and a half, you've started seeing gene AI become a critical part of how we all do our daily work.

Uh, so, you know, I think what I've observed in the last year that I've been super deep in, in that area, uh, and having been in around develop experience for a while, is every organization that I know of and almost every developer that I talk to is using generative AI in one capacity of the other or the other. They may be using it to get access to information, uh, try and figure out how something gets done or they may be using it to generate code. And I think where it's going, where most companies are, and a lot of folks are, is they're go at that stage where they've figured out what they themselves are comfortable with and kind of where they want to go over the next six months year.

Uh, so some of we are starting to see that shift away from curiosity to where the early adopters have found meaningful use out of it to where the next next stage of people and even beyond are starting to come in and make, uh, really good use of it. I mean, a great example of that is British Telecom. Uh, you've probably seen the story that they had.

Uh, they published a press release about using, uh, code Sper, which is our coding, uh, assistant from AWS and the kinds of success that they're getting, the number of lines of codes they've written in just a few months, the 12% productivity gains are getting from things that are not being automated. And that's just the start. You'll see, I mean, there's many other examples out there of people who are starting to do that, who are not just your startup in a garage or an individual developer.

So it's great to see Will the output continue to get smarter. And I'm asking the question because sometimes, um, the models are trained on lots and lots of examples of code pulled from all over the web. And some of it is, shall we say, inconsistent.

So, um, that results in output that might not be as efficient as it might be. It may have vulnerability. So, um, are we gonna see maybe, I don't know, large language models trained on narrow sets of data to increase the quality of the output?

Yeah, so the thing I like to say is whatever we are doing today with generative AI is strictly the worst it's gonna be 'cause it's only gonna get better tomorrow. Uh, and there's, and I think there's two or three reasons for that. One is the models keep getting better.

Just the, the bigger models get better, the smaller models get better. How we, but we've also, as a group of people and practitioners learned a lot about how to make them useful. Like I think the shift over the last several months has been away from, I have this great model now, great models help.

I mean, we saw all the new Claude releases on that we announced with, uh, on Bedrock with, uh, in partnership with Anthropic. Those models are strictly going to make your life better 'cause they are much, much better models in their capabilities. But I also think in how we use those models, we become a lot smarter.

Uh, I'll give you an example. Um, we have this capability called Q code transformation. So Amazon Q is our, the AWS assistant from AWS to help you be more productive at work.

And part of what it does is help develop software developers and what, uh, Q Code transformation does, it's a combination of using the LLM as is, but also using our understanding of how software gets compiled, how compilers work, how so you know, how you understand and correct, uh, errors, uh, et cetera. And all of that combined knowledge is what makes it so effective at what it does. Uh, and the fact that, you know, at Amazon we could do a couple of thousand packages in a couple of days, uh, that, that part is very important.

And, and I think that's what's going to be really impactful for customers. And what they're able to accomplish is a combination of better models, which model to use for what. It's not like to use just one model for everything.

You use the right model for the right purpose. So you can use a small fine tuned, very, very carefully built model like you suggested, that's trained on the right set of narrowly on the right set of data for a very specific task or a big general purpose model for these big reasoning tasks. And people are going to get that's going to, that's improving every day.

But also how we use those models, what we build around it is also getting better. 'cause we are getting better as an industry, as a company, as a, as development teams on using them and understanding things like how prompts work, how, how, what kind of resources do you need to support these models around them. And, uh, so I think every, it's, it's, it's fascinating to see how quickly we are learning these lessons.

Uh, sometimes it feels like you're learning the same old lessons again a second time, just in a different, uh, in a different, uh, year. But it, it, it's a fun place to be. We're clearly seeing how developers each one will be more productive, but are we also gonna see a lot more developers because the barrier for entry is coming down, and as long as I can figure out how to reason I should be able to build an application.

So I mean, are there gonna be, I don't know, thousands, tens of Thousands more developers out there? Yeah, I'll actually speak about it slightly differently. So the areas where we have seen, uh, immediate success and where we see impact is onboarding new developers.

So let's say you are at a company where you have a particular, particular way of development or particular set of languages and you're hiring people out of college. How quickly they become effective has always been a challenge. Folks at Amazon, for example, we have specific metrics around how quickly a developer can become, uh, effective and effective part of the team.

In the end. In the end, that's what we're looking at. Um, we've seen both here and with our customers that, that a lot of our, the focus is on how quickly that happens.

So that's one. So you're getting more, uh, developers getting effective quickly, especially folks who are more junior or coming new to a company. So that onboarding time goes down.

Let's say, uh, and I'll use myself as an example. Don't, don't quote me on it, um, but, or don't ask me to write production code in it, but I'm not a Python developer, but I have written Python code more, more, more Python code than I would have otherwise because now I can use Gen AI to help me do it. Um, there are things that I have to do to verify, et cetera.

And I, you know, in the end, you're still doing code reviews, you're still sending it to, for a code review to make sure that it's good code. That's what you should be doing. But the fact remains that I can be a much more effective Python.

I can actually work with Python code. When I, in the past, I would've said, no way. I can't do it.

So that's an example. Uh, there's an example of an, uh, of a team at AWS that actually ended up building a product in, uh, they were building the service in Rust. They're building it from the ground up.

And what it, what they realized was that very quickly that rust, rust is not an easy language to get going, but their entire development team got really proficient at Rust very quickly and effective at using Rust to build the service because they were taking advantage of gene AI tools. So the barrier to entry went way down. Now you can extend that further to what you just said is like, building software is not just about writing code.

There's other ways to write software and the best exam and build applications. And the best example I have for that is Party Rock, right? Uh, party Rock is, you know, we released it as a way for people to understand how Jenny I works to excite them and help them use it.

But we've seen people build some very meaningful applications that are meaningful to them, right? For example, it could be an application to help you build recipe to make, you know, decide what recipes you want, but it's an application. It's, you're not searching, you're building an app.

Uh, I know people who build apps to, um, write marketing copy. I know people who build applications to do, um, keyword based searches on certain kinds of data. You know, uh, so if you take all of these into account, I think you are correct that the barrier to entry for what it means to build is gonna go way down.

I think what's going to be interesting is how, in what ways it's gonna go down. 'cause it's, it's still early, you know, know, it could be party rock style applications or it could be everybody works with code suddenly because writing code becomes that much easier. And I think that's an interesting place to be.

What will be the downstream impact on DevOps workflows? I know a lot of those folks are watching the volume of code increase, the size of the bills are increasing and the multiple projects are being run more easily. Um, how do we kind of manage all of that flowing through our pipelines and what's, what are the issues there?

Yeah, and actually I think that's, uh, super important. I'll bring sort of the AWS point of view on things is, and I'll talk about Amazon queue. Amazon with Amazon queue.

Building writing code is only one of the things. It's, it's one of the things that Amazon Queue does because in the end, code is a way to build an application. It's a way to build a service, it's a way to build a reliable, you know, in the end you want to build reliable maintainable services like high quality, et cetera.

And if all you're doing is building a lot of poor quality code, it's increasing your work downstream. So we are, you know, the output of what we do is in the end a service or an application. And we are very focused on that.

So some of the ways that manifest is a lot of focus on test generation and test and testing. Uh, it's also one of the things developers hate doing. So if you can do that on their, you know, for them it makes life a lot easier.

Uh, I know devs at AWS for example, who do almost all their test writing using gene AI now. Um, but we've done more, for example, uh, on data on, uh, on our coding side, spent a ton of time on, on code code analysis. Like how can we make sure that code is good?

Uh, one, one of the ways we do it is a feature that we call automatic code remediation. It actually scans your code for, uh, things like nll point, exceptions unsafe code, so to speak. And then not only does it point out that it's unsafe, it also fixes it for you.

So it's writing the code is not sufficient, making sure that it's good code is where you want to be. And in the end, is it the right application? Does it meet the right architecture?

Does it meet the right style guidelines? Does it take care of dependencies the right way? Those are the kinds of things you want to care about.

Uh, on the operational side, uh, this happens more on q in the console. 'cause a lot of AWS customers do their operations in the console. We are less focused on helping you build an application and more focused on helping you troubleshoot an application.

So if you have errors coming out of Lambda, for example, Q will help you figure out what that error is. If you're seeing networking issues, Q will help you figure out why you're not being able to access, uh, an end point. And it uses things like the network reachability analyzer underneath the hood.

But you are, you are talking to it in natural language and it's telling you this is how you can fix a networking issues. So it has to be the end-to-end life cycle of not only are you building, but can you maintain it, can you run it, can you evolve it? And you know, is it a solved problem right now?

Not yet, but that's very much how we think about it, because Exactly. Otherwise you'll just have millions of lines of code that you don't know. What did that may not be really good.

There you go. What ultimately will be the impact on the, the job market for developers? I mean, you can't talk about this stuff without people getting worried about where their cheeses as it were.

But it also seems to me that there's gonna be less toil, less stress in the whole process. So, you know, what's your sense of where does the human developer fit in? Yeah.

And in fact, you just, uh, took the ma one of the, when we first started writing down what we should be doing as a team and what are the areas where we could have an impact. One of the senior engineers on my team literally wrote down on a whiteboard, eliminate toil. And the, and the way this was done was we have surveys and things that we've done inside a WSI think, uh, somebody showed a slide on that at Green went on.

Where do developers spend their time in a day and where would they like to spend their time during the day? And I think for the foreseeable future where AI is gonna help you a lot is eliminating to like the things that developers either don't like doing, or quite honestly, uh, AI is just gonna be more effective than humans are doing. And I'll give you a very good example.

And that's back to the Java migration example that I used. That's just one example migrating from say an older version of Java. Java, say Java eight to a new version, say Java 17 or 21, historically would be team by team.

Package by package would take, like there would be a campaign, there would be program managers running it across the company and it'll take you a long, long period of time. What is it do developers like doing that? Absolutely not.

They hate it. Second, it takes a long time and that's time that developers could have been doing something else. Uh, that's not just upgrading Java packages.

So the fact that now you with a very small team can migrate every package at a company the size of Amazon using genai, freeing up developers to spend their time writing code instead of upgrading Java packages, building a new service, upgrading, evolving an application, that's a huge win. Not just to the company, but to the developer themselves. They're doing things to actually enjoy.

Uh, and I think the other way to think about it is, I always like to think of, we think of Q as being, um, an assistant that becomes something that you genuinely enjoy having by your side every day because it helps you get unstuck. It helps you answer questions that may have you, you know, you have run to the library or spending hours on stack, you know, on one of these sites, whether on our forums, on, uh, stackable forms, asking questions or looking at documentation. And here it's just available to you.

Um, and it's available to you in context of what you're trying to achieve because it is aware of your environment, the things that you're trying to do. So I think, um, the way to think about this is they are there to help you get unstuck. They help you to make progress faster.

And the developers who they use that mental model is, are the ones I see being the most effective. And this is just on the development side, probably true in other places as well. Should we have set some expectations here?

And I'm asking the question because we always say, you know, development is a mix of art and science, and just because I got rid of all the toil, it doesn't necessarily mean that I'm gonna have that increased amount of inspiration to get to the great idea or the code is, I'm more or less creating the time to allow that to occur. But you know, it, it occurs between the the in that natural intelligence between the two ears of the developer. So, um, is there a correlation between reducing toil and increased innovation in your mind?

Or how do they relate Partly from the time perspective, but I think this is where the generative part of generative AI gets interesting. Um, when most people think of computers, they think of it answering a question. Exactly, right?

This is what I want to be. Generative AI is a little bit more like humans in, in kind of the fact that, I mean, generative AI models are probabilistic models if we put the right tools around it. Uh, an example of this is, uh, what we do with q feature development.

A Q feature development is less about write this code for me and it's, I have this problem that I want to solve. What are ways that I can solve it effectively? And today it gives you a plan based on what it knows.

Uh, over time these plans will get a lot more sophisticated. And there it's meant to be more collaborative. Like this is your partner in trying to figure this out and you can go back and forth and it could potentially take you in directions that you would not have done otherwise.

And I think that part of it is what's going to lead to the creativity over time. And we see this, I, I think that's why I said you see this a lot in the creative domain already. Um, you know, just if, if you're a photographer and you've played with some of the, uh, AI masking tools out there, they're really compelling and, and, and they actually help you be a better photographer in the end, at least according to me.

And I think these tools like q feature development that are there to not just say, oh, this is how you write a function that does X, but how to help you think through and point you in directions that you may not have thought of and give you ideas. And that's when then then natural human intuition and imagination can then take it forward. Um, so I, I see it from that perspective.

I think it's, it's probably the right way to think about it. Uh, and I, and I feel like if people think about it in those terms, they're gonna be more effective using these generative AI tools. Mm-Hmm.

We think a lot about the building of new applications, but we have tons of applications that are already running in production environments that everybody kind of knows could be improved. So will we be applying gen AI to app modernization as well? And might that not be one of the bigger opportunities?

Oh, I actually think it's one of the more near term opportunities. The Java migration thing. Java modernization thing is an example.

You went from old version of Java, but you're also fixing things like dependencies and using newer classes, et cetera. Uh, you're not doing it, you're doing it for any reason. Security improved performance, improved more efficiently.

The hardware, you can easily see that evolving into more interesting directions. We've already announced that we'll be doing T net where we will help you go from older versions of t net to new versions, but also help you go from t net on Windows to t net on Linux. You know, that's how you want to modernize.

Um, in fact, there's a customer I met, uh, at Green who reminded me, uh, they're an old a hundred year old company with lots of, uh, I'll use a pejorative legacy code. They, they reminded me they actually never don't write that much new code. They're always refactoring.

And I think that's the part, like, I think if we think about gen AI for developers, just in the context of writing code, we are looking at a very narrow slice of the problem. So the automatic code remediation, uh, example is also you, you don't have to do it with new code. You can apply it to your existing code.

What are all the code that we've ever written that is all these bad, like we can, uh, make safer? That's one Second, uh, with the Q code capabilities in the IDE, now, uh, you can do, uh, code explanation, which helps you, like what does this code do? And a great example of that is taking a language, uh, uh, like, which you may not, you know, it's a code written in old language, the person who wrote that has long since left the company.

But Gene is pretty good at helping you understand what that code does, and that's a great starting point to doing more interesting things. Right. So what's your best advice to folks who are still kinda, yeah, they're trial and erroring their way around this.

Is there a, have you seen any best practices or patterns that people are using for bringing this stuff in? To some extent, it depends on who you are. I'll answer it from an organizational perspective than the individual one.

Uh, I like using the metaphor of swimming pools. Um, you know, if you go to any of these swimming pool complexes, there are three types of pools. There's a children's pool that's like shallow and you can, you know, for the people that never swam before, there's the regular pool with shallow on one side, deep on the other side, and then there's Olympic pool, right?

And feed deep throughout. I would say there are some customers, many customers who are in the children's pool, what's been really encouraging over the last year, and this is what I would encourage people to do, is go into the regular pool, start walking from the shallow end towards the deep end. You, you'll find where you're most comfortable as you, depending on who you are, where you are, the tools that you've chosen to use.

And you know, we are there to help you through that process. And you could be a customer like, uh, someone like an Adobe or an Intuit that have, you know, embraced generative AI or even AWS or Amazon and you are in the Olympic pool. You under you, you, you understand how this works.

You understand the challenges, but you also understand the potential and you're willing and that it requires how you, you have to rethink about how you organize around it as a, you know, that's very important. Like how do you organize a company and your people around it? Um, for individuals, I think you have to start trying things out.

You don't expect, I mean, I wouldn't say that you should expect to solve everything. These are tools in the end. They're tools, they're early, but they're progressing at a rate that I've never seen anything progress.

And the reason we ship Party Rock was to show you the art of the possible, here's a very simple AI tool whereby typing in a few words, you suddenly can write poetry or build an application that builds your Animo DB table. And then you can go in and play around with the prompts and the prompt language and the model that used to building it. Just get comfortable with it.

And the more comfortable you get with these tools, the more effective you'll be with them. And I think you should be doing it. I'm, I, I drank the Kool-Aid a while ago.

I think everybody should be doing it. Um, but you don't have to, you don't have to go a hundred percent and you can start and steadily move your air. But I would encourage everybody to try because there's, there's a lot here.

Alright folks, I heard it here. It's just like swimming or anything else. The more you do it, the better you get at it.

And Exactly. And there you have it. Hey Deepak, thanks for being on the show.

Thanks for having me. All right. And back to you guys in the studio.