AI’s Impact on Application Developers and Software Engineers with Tabnine’s Eran Yahav

This is Textron tv. Hey guys, thanks for the throw. We're here with Eran Yahav, who's CTO for tab nine.

And we're trying to sort out what is the future of application developers in the age of AI because, well, there's a lot of talk about how much is gonna be automated and automated, just what is the job function gonna be. Ron, welcome to the show. Thank you for having me, Mike.

I think everybody has the same basic conversation when it comes to ai. They all have some understanding that their quote unquote cheese is about to be moved, but they don't know to what extent and are they gonna be completely obsoleted or is the role gonna change? And you have a lot of hands-on experience with ai.

So what are you seeing and what should we expect? Yeah, the role is definitely gonna change, but uh, AI is not here to replace you. AI is here to amplify you.

And in fact, the role has been changing for years, right? And maybe before we discuss, um, how know, how the role changes, we can start with the basics of what is the definition of the role anyway? What is the job of a software engineer, right?

Or we say that it changes, let's agree on what is it that is being changed. And uh, the definition on the job description has always been solving business problems using software. And that job description is not going to change.

We are still going to be solving business problems using software. We're just gonna do less of the boring stuff, less of writing the code itself, dealing with the syntax problems, dealing with program that does not compile because you forgot the semicolon, right? All this stuff is going away, maybe, uh, for many domains.

And we're just gonna focus on the higher level planning, on the higher level, what needs to be done, and maybe less in the nitty gritty details. So that's a high level answer and we can dig deeper. Definitely In effect, am I gonna become more of a supervisor of the development of a piece of software rather than spending all my time trying to figure out how to write a specific line of code better or testing this, that and the other, or creating the tests.

And there's gonna be this kinda series of AI agents that will take care of a lot of these functions. How will that kinda work out in your head? Yeah, In my head, I think it's, it's gonna be basically the shift from an individual developer.

Every developer should think of himself or herself as, um, engineering manager or as a team lead. And they're going to be a team lead of a team of AI agents, basically, that are going to be delegated certain tasks. And hopefully the AI agents can complete these tasks with, uh, high reliability, producing high quality code.

But this is definitely not always going to be the case, right? And just like any team lead, you'll have to make sure that the problems are well-defined when they're delegated, and you'll have to make sure that what you're getting back is actually satisfying the requirements, right? And as any team lead, you'll have to say, Hey, you know what, Mr.

Ai engineer, you kind of missed on this part here and that should be improved and have another iteration with the delegated tasks, right? So it's really more a managerial responsibility, architectural thinking, uh, understanding your requirements, the constraints, the trade offs. These are still going to be very much at, at this point, uh, human tasks, right?

And then we're gonna basically have the AI agents deal with the low level details of the implementation coming back to us with something that hopefully works. There's a little bit of cynicism in the air when it comes to ai. Folks are basically saying that instead of going to some website and hunting around looking for, uh, code that I'm basically just gonna copy and paste, I'm now using an AI agent to do that.

But the problem is, um, the AI agent doesn't have any context for the production environment. So when I go to debug the thing, um, I don't know how it was written and I'm more or less confused by all the code written by the machine. So I kind of throw up my hands and say, well, maybe I was just better off doing this myself.

So I know we're on the journey here, but what, what are the practical use cases for AI these days? I think we're already seeing AI pop across the entire SDLC. So we have AI generate code, generate test, generate documentation, review the code, uh, solve review comments, all those things are done by ai.

I think, um, the cynicism is understandable and expected, um, given that we are far from perfect on all of these tasks, right? So part of the human skill in the coming years is to know where to apply AI and where to actually dig in and do the stuff yourself, right? Because the AI is either gonna give you a partial result or something that you'll have to debug very carefully anyway, right?

So I would not use it for my super duper one-off sophisticated algorithm that requires a lot of, uh, contextual understanding of my company, et cetera. That's maybe not the best use case right now. It will be, things will mature, uh, but we're already seeing, again, increasing autonomy across tasks across the entire SDLC.

Things are improving all the time. Uh, I think things that were science fiction just a couple of years ago, like having ai writing entire application from scratch are pretty commonplace now. As soon as, as long as the specification is clear enough and the environment in which this is happening is isolated enough.

So there's still way to go in terms of ai, understanding the context of your organization, right? Understanding what code already exists, what libraries should be reused, um, how to not generate things, uh, if you can use an existing microservice out there in the organization, et cetera. And these are part of the things that top nine is focusing on making top nine, making the AI more contextual to organization, having the AI engineer in a sense onboarded to your specific org so it really does work that could be maintained, uh, downstream, right?

Something that is really matching the best practices of the organization writing code that is not foreign. To your point, it's very easy to generate code that may be functionally correct, but is completely foreign to the organization, right? Something that doesn't use the right library is something that doesn't call the right microservices.

So it may work, but whoever needs to maintain it downstream is gonna have a hard time At the other end of that spectrum. I can't help but wonder how quickly are we gonna move to a scenario where people may be skeptical at the moment, but I can see a world where I may not wanna be an application developer without using these types of tools because there's a lot of toil involved in being a developer that's not a lot of fun. I think like previous tools in previous, you know, revolutions, let's call it, uh, I, I think the main thing we're gonna find out is that we can accomplish much more and it opens the kind of the way to new ways of doing things.

Like, let me give an example. Today when I have to develop an application, I maybe have some wire frames. Somebody gives me like the basic flows of the app, et cetera, and I develop it and I get one application because the cost of the development is so high that I basically get one bite at the apple, right?

It's kind of, this is the application, but imagine that the co co cost of generation is like low, is like maybe a push button. I can generate a hundred different variations of this application with different ui, with slightly different behaviors, with slightly different logic maybe. And now the new developer, the the empower developer by I can basically spend time choosing the right application out of the 100 iterations, refining it, maybe iterating on it some more, so you can produce maybe 10 x or a hundred x more options than what you used to be able to do without ai.

I think this is really the way in which it's gonna change. And maybe you can even have multiple versions of the same app that are personalized for different personas, right? This is the kind of empowerment that you're getting from AI that you can basically get stay in the spec level, kind of define what is that you want and maybe get the rest done by the AI agents.

We all talk about, well, isn't it gonna be awesome that AI can generate code? But you know, I feel like that's just going back to the old, you know, let's measure people on how many lines of code that they're actually creating. And I kind of think, I thought at least that we kind of proved that that was pointless.

There is this goal that says we're gonna build more applications per developer, but each application is a problem that needs to be solved. And what is the cognitive load that a developer can really carry simultaneously? How many problems can I solve regardless of how much AI I have?

How much can I carry that, um, thought in my head at the same time? Can I really work on instead of three things, 20 things at the same time? Or is that just not how it's gonna play out?

Yeah. So again, I was not implying that you'll have to like, you know, really fully develop a hundred AppSec and deal with all the details, but really, but, but there are many variations that are relatively small as, um, consumer of the app, right? As a user, but may make a huge difference in like how you code the app, right?

That was what I was referring to. It, it is, maybe it is the same calculator app or it is the same Tetris game, but maybe the it's 3D instead of 2D. Maybe, you know, the buttons looks slightly different.

Maybe the the keyboard or the mouse controls are different. These are, it's the same app, but kind of you get a lot of variation, a lot of strength or power uplift by the AI to try these different things without having to rewrite the whole thing yourself manually. So again, cognitive load is definitely a thing, though.

I don't want to dismiss that, right? Like imagine, uh, I'm now, I run and I'm using ai, I give it some tasks and it comes back with 1 million lines of code that it wrote for me, right? Um, now what, right?

Uh, if there's a bug there, who's gonna maintain it? Who's gonna understand it, right? To your earlier point.

And so I think one of the challenges that we're facing as we're moving into this more independent or autonomous agents is how can we trust the results that we're getting back from ai? How can we guarantee that the code that is being generated is actually an asset and not technical debt, right? That's a central question.

And that I get asked by engineering managers as well, right? I have top nine, it generated whatever, 15,000 lines of code for me. Like how can I guarantee that this is something that we can maintain, uh, down the road?

And this is why we put a lot of emphasis on what we call top line coaching, which is a way to define rules and best practices that are enforced by AI as code is being generated and also actually for human written code, right? So you want to be able to define kind of like what are the rules for acceptable code in the organization? And these rules are org specific, right?

They're not the kind of rules that say, you know, your function has five nested if conditions. So it's a, so it's a bad function. This is not the kind of code review comments that you would like to get from top down.

It's more like, Hey, you know what Mike looks like you are sending customer data without encryption. This doesn't look like good practice in organization, right? Or looks like you're accessing the database without authentication or without logging.

This is again, not good practice. So you want all of that level of control, um, and oversight to be built into the AI product such that the human engineer can have more control on what's going on and not just watch the AI generate arbitrary code. Uh, that may or may not be maintainable downstream.

To your point, it seems also that people are struggling a little bit with the LLMs are probabilistic and so they're scratching their heads going, well, how come it doesn't gimme the exact same code? Every time I send a prompt, I get a different answer to every use case. And, and they're kind of like looking at it going, a lot of what we do is deterministic and I kind of need the same exact piece of code the same way every time.

So how do I kinda marry these things together? Yeah. First of all, again, think about it as delegation, right?

And so if I as an engineering manager delegate the tasks to five different developers, they may give me five different pieces of code that again, functionally may be equivalent or nearly equivalent for the requirements that they've been given, but they are gonna be different. And so having different results is not necessarily not inherently, uh, a bug or, or bad thing as long as it's, uh, it's gets a job done. Uh, but being contextualized on the organization, getting like the right code for my setting or a right code for the, for my setting, it may not be, again, like a single unique solution.

Maybe multiple solutions, but something that matches my specific org requirement. I think this is something important, uh, to, to get from the ai. Yeah.

So if, if you go again, if you go to say charge GPT and ask it to write a piece of code to accomplish a task, it does that completely in a way that is completely disconnected from the context in which you are operating, right? So let's say you ask it to write a sorting function so it will generate a new sorting function, maybe quick sort me sort whatever it is that is the favorite algorithm that you get from Chad G pt. But maybe you already have mic sort somewhere in your code base that is your specific sorting algorithm with all the edge cases and and whatnot for your organization doing it in, yeah, maybe sorting is too generic of an example, right?

But you, you get the idea like something that could be specific to your org creating employee or something like that, rather than the generic one that you'll get of a developer that has not been onboarded to the organization. So AI, just like any other engineer, has to be onboarded into the organization in order to give you, give the consistent relevant results that, that you'd like to see from an employee. So in that example, might it not be a better way to say, okay, well maybe I will ask four or five different AI models to create something for me and then I'll compare the output and I just need a platform that lets me kind of navigate those things to determine which of those outputs actually fits better in my use case.

'cause it seems to me each of these LLMs is slightly different and maybe better at a given task at any given moment. Yeah, and I think it's not as much a question of the model because the models, of course, they have their nuances and difference in, in behavior. Uh, but it's more a question of the context that is given to, to the model, right?

Uh, you can run the model with very different context and get dramatically different results. Uh, example is, again, what code base already exists and can you contextualize on the organizational code base for getting new code to be generated and maybe, you know, in one code base, the result is calling an existing microservice in another code base. The result is generating a new microservice, right?

Depending on in what context exactly we're operating. So to me at least, it's more about the context than it is about the specific LM and LMS are, you know, rapidly changing, improving all the time. And it's, uh, it's more the question of what you feed them when you're looking to accomplish a task.

We of course, have been talking about shifting as many things left as we can possibly do, and developers of course don't always appreciate that 'cause they're like, I am trying to write code and I have a few thousand things I'm trying to do here and you're adding one more. But where is the right place to like add things like testing and observability. Now in the age of ai, can I just drop that into the IDE so it sits alongside in real time as I'm developing and it's just something that sits over my shoulder and I don't need to do those things later in the cycle?

Or am I trying to just do those things everywhere across the entire cycle? Yeah. So actually, you know, you can shift further left from the ID to the, the requirement stage, right?

And you can maybe, uh, just write the requirements in the detailed enough way so you don't have to write the code. And the AI will write code that already has observability built into it, already has some, uh, security, uh, hardening built into it, et cetera, et cetera. So a lot of the, let's call it non-functional properties that they'd expect from the code being generated could in fact be handled by I and expect observability to be one of the first ones that, that gets integrated automatically because that is first annoying to the human developer and second, easy enough to, to get pretty decent results with automation.

So we started this conversation talking a little bit about what an application developer does, but in the age of ai, well, won't everybody more or less become a software developer? I mean, what is the, the scope of that role could expand considerably? 'cause all I need to know is how something works then maybe, you know, mere mortals like me or application developers.

Yeah, I think the, the, the realm of like system thinking, large scale software design, architecting the system, all those components are very much still, uh, human endeavor. Uh, understanding, you know, also existing code bases and how to interoperate with that is a combination of what the AI understands together with some, uh, human guidance, um, there's still gonna be a bunch of like understanding and improving and integrating the AI generated code, right? We're still not in, in a place where, you know, we push a button and get really an entire enterprise scale system generated by the ai.

So there's still gonna be a lot of putting together the components, whether they're human created or AI generated. So I think we're just moving one level up in terms of engineering, just like we don't have to worry for the most part about memory management anymore. And we don't have to worry about many of the low level details that I had to worry about when I started, uh, programming.

Uh, I think it is the same way we just moved up. One level, don't have to worry about syntax. Maybe programming languages become less foreign because, you know, maybe you don't really care if it's python syntax or c plus plus syntax.

It's roughly, uh, equivalent to the ai, right? You write one level higher, not necessarily completely natural language, but something that is between natural language and problem languages at this point, some mixture. Mm-hmm.

So there's still a lot of work for the human to guide the ai, understands their, understand the result, uh, refine them, integrate them, et cetera. When I talk to a lot of folks, they're a little intimidated by a lot of this and they don't know where to get started. So what, I mean, what's your best advice other than the fact that I'm gonna go tell a bunch of developers to start experimenting with things?

Is there kind of a, a method to the madness as it were that, you know, you're seeing people put together that has a better outcome? Yeah, I think that the first step is to start using something, get something in your id, uh, get familiar with the capabilities in, in a very real sense here, resistance is futile, right? Like AI is here, AI for software engineering is here to stay.

Um, the, the main thing in getting experience with the tools is to understand where they do not work right now, right? Like everything else, the hype is over the top. Uh, the skepticism is under the floor and the, the truth is somewhere in the middle, right?

And so kind of finding for your world, whatever your job is, what AI can do for you today right now, and kind of starting to refine your intuition on, oh, oh, here is a place that I should actually use AI and no, this is not a good place to use it. I think that's the, the most important thing that people should do right now, really start collaborating with AI and understanding where delegation to AI makes sense and where it does not make sense, right? So that's, that's really the first step.

And the earlier you take it, uh, the earlier you get on the train that is, has definitely left the station, right? This train is just accelerating and you need to be on it. Alright Folks, you heard it here.

The only wrong answer when it comes to AI is doing nothing. Hey rah, thanks for being on the show. Thank you for having me.

All right. And back to you guys in the studio.