Sauce Labs Warns AI Coding Is Outpacing Software Verification

Hey guys, thanks for the intro. We're here with Prince Kohli who's the CEO of Sauce Labs, and we're having a little chat about some research that they have about the impact that AI coding tools are having on the quality of the software and applications that we're building, because, well, it might turn out to be too much of a good thing. Hey, Prince, welcome to the show.

Hey, Mike. It's a pleasure to be here. Walk me through what's going on.

What are you guys seeing? Because we are at least hearing more reports of issues in production environments stemming from AI coding tools. Is it the fact that we're just not testing enough at this point and the testing hasn't caught up with AI coding, or is there something more amiss here?

So, there is an arc in verification. And the arc is that if you go back, software creation and writing code has always historically been a little bit faster than software verification. But around 20 years ago when test automation frameworks were invented, these things could more or less stay at the same pace.

Both got better over time, but the change in pace was not significantly different. So you could keep up with the traditional methods of verification. But with AI-written code now becoming, depending on whom you ask and how people do it, 5 to 10x, if not more faster, the older, the earlier verification methods are simply not able to keep up.

And there's multiple reasons that we can get into why that is the case. But that lag has become now, it's not just again of a 10% lag, it's a 10x lag. So verification is becoming the very long pole, and what we have seen in our customers with the report that we did was they're reporting that while code creation is becoming 10x or so faster, release velocity is only increasing by 20%.

But the rate of bugs is also going up, and also in the AI-generated code, the rate of bugs is going up. And the costs are not coming down. Productivity is not where you want it to be.

So lots of very interesting data points. " So at some point, the issues in the production environment kind of negate the benefits that I got from the AI coding in the first place, and we're kind of on a zero-sum game here. Yeah.

See, I think that is well captured, I would say. " And people often conflate code velocity with product velocity. And that is not the case, right?

" I can go into Cloud Code today and say, just give me this app. It'll create something, right? It might even seem to work on day one.

But there is code reviews. Humans need to be involved in many cases. You need to make sure the verification is done by a different LLM than the other one.

You need to make sure that test authoring is done at the same pace as code creation. You want to make sure that verification is done on the right devices in the right environment. Those things don't magically disappear just because you create code.

So hence the reason that at least the responsible organizations, they're not releasing at the same velocity as they're creating code. It's a little bit better, but not as much as it needs to. But when there is pressure on them that says, go fast because now you have AI, then everyone tends to take shortcuts.

Then you have reviews done by agents that don't always understand what the code was. You have verification done as a optimistic verification, right? Happy path verification.

This seems to work in this. It should all be good. Then they release code.

And then because you have more code going out, you have more bugs going out, and because it is less verification, the bugs can be very subtle as well. And therefore, the net cost is probably worse than before in many ways. Yeah.

One of the things I hear is that the developers themselves are kind of, well, they're not writing as much code, they're reading more code. But they're reading code that they don't really understand because they haven't had the context for how it was created. That's right.

But at the same time, they're under pressure to ship more code, so they skip the application development processes or the testing processes anyway. And that may not be net new per se, but it seems to compound over time. And so is this exposing a lot of the weaknesses we had in application testing in the first place?

Yeah, I think so. I would say yes, and I would say it's a big yes also because, see earlier, because humans, people like humans seem so weird. People like us, you and me, we were writing code, right?

And because we were writing code, we knew what we were writing, and therefore, we knew what it is that we have where it needs more analysis and more review because it's a complex algorithm or a complex heuristic. And we would, in the code review time, we would have the team look at it. So there were, by design or by accident, there were processes that were catching soft areas of the code earlier.

And wherever as a developer, for example, if I felt this code is very complex, I need to make sure really it works in production or in testing I would inform my QA team, or I would help them create the right kind of framework for the testing and to spend time on this aspect. But what's going on now is there is too much code being written for these processes to work because these are manual processes. You just cannot keep up.

Right? Mm-hmm. And therefore, reviews have become automated.

Testing is functional, not user journey driven. It tests one particular part, not the entire complex end-to-end. And because traditionally, authoring was also done by the developer or by the QA, and that is just a slow process.

So unless you change that process, you cannot keep up. And that's what the companies are discovering now. Hmm.

I also believe, and I think it's true, that the latest generation of the AI models are getting better. So theoretically, at least, the quality of the code being generated improves. But the cost of using those models is also higher.

So what is the balance here between writing great software using AI and the total cost of the exercise? Yeah, and that is the latest thing, right? Which is how many tokens is worth one engineer?

That's the latest debate, which I find it's either amusing or sad, depending on which side you are on. But I think we mix up two things that should never be mixed up. See if you go back any number of decades in any industry, you cannot weigh technology in terms of number of humans it takes.

Although, you can do a special exercise. These technologies are meant as assistance to make people better. Right?

They are not really meant to replace people, and sometimes it's too easy to do that on a piece of paper, but that is not what it is. So, the way I look at this is that a good developer knows that they have great compilers, they have great tools too for CI/CD, similarly. Now they have great tools for helping them code.

But the ownership of that code is theirs. They have to make sure it works well, and good developers will use these tools to create code at faster velocity, but they will make sure that the way they write the code is written, the way it is reviewed, the way the prompts are done, the way you do cross LLM check on a code, those processes are also of high maturity as well. And while any LLM will get better, and you will see fewer silly bugs, but the severity of bugs will become worse over time.

And humans will have a very hard time finding subtle bugs. It's always been hard finding subtle bugs when you have written the code. Now, when the architecture is someone else's, the code is someone else's, it's always been the case.

With someone else's code, it's impossible to figure out what's going on. And AI-generated code is also a little bit like spaghetti, right? So it just becomes much worse.

And this seems to be playing out in spades on the security side of the equation because we are seeing AI models discover more vulnerabilities and weaknesses in code. And we also can't really rely on the severity ranking for any given vulnerability anymore because the bad guys are figuring out how to daisy chain a bunch of low-level ones together. So- Mm-hmm ...

do I need to kind of rethink my approach to testing and with an eye towards security because some of the constructs that I previously used just don't work anymore? That's right. And the constructs don't work, and one of the reasons cybersecurity wasn't as worse as it is now is it was hard to do, right?

It was just complex. Understanding a buffer overflow and chaining them together, like you said, those required a lot of work by very smart people. But quite a bit of it also was mechanical, and the mechanical part has now been taken over by the LLMs.

So therefore, the bar for code verification and shipping high quality code free of bugs that can be exploited by black hat or white hat people, that bar is actually much higher now because there are more people, more automated agents trying to find bugs in your code, smart agents, too. So, you also have to know that earlier bugs that you could ship with, you just cannot anymore. Hmm.

Now, for a long time, we talked about shifting testing left. And conceptually, that all sounds good, but I think what happens, though, is in the age of AI, the developers are using the same AI agents and models that they use to write the code to test the code, which is essentially asking the AI to grade its own homework. Is that fair?

It is well put. So the modern way now with AI tools, the right way to, that I think our largest customers who are the most mature are thinking about this, is along the same lines. " Doesn't look at the code because the source of truth should be the intent of the application, not what the code does.

And auto author, auto understand intent, auto author tests for the right kind of environment, auto run tests, auto troubleshoot tests, and that allows it to keep up with the AI-generated code. But more importantly, also is a second check. Right?

It's It's a different cousin not doing it. It's a different family doing it. So you can control that if, let's say you wrote code with cursor, then you can use any of the LLM that we can provide, right?

Whether it's Gemini or Cloud or whatever it is, and with our own secret sauce on top. And say these two are going hand in hand to make sure that the code does what it's supposed to do. So all our biggest, I would say every one, every last one of our largest customers, who are major banking, financial, large enterprises, who are very careful about what they do, they are following the same path.

They are absolutely avoiding using the same model to write and to verify. Mm-hmm. All right, I get you so far.

So what's in the report that surprised you most or kind of leapt out at you and said, wow, I didn't think I'd see that? Yeah. So I would say that in many cases, I thought the numbers would be closer to the middle.

But, an example of where I was surprised was 80% of organizations said that they traced a production incident or outage to code that was generated by AI. 80%. That's higher than what I expected.

And 90% said that there was serious business impact from these issues. 66%, and I'm giving you these numbers because all of them are just higher than at least what I expected, would love what you thought. Two thirds of them admitted that they had to compromise quality or testing standard to hit a release deadline that was imposed because they thought AI code was supposed to go faster.

So across the board, and by the way, there is another one, 92% admitted that they don't have safeguards to catch failures from AI code today before they go to production. So, scary. Very scary.

But early days, my hope is that people who read these reports start becoming just a bit more cognizant of the risks that they are taking and hopefully do something about it. Yeah. Are we going to have that moment, and what kind of drives that conversation?

Because I think right now there's a tendency where everybody's kind of AI happy and they're all, we're generating more code than ever and this slapping on the backs and everybody's kind of doing this, that, and the other. " Yeah. You see, what I've seen is there is always a bit of a cycle.

It's a sine wave. Initially there is always excitement about what you can do, and that's how it should be. Nothing wrong with that.

So everyone tries new things, right? And then you have experiences like Samsung did and others had, where data starts getting leaked or bad security problems hit, or suddenly your customers start getting just truly upset at what they consider AI slop. And I think the turning point will be when you get just like what is happening right now in labs, where you are seeing the security, the agents escape, and hack their way out of sandboxes.

You're going to start seeing the issues being exploited by agents on real production code for real applications. And it's going to cause a huge amount of loss of face, loss of money, and loss of brand. And that's when people will realize, okay, or take this seriously.

There is no magic to it. There is hard work involved. And whether it is agent of agents model, supervising agents model, but however you do it, take verification seriously.

But it is 100% a given that's going to happen. It's going to happen in the next very short order of time, weeks or months, not years. I think, too, one of the things that maybe we're not counting on is the fact that the end customer now has AI, and they may use that AI on the software that's being shipped to them to evaluate it, and essentially they're running their own tests, and they may have a thing or two to say about it.

That's an excellent point. Yeah. That's right.

The discovery has become easier, too, right? So yeah, it's 100% true. All right.

So, ultimately, what's your best advice to folks about how to get in front of this and maybe start this conversation? Because I don't think anybody wants to be the person who's trying to bring everybody down, but at some point, somebody has to stand up and be the adult in the room, right? Yep.

So my advice is twofold. One is, do not be scared of AI. You must absolutely use AI to make things better, right?

Because the point is not to step back. But remember that as you're using AI to generate more code, it must go hand in hand with making sure that what you ship, your responsibility is to make sure that it does the job well with high quality security issues being taken care of. So invest on both sides of the house, code creation and code verification.

And that is what solves the problem that they're all being asked about, which is how do we get more productivity as opposed to paying the cost that you don't realize today, but tomorrow you will pay. So go forth, conquer both the creation and the verification. And we are happy to help, right?

If someone needs a expert on the table. All right, folks, you heard it here, and there's no reason not to go faster. You just got to go faster, smarter, and with some care.

Hey, Prince, thanks for being on the show. Awesome. Thank you so much, Mike.

All right. And back to you guys in the studio.