Snyk CTO Danny Allan Warns of a Potential Software Security Crisis Amid AI Vibecoding Boom

Hey guys, thanks to the throw. We're here with Danny Allan as the CTO for ny and we're talking about vibe coding and the security implications thereof because, well, we've seen this play before and it didn't end well, so maybe it'll get better this time. Danny, welcome to the show.

Ah, thank you Mike. Always a pleasure to be on with you and speaking with you. Alright, so back in the day, we had low-code and no-code tools and people used those to build applications, so-called citizen developers and a lot of those things, well, they weren't exactly pretty and they were very rarely scaled and they certainly contained a lot of vulnerabilities.

So now we have vibe coding tools that appear to be the next generation of these things and we're gonna use AI technologies to rapidly iterate applications and what else could go wrong, but is there a way we should be thinking about vibe coding and security maybe beforehand and not chasing in after the fact? One more time. Well, vibe coding is here and my belief is that it's here to stay.

It gives a lot of productivity to developers, especially when they're building applications that are CRUD type applications, create, you know, review, edit, delete type applications. I, I don't think it's going away, however, because it's trained vibe coding because the LLMs behind them are trained of course on open source code. There are, as you say, lots of vulnerabilities and I think there's an opportunity, uh, to build security in at the very beginning.

We call this security from inception. Um, we're not doing it right now to be clear, but there is clearly an opportunity this time to do things different than back when we had no-code, low-code. We've been talking about shifting left for some time.

So is this just a continuation of that and how far left do we need to go? I would argue that this is further left than the previous iterations that we had. So when we said shift left before, most of the time it was, I want the developer back when their writing code to be doing security testing within the IDE and sny obviously did that with all of our, our customer base and that was kind of considered shift left going back the developer's desktop and testing from there and of course through the pipelines.

But when you're vibe coating, there's no actual code at that point. Literally you're interacting in natural language with your cursor or your windsurf or whatever the coating assistant is, and you're saying, I would like an application that does X. So you actually haven't written code, you're actually generating code.

And the opportunity here is to inject security into that process. And so I, my way of thinking about it is, this is security from inception, this is even further left than we had in the past, and that's a good thing. To that point, therefore, am I gonna build some sort of AI agent into the vibe coding tool that will monitor the security of the code as it's created?

And I'm assuming that I don't necessarily want that AI agent to be based on the same LLM that's being used to write the code. 'cause otherwise it'll just confirm the mistake that was made in the first place. Maybe I think there's two phases that we're gonna go through.

Um, the first phase is that there's gonna be generation of code and then the code will be tested. And we're already doing that right now, by the way. So if you're using one of these generative coding assistance, it's generating code and at this point there's a fork.

You can either tell the LLM, which was what you just indicated, you can say make it secure. And some interesting stats there. com where they test this and g PT five, when you say just generate the code, uh, 54% of the code that is generated is both what you want and secure.

When you prompt it to do security, it actually drops from 54% accurate and secure to 44% accurate and secure. And my belief is that it is because, um, the LLM in the model has been pret tuned for security. And so when you give it instruction, it overrides, however, that is one fork in the road as you're generating code, you give it a second prompt to say, make it secure.

Another way of doing it is to send it over to an alternative tool. And I think this is the better approach. Um, I quite honestly, because you want to use Belt and Suspenders, if, if the LM already generated in secure code, you probably want a secondary model to validate it.

And you can do that. Again, snyk already is doing this, um, V-R-MCP server to, you know, exposing it to any of these coding assistance. However, my beliefs is is that that is really phase one because what you're doing is you're iterating, you're generating code, then you're testing it, securing it, you know, generating it again, testing it, securing it.

I think ultimately we're gonna go to the point where the LLM itself that is generating the code actually generates secure code. So rather than being trained on a data set of all the open source code, which has all the vulnerabilities that LLM or the, the open source code will be curated into secure code only so that what is generated is actually secure. But I do think that that is several years out.

I don't think that is logistically likely in, in the next few years. Well, it makes sense. So challenging.

'cause in theory you would just kind of go find the examples you were looking for and, uh, hire a bunch of people to show the LLM, this is good code over and over again. But what makes that more challenging than we think? Well, if I ask you, Mike, to go through all the open source code over the next month and, and secure it all, you'll probably realize what makes it so challenging.

The good thing about open source is there is so much code, the bad thing about open source is that there is so much code out there. Um, it does, it takes, it takes time intelligence to go through and verify it and make sure that there's not any false positives and false negatives. That is actually a very, very time consuming thing that I do think will happen over time.

But it's going, it will take time and it will take people that are security aware. Um, most developers, they know about the common vulnerabilities like, you know, cross site scripting, SQL injection. But when you get into these complex authentication authorization, cross file, you know, access type scenarios, it, it really does require someone who understands security at its heart to make sure that it's done properly and correctly.

Mm-hmm. What are the bad guys doing about all this? I'm sure they're watching closely and they're probably elbowing each other a little bit going, can you believe what they're up to now?

But, um, you know, can we secure all this stuff from them because they'll go after this stuff in a heartbeat, right? Yeah. Well, the bad guys are not unaware of AI and they're using AI often call this the perfect storm because you have, uh, more code than ever because of ai.

You have more complex code than ever because there's an increased attack surface. But then, uh, there's also attackers that are using ai and we're seeing this in some of the recent 10:00 PM supply chain attacks where there, they're actually worming, I don't know if that's a word, but they're, they're creating some of these supply chain things and, and putting in code that propagates itself. And it's very clearly using AI techniques to do this.

Um, so, you know, on one hand it's easy to be frustrated and, and propagate fear, uncertainty and doubt. But my belief is that actually on the, on the plus side of this and the white hat side of this, we have the tools that are disposal to actually make things more secure and more secure at inception. And that's a good thing.

Mm-hmm. Are we taking this seriously enough now or is this one of these circumstances where we're about to go build and deploy a bunch of software and then there'll be some cataclysmic event that everybody will wake up and say, oh wait, we gotta go do this smarter. Well, I would love to believe that this time will be different, that we will do security, that the market, the industry will think of security from inception and do things the right way.

So far in the last 12 months, it has not proven to be the case. Security tends to be an afterthought. And I think that's just human nature, Mike.

We, we, um, like to build things and so we get excited by using AI to generate code or do new types of software that you could never do before when it's backed by LLMs and security tends to be an afterthought in that environment. So I think we have the possibility to do things right. It doesn't seem to be trending in that way at the moment.

Now there are areas where that is the case. If I talk about our financial services organizations, our financial services customers or technology, they do tend to think at this at the very beginning. But one of the things that AI is doing is it's lowering the bar of entry.

And so everyone is becoming a developer. The, you know, the the mechanic shop down the street is now writing software and doing things, and of course they have no knowledge or awareness of security. And so the best thing that we can do is actually to build security into the underlying foundations.

I always said how did we eliminate buffer overflows? It wasn't by educating everyone on how buffer overflows worked, we went to memory managed languages like Java. And so if we can build security into these coding assistance and into the AI itself, we will come out far further ahead than if we were just trying to educate and pressure people to do the right thing.

Mm-hmm. And isn't that ultimately gonna have to be the end goal? Because expecting developers, nevermind citizen developers to know about what it takes to secure their code is just, well, it's nice if they do, but it may be just too big an ask.

Yes, I, I do agree with that. I think it is too big enough to try to turn everyone into security people and we shouldn't expect that their goal is to build software to, you know, make their beer taste better, is what, uh, uh, Jeff Bezos used to say. That's what you need to focus on, do what your business does really well.

But what we can do in the security industry and what we're very focused on here at snyk obviously is building security in, in a way that it adds no cognitive load, that it's just part of the process. So if you are vibe coding to use that term, security guardrails are just built into that process as it goes into the pipelines, it's part of that process as it gets deployed and the security controls are, are put in place. And so while I do think education is important, I also think we can do so much more to embed it functionally within the foundations of, of this new era.

Mm-hmm. What's your best advice then to the security folks who are probably looking at all this and they can see that there's clearly a tsunami building, it hasn't quite arrived yet, but, um, what do they need to do to get prepared for this? Because, well, the first wave of this stuff at least is almost guaranteed to be insecure.

Well, the first thing is embrace it. Don't resist it. When you begin to resist these technologies, all you're doing is creating shadow it.

If you say you can't use this or you can't do vibe coding, what is going to happen is developers are going to find a way around it. So you should not resist it. You should embrace it, understand it, learn it.

The other thing, um, that I think that they can do is collaborate with the business. Recognize that security is not a, a department of no. It's how do I collaborate and put in the guardrails and the checks and balances to ensure that we can actually trust the ai.

The exciting thing is AI does unlock a huge amount of opportunity, and when we can trust that the software coming out the other end is secure, then you know, we can do so much more. And yet, when we talked to CISOs, we did a survey earlier this year, 96% of CISOs said that AI was one of their top security concerns and 70% of them actually had said that they had an AI related attack within the last year. And so my point is this, that we need to collaborate with the business to help them embrace it, but doing it in a way that we can ensure trust to the, to our executives or to the board of directors.

Mm-hmm. Aren't we just moving too fast? And I asked the question because, well, nobody got out of bed this morning and said, yep, I know what I wanna do, let's go build some insecure software.

But they seem to do it anyway. And I think part of that issue is that there is this sense of, you know, we gotta complete this thing now and not take the minute or the day or even a couple of days that might be required to think about the security aspect of it. But, um, is there just something wrong with our culture?

I think there's two answers to this. If you're in a well-established technology space that has been building software for a very long time, sometimes it actually makes sense to slow down before you go fast to make sure that you get things the right way and do things the right way. However, I don't wanna ignore the other part of the industry.

We have all these companies that are, that are being built right now with billion dollar valuations on top of ai. And their moat really is speed. The, the reason they get to the size that they are, the reason that Cursor has gotten to the size that they are is because of the incredible speed that they've used on top of ai.

And another way of saying this is there's not technology differentiation. There is only speed differentiation. And so it's really hard to go to a company like that and say, oh, you need to slow down because they'll get passed by three other people who are, you know, fighting to get to the top of the mountain.

So I I, I think of the industry in two different ways. The AI native companies that are moving fast and pretty much have to move fast. Um, and then you have your well-established companies with, with a lot of intellectual property.

And for them sometimes I think you do need to slow down before you go fast. Mm-hmm. Do you think that the regulators and the auditors are starting to figure all this out and looking at this and going, well, on the one hand they're probably freaking out 'cause they're like, how much stuff do we gotta review now?

And the second part of it is, well, there's just gonna be more things for them to find. So, um, are they gonna start, you know, aggressively analyzing more and more of these projects? Or is that still, in your opinion, I don't know, maybe a year or two away before they wrap their heads around it?

I think we're a year away at year or two away from regulators getting their hands around it. I mean, I'm in the technology industry and it's hard for me to keep on top of what happened in the last month. You did.

And I live and breathe this every single day. Um, regulators historically have always been slow to the market because they don't live and breathe it. They're regulating and legislating and doing all the things that regulators do, and they don't even necessarily understand it.

So I think the, the regulatory part of this is not going to drive the industry. I think it is up to the technology companies, up to people who are on the cutting edge of this, the financial services, the insurance industries to really step up and be the leaders on how do we adopt AI but do it in a secure, trustworthy way that doesn't put us in a, in a worse place. Mm-hmm.

And who should be in charge of all of this? Because we have the people building software, we have the security teams, we also have a bunch of AI teams in a lot of organizations. And then, um, but I feel like, you know, as we kinda get the village together to build the application, nobody's in charge of security so nothing gets done.

I am a big believer in small, tight consortiums of leaders that get together. You never want a single company owning it. It's a bit like a fox in the head house because it's likely if it was a single company to be one of these model-based companies.

And so consortiums actually do very well and SNY is a member of one of these, um, cos I is the ones the consortium for security ai. But, but I'm a big believer that these types of organizations with a limited number of members who have the best interests of the, of the industry and the public in mind are probably the most effective way to drafting the initial models and architectures for security. And actually to this end, we're participating in this conference that is happening in October called the AI Security Summit out on the West Coast.

And that's exactly what it's about. It's not a single company sponsored. It's about bringing together industry leaders from everyone from coding assistance to security companies, to data companies, to, to develop and draft kind of a thought and a proposal for how the industry is shaped going forward.

Mm-hmm. Do we also need to be a little more cognizant, maybe even careful of how we're going about building all this software using these vibe coding tools? And I asked the question 'cause twofold.

One is, um, it seems to me it's ripe for building a lot of duplicate applications within an organization. And as such, every time I do that, I increase the attack surface. But also the amount of code being generated by these tools is, shall we say, verbose and debugging that and understanding where the dependencies are is that much more complicated.

And so therefore the attack surface also got deeper and wider. Is that fair? Absolutely.

It's something that I've actually been speaking about for the last few months or so because it's deeply concerning and, and it hasn't been raised a lot for the last, I don't know, I'm gonna say five, 10 years. What people have been doing is going to open source to, to use open source components. And the good thing thing about that, if there was a vulnerability in the React framework, you know, and it was discovered it would get patched and people would be secured.

'cause everyone is using the same component to do that similar function with Vibe coding, there is a real danger that we fragment the code that is being created because rather than going and looking for an open source component, what the Vibe coding tool will do is generate its own code based on the open source. And so now you end up with 20 companies because they're using a non-deterministic model that have slightly different code for the implementation of this particular function or feature or capability. And 20 different code bases means that you could end up with 20 different vulnerabil vulnerabilities implemented in 20 different places for which there is no future patch.

And so I actually worry a lot about this, that when Vibe coding we end up with a greater amount of code that quite honestly would've been much better to have been consolidated on an open source component that is more easily managed downstream. All right. Hey folks, you heard in here five coding, it's not going away.

It's gonna be everywhere soon. The issue is of course, that all that code is also gonna be everywhere soon. And you, we may not know even where it's running or how it got deployed until it's too late.

Hey Danny, thanks for being on the show. Ah, thank you. Pleasure to join you.

All right. And back to you guys in the studio.