Impact of AI on Developer Experience and Application Security – Randall Degges, Snyk

This is Textron tv. Hey guys, thanks for the, we're here with who's head of developer relations for, we're talking about the impact that AI is having on developers. Randall, welcome the show.

Hey, thank you so much for having me. I'm be here. I think it's already clear that developers are using the latest generative AI capabilities, things like copilots to write code.

The question is, is it making them more productive and in what ways and wonder are some of them may be the inherent risks that go with that that we're not considering? Great question. So first off, I've been writing code for about 25 years or so now, and I think the, the initial part of your question is, are large language models and their practical applications like GitHub, copilot chat, cpt, et cetera, are they making developers more productive?

I think the answer is absolutely yes across the board. Um, speaking from personal experience, the amount of code you can write and the amount of learning you can do using some of these tools is extraordinary. I mean, previously, if you were trying to figure out how to solve a problem as an engineer, um, what you might do is you might say, okay, maybe I need to implement a, uh, a registration form on my website.

You're gonna go off and google the name of your framework plus, you know, registration or password hashing or whatever that particular thing you're trying to solve was. You spend a while looking through some blog posts, maybe find some stack overflow questions and answers you use as reference nowadays. You can go straight to chat g p t, ask it there directly and get back some really useful resources and information.

So it expedites the learning process for engineers. The same is true for writing software. So in addition to learning about it, they can also now just output a function or a method or a class or an object, or sometimes even ready to go scripts.

They're gonna help do exactly what they need to do. So the first part of the question is, does enhanced productivity? I'd say absolutely.

The second part of the question though is does it introduce additional risks? And to really understand that you have to put a few pieces of context together in your head. Um, the first thing to know is that regardless of how software's being created, so if I'm personally writing software and Googling things and learning through Stack Overflow and then right handcrafting the code myself, you know, some organic artisanal handcrafted software, if you will, or if you have copilots shooting things out or other pieces of software, at the end of the day, security is a really complex topic, and if you're trying to build secure performance production software, you always need to have review, and you always need additional tools to make sure that the code you're shipping doesn't have additional risks in it.

And the same is true for code generated from anywhere. So AI is no exception here. The biggest risk vector to ai, I would say in the development of software is that people are now churning out code a lot quicker.

And when they do that, they're oftentimes either not paying as much attention to the quality of the code that they're creating, um, or maybe they're sort of blindly trusting that the code that they're getting back from these engines is really well, well written, really secure, et cetera, which is often not the case. And so the short answer is in terms of software security, you still have to follow the rigorous, you know, proven over and over again. Systems of building software in production environments, if you want to ensure the things you're building are safe and secure and AI is no exception, Do we trust these LLMs too much?

Because to your point, it seems like one of the issues is their general purpose platforms, and they were trained by essentially hoovering up all the code they could find, and the code was of bearing quality. So what they generate will be randomly different at different times. So, um, having too much faith in these platforms, Uh, I think the answer is absolutely.

So Stanford did some research a while back and basically found out that people using AI tooling to create software tended to have more security vulnerabilities in their code than those who do not. And even though you can ask developers and pull them and say, Hey, do you think the quality of AI generated code is better or worse than what you're generating yourself? A lot of people assume it's better.

The reality is that due to the very nature of the way large language models work and the way they're generating text and code and all these different things, um, is that it's never gonna be perfect. There's always going to be issues. And if you think about it from how these things are built as a point of view, we'll, you'll get it right away.

You know, so the way these things work is open AI or any of these large language model providers will go out onto the internet and crawl massive amounts of information. They'll download lots of questions and answers from stack overflow, lots of threads from Twitter, uh, lots of information from blogs, right? And so they have all these examples of text.

What they do from there is they take that information, they do a bit of data cleanup and sanitization on their end with human researchers, but after that, they essentially compile a statistical model and say, okay, um, given this particular, you know, chunk of text, what is the most likely next chunk of text to come after it? And the way they're generating that is by analyzing all these existing sources of information out there on the internet. So that's how large language models work.

Like if I'm talking to GitHub copilot or chatt pt and I say, please write a Python function that is going to generate a secure login page from my flask web application. It's going to be predicting that based on things that's seen online already. Now what's the problem with that?

In theory, there's no problem if all the information online on the entire internet was always accurate, but as we both know, that's just not the case, right? Like there's tons of information on Stack Overflow where maybe it was accurate a couple years ago, but today that information has security issues in it or just isn't accurate at all. There's always these gaps in knowledge, right?

And so blindly trusting data that you get from a large language model is just a really, really bad idea. And I think more and more developers are coming around to realize this, especially when you're using, you know, static analysis tooling like sny or other companies or even open source projects out there to help vet the quality of your software. It becomes pretty obvious after doing this for a bit that you can't blindly trust code that's written by ai.

Are there different use cases where I might trusted more? So let's say I'm trying to create a script for something that's running internally, and I just wanna automate some DevOps workflow versus code that might wind up in a web application that's externally faced. A hundred percent.

I mean, at the, the end of the day, it really comes down to risk factors. There's certain types of things you can do with AI where the risk of causing a a an issue that's going to impact people is extraordinarily low, right? Like for example, I was using AI the other day to take some video files that I had locally on my computer and to strip out the metadata and change it with something else.

And I essentially went to chat to U P PC and I asked it, Hey, please create a command line script using the FFM P command line tool that is going to do these things that I just described. So let's bat back an FFM P command line for me to copy and paste into my terminal. Now, the risk there, I would say is pretty low because first of all, I asked it a very specific question about the tool that I already know what the tool does in a vague sense, right?

So I don't have to understand every single option that it's giving me back, but I have to understand the basics of it. Secondly, I'm only going to paste that one f m peg command line option that is sped out to me into my terminal. And so I have a pretty good amount of confidence what it's going to do, and that if there is an issue with it, or if the data or if the information I got was incorrect, the scope of that issue is very limited as to what, what problems it can cause.

Now this situation could be radically different. Imagine your developer working on a medical company and you're help building tooling that is going to take in patient information from a patient's, you know, uh, uh, charts and, and, and you know, history and then make medical recommendations for changing medications or going in for routine lab work. If you're performing actions like that and you get incorrect information, you know, the consequences can be catastrophic.

And so you really have to just, uh, put on your risk management hat whenever you're using tools like this and just think, okay, how valuable is what I'm about to do? What are the potential things that could go wrong if I do it? And I'm really just way off base?

And better yet, how do you write your questions, the scope, the types of things you're asking AI for to help you out while reducing that risk? I feel like the pace of innovation for developers as far as pacing the innovation being seen by the DevSecOps teams, are we looking at a situation where the amount of code that's being generated and the code bases are just gonna grow exponentially and then weeks and months ahead, and I don't really have the tools to cope with that, and I don't have any generative AI capabilities, or do I, what's the state of the, um, frameworks for helping me cope with all this? There's a lot of things to consider there.

I mean, let's just take it from the policy perspective for for a moment here. Um, at a lot of larger companies right now, CISOs and engineering leaders are freaking out. And the reason they're freaking out is because they have employees, just random developers who are using tooling that they haven't approved or vetted whose information might con or whose request might contain sensitive company information.

People are doing these things today, and so there's a lot of like weird things happening from a policy and enforcement perspective. Um, at, at sny we talked to a lot of large companies with advanced security departments, and in those cases, basically everyone agrees, Hey, developers usually using these tools already. The solution isn't to enforce, uh, restrictions on what things you're able to use.

The solution is to make sure that we are extremely rigorous with our software development practices. If you're going to be building software, it's more important than ever right now to follow strict, you know, best practices for building software. What that means for a lot of people is if you have an engineering team and you're writing software, you need to be using static analysis tools to take a look at every single piece of code you're writing, ideally, as soon as you write it and try to detect security issues early on.

Um, you're very right in the sense that the innovation for outputting code is vastly, uh, surpassing the innovations for securing code in in general. However, if you're using the right tool and you can avoid a lot of those mistakes. Now I'm gonna mention sny briefly because I work here and that's what I'm most familiar with, but the approach we take to doing this is twofold.

First of all, we use large language models ourselves to show people how to actually fix a piece of software that we know has a vulnerability. So let's say we're looking at your code mic and we say, Hey, this, this one Python file that you're writing has this big, you know, path traversal vulnerability in it, and if you allow users to actually view, view this page of your site, there's a big risk, uh, of data leakage or other things bad happening. So we can take a look at that.

Now, in the past, we were able to successfully identify that stuff by using our own symbolic AI engines. Now, these aren't large language models, these are things that are trained by actual human security analysts that are highly accurate. And these are two fundamentally different approaches to ai.

One approach is to predict what thing is gonna come next based on a lot of input, which is what large language models do, the others that have like actual people creating and crafting rules to detect things that are very specific and niche. And so at nyk, we take a hybrid AI approach. We use symbolic AI to help detect issues in code with a very high degree of accuracy.

Then we use large language models to generate potential fixes. And then once we generate these potential security fixes, we then use our own symbolic AI tooling to validate that those fixes will actually solve the problem and aren't introducing new issues or new security problems. And so it's really only through a combination of using different tooling and having different like fundamental philosophies on securing software that you're able to deliver a single product that's going to comprehensively help in this regard.

Will this something get better if we start using LLMs that are trained specifically to write better code and code? We've seen some examples with that is the l l m is much narrower. It's not a general purpose.

It does one thing well, and so ultimately, might this someday get better? So there's no doubt things are gonna get better. However, if you're still thinking about it, the fundamental problem is to have a successful L L M need a lot of data and a lot of examples.

That's the only way to generate really accurate predictive texts. Now, the problem there is software is a very niche field. All things being considered and software is not typically written the same way.

Like if you take a look at and you analyze every single book in the Library of Congress and read all the texts of those books, the English language follows a lot of straightforward rules and you'll see a lot of the same types of things, right? Like you'll see similar paragraphs, similar explanations of things there. And so you can train models on English text fairly well, code's a completely different story.

Like if you go on stack overflow and look at examples of how to properly implement a b crypt password hashing algorithm in 2023 or, or whatever the modern times are, you're gonna get back a few examples of this, but there's not a lot. And so the fundamental amount of data you're working with is a lot less than other domains sometimes just by the very nature of the types of work we're doing. And so because of that, it's going to be a big challenge for companies like OpenAI and other large language model providers to build these comprehensive data sets to get better in these very particular niches that, that we really care about, right?

And so it's definitely feasible, feasible for that to happen in the future, but I think we're still personally a long way out from having these rock solid l l m style outputs. And again, due to the very nature of the way LLMs work, there's always going to be a need for different types of solutions there. And that's why we're heavily investing in not only the large language model stuff at sny, but also symbolic AI to make sure that the things we're actually generating are accurate, uh, production ready, safe, secure, you know, all all that stuff.

So, so short of outright banning usage of some of these tools, what's your best advice to organizations about how to go about, uh, having this conversation with developers? I think that if, if anything, if I've learned anything in the last, you know, 10, 15 years from the security space, it would be you need to establish clear processes and security is something that every single person in your organization needs to think about. You know, up until a year ago before like chat c p t hit the scenes, security was already, uh, an under-resourced part of businesses.

You might have one security engineer to a hundred developers at an organization, and that security engineer would be responsible for monitoring software, coordinating with software developers to make sure they're fixing things in a prompt, uh, fashion, et cetera. Nowadays, the importance of that has multiplied, you know, a hundred x. And so the best advice I can have is if you're working in security on an organization, build a really clear, simple process for your engineers and don't just give them a clear process that says, Hey, every time you write code, make sure you run this tool to scan the code for issues before you check things into production.

Um, you need to go a step further. You need to have security implemented at all the different parts of your software development lifecycle. Ideally, if you're writing code in i d e, you'll have a plugin, like the sneak plugin that's analyzing the code you're writing in real time and giving you security fixes and recommendations.

That's the first part of it. The second part is to instrument your continuous integration and delivery pipelines with security tooling. So that way not only is, are you looking at the code you're writing in real time, but also you're looking at the entire code base all the time in a recurring fashion to figure out are there new newly introduced security issues that you weren't aware of before?

Or was there an a security issue disclosed recently that is now impacting you that no one even knew about before? You have to have these things in an ongoing fashion, and there's no way to like short circuit that, you know. So the best advice I have for security teams, again, is build a simple process.

Make sure everyone in your organization, every single developer is aware of what that is. And then finally, you should be doing enablement for your engineering teams and showcasing to them how to actually use these tools to be productive. It, you know, the best security tooling in the world isn't going to help if no one's using it and understanding it, right?

So you have to not only have the tools and have the process, but you actually have to understand it as well. And that's, you know, the sneak advice for security teams. So, So ultimately, do you think we're making progress on DevSecOps, or is it still in the realm of sometimes I feel like, you know, the boss shows up and gives the sermon and everybody nods their head and then goes back to living their lives as they were before.

I mean, anecdotally, what I've heard over the last year, just from peers in the space is people care about security a lot more. And I think your average developer is much more security conscious today than they were a year ago than they were 10 years ago. It's become a more and more critical part of software development, and people are just more generally aware of this in the industry, which is great.

Now, in terms of tooling, like is the DevSecOps space actually getting better for users? I would say absolutely. Uh, I think one of the biggest and most exciting things to happen in the entire security industry for the last like 30 years or so, is the advent of these LLMs becoming, uh, commercialized, if you will.

The reason why is because we're getting pretty close to the point where today, you know, a lot, a lot of security tooling is focused on, first of all, identifying problems. That's like the first thing security tools focus on, right? The second thing is on helping people fix those problems, which is what sny and other vendors do today, right?

Like we identify the issue, we show you how to fix things. The third part though, which is quickly becoming a reality, is the ability to autonomously fix issues. This is something that has only been a dream for people ever since, you know, I've been in the space, you know, what if as a developer you could write some code, it had some security issues in it, and in real time you are getting fixes for this code delivered directly to you.

Or maybe, you know, maybe right now those fixes get put into your I d E as you're writing code so you can understand what's happening maybe in the future. Those fixes happen instantaneously, and you don't even need to look at them. We're, there's so much innovation happening in the space right now, it's a really exciting time to be in security, and I would love to tell you about a lot of these sort of interesting projects and experiments we're doing internally.

Some of them are, uh, going to be announced in the future. And so I, I don't wanna spoil the fun, but there's a lot of really exciting things happening in this space, for sure. All right, folks, you're heard in here.

Things may get a little bumpy in the short term, but long term we get better with every cycle and every revision. Hey, Randall, thanks for being on the show. Yeah, thank you so much for having me, Mike.

Appreciate it. All right, to you guys in the.