We Analyzed AI-Generated Code: Here’s What You Should Know | AI in Action 2023

Hi there. I'm Jaime Jorge. I'm COA Co-founder of CO C.

And I'm here to talk to you, uh, today about analyzing AI generated code. And so, um, this is kind of a presentation that, that, um, that encompasses many things that we've been, uh, gathering in kind of a lot of our experience. Uh, building code quality and code analysis and security.

Um, and let's start with a, a good introduction. So obviously what we're, what we're seeing is, is ai, um, reaching out to every single, um, possibility realm of, of, of our existence. Um, and coding and programming is no exception.

And there's an estimate here that we're seeing that, um, GitHub copilot being kind of the, a good representation of using AI assisted tooling for development. They're reporting 88% productivity gain, um, when developers are using tools like that. And so there's, there's been a wave of adoption and excitement, um, of AI ss tooling.

Um, and it's been very exciting. And, and I think this opens up tons of new possibilities, toolings, IDs, um, and at the very least, uh, a whole new realm of productivity. Um, and so we know that is efficient to write AI generated code.

The question is, is AI generated code reliable, secure, and effective? And these are kind of three questions that we're gonna be asking throughout this presentation. Um, and what we, we, we start ourselves to ask, and, and, and this is really important also because our customers, um, are asking the same questions.

They are asking, um, whether the development teams should be using, um, AI generated assisted tooling. And so we devised an experiment, um, to try to answer these three questions. And the, an experiment is quite simple.

So we did was we generate fully executable code using ai. In this case, we used the, um, OpenAI, um, API to generate code GBT four. Then we ran our static code analysis, uh, code of CCLI tool that applies 14,000 code analysis rules to any single line of code.

So it's quite comprehensive and useful not only for quality, but also for security. Then we would, um, extract and evaluate results. So kind of get results from, from, from that analysis.

And then we would do this at least 10 times. Um, and the objective was really to try to get, uh, quality slash security slash effectiveness, um, of AI generated code, right? So this is kind of the experiment that we did.

We did a whole, um, script that would run these things in parallel. And then we did analysis, um, and the results were quite interesting. And so that there's been three ques, they, there's a few key insights that we're gonna be sharing with you today.

Um, I'm gonna share with you exactly the prompt as well as a piece of code that is, uh, an example of generated code. So the prompt was to try to get it's first in Python, um, and we wanted to generate five python functions that received plain text and then hash encrypt open the file set to the file. Um, and that was kind of it, right?

So we were trying to get, um, basic library, um, usage as well as io, um, and a few, um, and, and kind of basic, uh, python code generation, right? So this is one of the samples of the 10 that we generated analyzed. And so this serves as an example of what, how we're of some, some code that we generated here.

So the key, um, takeaway number one, and this is a conclusion, um, the, the first conclusion we take is that the design decisions of using AI generated code are implicit unless you make them explicit design decisions when using AI assisted, uh, dev tools are implicit unless you are explicit. And so, as an example of, um, in the code that we generated, only one of the files actually had error handling. We didn't ask for it.

Um, but one of the iterations, the, the models decided to include error handling, which is something that we should be adding. Um, even the, the, the, the error handling that was included was not really robust, right? And so it's, it, it really, there's a lot of, um, difference in all the code that you generate.

And so unless you're very explicit in the things that you want, um, you're not gonna get, um, those outputs and outcomes. Um, which is interesting as, so as soon as we enhanced prompts and enhanced those prompts and asked for better enter handling and paranoid security concerns, uh, or behavior, then codes started becoming a bit better, more protected. So it's a really important thing, which is there needs to be a lot of effort and, um, intent and thought in designing those prompts.

And so prompt engineering is extremely important because design decisions will be made for you if you don't, uh, if you're not explicit. Um, and that's a really interesting, um, there's an interesting consequence here, which is, apologies if if you invest enough time in prompt engineering, you might actually surpass the, the, the effort of actually doing the solution without any ai. Um, and so one of the reasons why we're seeing a few senior developers not incredibly excited about AI is because as soon as they see a solution being developed, and they already see all the kinks and, you know, small misses and, and bugs and, and potential security problems that these solutions are, are having, um, and so it's great if you have a really small closed, um, kind of a, a small problem really understood.

Um, it's a great, um, way to get solutions to solve those problems. But as complexity of the problems increase, it's gonna be more and more difficult for it to be, um, more effective than the good old brain powered developing software. Right?

So are we asking, you know, the, for the first question, is it reliable? You know, we have questions, uh, because can we, um, with our eyes closed completely trust that the code that we're generating is gonna be reliable and, and, um, and, and, and protecting ourselves in all these different facets of, of software development? And the answer is, unless you are investing in prop engineering, it's gonna be a no.

Um, key takeaway number three for us is that AI generated code has risk and has low quality. So what we see is, is that, um, AI generated code lacks complete code sensitization. So you get a, you know, very different solutions every time you ask for the same prompt, but it also lacks clean code and, and code quality principles.

So things like co style best practices, even basic security will be concerns, um, that you will have with the AI generat code. Um, most serious of them will be related, for example, with deprecated libraries or libraries that you shouldn't be using. Um, and this is due really to, uh, training cutoffs.

So these models are trained in a particular data or have, uh, data up, up until a certain point or date and time, and the world will change based on what happens, um, after that, right? So, so to give you an example, since the new cutoff of, of the models now supported by OpenAI, which is around April, 2020, uh, April this year, there's 19,000 new vulnerabilities that were identified. So even when the model's already updated, there's every single day and every single month new CVS that you need to be aware of, um, and that the models will not be updated.

So there's an intrinsic risk associated with using AI generated code that's gonna be very hard to, um, completely distinguish. Um, so that's an important part. And, and, and even if you have the best prompt design, even if you put all, you know, you know, an amazing effort in designing, uh, prompts to protect yourself, to have the best principles and software development, um, your code security will always be at the mercy of the training cutoff.

And that's really important. Um, and so it's, in our opinion, it's so important to allow yourself with more tools that allow you to go really fast with AI generat code. But, uh, but, um, you know, don't, don't expose yourself to, um, this lack of security.

And so if you think about, uh, the second question that we have here today is AI generating code secure, and the answer is no. Uh, uh, and so no one should be generating code with AI and putting into production without any review or static code analysis. Key takeaway for us, number five after this experiment is AI general code is not standard nor effective.

So as we've seen every single time we asked for a solution, um, to the same problem, to that, that prompt that we gave the models the code is different, has some sort of difference, um, in how it was designed. And again, this is because we're, we're, we're dealing with models that give you different solutions every time. Um, and so that's an important thing, um, that that is, uh, quite, quite, quite important to consider, frankly.

Um, because if, if you think about this scaling to many, uh, developers inside your, you know, in your team, um, code sensation is something that's really quite important because it allows you, for example, to have people mobile within your company and understanding different parts of the code base. So if all of a sudden these code bases start diverging, um, you know, exponentially, it's gonna be very difficult for you to maintain them over time. Um, correctness is also a very important thing.

There's a few papers that appeared, um, during the course of this year, if I'm mistaken, that that showed that only 65% of the code generated by, um, JGBT is correct. Um, and in our particular case, all of the 10 solutions that we generated, um, after running dynamic analysis and a few other, um, tests, only one solution really worked or was, was correct. And so it's, it's important to understand that yes, while it's extremely efficient to generate a code for the problems that we have to, to, to solve, um, we run the risk of going nowhere really, really fast because we're generating code that might not be correct.

Um, and it's definitely not consistent or standardized with how, how the company is producing software. Um, and so a really important thing for us is, is that we communicate this after this experience, is that this, this piece of beautiful technology, um, has, uh, you know, a lot of purpose and is changing the world, but as, as it touches software development, we have to advise that it should be used with well-defined low risk problems, or these problems that were broken down into well-defined and low risk problems. And definitely on mature stacks, because it would enable you at least to have more, the models will have more context to help you.

So if you have you a stack that doesn't have any context or is really fresh or new in the industry, it's not gonna provide you great answers and, and, and it's less likely to be correct. And so, answering the, the third question, is AI generating code effective? Our answer is well out of box, no.

And so I, as, as we look into these, these three conclusions, um, uh, a, uh, a fair way to look at at, at this is to say that, um, there's ways to make the, um, efficiency gains, the productivity gains of using, um, AI assisted development. Um, there's ways to make the, make these much more effective. And so prompt engineering is extremely important.

The more you invest in, in your prompts and, and, and helping the model understand, giving it context, giving it the code you're trying to change, giving it all the, the nuance that you, that you need explaining like you would to a general developer, junior developer, it really will pay dividends over time. The second thing is static analysis is really an important thing. Good old rule-based static analysis is, is is extremely valuable nowadays because it has all the updates of the industry.

It's all the rules, otherwise it doesn't hallucinate static cord analysis. It gives you all the results that you need in order for it to be safe. So in the case of, for example, deprecated libraries, this is such a low hanging fruit for static analysis and, and, and, and CVE checking and all these things.

Um, and then last but not least is reviewing code. We've, our industry built a process around protecting ourselves, um, and, and really spreading the knowledge of design decisions. So now more than ever, co review and automated co review as well becomes super important to make sure that, um, other people are understanding what's happening in our code base, and that we, we catch errors and bugs and security problems as early as possible.

So if you don't do these things, then we should not be expecting AI general coded to be reliable, secure, and effective. And you run serious risks, uh, because of, because of this. And particularly when you have a team of developers, um, and, and everyone is doing this every single day, then the risk is compounded, which is, uh, scary.

And that's that. Um, thank you so much for your time. Um, we're the makers of codine, so we, we know a lot about the code analysis and code quality side of things, and, uh, I hope you learned something today.

Happy to answer any questions. Thank you.