Riding the Generative AI Wave: Navigating the Impact on AppSec and Your Organization | AI in Action 2023

Hi everyone. Thanks for joining us, uh, for this session as part of the great conference from Textron. Okay, I'm, uh, or Bandit, uh, happy to be here with you in AI in action.

Um, I'm Ori I'm the VP of Product Management at Checkmarks, and together here with me today is my good friend, dine Dvar. You want to present yourself? Hi everybody.

Um, you know, tech and strong. What a combination, right? So, I am Dine Ani.

I work with Visa. I am the director for, uh, application security. I run the global team for application security.

Awesome. Thanks D Warren. Thank you for taking the time.

So today, um, me and Dine are gonna talk about a few things that you need to consider, um, specifically around AI and Gen ai. As you know, you are going to, um, implement and roll it out into your, um, organization. I'm going to share my feedback and perspective and insight based on dozens of discussions I had since early this year with our customers.

And the is also gonna share his insights from what they are doing, uh, in Visa. So, getting started, um, the way that we look at check marks, um, on how to approach Gene ai, uh, from a technology standpoint. While, you know, AI is very disruptive and it has many implications on almost anyone in your organization.

In the next few minutes, we're gonna talk mostly about development team and application security teams. I think the n the first thing that, you know, we kind of discussed between us is the fact that Gen AI is actually changing the developer workflows, right? If in the past it was, you know, mostly IDs, some stack overflow or some, um, internal sources, now we see that the shift is into GPT Co-pilot and other, um, solutions.

I just named a few. And it actually means that from security perspective, we need to be aware that the workflows are actually evolving and we need to be aware of that. So, so what is your take here, dine, and how are you, you know, handing that in Visa?

See, um, again, my perspective is not just with Visa, but also on my independent research and my own personal perspective. So, you know, it, it is an amazing time to be in tech. It's an amazing time to be a developer because gen AI is gonna change the way we code, the change, the way we push to production, the to market is gonna be phenomenally reduced.

Said the biggest thing is how are we going to use this Jedi? How are we going to consume the velocity of the code and deliver products faster? And train the developers not to find and fix vulnerability earlier, but also think like a hacker think like, how can I code securely?

So whether it's GPT or copilot, they are all the tools that gonna help us to not, again, as I said, to reduce the meantime to market, but most importantly, continuously ship secure product. Absolutely. And, and, and you know, it, it's, it's, it, it raises another point that, and I had many discussion while developers are getting effective, right?

Because with gen ai, the, the co-generation part of it is going to increase exponentially, you know, developers are Anything between 30 to 60% more efficient, okay? Right Now it's great because we all want our developers to be better. But as acuity staff or personnel, our tools, unfortunately, at least not yet, and we're gonna talk about it in the, in the coming minutes, is still growing a bit, you know, linearly while we were already understaffed, right?

Anything from one to a hundred, one to 150. So this gap, you know, between developers and developer efficiency compared to application security efficiency is going to grow. And we, we need to, to see together how we can close or at least minimize the gap, Right?

And I, I think it boils down to, again, two basic things. You know, I can talk about, you know, indirect object reference, a big name of, of vulnerability. But if I boil down to its basic rule-based access control, you have to, you know, train your developers on three basic things, uh, to build the basic foundation, input, validation, SQL handling, and role-based access control.

And in parallel, you gotta have those application security testing tools embedded into IDE. 'cause all of the Gen ai, I think if you remember when the death movement started, everybody was gungho about ci, we will fix the vulnerabilities before the build is done, fail the bills. But this, with gen ai, we are shifting extremely left where you are looking into an input coming in from generated ai.

And then you're, you are dependent on two important factors. A your developer, you know, you should be kind of trained to understand what is hallucination, how are we going to work on prompt engineering? You know, again, boiling down to simple things, in my mind, my simple mind says garbage in, garbage out.

So with that said, we have to be in a position to embrace ai. Uh, it's, it's, it's a new industrial revolution. It's gonna phenomenally change.

You cannot, uh, escape it every, it's AI everywhere. And you, if you look at, um, my employer Visa has been using AI for the past 30 years for risk and fraud and s other things. So now it's time to use it for other purposes, security being number one, right?

So as we were talking earlier, it's again, the combination of three things. You need to train your developers, you need to train your tools, and most importantly, you need to understand no bug can go to production and how you engage, enable and empower your developers. That's gonna be, you know, the game changer using ai, Ai, uh, I, I, you know, spot on.

I think you touched two, two points that I would like to, to, to elaborate. You know, you mentioned AI hallucination, prompt injection, right? I, I think AI and specifically ai, gen ai, and you also said it, you know, AI is, is not a new technology.

It's been around since the, the 1960, something like that. You know, we all, uh, as young kids, we were used to play, you know, FIFA or, or NBA or whatever. It had basic AI capabilities.

So the technology itself, it's not new, but what is now new is the fact that gen ai, which is a subset as rich mass market, you know, my, my mother uses GPT all the time, which to me is simply crazy. But it shows how much of, of adoption it gets in really, really fast. So, and with this type of technology, like any new technology, we need to be aware of the new types of attacks.

And you talked about developer education now, you know, we see almost every week new types of attacks. A high hallucination was the first one. Uh, we now see malicious LLM, um, injection or whatever you want to call that.

Uh, and I think it's a great example of, you know, if you are a Spider-Man fan with great power counts, great responsibility, because gen AI is awesome. I mean, it's a great piece of technology, but we need to be aware of the risks. And education is key, by the way, not only for developers, probably for the, you know, entire organization from your CFO, your financial controllers, HR, and, and everyone in your organization.

That is true. You know, in my private research, I was able to use or, you know, pick a library which was not supported for the longest time, and I was able to, uh, change it. And I was able to, you know, convince one of the AI gener code generators to give me that library.

And I was able to do that. And luckily all of the, you know, associated SCA that I use on my personal laptop, I was able to find how you can find, right? So as you said, hallucination, prompt injection, as well as, um, malicious engineering are going to be one of the key things.

And again, you have to, as I'm, I've been kinda harping on this for so many years, you know, as Steve Waler used to say, developer, developer, developer, right? So we have to focus on a developer. How are we going to train them, and most importantly, embrace this technology.

So developers are going to use it. So, and this no matter What, yeah, No matter what. So this is where you need to embrace it.

This is where you need to make sure you have the con proper controls in place to make sure the, the code that you are generating using AI is secure. You know, again, it is from code to click, the whole journey is going to be somewhere, uh, um, pampered with ai, whether it's generation, whether it's, uh, vulnerability detection, or whether it's anomaly detection or threat. Mm-hmm.

Hunting anywhere you will be, we all will be using ai. So now how we, are we going to expand our existing processes to include, uh, the additional code, uh, include the, uh, smaller reduction, the huge reduction, rather in our meantime to market? How are we going to work with, uh, our detection tools, whether it's, you know, uh, vulnerability detection or threat detection out there in production?

Because if I'm using ai, the bad actor is also using ai. And my biggest fear, if you ask me as a person, as a security person, is not zero day, is essentially what a script kitty can do. Yep.

So with chat, GPT and other generated ai, it, the, the threat actor can weaponize everything. The whole tail chain is kind of needs to catch up. Yeah.

I, I completely agree. And, and you know, we talked so much about developers. Let's think, you know, what can application security teams do to help accelerate, you know, their work, the application security part, not development part, like the, the side of, of the security teams by utilizing Gen ai?

I can give you one example that, you know, when, when Checkmarks launched, um, our early access, we introduced a concept of guided remediation. Like, you know, we know that developers interact with GPT, it gives them a lot of advice. And we said, okay, let's take that, but let's take it to the next level and provide context aware guide the mediation.

So it's kind of, you know, you can think of it as a wrapper around, um, GPT or open ai. So it would give the developers everything that they need in order to remediate right in the idea. And you talked about, you know, shifting further left.

And I think it's exactly that. So how, you know, any more ideas on how security teams can utilize Gen AI to help them become better? Because remember, we talked about the gap that is getting bigger, right?

And we're catching up. We, and we are always, as security professionals, we are always behind the eight ball. I, I think one of the use cases, which I think it's gonna be very helpful is code quality, right?

One of the things back in school or whenever, in our earlier days of our development career, the thing that we kind of emphasize upon is unit testing. I, that's something I, uh, I think it has a big use case. The code qualities should, will and has to be improved using GPT mm-Hmm.

Or any other code generation, uh, techn AI technology. And most importantly, how are we going to test those unit tests? And if you are talking, um, you know, not only the libraries, not only the developed code, somewhere, we have to find a way to, uh, have some sort of, um, even runtime vulnerability detection further left, as left as we can, right?

Uh, what about authorization and authentication, right? Mm-Hmm. How are we going to, you know, work towards APIs to find and fix the vulnerability earlier, right?

And the other part is Providence. We also need to know what code has been generated using ai because there are certain, uh, legal and copyright, um, implications of, uh, AI generated code. So we do need to find out a way to, uh, be able to have, uh, uh, indication, is it our i our own ip, or a combination of both things.

And then we also need to make sure the LLMs that are using are secure. Yep. If an enterprise uses uses GPT somewhere between their, uh, internal and external, there has to be a proxy.

You have to, you know, detect what are the developers doing, you know, in browser you can stack overflow, you can have, you can instrument and find out what they're doing, but in terms of LLMs that you are using for, uh, code generation, we do also need to see what are they doing. And I, I think you'll find interesting insights. You'll have different, different groups.

Y you know, because coding is just like language. I speak in a certain way and I'm gonna code in a certain way. So getting that metrics from that engine is gonna help, uh, A, our kill chain.

And b importantly, understand where do our developers need a little bit of TLC in training them how to A, develop better code, and B, how to use this AI tool in developing better code. I, I completely agree. And, and you know, you, you said it again, it's, it's all about education.

Okay. Now, I, I want to take it into kind of, you know, we, we, we talked about the risks and everything that we see, but you know, if, if I'm a, an enterprise and I'm probably, you know, whether it's my CTO or even ACIO initiative, uh, I had many of those discussions this year, how can they get started? And, uh, I think that, you know, um, it really reminds me of the early days of, you know, when people started to use cloud native technologies.

Um, here, it's to me, uh, mostly about assessing, right? You need to find the right AI and gen AI solution to each use case because you can't force your developers, for example, to use this GPT, and you can't force any other department to use with copilot. And just giving doses to examples.

So the first thing that, you know, if our customers, like check mark customers are asking me is try to assess, map their different use cases that you have in your organization. And, and by the way, code generation is one of them. GPT is more of a, you know, general purpose, gen ai, but there is also music generation.

It's, uh, visuals that, you know, you have Dali, you have me journey. Those have those, all those risks as well. And then once you have those, and my recommendation and interesting, uh, to hear your thought about it is there is no one size fits all.

Okay? Right? Whatever works for your development team doesn't necessarily work for the rest of your organization.

So, so what is your take here? I think, uh, we shouldn't be scared of this technology. It is a ti this is the perfect time, you know, for the fans of Spielberg movies.

Uh, the, the reality is it is an amazing technology. And as you mentioned, assessing, you know, there is no one size fit, even in our, in, in any enterprise, all teams are going to be embracing this differently. Uh, you gotta assess what's the impact.

Like, for example, there was some company where some developers used their, uh, um, uh, own code and put it out to GPT, it was exposed and all that. So let's learn from that, right? Let's learn from that.

And also don't be afraid of it. See, what can you, you know, get out of this, right? And for example, I want to, to use this GPT for threat detection.

I want to use GPT for penetration testing. I wanna use it for runtime on detection and or code generation. We have to experiment, we have to learn.

And most importantly, like any other technology, you have to get businesses approval. Businesses buy-in, and your executive buy-in. And again, when we, if you remembered, you know, the early days of shift left, not only we were work focusing on developers, we were also focusing on our product teams.

How do you emphasize why security is one of our important setting proposition? So we have to join forces with business and un help them understand that this is a technology that's gonna stay and improve their goals, where they can hit production faster. Meantime to market is faster.

And most importantly, keeping our products secure, reuse using this technology. I, I completely agree. And, and, you know, time to market, this is what drives, you know, what it, what drove cloud native technologies in the beginning and what now helps enterprises go with Gene ai.

I, I think one of the things that many people are currently having a challenge to do is defining the policies. Okay? Because, you know, at the beginning we heard, you know, like there is the famous, um, article that said that Italy, the country, right?

Completely banned, you know, GPT, which to me, as, as we just discussed, I think blocking technology is never the solution. Okay? Right?

Like people would find way you, whether it's with VPN or other, and, and I heard also Enterprise is doing that. And I think blocking is never the solution. But you need to do, I mean, what we all need to do as an enterprise, as as an organization is as you said, assess, right?

But then define the right policies, the right tools. We already see, um, you know, like the early adapters or the early access for some protection, like check marks introduced, check ai, it's completely free. So, you know, um, everyone can use it.

And we did it simply because it was too important to, to put behind the gate, but also education. So it's policies, tools, whatever is out there right now. And also education.

So how would you go on defining those policies and tools and everything? You gotta look at your specific use cases, right? So let's go one by one.

Let's take for example, our code generation, right? And I'm gonna say, as you mentioned, 30% increase in your product efficiency, right? There'll be 30% more features coming in.

Now you have two problems to solve. Now, one is how are you going to ensure that additional 30% is served properly, where you have the scan engine capacity to scan, you have the process capacity to review and find and fix those vulnerabilities. And you have the, the database capacity to consume those, um, scans and, you know, uh, provide them when required by audits or something of that sort.

And the other part is how are you going to improve the efficiencies of your developers in the IDE? If you look at it currently, if you're using an IDE, uh, compatible testing tool, how many pers what percent of people are using anything between five to 25? Now, that's not gonna change your, the give the, the, the needle's not gonna move for your efficiencies in code quality.

Now you had to focus on your developers using those IDs. When the code generation comes in using any generated code to ensure it is free of vulnerabilities, there's no hallucination, there's no nothing that should be, uh, a part, a part of the problem, rather not part of the solution. Mm-Hmm.

And on parallel, you gotta make sure your lms, the, the, the generations from the generative tools of the code are secure, right? Then you have to take care of your LLM security, and then you have to take care of your network security. So it's not gonna work in, uh, in itself.

You gotta have different layers of protection. Now, if you shift on the, uh, customer service side, you know, AI tool support and all that, you can, uh, you can mess up a lot of things if you don't, uh, put them properly on because it's customer facing. So you have to think beyond not just a simple process, but the whole different layers that are attached to it, and make sure your partners are included.

If we, going back to the development, um, features, you already working with a lot of technical debt with gen ai, you're gonna add to it. You have to work in partnership with your product teams to make sure we have certain bandwidth to take care of that technical debt. And again, I, I think I cannot emphasize enough, security is a, uh, you know, a community sport is a team sport.

And it takes a village Yeah, village. It takes a village. And most importantly, uh, secure products are, are, are everybody's unique proposition, selling proposition.

If you're e-commerce, if you're any entity on web, be even, even a hospital, we have to mm-hmm. Secure. So we have to work with our product development teams and executive sponsors to make sure we are enhancing the processes, we are understanding the complexities of those processes and injecting the understanding.

Don't run away from this technology. It's here to stay. You can use it to stay ahead of the game.

Absolutely. And, and, you know, you talked about the layers and, and one thing that I recommend every customer that I talk to, there is kind of the feeling or the hope, you know, because it's machine generated, it's more secure. But as you said in the beginning of our conversation, it's, you know, garbage in, garbage out.

And, uh, you know, if, if everyone that, that in the audience that listens to us right now, uh, my at least one ask is whichever DevSecOps processes that you have to keep them, okay? Exactly. Don't drop anything.

And if you don't have those in place, which it can happen, gen AI is a great opportunity to also include that. So whatever you have running in your pipelines, in your pool request, keep those. Don't think that because it's machine generated, it's more secure.

And if you want to prove your point to, to your, um, executives, take a look at the Stanford research that shows they, they prove that secure machine generated code is by definition less secure. So this is like, you know, one takeaway message from me, uh, just before we wrap up. So yeah, for any final Points, I think you hit the nail on its head because don't fix it if it ain't broken, right?

So all your processes that you have worked very hard to build, you have to not only secure them, but also enhance them. And I, I didn't wanna say this, but since you said this thing secure, uh, the, the code that is generated by our generative tools are by definition not that secure, right? So how you enhance your existing processes to a identify what's being generated by the machine, and how do we include everybody in the process to make sure, uh, it's found and fixed earlier, and most importantly, we learn from it.

I cannot be having a developer, a development team using, doing the same mistake over and over. Finding and fixing a vulnerability is not a problem. Changing the behavior is the challenge that we all have to think about.

Yeah. So I definitely agree with you on that. Absolutely.

And you know, I, I like to say machines are great, but don't forget to keep the human in the loop, so Oh, Absolutely. They're there to be used. They're there to be kind of enhancing our work, not replacing us Abs maybe one day, but not, not at times.

So, okay. So everyone, thank you very much, uh, dine. Thank you.

It was a great conversation. You can hit us on LinkedIn, on Twitter, wherever, uh, you can find us. And looking forward to the next time.

So Dine, thank you very much. All right, thank you.