Tips for Securing AI Applications – Techstrong AI Podcast EP57

Hello and welcome to the Techstrong AI Podcast. I'm Amanda Razani and with me today I have David Broer. He is the technical director of NCC Group.

How are you doing today? I'm doing pretty well. How about yourself?

Doing well. Alright. So can you share a little bit about NCC group and what services do you provide?

Sure. NCC Group is a global cybersecurity consulting organization. And so what I like to describe us as is hackers for hire.

So my main line of work is I go in, I test the security and CIE of application environments and we determine whether or not there are major security vulnerabilities that could be remediated by the development team. Really exciting, fast paced line of work and I love what I do every single day. Wonderful.

Well, our topic for today is tips for securing AI applications. So I imagine you have some good advice as far as that goes. What are some of the top concerns and security issues when it comes to AI applications?

Great question, and I think I'll address the elephant in the room just from the get go that a lot of organizations or approaching AI from the perspective of prompt injection is this big boogeyman that we're afraid of, hackers are going to get our AI to say bad things, it's gonna damage our company's reputation and who knows what's after that. And so to some extent, right, I have good news being that there are things you can do to secure AI driven applications. The bad news is the impact if you do it wrong, is a whole lot worse than your organization's reputation being damaged.

So throughout our consulting, right, we interact with a variety of organizations who have used ai and I would say that the vast majority of them have not added security considerations like depth level security controls into their AI driven applications. And so as a result, we're not just seeing organizations get their reputation damaged, but we're seeing these AI systems give up sensitive information, uh, give us access to users accounts that we shouldn't have access to, and overall are really running havoc because developers don't understand the security implications of where they place a large language model within their application environment. So what advice do you have as far as these issues go?

Yeah, so the biggest piece of advice is that we need to stop thinking about ais within our applications as a traditional software component, I describe what most developers think of when it comes to remediating vulnerabilities. As a point in patch model, I find the issue in my code or a series of issues, I fix those issues and then my code is good to go. AI is much more similar to a person, I describe it as the intern in the middle scenario, having an individual in your environment making decisions, and if they can be influenced by a threat actor, then they become an agent of that threat actor.

So for instance, let's say that we have a bot that's like summarizing reviews. If I put a review out there, you have to assume that as soon as it reads that review that I wrote, it's going to do exactly what I instruct it to do. So if we're not building in access controls for our systems that adjust based off of what these systems are being exposed to, we're already 10 steps behind in the security race.

So we hear a lot about the, the threats, um, from ai, but um, we also hear how we can utilize AI to, um, to fix these problems. So, um, what can you share about using AI to fix the problems with ai? Sure.

So for starters, when it comes to governance of data within an organization, we need to remember that we've been using AI and machine learning tools for over a decade. Now, some of our most robust security solutions for detecting malicious activity and responding to them dynamically have been powered by machine learning algorithms for a long time. Now, in the modern realm of say, large language models, we're usually talking about something in the range of using AI to parse through tons and tons of text that a person might miss.

And that's really the optimal use case for a lot of these systems is for bulk, uh, evaluation where a human, they, they're going to get fatigued, it's tedious, uh, whereas an AI can just crunch through gigabytes and gigabytes of data the exact same way every single time. Now, the downside is that humans usually have a better level of assurance than AI systems. If I have a high level employee and I ask them to do a job, I have more confidence that the employee's gonna do it correctly than I would say a large language model.

So we need to balance the assurance of the situations that we put these systems in with the amount of usefulness that we get by, you know, getting access to rapid processing versus having a person go through all of that data by hand. What advice do you have for business leaders then when it comes to restructuring teams and having a balance between that human in the loop and the technology? Great question.

I think that for leaders, one of the most important things you can do is have an acceptable use policy. We hear about shadow AI all the time, and effectively employees are going to use AI to accelerate their jobs. It's more or less just a fact of business.

Now the question is, can we control and govern the way that AI is used in in, in a mechanism that doesn't put our organization at risk? So for example, if I go out and ban AI wholesale, but my employees figure out that, hey, I could do my job two or three times faster. If I have access to this AI tool, they're gonna run behind my back and use them without my organization's knowledge, and I could go and play whack-a-mole and eliminate the solutions as best I can, but I'm fighting a losing battle.

What we should be doing is giving organization sanctioned solutions that we have agreements in place for data confidentiality and that offer the solutions our employees are looking for so that whenever they think, Hey, I need an AI tool to accelerate my job, they don't go running to the first Google result they can find. They think, okay, we have a tool for this. Let me go see if the tool meets my needs.

And if it does, employees will follow the rules. That makes a lot of sense. Well, you know, technologies are advancing very rapidly.

We've seen it with ai. We're starting to hear about quantum computing. Uh, what are your thoughts as far as the risks that are gonna be involved once we start implement implementing Quantum on a more wide scale?

Yeah, quantum is an interesting question because on one hand it feels like it's a little bit like nuclear fusion. It's always 10 years away and that kind of thing. But on the other we are seeing breakthroughs in quantum computing and I think a lot of people misunderstand what it's going to be used for.

The primary risks of quantum are that our security algorithms that we're using for encryption at, at least from the perspective of a security guide, for me, that's the most important risk that we need to be thinking about. And so that's really a job for the, uh, excuse me, for the protocol developers, those who are going out and plotting what types of communication algorithms we're using to secure our network traffic. So businesses as a whole just need to make sure that they're staying on the cutting edge of developments, making sure that once quantum computing becomes a thing, we have post quantum algorithms to implement.

So as long as we're not falling behind and we're keeping our security policies and standards in line with the, the standards of the security community, we should be okay. Even as this technology evolves. All right.

Well, if there was one key takeaway you could leave our audience with today, what would that be? Sure. The key takeaway is that AI systems are not traditional code and we need to think about them differently when doing our threat modeling and when doing our development practices.

Whenever we throw an AI into the mix of our application, we need to remember it doesn't behave the same way under every class of input. It's a statistical system. And so our job is to make sure that even if a threat actor gets their data into our AI's context window, that the surrounding systems don't provide it a mechanism to give that threat actor access to data or resources that they didn't already have.

All right. Good advice. Thank you so much for sharing your insights with us today.

Absolutely. Thank you so much for having me. All right.

And thanks to our audience, stay tuned. There's more.