Challenges of Securing LLMs with Perception Point’s Tal Zamir

This is Textron tv. Hey guys, thanks for the throw. We're here with Tal Samir, who's CTO for Perception Point, and we're talking about how to secure all these large language models that companies theoretically are gonna build.

And that's gonna be a bigger challenge than anybody realizes. Tal, welcome to the show. Thanks.

Thanks for having me, Michael. And, uh, great to be here. I think it's fair to say that there's a certain amount of irrational exuberance in the world when it comes to all things generative ai.

And I think the issue is though, in my mind's, there's gonna be a lot of data pouring into these large language models that eventually organizations are gonna wanna either build on their own or customize to some degree. How does that all get secured? Because the LLM is only as good as the data that's in it, right?

A hundred percent. And I think, uh, with the recent announcements that you see coming out from Microsoft with its co-pilot and others like bd and of course open ai, uh, there's gonna be a lot of challenge securing that. Um, just, you know, those chatbots or whatever you might build on top of these models, they're gonna have access to your enterprise data and in unexpected ways might mix different types of data from different clients or different stakeholders within the organization.

And it's a true challenge to grasp how to secure this new, uh, brave world of LLMs working on enterprise data. And at the ends of, uh, every employee in the company. Do the folks building the LLMs have an appreciation for that?

Because it seems to me we're already challenged trying to teach developers security and now we want data scientists to figure it out. And it seems to me they know even less about it than developers. Yeah, definitely.

And I think, you know, um, they're trying, definitely putting safety, um, as part of the requirement for whatever they build. I think more, uh, from Microsoft and with its recent, um, you know, announcements, I think they are more minded to that, but of course it's new to everybody, so it's gonna be, uh, with gaps initially at least. So what exactly are the bad guys gonna try to do?

What do we need to think about here in terms of their attack power? Definitely. So we can take, for example, you know, the recent announcements from OpenAI as a, as a case study, you know, they're just in the last couple of weeks, uh, announced, uh, GPTs gpt are basically custom bots, which are similar to chat g pt, uh, but you can as a, you know, anyone actually, uh, customize what they do, what knowledge they have, access to, what actions can they do in the world, which is great.

It really makes it easy to build your own custom chat bot. But if you're putting the kind of black hat on, uh, and you think, what can the bad guys do with that technology? Um, so imagine social engineering on steroids.

'cause I could send you as an employee in a company, uh, a link to a chat bot, uh, that will, uh, impersonate a customer service representative or a help desk representative. And because those chat bots and Nels are so effective, uh, with language, they can really convince you that they are human operators of that customer service or help desk, um, and convince you to provide your credit card number, your password, your credentials, or other sensitive information. So I'm sure we'll see this type of, uh, very advanced phishing social engineering with those types of new bots.

And you can even take it to, to another example of, you know, I might send you a, a link, uh, to, uh, an instant messaging app looking like WhatsApp or something like that. But, uh, convincing you that I'm your boss, uh, talking to you right now. And because those chat botts are so easy to build, you might be fooled that, uh, you need to transfer funds over, um, kind of the business imm compromise attack.

But again, who overcharged with those LLMs and convincing people that're talking to a real person Also seems to me we might see some out and out sabotage. If I've got a process that's dependent upon the LLM, somebody will try to figure out a way to poison the data pool that I'm using. So how am I gonna approach that?

'cause frankly, we're not all that good at data management. Definitely the scenario that're describing is definitely a reality. And, um, customer companies would really be careful about what access they provide to those, um, LLMs, GPTs, what have you.

Um, once they have actions, uh, that they can do in the world, and that's, by the way, another thing they introduce through those g GPTs, they can actually send email in, in your name. They can get access to your data in your name and so on. Um, they can practically go viral by sending emails across the organization, or again, share data inadvertently to the, uh, wrong people.

So, you know, I would approach this very carefully. If I'm an enterprise, then I would really pick, uh, narrow use cases to experiment with and limit what those, uh, LMS can do in the world. Uh, let's have them first provide, uh, help for departments which are not handling sensitive data and are more in the, let's say, marketing world or, uh, writing content, which is not mixed with sensitive data.

Um, I hope it makes sense. I see a lot of organizations are at least experimenting with the idea that I'm gonna put my sensitive data in something that feels like a vector database and show that to the LLM so that, um, my data doesn't wind up in the LLM, but I can still take advantage of its capabilities. But invariably there's gonna be a network involved in that conversation.

So do I need to secure that as well? Because somebody will start going after that vector database will be essentially a very sweet target. Yes, definitely those types of, uh, vector databases that are, uh, now being built and, uh, when the organizations are converting their knowledge and data into those, uh, vector domains, uh, to be able to find similarity and so on, that's something that is, uh, super valuable.

It allows you to, uh, get access to data, uh, which is similar in meaning, uh, across your, uh, database, your knowledge base in a sense. Uh, so I believe it'll be, um, something of focus again with third parties. I don't think that companies should build their own security around those vector databases.

There will be companies that are already seeing now, uh, emerging from self around protecting those databases, um, also doing prompt security. So when people provide prompts to those lms, how do we secure, uh, what's coming in and coming out of those, um, models. So what is your best advice to folks?

There's a lot of nuances. How do I approach this in a way that, you know, 'cause ultimately when something goes wrong, it's gonna be the security guy's fault. Definitely.

So I think the main, I, if I'm looking it from the, the, uh, perspective of perception point, uh, where I'm, uh, the CDO and basically we focus on, you know, work-based security, which is everything the user and employee in the company does be it, uh, email primarily, um, your, the websites that you visit, the cloud apps that you're using. And from our perspective, the way this will get abused, and we're already seeing the start of this right now, is by raising the bar on what's out there in terms of email attack, right? So attackers have access to this amazing capability to create super sophisticated, um, impersonation on email, which is based on text, based on language.

And this is like their playground. And now with those tools, they can really perfect their attacks a much faster at scale and target a vast amount of organizations for any size with these new tools. So I think the first thing we need to be aware of as a security industry is that, uh, attackers are really raising the bar, and we need to put defense that is able to also use those LLMs to understand the meaning behind an attack and to understand it's a malicious intent, uh, and move away from the traditional way of detecting, uh, malicious email, which is based on, you know, patterns and keywords and so on.

We're now moving to a whole new realm of meaning, uh, that we need to be able to understand as, uh, security vendors and companies that are looking to protect themselves against those types of, uh, views of, um, machine learning. Large language models, um, should look first to protect their email in a more advanced way with solutions that can get into this level of deep understanding of language. Uh, and to think as a user, as a human when looking at emails, those, um, protections need to be in that state of mind.

Uh, at perception point, that's exactly what we've built. We are actually, um, looking at emails to a website that you visit and so on, just like a user would look at them, them with the understanding of the meaning, uh, of, uh, those texts or how they look. Uh, when you look at a website, is it looking like a phishing website?

Is it looking like a phishing email and so on? How sophisticated are the bad guys about all this right now? Are they already using these tools or are they still studying this as well?

Yeah, so the, the interesting thing is that we cannot be sure if they're really using it or they're just writing really great emails, uh, or phishing websites that look exactly like something that Genive AI could write. You know, you can't be sure a hundred percent, it's all kinds of detectors that try to detect whether something was created with gen ai, but it could all be, uh, you know, uh, fully, uh, legitimate and someone really having great English, uh, when they're building their, uh, email, uh, and attack campaigns. But we do see interesting things like we recently introduced our ability to see similarity between different emails based on meaning, based on the semantic meaning of a, of a, of an email.

And we suddenly found, uh, the same scam completely written in a different way, uh, in other organizations or in other people in the organization. Um, we suddenly surfaced those new types of, uh, emails that we haven't seen before as malicious, but because of this, uh, understanding of language, we see FMBA whole family of emails that are same in essence in the true meaning of that. Uh, and we believe this is generated with, uh, generative ai where the attackers built some kind of automation to create variations of the chain attack because they know that the traditional security controls will look for specific keywords or will not go to the semantic layer.

But once we've built this capability, we suddenly saw those thousands of new variations emerging, whether they're really variations of people actually typed manually or generated with gen ai, we can't be sure. So have we somehow or other stumbled into a cybersecurity AI arms race thing? Completely.

Uh, it is a hundred percent like that, you know, and the worrying part is that, uh, those models are out there, the genie is out to the bottle. There's no way to put it back. It's open source, uh, to like Facebook meta.

Um, and others are releasing their super advanced model that took, uh, hundreds of millions of dollars to frame. They're just putting them out there for anyone to download. You can just download the file, which is gigabytes size, but you download the file and that file is, uh, in a sense, uh, the human knowledge in a single file.

And you can use that to build whatever you want and have it generate content for you. And this can be done offline without any permission from anyone, and also be abused for any purpose, um, that the attackers, cyber criminals might, uh, only use it for. And then the implications are, uh, worrying.

But the good news is that, uh, you know, the defenses and the, the security vendors can use the same technology as well. Ultimately. Do you think that this might benefit the defenders more than the attackers?

Because right now the playing field is uneven, so maybe I'll get some AI cybersecurity tools and I'll be better off. Yeah. The, the nice, uh, part of this is that if I'm at the company, uh, deploying solutions that use machine learning, I have some kind of unfair advantage, uh, versus my adversary, uh, because I know, uh, things that attacker could never know and I can feed that information to my, uh, machine learning, uh, models.

For example, as a company, I know the relationship between people. I know who sends who, which email and who's the boss of who, and, um, you know, all kinds of internal data that is not exposed to the attackers. And then if I'm fitting this into my machine learning models, they can be, uh, more effective in detecting the anomalies and understand that something goes wrong.

'cause they really understand how the organization works. If I'm sharing this data only with my security vendors and that hack don't have that, I have this unfair advantage. So hopefully, uh, we can be, uh, with the upper hand here.

Do you think that governments are wrapping their heads around this and they're gonna force the issue because they're more concerned about AI than anything else and they'll just wrap security around that as a framework? Yeah, it's a, it's an ongoing dilemma of regulation and, uh, how far will they take regulation around, uh, AI and gen ai? It's not a so thing, there's something in the EU coming up around, uh, regulation for, uh, generative ai.

But, uh, you know, there's a lot of money at stake and, uh, a lot of, uh, you know, interest for the folks building those, uh, AI technologies. Uh, not to restrict it too much, but I think everybody understands that, uh, safety is critical, uh, for this, uh, to be successful. So how do the security people insert themselves into this conversation?

'cause a lot of times, um, shall we say that they are the person that is voted to be most avoided in most organizations, and there's a lot of, um, folks that are rushing to go build things and, you know, security's kinda, uh, sometimes viewed as an obstacle. So how do I get in the middle of that conversation in a constructive way? Yeah, it's a good question.

So I think the first thing, uh, in most organizations, you, you can't really just block gen two AI websites and chat GPT and the likes, uh, and employees, no, you cannot get access to that. This just won't work. You know, people will find a way, there's already a term for this, which is called shadow ai.

You know, if an employee wants to use ai, they'll find a way to use that. There's so many services out there. Uh, but you can definitely, uh, first educate and sometimes use tools to educate.

Um, for example, you know, with our own solutions, we have a, a browser security solution as part of the perception point portfolio. And you can, um, have this solution warn users before they, uh, enter, uh, generative ai, uh, website so that they can be more careful and then, um, you know, warn them before they provide PII personal information, credit card numbers, passwords into those genive AI websites. Uh, so this way you can enable this in a safe way without, uh, you know, stopping people from doing that.

Uh, and another thing you could do, um, you know, there's so many different genive ai, uh, services, but you can pick a few that are more enterprise oriented and have them as the sanctioned legitimate, uh, go-to solutions for your employees. Uh, Microsoft, again, is leading the way here with its own co-pilots, uh, set of, um, solutions. Uh, and definitely if you go there, you're getting more, uh, enterprise controls around those solutions.

So you should enable Genive ai, but put the controls in place, put the education in place so that people are aware of what they're doing in that, uh, those applications. All right, folks, what you heard in here, AI is a journey and we're all going on it together. The issue, of course, is that somebody needs to protect the proverbial wagon train, so we don't all wind up getting an outcome where things could just go horribly wrong.

Hey, Talal, thanks. Being on the chair. Of course.

Thanks so much. All right, back to you guys in the studio.