AI Agents Expand Enterprise Security Attack Surface

Hey guys, Thanks for the Throw. We're here with Emmanuel Salmon, who's the CEO for Nagomi Security, and we're having a little chat about AI agents, infrastructure, and security. Emmanuel, welcome to the show.

Thank you so much, Mike. Great to be with you today. I feel like if I look back in the last 10 years, and it took a long time, but we made a significant amount of progress in securing IT infrastructure environments.

There are still plenty of issues to go around, but hey, things could be worse. And yet, now I look at the development of these new stacks of IT infrastructure for running AI, and I can't help but wonder, are we just making the same mistakes over again? And have we not thought through the security issues well enough?

Well, I think that everyone is worried about security. Very worried about it. But I think that this is even a more complicated issue than we had in the past.

In the past, we had new attack surfaces popping up. If you think about 10, 15 years ago, we had mobile, and then we had IoT, and then we had OT. But with AI, it actually is much, much more complicated just because those AI agents are taking so much more than just an attack surface.

So I think the challenge here is much bigger than we were facing in previous times, where we just exploded with an attack surface. So it goes on multiple levels then. It goes from all the way from the infrastructure itself to the AI agents are essentially an application running on top of that infrastructure that, shall we say, is easily manipulated by somebody using some sort of prompt engineering attack.

So what are the layers here that I need to think through from a security perspective? Well, I think that we need to think about AI as a multifaceted attack surface that we'll need to protect. There's obviously the identities we'll need to take care of, right?

Those agents and AI agents are going to be taking roles of identities. They will be performing tasks on your behalf, on my behalf, obviously on systems' behalf, and we'll need to be able to secure that. We'll need to be able to govern the security, so to have the visibility and the governance of it.

And we'll need to be able to put guardrails and actually figure out whether those guardrails are actually working as intended. So we're going to see here almost yet another section of security that was just not covered so far. Think about enterprise IT as a big one, think about cloud as another big one, and think about yet another big one, AI security.

So if I think this through, are the adversaries now looking for these particular stacks of software? Because at the end of the day, they all represent something that feels like a target-rich environment. These are things that people are running that are almost, by definition, mission critical or definitely provide access to things that are worth compromising.

So pretty sure everyone is looking around those type of assets, like those agents that are running around. However, if you think about the percentage of the attack surface that is currently under AI versus the other segment, which is everything we've been working on in the last 20, 30 years, all the infrastructures we already have now, what we see is that attackers ultimately go after the weakest link. They're not looking for the next frontier.

They're not looking for the fancy new toy. They're actually trying to get in, get into the data, extort whatever value they're trying to get. And so I think that there's going to be more attacks that we're going to hear about that are going to be AI security, but they're not going to be the vast majority of attacks that we'll hear about in the next six months.

We will probably have one big issue, one big event that everyone will talk about that will create that awareness that we're all expecting. But I think that still the bread and butter of attackers is going after identities and phishing and ransomware and old infrastructures and all that good stuff that we know for years now. I could think that through for a minute, but it seems to me the AI agent is relatively simple to compromise, so maybe this is the new easiest path of resistance, or the least path of resistance for the attackers, because as far as I can tell, creating a malicious prompt for an AI agent to tell it to go do something is relatively trivial.

Yeah, it might sound as trivial. The question is, can the AI agent actually get you what you're looking to get? So I think that as we as defenders are learning what we need to do in order to defend against attackers of AI agents, think about it on the same way from the other side of the fence, the attackers, and what they can do when they actually do a prompt engineering or any type of attack around the agentic part of the attack surface.

And at the end of the day, AI agents are just non-deterministic. You can tell them to do something, it doesn't mean they will do exactly what you want, right? That's kind of the fun part of it.

They have reasoning, they have memory, they have tools, and you don't always get it right, even if it's your agent. And so think about it also from an attacker's perspective. They might get things wrong as well, and things might go bad, and they might get detected.

So I think there is also some science that is going to go through from the attacker's perspective before we see widespread attacks just on agents. So to your point, the cyber attacker might wind up being just as frustrated as the average end user is when it comes to AI agents because they don't always do what they're told, right? Yes.

You can think about it that, and it changes by model, and it changes by type of the level and the maturity of the model, and it changes by type of prompt, and it changes by the day almost. So I think that there is definitely going to be things that we're going to see more and more, but if you're asking for my prediction, I think we're going to hear more about a huge mistake that actually happened that created a huge leak of data, whatever that may be, rather than something that is malicious, that is something that is maybe nation state. Especially because we have so much, and there is unlimited opportunities for attackers right now to go after traditional attack surfaces, that even if they would target AI, it would just be yet another percentage that they can target versus the 99% they can just go after and they know how to do.

And there may not be enough of these AI agents out there yet to make it worthwhile to go look for, given all the other options they have. Correct, and I agree. And maybe it's not enough.

Maybe we haven't granted them enough data, and maybe we need to be able to realize who gets them and who has them, et cetera. Think about who's using agentic capabilities. If you want to go after the big money, think about large industries, they're still at the beginning of their adoption of AI, right?

Think about the big corporates that you know. The vast majority of companies that are really deep on the adoption of AI, those are really young companies. Think about startups.

Think about tech companies. Obviously, they're a target, but this is just a smaller proportion of the industry as of yet. Now, that's going to change very rapidly, right?

Six months from now, 12 months from now, we're going to see a different picture. But if you think about where we are today, I think that companies that are trying to go after big money, or sorry, attackers that want to go after big money, they still have a lot of opportunities to go after their traditional infrastructure, and maybe opportunistically around their agentic and AI infrastructure. And you seem optimistic that we will all learn from somebody who has this catastrophic event.

But in my experience with security, I can't remember the number of times that somebody woke up and said, "This is definitely a wake-up call for everybody," and then everybody just rolled over and hit the snooze button and continued on, and then the next one happened. So what makes you think we're going to be any different here in AI that we're going to learn? So I do think we are going to learn here on this one because I think this is a transformational shift.

This is not just yet another thing. This is not just moving to mobile or moving to IoT or OT. It's much more similar in the sense like moving to cloud, where everyone knew that this was a big transformation that would enable a lot of opportunities, but it would actually expose us to a lot of other types of attacks.

And so when we start seeing those attacks happening, and people understand the business opportunity of leveraging AI, they will have the usual friction between the CIO or the chief AI officer of the company pushing on doing more with AI, and then we'll have the CISO, the CISO security leader, that will try to allow them to do it, but do it under specific guardrails. And that will be the friction that we'll see. But right now, I think we're seeing a very strong push from the board and from CEOs to adopt more AI because that's just the mandate that we're getting.

So we're just going to see more and more, and the adoption is actually much higher than we saw in many other transformations in the past. One of the issues you do hear a lot about lately, if you're paying attention to this space, is that it's hard to observe the AI agents. It's hard to understand what it is they actually did, and I don't even think there's a log somewhere you can go look at.

And so if something bad does happen, determining exactly what happened might be exceedingly difficult. So do we need to kind of solve some additional issues here that go beyond the security of the AI agent, but just the overall observability of the infrastructure they're using? 100%.

I think observability on its own for AI is already today a big category, and there's also been acquisitions announced in the last six to 12 months by large vendors. So I think that everything around AI, even the visibility portion and the observability on its own, it's a big need and it's a big opportunity from a business standpoint. And that is a foundational element also for security, right?

If you don't know what they're doing, if you don't have visibility, if you don't have observability, how can you secure it, right? So we're seeing that, and that is actually ramping up, and it's getting better quite quickly, but it's still not where we are on the infrastructures that we were used to manage in the past, right? If you think where we are today with cloud security, the amount of visibility that we have and observability is significantly higher than we have for AI.

But that gap will be closed probably very quickly, not only because of security, but also because of just understanding what's going on in those adaptive infrastructures, and especially because those agents get capability of actually moving around and performing actions, and they have the reasoning of building and kind of moving from one action to another. And so observability becomes really a foundational part of the story. As you kind of think about all this stuff, will it be a single event as you describe, or is it more likely to be death by a thousand cuts and then we figure it out because the bad guys are going to go after some small stuff and fly under the radar for a while?

Yes. I do think that in the beginning, it's going to be death by 1,000 cuts. I think that there's so many angles here that we need to sort out.

There's so many types of agents. There's so many areas of AI that we need to secure. We'll need to be thinking about the identities.

We'll need to be securing their tools. We'll need to have the visibility, like the observability. We'll need to be able to understand, are they doing actually what we're expecting them to do?

Which is obviously, to your point, the prompt injecting type of examples. So there's going to be many angles here. And I have to say, personally, sometimes people talk about AI security broadly, but I do think that this is just too broad of a word, right?

Because it's just a big ocean of things that needs to be sorted out. And now we're actually starting to see sub-companies. A company that is focused only on the governance side, and another company that is focused on the identity side.

And we're going to see that segment again and again until we're actually able to figure out what does it mean to actually protect this new attack surface. So what's your best advice to security people? Because I think, on the one hand, they have this probable inclination to tell everybody, maybe we shouldn't be using these things.

But then again, nobody wants to be the person at the party telling everybody to put down the punch and the fun's over. Yes, 100%. I think that we need to always remember that security, it's a risk function.

Risk means for everything we do, when you are taking a risk, like in everything in business. And the question is, are we, first of all, as security leaders, understanding and explaining the risk to the business side? At the end of the day, when we move fast, we might break things, and maybe that's okay.

And that's something that I think today, especially with large corporates and CEOs and executive teams, we're hearing more and more the will of going in and moving faster and even breaking things if that's the case. So as long as we're able to accept the risk, as long as we're able to have the visibility and understanding of what may go wrong from a security standpoint, I think that people will continue to grow and to move further with this AI transformation. Now, as quickly the AI on its own is evolving, AI security is on its own evolving very quickly.

So it's going to be a big portion of their attention to just figure out what does it mean to actually defend and to actually protect AI. That's going to be a major section of their responsibility. And we will see one to three years down the road where that's going to be formalized.

People will know what are best practices, which I think today are still being formed. I think one of the challenges I also hear people wrestling with is that they're not quite sure, to what degree can I count on my existing tools and platforms to be extended to address this issue, versus do I need new tools, new platforms that I got to go layer on top of what I already have, and then I got to go explain to management why we need to spend more money on security? Yeah.

That's usually the case, right? Why do we need more tools if we have all the tools from the past, right? I think AI is different in the sense that on the first thing is those AI agents are actually using almost the same infrastructures and identities and tools that humans have been using.

So if you're putting the right guardrails around your email, around your endpoint, around your network, around identities, around all of those areas in security, you're already in a better shape than if you had holes in your security in the past. However, AI is putting something which is different, and we will require new type of security tools and security controls and capabilities that we don't have today in order to put the right controls in place for those AI agents and those capabilities. So, there is no other way around of having something on top.

The question is, we cannot rely on that part without having all the rest working at a baseline that should at least protect the foundations. Also, fundamentally, is the nature of the job going to change because it was always stressful, but I feel like when I look at what's happening with AI, the attacks and the response are all going to be happening at machine speed, and it's not clear to me that humans have the ability to process all of that. So how am I going to approach all this?

Yeah, I think that we're sorting this out, and I think that it's clear that we're going to have a lot more automation, both as a defender. One of the things that I'm a big believer in is we're going to be fighting AI by using AI. And we still need to have that confidence level where a human is able to actually see and validate a decision made by an agent.

" You'll still probably want to have some sort of guardrail that tells you, before an email goes out, I have the ability to review it, click Approve, and then it goes out. So we're going to see more of that, and I think that one of the things that I'm hearing more and more is that humans actually need to be able to manage agents. So think about humans being promoted to be team leaders of agents.

Each agent might have a different function. Each agent might have a different part of the organization. You need to know what type of guardrails and how to manage them and how to get them to improve all the time.

And that is where us, as humans, need to become much better in harnessing that power. Otherwise, we're going to be sucked down by just chasing ambulances all the time and mistakes that might happen in those environments. I think the challenge, though, is that the AI agents are, shall we say, aggressive, and they're likely to do something first and tell us about it second.

And humans, at least, hopefully, have been telling each other first before they go do something. So how do I manage that in a world where I can come to work in the morning and the agents will have done 50 different things that I didn't necessarily know about till later? Yes.

This is an ultimate decision that you can think about it as who's probably more susceptible of making a mistake, a human clicking a phishing link and actually approving a malicious activity going in, or an agent doing something like this? But I think it's ultimately going to be very similar to what we're seeing with autonomous cars, like Waymo in the Bay Area, where we think about Uber versus Waymo, human drivers versus autonomous drivers. And at the end of the day, we'll see there's statistics that show us that the differences will slowly catch up.

And at some stage, we'll see agents just performing on specific tasks much better than the average human. And that will actually allow us to take that leap of faith and move forward. And that's just a matter of time.

So is that a function of good engineering discipline, because we're going to make sure that the AI agents are narrowly focused on a specific task, and there may be 20 of them doing various tasks, but none of them will have godlike powers to do something dramatic, but collectively, they will be orchestrated in a way that accomplishes the mission safely, hopefully. Yes. And I think that even from an architectural standpoint, think about the ability of creating one agent that is actually performing the task and is acting permission from yet another agent that has just a QA performance capability and is just performing actions of verifying the other agent made sense.

In our mind, we're thinking about it as AI is validating whether AI is doing a good job. But those are different agents with different memories, with different sets of commands and capabilities, and this is actually how it could become something that we start trusting because there's multiple layers of testing that we do before we let someone or an agent do something and maybe break something. Let me ask you this one question, then.

Do you think it will be an incident that forces all these conversations in these organizations, or is it more likely that the auditors will be coming by one day and saying, "Folks, you need to explain how all these AI agents work and how they're secure," and that's what's ultimately going to force the conversation? So I do think that auditors are usually coming in late to the party. There's still auditors that I'm speaking with in large banks are still talking about CVs that are 20 years old, and they're still sorting out things that we may, as an industry, already sorted out five, 10 years ago, but still the same questions, because ultimately, auditors go into more traditional regulated industries.

I think that once we'll see that big incident or that big break happen, it will get regulators and auditors as well to start thinking about it and putting some guardrails around it. But it's ultimately going to be always late to the party. So I think that us, especially as security leaders, the industry vendors, and the innovators, we need to figure it out and lay down those best practices as soon as possible, knowing that we'll make mistakes, but allowing also the innovation to go ahead as quickly as possible as we're all enjoying it right now.

And the auditors, I think, will come in, but a bit later to the party. All right, folks. Well, you heard it here.

Hey, maybe the best thing you can do as a security professional right now is start deploying your own AI agents because that's how you're going to figure out what the best practices really are. Emmanuel, thanks for being on the show. Thank you, Mike.

It was great to be here with you. And back to you guys in the studio.