Arcova Warns AI Agents Are Outgrowing Security

Hey guys, thanks for the intro. We're here with Joe Perry, who's the advanced services lead for Arcova, and we're having a little chat about OpenClaw. If you've been hiding under a rock, you may have noticed that these things are proliferating all over the place, but there's other similar types of AI agents that are out there, and well, it's not quite clear how secure they are, but they're certainly being more targeted.

If that's going to be an issue for everybody, we're about to find out. Joe, welcome to the show. Hey, it's great to be here, Mike.

Thanks for having me. So what exactly is the issue here? Are folks deploying these things on their own?

And OpenClaw itself seems to me like there was very little concerns about security controls. It was basically a hobby project. But are these things everywhere now, and are the bad guys looking for them?

What's the state of the current cybersecurity landscape? Yeah. It's a fantastic question, and I think what you said in that question hits on the core of the problem, which is it's a hobby project that has gone so far beyond what anyone could have expected a hobby project to be.

And so now we're seeing this massive explosion of use. We're very familiar with the problem of shadow IT in cybersecurity, and now we're seeing this problem of shadow IT in the form of an AI assistant that has access to more or less everything that a given user has access to, often including a great deal of their personal data, their financial data, all of this. And so, as I'm fond of saying internally to clients, there's a fun, weird fact in nature, which is that the main way lobsters die in nature is they get too big and they can no longer successfully molt.

And that's exactly what's happening with OpenClaw, is that it's wildly outgrown its ability to sustain itself and has wildly outgrown its ability to keep its users safe. That's not even necessarily a problem of OpenClaw. It's not some fault in the designers.

It's just a reality of it growing so phenomenally quickly without having that security baked in from the very beginning. And so now we're looking at an industry-wide problem. What are security folks supposed to do about this?

Because I think we've moved beyond the office of no, and a lot of folks are now being tasked with, well, how do I enable somebody to do something similar in a way that is safe? So should I just be out looking for AI agents that are more secure to replace OpenClaw, or what's my strategy here? Yeah.

So step one is definitely specialize and localize what you're working on. So you're right to say the office of no doesn't work. Just telling your users you can't have any AI very rarely succeeds in any organization.

On the other hand, what I have found works extremely well is basically doing exactly the opposite and saying, "These are the specific ways in which you can use AI that will be successful, that will be safe, that will be permitted within the business. So yes, you can use this version that we have, whether it's Claude, OpenAI, so whatever the case may be, that we have a corporate license for, that we have all of the guarantees that we need for our data to be safe. You can use it in these ways.

" So it's less about just saying no as a blanket and more about saying, "These are the places where it can actually be useful to you. And so we're going to build those paths in and tell you yes out of the gate for those," with that blanket no being for everything else. Especially when we're talking about tools like OpenClaw, which again, have no corporate license available for them.

Some folks just seem to be under the impression that I can securely deploy OpenClaw on a bunch of isolated Mac Minis, and I can control what it will have access to. But as far as I can tell, some of these AI agents are, shall we say, pretty clever about finding ways to access things. So is that a viable approach, or is that just postponing the inevitable?

Yeah. It's a great question. This is where we get into the old questions about zero trust, where if I implement zero trust, am I safe?

Well, yes, if you do this perfectly without any flaw, you are theoretically capable of creating a risk-free environment. Have I ever seen anyone get even vaguely close to that? No.

No one's anywhere near being able to succeed at that. " But if it has a path to touching those, it's eventually going to. As you pointed out, cleverness is one of the things where it decides, "I need to do this.

" Accident is also a huge source of the problem. " And it's usually about four or five prompts later that Claude tries to take that specific action. And that's why keeping it in, for example, planning mode versus edit mode is a really important process for people using it.

And so that's something that is not unique to that tool. It's true of most AI tools on the market, that you can't actually give them that kind of hard and fast rule and expect it to be perfectly implemented. More or less, that's what stochastic means.

We've also seen maybe things like Perplexity Computer arise, where that feels more like a SaaS application environment. Is that any more secure? Is that one way of thinking about this, or is that equally insecure, just differently?

That is, again, another great question. It is definitely a step in the right direction, is what I'll say for it. I am still very nervous about from a fundamental philosophical level, putting all of your data and all of your security in any bucket, no matter how fancy that bucket may be, just is, to me, a bad decision.

That being said, Perplexity, there we go, is, as you're saying, they take a much more SaaS approach. They're actually securing interaction points. They're using more what we might call mature infrastructure.

That's a long way from me saying I endorse that process, or I endorse that method of doing business. But it is, I would say, not equally insecure to an open source project on the internet with security as an afterthought. Will there eventually be, I don't know, AI agents for security folks that can be used to secure the AI agents that the average folks are deploying?

And is this going to be some sort of ecosystem of AI agents that make up checks and balances? I am sure someone will try that. I'm sure many people will try that.

I have absolutely no confidence that will succeed, and for kind of a very basic reason. So if we think of ... temperature, right?

The idea of randomness for an AI model. You can scale it from zero to one. One means you're always going to be very predictable.

You're going to do the same thing every single time. You will be predictably able to achieve the outcome I'm looking for. Now, that is not where most people keep their AI because that's not really useful for the kind of chatbot interface that we're used to having it.

People get a lower temperature, and they have it much... I think I'm reversing this. But people have a temperature that is much more random or at least a little bit more random and allows for more, let's say, unusual responses or more unpredictable responses.

And that's what creates that spark of creativity that makes AI tools so useful for brainstorming and so useful for accomplishing creative and difficult tasks. But when we're talking about security, security is a rigid task. It has a rigid problem set is, will you secure this data from attack, yes or no?

99, adding another AI tool on top of that is just multiplying values less than one, which is to say, making it less secure. So if we have two AI that are watching each other and they are both 99% secure, well, the result of that is actually less secure than either of those devices. This is something that's been demonstrated repeatedly in AI security testing in labs.

AI securing other AI makes both systems less effective consistently. Now, it's a pretty pitch. It's a nice idea of, well, let's just take those problems and abstract them out to another tool.

But the material sort of mathematical reality of it is that the introduction of randomness makes AI useful in every other domain and makes it not useless in security, but means it will always rely on human decision-making in the event of security. As you kind of look at where we're headed with all this, are there soon to be, I don't know, hundreds, thousands of these AI agents running around? And are we just waiting for some sort of catastrophic event?

Unfortunately, there are many more than thousands of these running around. I think the OpenClau is the most successful project on GitHub in terms of downloads and stars. It is wildly, phenomenally popular.

People are using it all over the place. And we are, in fact, seeing pretty substantial impacts of that. Now, this is where the good news is the AI industry is just never quite as big as it claims to be, and it's never quite as impactful as we hope it's going to be in the future.

So no, we're not looking for, like, a world-ending cataclysm. This isn't a case where OpenClau is going to take down the financial system or cause all the banks to collapse or anything like that. But it is absolutely a case where individual users are suffering identity theft.

They're suffering major financial loss. They're suffering loss of their data. They're experiencing very difficult-to-recover-from errors as a result of overly trusting an AI tool that they do not much understand.

" Never interrupt your enemy when he's making a mistake, right? Right. Yeah, absolutely.

I think there are two ways that attackers tend to be using AI right now. " Where they succeed at that, it tends to make the front page of the news, where people tend to get very excited about it. Where they fail at that, they just kind of abandon it and go back to their normal practices that have already been working for them very well.

But I think when we're talking about stuff like OpenClau, I'd be shocked to find out there aren't at least a few attackers trying to get their own version of that seeded, in the same way that we've seen for other tools that have become popular over the years, where getting, whether it's the exact same tool and just learning how to exploit it, or whether it's releasing versions of that tool that have malicious code baked into them. That's one of the things that we've seen in a lot of testing with clients or working with different clients of AI assistants that are theoretical competitors to a Claude Code or theoretical competitors to an OpenClau that are, in fact, exfiltrating users' data. They don't tend to last very long on the App Store, but they crop up with alarming regularity.

As you kind of look at all this stuff, will there be more audits of some of this stuff in the future? Because I know we don't have a whole lot of AI-specific regulations on the books, but we sure have a lot of other regulations in the books, and they seem to apply to how we access and manage data. " There are few things I would like more in this world than the AI industry to receive proper auditing.

I think that where we're at right now is the AI industry has learned a tremendous amount from the crypto industry and a tremendous amount from the crypto banking industry in particular about how to develop very friendly relations with regulators and very friendly relations with governments, such that it's only the biggest players, the most dramatically out there, the FTXs of the world, who actually get taken down. And the folks who are just kind of doing the garden variety, chugging along, and moving fast and breaking things, they tend to have a much more friendly relationship with regulators than, for example, financial markets did 20 years ago. And so I think that's one of the places where I would love to see it.

I frankly have not seen a lot of signs of anybody getting very serious about auditing AI usage from a government perspective. There's certainly a lot of movements about sort of getting in that direction, but there's nobody who's taking a truly hard line about it right now. We talk a lot about guardrails in the age of AI, and yet what I've seen so far of the guardrails is they don't really work.

So is the question going to be, do we need guardrails, or do we just need better controls on the data that's being accessed by the AI agents, and that's where the focal point should be, even though we haven't really been doing a good job of that ourselves for the last two decades? So will we get serious about it? Yeah.

Man, if I could answer that question, wouldn't I just be the richest guy in cybersecurity? No, I think you're right on the money there in that the problem is an old problem, right? It comes down to securing data and only allowing it to be used for permissible uses.

And theoretically, kind of going back to my joke about zero trust earlier or DLP, UEBA, we've had many solutions over the years that are intended to take bites out of that problem, and I think AI has the potential to be really useful in the course of solving that problem. But it's also right now proving catastrophic towards trying to kind of get an answer. And I think most folks might be an overgeneralization, but certainly a lot of folks in our industry ...

are of the belief that AI is fundamentally new, and the problem that it poses is fundamentally different from any other kind of problem we've ever solved before. That's just not true. It's not even sort of true.

In reality, what you said is right on the money. We are looking at a problem of how do we secure data while still making it accessible for the use that it's appropriate for? And so whether that's a human accessing it, whether that's an AI agent accessing it, whether that's an automated Python script accessing it, whatever the case may be, it's about that evaluation of need and that evaluation of appropriateness.

It's fair to say that in the age of AI, it might need to happen faster, might need to happen across more individual user profiles. But those are mostly a factor of the scale rather than a factor of the technology. As you look at all of this for a minute, do you think that AI agents, are they extensions of humans, or are they some type of non-human identity?

Because we've had trouble trying to figure out how to manage human and non-human identities, but I don't think anybody's quite clear what an AI agent is, because it inherits permissions from a human, so some people think of it as an extension of the human, and others are saying it's a type of non-human identity, or is it some sort of hybrid that sits in between? Yeah. This is one of those questions that I think has maybe been slightly overly philosophized, because I think the answer is, how are you currently handling identities in your organization?

If you have a really robust management of roles and role delegation, this is probably an easy problem to solve in terms of timing access packages, in terms of specific units of access, workspaces, resources, whatever the case may be. If, on the other hand, every user in your organization currently has a local admin, probably AI is going to hit you like a horse. And so it's really going to depend when we're talking about is it an extension of a human, or is it a separate non-human identity?

" So regardless of what the agent could do theoretically, in this time of use, it is only able to act within this package. Which is really where we're seeing most privileged access going in modern ecosystems toward that just-in-time package, and that checking things out, performing the action, and then the checkout is over. I think a very similar sort of process with AI agents, just an extra layer of scrutiny in the process of checking out.

Will we need to figure out the relationship between these AI agents? Because it seems like a lot of times they are, shall we say, programmed to be aggressive, and they will find ways to access things because they've been designed to find the data or accomplish the task by any means necessary. And they're clever, it seems.

They go out of their way, and if they can't find one way, they go find something else that has permissions, and that may be some other AI agent that they wind up talking to. So will we need to map the relationships between these various AI agents as they develop dependencies on each other and try to find ways around our controls? Short answer is, I think that would be impossible to do.

I think what has to be done is much more of a sandboxing approach. Trying to map end-by-end relationships scales in complexity incredibly rapidly and reaches a point of really no return almost immediately. Whereas instead, if we say that all AI within a given organization can only exist within these parameters, can only be used from these platforms on these tools, which themselves only have access to data under these specific contraction permissions, then we remove the question of, well, AI might try all these different things, but it's trying all of those within that closed circle, that closed ecosystem.

One of the things that you mentioned there, AI seems very clever in its ways of bypassing controls. " And so that's what we see a lot with AI tools, where they have this long list of things that will theoretically accomplish the goal, and they just work their way down that list until they find something that works. So as long as we have limited their capacity for using tools, that's things like restricted shell, that's things like sandbox environments, that's things like rapidly torn down environments when they start to spike in usage or when token costs start to spike, which tends to be evidence of malicious or atypical activity.

" Like I said, trying to solve that each AI's relationship to each other AI in that end-by-end matrix isn't really something... All other points aside, just the storage of data for the information would become problematic over time. Ultimately then, what's your best advice to security folks about all this stuff?

Because to your earlier point, nobody wants to be at the AI party telling everybody not to drink so much punch, but they also know that there's likely to be a bad outcome. So how do they have this conversation? Yeah.

The short answer is you need to be a skeptical user, and both of those words are really important. You need to be skeptical about AI, and you need to make use of the tool so that you understand the things that it realistically can and can't do. There are tons of ways that AI can be incredibly helpful to an organization, especially a security organization.

I use AI pretty much every day. Most of my colleagues use AI pretty much every day, from things as small as, "Hey, I'm looking for an article about X. " To, "Hey, this KQL isn't working.

" There are a million simple uses to it. Where the problem comes from is mystical thinking. " If you are trying to, especially for a security person, having a very deep understanding of where AI touches your organization, where that risk is created, where that risk surface exists, is going to do you a lot of good in talking about, "Hey, these are AI tools that it totally makes sense for us to bring into.

If the marketing team wants to have an AI tool listen to their meetings and summarizing it, that's actually okay for us to do. They're not saying any private data that's going to get us in trouble. " And that's not necessarily universally true, but that's kind of a general example of it's not about saying yes or no, it's about understanding the actual risk surface and making very informed decisions from there.

That's true for AI. It's true for pretty much any technology. If you're the CISO that tells everyone no all the time, you're not the CISO for long, and if you're the CISO who tells everyone yes all the time, the company tends to show up in the news for a security breach before very long.

So it's finding that balance and always trying to be as informed as possible when you do it. All right, folks. You heard it here.

Hey, it's kind of like driving. Nobody wants to take directions from somebody who doesn't know how to drive, so AI agents are the same way. If you don't know how to use them, no one's going to listen to you anyway.

Joe, thanks for being on the show. Thanks so much for having me, Mike. It's been great.

All right. And back to you guys in the studio.