Mitiga Sees J-Space as New AI Agent Telemetry

Hey guys, thanks for the throw. We're here with Ariel Parnes, who's the COO of Mitiga, and we're having a little chat about, well, an interesting capability that Anthropic has described. It's called Jspaces, and it kind of gives us the ability to see what's happening between the reasoning of these AI models, and well, apparently, this also has some security implications.

Ariel, welcome to the show. Hey, Mike. Great to be here.

Thank you. Good to see you. Walk us through what's going on here.

What do you know about this, and how is it that this might have some relevance for the security community? Yeah, sure. So, a few weeks ago, Anthropic published a research where they update the industry about a structure that they found inside Cloud.

I would define that as a workspace where, at the very high level, the model's thinking happens. Now, what's interesting in it is that it wasn't designed by anybody. It just emerged through the training of the models.

And what they've been able to do is to develop a methodology, a tool, they call it the Jlens. So Jspace is the space where they believe the thinking happens, or they show that the thinking happens. Jlens is the tool they use to look into that space and understand what happens there.

And what they realize is that they can see or extract words that represent the thinking process of the models through their analysis. So for example, let's look at just any line of thought that they could have seen through a chat communication. " So obviously, the answer to that is Buenos Aires.

That would be what you see as an output of the chat. If they look into the Jspace through the Jlens, what they would see is Messi, Rosario, which is the city where Messi was born in Argentina. It never appeared in the output.

It was seen through the Jlens in the Jspace, and this is why they came to the conclusion that that's the space where the thinking happens in a very simplified way. Hmm. Some other examples where they exposed these thinking processes, when they gave the model a code with bugs, the word error appeared in that space.

Or they gave the model a prompt injection, the word injection appeared there, even though the output never showed that. Even more interesting, through one of the tests that they did to the model, when the model was asked to perform a certain function, the model decided to cheat and bypass some of the test regulations so that they could get to the outcome of it, and the word manipulation appeared there, even though the output looked perfectly correct. So this is a very interesting finding.

I have to admit or recognize that at this stage, it's only research. It is not yet engineered, but this is where I believe the industry is moving towards. So do you think that something similar exists in the other AI models that just hasn't been discovered or labeled yet?

Is this going to be a pretty common thing? I believe so, and even more so, we have proof to that with DeepMind. The Google team did the same analysis and came to the same conclusion or the same finding.

They used the same model that was published by Anthropic, and they reached to the same conclusion and the same findings. So I expect to see that in every single model based with the same generative AI methodology. So what are the implications, because what you're really saying is these things aren't as much of a black box as we think, and today, we kind of just monitor the inputs and the outputs and hope for the best.

But if we can see inside the models and how it's reasoning, what are the implications? So if we want to monitor and control models or agents operating in our environments, what we have today are only two type of signals. The first is what the model is saying, and the second is what the model is doing.

Both of them are post-action. So whenever something happens and the agent decides to do something, then it would probably express that through words, and then also execute it. The new research, the Jspace signal, provides something that we didn't have yet, which is intention.

Intention comes before the action, and that allows us, once we can consume these type of signals, and the industry is already working on that, I'll talk about that in a second, allows us to get closer to the ability to detect and stop malicious or risky activities before the damage happens. Right? So I'm assuming that this is all going to be happening at machine speed, so how do we get in there and kind of understand those signals in a way that turns them into something that is actionable?

Because a lot of this stuff is just happening faster than we humans can keep track of. So what will keep track of this? Right.

So in my opinion, there are three layers of leverage or usage that we can do with these type of signals. The first one, which remains completely internal for the vendor, for Anthropic, OpenAI, and others, is by using the signals in real time from the moment that the word appears in J space to the next token, the next steps. This is something that can only happen within the ecosystem of the vendor because it happens in milliseconds, so there is no real feasible way to share that, and use that to detection response.

However, what we know is that agents do not have hands, right? So in other words, what happens is that from the moment that the agent thinks about something, then moves to the next token, and then takes an action, the action goes through a gateway to some sort of tool or application. So let's go back to the story of Messi.

If I ask the model, "I need to buy a flight ticket to the capital of the country where the best soccer player ever played lives," then within the J space, we see Messi, Argentina, then the output is a word, Buenos Aires, and then there is an action. Mm. This is where we have the opportunity to leverage these signals to stop the malicious activity before it happens.

And this is why I am arguing that we need to share, that the industry needs to share these signals, and they can be significant in improving our ability to stop the damage before it happens because the damage happens through the leverage of tools, and that gives us the time that we can react and stop the malicious activity. Then there is another component, which is the postmortem analysis, the forensic investigation, which is extremely important for us to improve our guardrails, to improve our security through learning from every single incident, every single investigation. There is now an initiative through the Linux Open Foundation, it's called SAFE, where the industry is trying to agree on parameters to share information about AI bridges precisely for that purpose, to be able to understand what happened and then improve our models, improve our security.

The more we can collect and share forensic information, the better we can build our security ecosystem. This signal would be significantly important in providing this full understanding of what exactly happened. So there is a limitation of what we can do, but there is much we can do with these signals when they are shared, even though these attacks happen at machine speed.

Do you also think that this is going to be a treasure trove of data for the compliance folks? Because I think a lot of them are trying to figure out, how is it are we going to understand what AI agent did what to whom when? Exactly.

And I think generally speaking, with this new source of telemetry, there is much we can do. We need to build a framework to understand that. I just came back from Black Hat and I read about this new publication, a new research that Professor Meshi from Ben-Gurion University shared about offering a framework that would be able to translate these signals into detections, into insights through something that looks like detection rules that we used to do in regular security world like Yara and SNORT and others.

So this is the type of stage where we are when we need to build this framework to allow us to consume these signals for different purposes. The value or the importance is how can we cross-correlate different signals to build a full picture. So it's not just about intention, it's about action, it's about action across the entire estate, what's happening with the agent in the cloud, on premise, in the device, in the SaaS applications, and so on and so forth.

So I truly believe that it's all about having panoramic visibility, and this sort of signal is or can be significantly valuable for that panoramic visibility. How might that be achieved? And I'm asking the question because sometimes I feel like we ask the AI to check its own homework.

" But in reality, it might be very different. To your point, they tend to hallucinate. So do I need to create some sort of other LLM AI agent framework that is going to check the homework of the other AI agent and LLM so I make sure that there's somebody who's validating this thing who's not checking their own homework?

That's a great question, and let me break it into two different components. One is, yes, in order to understand in a relevant timeframe what an agent powered by LLM is doing, you need another agent. You cannot beat a bot with a human.

It's bot versus bots, and that's given. However, the other side of it is that when we look at these J space signals The research shows that it's very hard to trick them or to trick that signal. And basically, through their research, what the Anthropic team realized is that when they turn that off, or when they remove that component, the capability of the agent is significantly reduced, the ability to reason.

So it is very hard to manipulate this specific signal, because once we think about agents analyzing what other agents are doing, the immediate question is: Can we trick the agent? Can the agent trick us or trick another agent? And they certainly can.

When we look at actions, they definitely do, when we look at what they are saying they're doing. There is enough research. It's very hard to do that when we look at the J space, hence the value of this very unique source of data, because this is where the agent is thinking, and it's very hard to fabricate or to manipulate that.

What level of scale are we talking about? Because in my mind, at least, the future of workflows is multiple AI agents are working on different things in collaboration with each other. And at some point, I want to be able to compare and contrast what they're doing.

So, what level of scale are we talking about for capturing signals here, and what's feasible? In an average environment today, and it changes almost on a daily basis, we have hundreds, already hundreds of agents operating. And that's at the very early, I would say, stages of adoption of AI tools across the industry.

So we're talking about hundreds, thousands of agents operating constantly. It is vast amount of data. This is one of the reasons why you cannot beat that just with humans.

But I believe that it is a scalable problem when we put the same technology in both sides, both in the production or the action itself and the monitoring and control. So the short answer to your question is it's scaling up, it's growing tremendously, but it's still feasible. Mm-hmm.

I would imagine there's going to be a massive amount of data to collect and analyze, and we're already struggling with that as it is in just standard telemetry. So what might need to happen, or is there some sort of technology innovation in terms of our ability to collect signals that needs to occur to make all this happen? So there are two problems here.

There is one is to collect and process the signals, and the other one is to store them in a way that can support the usage of them through analysis and investigation. These are two different engineering problems that the industry is dealing with. Mitiga and other companies are addressing these type of challenges.

When we talk about the storage, it is all about leveraging smart technologies of distributed data lakes, with different layers of data aggregation, so you don't necessarily need the original signal every time. You can, through aggregation, optimize the storage. You can, through aggregation and through layers, optimize the storage.

You don't need all the data available in real-time all the time, so you can separate between hot, cold, warm storage. So that's the area of storage. And the other one problem is obviously, or challenge, is obviously the processing of the data, where again, through optimization of data processing, through smart data processing, through obviously scaling up compute capabilities, the engineering allows a scalable solution.

Again, different technologies, different vendors are dealing with that. We at Mitiga build our product around that because we needed that exactly for that purpose. But there are so many other solutions, and yes, it's a challenge, but it's a challenge that the industry is managing.

I think a lot of security folks, and even compliance folks right now are a little overwhelmed by everything that's currently happening. What's your best advice to them about how to approach all this? Because the agentic workflows and the adoption is moving a lot faster than they're capable of responding.

So, what should they be thinking about now, and what kind of strategy should they be putting in place? That's a great question, and there are so many things that can be done. If I needed to focus on something, I would put the emphasis on thinking about these LLMs agents as identities in your ecosystem, the same way that security teams are looking at human identities in terms of permissions, in terms of understanding what they're doing, why, what access they should have and why, what permissions they have and why.

Tracing, auditing, looking back, and so on and so forth. The same type of lens needs to be applied when we're looking at agents, non-human identities. It is much more challenging, obviously.

But when you look at it from that perspective, you can start to arrange your security ecosystem around that. Again, through visibility, which is the most important thing, through awareness, through controls, through monitoring, through continuous detection, all the way to the ability to conduct forensic investigations. All right.

Well, folks, you heard it here. Hey, you may or may not agree that Messi is the greatest player of soccer or football, depending on where you are, of all time, but probably one of the greatest. But at least we can at least think about now maybe getting inside that AI's head to understand how it might have come to that conclusion, and maybe we can just have the debate, but we'll see how it goes.

But one way or another, the good news is, hey, we're getting more visibility, slowly but surely, into these workflows. Ariel, thanks for being on the show. Thank you, Mac.

Thank you very much. All right. And back to you guys in the studio.