Why Cyber Ranges Are Replacing Tabletop Exercises for Security Readiness

Hey guys, thanks for the throw. We're here with Peter Lee, the CEO of sim space, and we're talking about, well, cyber ranges as it applies to cybersecurity and training, and maybe just modernizing this whole process. Peter, welcome to the show, Mike.

Thanks for having us. You know, I just saw recently somebody going through some cybersecurity exercises and it was a tabletop, kind of a version of a tabletop game, and everybody had their role to play and you know, as I was watching and I couldn't help being reminded of, you know, a clue, you know, Colonel Mustard did it in, uh, a library with the candlestick. And as fun as it was, it didn't seem very realistic.

So, um, what should we be thinking about here when it comes to training and modern attacks and resiliency? Is there some way to think about all, all this that's just gonna be maybe better than what we have been doing because, well, I just don't think the bad guys are sitting around playing table talk games to attack us. Yeah, you're absolutely right.

Well, first of all, I think the whole notion of training has to get redefined as humans plus AI working hand in hand. It's no longer just humans. And I think, um, I've been shocked at the weaponization of ai.

I mean, it is unbelievable today. There's a lot of skeptics, still people who believe it's at the early stages, and that may be true, but we are moving very rapidly into an age of autonomous age agent attacks at scale. You know, they're gonna do reconnaissance, vulnerability, exploitation, moving laterally, you know, the whole full, uh, kill, the full, um, full value kill chain.

And so I think it has to be humans plus ai. And I think that that notion of training is critical. And I think a cyber range is the cornerstone of rebalancing the fundamental asymmetry of offense, cyber offense, and defense, right?

It's always cheaper, faster to attack than to defend the attacker chooses when and where the defender has to prepare everywhere. Um, so that's, by the way, that's half the story, Mike, and we can get into the other half, which is testing of tools, agents, capabilities. So yeah, as one old boss of mine said, you know, it's a lot easier to throw grenades than it is to catch 'em.

But, um, when you think about this for a minute, uh, I'm not sure everybody knows exactly what a cyber range is. They're familiar with range as in like the military has an ex exercise, they go out to the range and blow stuff up and they learn how to do things. But how does that concept apply to cybersecurity?

Well, I think we should define the cyber range. First of all. Um, a realistic and intelligent cyber range is going to be one that enables you to create a realistic replica of your production environment.

So this isn't gonna be some pre-canned laboratory. It's going to be a very rich, um, replica with full network topology. It's going to have integrated attack and activity emulation with your actual security tools and user behavior, which is chained in some ways and interactive with attack behavior.

You're gonna have a very significant, um, we don't like to use the word digital twin because of that implies, um, a layer of cost and complexity, which is diminishing marginal returns for what I think a cyber range, um, can provide. But yeah, absolutely tabletops, simple labs, individual training, these things are obsolete. They're not gonna move the needle.

You absolutely need to have a realistic replica of your production environment. And I'd say as I opened with my comment, not just for humans, but increasingly certainly what we see in our business is that, um, vendors are using it to train AI models, to test AI agents and to validate agentic capabilities. And you can't do that in an environment that doesn't closely, um, mirror your own enterprise complexity.

It's gotta be like any other type of agentic or um, AI modeling. It's gotta be very, very, um, tailored to your specifics. And how easy is it to do that these days?

I think people think that there's a level of complexity involved that's too challenging. Yeah. And then how do I test those AI agents once I set that thing up?

Because, well, every AI agent I've seen so far does things that are unpredictable. Yeah, those are very good questions. Uh, so on the first one, I think that, um, there the answer really depends on the degree of fidelity that you want for your, for your environment.

So we can have, um, realistic replicas of production environments set up in a matter of a couple days, or it could take several weeks depending on the level of fidelity. And that'll depend on the use case as well. Um, in terms of the testing of agents, it's really important to step back and zoom out.

One of the things that's really, um, critical in terms is the ability to get clean labeled data to be able to actually run full kill chain threat attacks in varying user environments with every single conceivable permutation of your network topology and the different tools you use be able to collect and synthesize that log data and figure out did this agent actually work? Thumbs up, thumbs down. And you can imagine doing those training runs thousands and thousands of times in order to begin to bring that capability into bear.

And of course you could think about dual, you know, traditional data science measures precision and recall, am I really tuning it so that it absolutely is gonna detect every single type of threat, but it may throw off a degree of noise or am I gonna reduce the noise, but I'm gonna let some of these threats through. So there's, I think there's some significant capabilities and, um, tailoring and customization that are capable in the cyber range, um, uh, uh, platform, but that's going to be user and use case to bend it. Mm-hmm.

Have we gotten to a point where we can't quite just put up a standard red team because the number of vectors that could be exploited are just too broad and there's too many things that could possibly happen and who knows, maybe we'll use AI agents themselves to create the attacks, but as one security expert said to me once, if you can imagine it, somebody's probably trying it. So how do we deal with all the exponential possibilities? Well, I think if you, on your original question, can we, uh, are red team still useful, let's call it or, or rely on?

Absolutely. I think that, you know, again, I I would go back to my, my touchstone of humans plus ai. There are things certainly if I can speak on behalf of our red team, that they're capable of doing that are extremely sophisticated and would, um, would not be, uh, available for agent training.

They haven't, the data doesn't exist, the penetration methods, the tooling that they've developed. That being said, I think what we see is we have actually, um, a fairly expansive set of ecosystem partners. There are a number of red team agent businesses that are beginning to, um, develop their capabilities in our cyber range.

And what they wanna do is they wanna partner with us. So as we run these realistic, um, training exercises, team training exercises, they're going to be able to, um, have a menu. You could think of them as a, to offer capabilities against the, uh, for the enterprise clients to test test their teams and tools.

And so I do think that that is going to be the future where you're gonna see a variety of different, um, scenarios. And I think the, um, the cyber teams that are really focused not on compliance and check marks, some say not on individual training, but on realistic team training that actually helps to outsmart adversaries in any cyber terrain. I think you'll see a tremendous amount of, um, interest in both red team and blue team agents.

We're seeing both, um, both parts of the ecosystem get attracted to our, our cyber range, um, and of tremendous interest to clients who want to do team training at, at, uh, at realistic scale. Some folks are concerned that perhaps we're a little bit over our skis when it comes to AI agents and we're deploying these things without thinking through the security issues. Um, are you at all concerned that maybe we're just waiting for some sort of catastrophic event before everybody gets serious about AI training and testing?

Mike? It's a great, it's a great question. I think one of the other elements of that, what we are actually doing is to help, um, validate the, uh, AI agents themselves.

Anytime you introduce a new capability, you open up a new tax surface and the reality is that, um, bringing on a blue team agent to help with your SOC preparations, but opening up new attack vectors without being prepared for it would be a terrible mistake. And so we are actually creating kind of an agent reading system or validation system, and we're, um, taking a hard look at the, um, at the vulnerabilities that the agents present themselves is a fantastic question. Mm-hmm.

Do you think on the other end of this that auditors might start using these platforms to test and validate the environments that they're being asked to vouch for? You know, that's a fascinating question. We actually see, um, it hasn't yet proceeded to the auditor level, but certainly the cyber insurance providers are taking an active interest and we've had several of our enterprise clients let us know that we've been instrumental in reducing cyber insurance premiums because they can actually demonstrate progress on a longitudinal basis against the varying stage and progression of threat actors.

So I do think that the insurance market is going to be the leading edge, but I would not be surprised if we see, um, multiple other, um, uh, constituents take advantage of that. No doubt. So as you look at it and you think about testing and training and everything that goes with that, what's that one thing that kind of just makes you shake your head a little bit and go, folks, we need to be a little bit better than that?

Oh yeah, sure. That's an easy question to answer. I think if you look at our large enterprise clients or large government clients, they are literally using hundreds of tools.

They are never going to be optimally configured. They are in a continuous, what I call process of selection and optimization. Many of these tools have overlapping vectors.

Um, they're continuously evaluating and adopting new tools and there isn't a programmatic, um, value-based, efficacy based way to actually, um, frame decision support around how do I configure and optimize and rationalize my tools in order to improve my security posture. And so I think that to me is just a continuous, um, cyber range use case that we're, we're really excited to, um, be engaged with our clients around. All right.

Hey folks, you're heard in here. If your training and testing is rooted in the last century, you're probably not gonna get the outcome you're hoping for. Peter, thanks for being on the show, Mike.

Thank you for having us. All right. And back to you guys in the studio.