Root Evidence Rethinks Vulnerability Management

Hey everyone. Welcome back here to Techstrong TV. It's a big week for security whenever we gather in the desert of Vegas in August.

I don't know whose idea it was to do this the first week in August in Vegas. It's 115 degrees. But luckily, we recorded some great content prior to heading out to the heat, and this is one of our specials for Black Hat Week.

I want to introduce you... Well, they don't need an introduction if you're a cyber person, you probably know these people. But if you're not a cyber person, these are two of the really big thinkers in cybersecurity in my mind, people I've worked with, watched.

In some ways, Jeremiah, man, I think I first met Jeremiah, he must've been 25 years old or something, right? Probably. Yeah.

And Rsnake as well. Let's just say we all had a lot more hair back then. Anyway, let me introduce you to Jeremiah Grossman and Robert Hanson.

Jay, and Rsnake, as a lot of people know him. Guys, welcome to Techstrong TV. It's great to have you back on.

Good to be here, Alan. Good to see you. Thanks for having us.

So what have you been up to? Nothing much. What do you mean?

Nothing much. Nothing much. Just hanging out.

No, seriously, we're going to talk about what you've been up to. But you know what, guys? Not everyone who watches this is a cyber person.

I got a lot of DevOps people, cloud native, platform engineers, everybody's an AI engineer today, or whatever. For people who aren't familiar with your backgrounds, you know what, Robert? I'm going to ask you to go first.

Give them an idea of who Robert Hanson is. Well, I've been in computer security for 30 years now. I'm actually coming up on 31 pretty quick here.

Most of that was spent in the red teaming offensive side, so I try to break into stuff, like banks or flight control systems or whatever, right? Just anything a customer would ask me to break into. So we had a really storied career, but gradually I became more on the defense side, worked at eBay and other places, and Jer and I got together through a bunch of mutual research we did, and we started a couple companies together and wrote some books and did a lot of offensive research, created a lot of vulnerabilities over the years.

And now I'm sort of more on the entrepreneur side and more on the CTO route. But it's been a crazy, crazy little 30 years. It's been a hell of a ride.

What a long, strange trip it's been, as they say. Exactly. Good for you, man.

Jeremiah, what about you? How would you describe yourself to people? Oh, usually when I'm describing myself, I like solving very hard cybersecurity problems.

It is a space that has the most interesting, challenging, world-changing problems that I can think of, and we get to play with tech. So yes, we could break into things and find vulnerabilities and things like that, but Robert and I, we get to talk to every company imaginable, every one of the Fortune 500, every university, every retailer. They all have very similar problems.

" And that's what we like doing, starting companies to solve problems. And the latest one is vulnerability management. It's a 25, 30-year-old space, and no one has it right.

So it was time someone took a real close look at it, so we dove in. I think it's older than 25 years. So when I was at Still Secure, we came out with VAM in 2003.

And we weren't first. Foundstone was already out there. EI was out there.

Philippe was telling people to put vulnerabilities. It wasn't even a cloud. Yeah.

I don't even know where he was keeping them to tell you. It might've been in his closet, I don't know. But he was moving vulnerabilities off of prem onto somewhere.

But in any event, you're right. It's been a problem that I think we never had an elegant enough solution for, right? And for a lot of reasons.

The finding vulnerabilities was one, how you do scanning internal, external, quarterly, yearly, continuously. What do you do once you find the vulnerabilities? Prioritizing them, which ones are real, which ones are paper tigers.

Who should be doing that work, right? The security guy finds the vulnerabilities. The help desk gal, is she responsible for fixing them?

Do we automate all this? Who trusts automation? In a microcosm, it's the theory of constraints, because every time we seem to solve one problem, another one reared its head, right?

So is the answer root evidence? We think so. Remember, the only reason vulnerability management has a prioritization problem is because we have more vulnerabilities than we can fix now or anytime soon.

Otherwise, 25, 30 years ago, we didn't have a prioritization problem. You scanned, you patched everything. That's what you did.

But then we got so many vulnerabilities, we go, "We can't fix them all. " And since we really didn't know, everybody guessed. We came up with CVSS scores and traffic signal security, and orange, yellow, red, and we fixed it on that way on just best guesses.

The entire thing was a guess. And a strange thing happened over the last 20 years. All these vulnerabilities we're finding, the vast majority never got exploited.

The adversary didn't care, which is the whole point of the exercise. So Robert and I did something only crazy people do. We asked the people with the best actuarial data in the business which vulnerabilities matter, the insurance carriers and their DFIR vendors.

They respond to these. And we ask them very simple questions. Which vulnerabilities lead to breach and financial loss?

And Robert will tell you what we found. Yeah. It's a shockingly small number.

6% of all vulnerabilities had led to a breach and/or a loss. 4 for a while now, which means that- Basically, this AI revolution has created a whole bunch of vulnerabilities that no one's using. And so the amount of vulnerabilities that are being used as a percentage is decreasing, not increasing.

It's still increasing, it's just not decreasing as a percentage of the total. 6% of vulnerabilities are not being used. From anywhere we can identify or anybody who we talk to, all the DFIR experts, all the people who actually pay out when this bad stuff happens, just the losses aren't showing up anywhere else than this very small list.

2%, that lead to a breach and a loss. We're talking about hundreds of vulns here, not thousands. Definitely not tens of thousands or hundreds of thousands.

So that seems to fly in the face of, what was it? Microsoft put out 500 and something patches for Patch Tuesday last month. Mm-hmm.

I think Oracle, also some crazy amount. Google. Google put out, I think it might have been 1,000 from Google.

It was another just crazy number. Why? Why are they putting the patches out?

Yeah. If such a small percentage of these vulnerabilities really matter. Right.

Really matter. Is it a question of, well, it's a small percentage, but I can't pinpoint which ones those are? 4% of those needles that matter?

Or is it something else that it's just we're Pavlovian dogs who have been programmed to patch a vulnerability? Those things that they're being patched, Microsoft, Oracle, and everybody else, they're probably software bugs. We really don't know if they're exploitable.

That's a whole other matter entirely for the adversaries. It takes a lot of work to stabilize and create a weaponized exploit. So those guys are just offering patches for a bunch of things, but the adversary is the one that gets to decide which ones matter.

Not Microsoft, not Oracle, no one else. The adversary does. You can have all these vulnerabilities, but the adversary, "You know what?

" And like Robert said, it's really only about 600 total out of 350,000 have ever led to a breach of any kind. 370,000. It's growing.

And that's the thing. So this part- But is it a steady 600, guys, or does it change? It grows like two, three a month.

Really, really small numbers. Two or three a month. So while we're chasing millions of vulns, those are the only ones that actually matter.

And we just didn't know that five, 10, or more years ago, how concentrated the set that actually mattered was. That's what's new here. How we found some of this out was, JR actually tasked me with a very simple graph.

Just show me the trend of CDEs over time. So I did that, and then I started collecting. To do that, you have to collect all the data for CDSS and all that information coming from CDE List v5 and NVD.

And then I'm looking at all that data, and I'm starting drawing concentric circles around CDSS scores, and then EPSS, and then VulnCheckov and CSACV, and what does Mandiant say? And it starts getting all these concentric circles, and where do you see overlap? And the scary thing is, we basically found there's no overlap.

Basically, no one agrees. Which means that either everyone's wrong, or just one of them is right. And so finally, we found two groups that did actually agree, but only mostly agreed, and that was the DFIR guys and the cyber insurance guys.

Which makes sense. They have the actuarial data. They have the actual data at the end of the pipe.

Everybody else is just guessing as far as we can tell, and not very well either. So Root Evidence ends up being, to the same name of the book, it ends up being the end of guessing. Let's not guess at what the adversaries are exploiting, strange rating systems.

Fix these vulnerabilities on all your systems. These. And it's not that many.

When we're scanning organizations, let's say mid-size organizations, thousands of them, we're finding five or ten of these a company. Not thousands, not millions, just these. Mm-hmm.

And what about the two or three every couple of months that get added? They get added to your Root Evidence dirty dozen, or whatever you want to call it. Yep.

Those get added in. And so one of the major things that we have, the big breakthrough I think that we made as a company, is creating a warranty around what we're doing. So if for some reason we get it wrong, and this is what everyone's worried about, this is the whole AI big kerfuffle.

What if you get it wrong? What if the AI is able to chain 15 vulns together and exploit me? Well, A, we have no evidence that any of that's actually happening in the wild, at least not at any kind of scale.

It might be happening onesie, twosie, or something. But B, if we do find on the other end, if let's say the cyber insurers say, "Okay, these three CDEs or 10 CDEs were used in a financial loss," we just add it to our list. And then we're like, "Okay, well these, for whatever reason, seem to be attractive to the LLMs," which are acting as a proxy for the adversary.

And then everybody's inoculated. We basically scan for that thing, and then everybody gets the benefit of this new knowledge. That warranty becomes a feedback loop to make us better and better and better over time.

" So we have a lot of incentive to get it right, and continue to track it and make it better over time. Five million incentives to get it right. And it's important the way we set it up.

So it's a $5 million warranty that if there's a breach due to a vulnerability that we didn't report, we'll pay out. And our insurance carrier partners are underwriting us. It's not that they agreed with us, we agreed with them.

We're using their list of vulnerabilities to do the scanning. That's the only reason they could underwrite us is because they're already sitting on all the same data. So they know that we're not making risky bets with their money because they have to back us.

They have to underwrite us. And they know that we don't want our premiums to go up, so they're taking us very seriously because we're using their data. There's nothing up our sleeves.

It's like, well, you handed us this data, and we're using your data. And so it makes it really easy to... If you think about it, let's say that their total losses are 100%, and then CDE-related vulnerabilities are, call it 30%, or whatever it is.

So we're already, just by our warranty only covering the things that VM covers, we're already just 30% of the losses. And they're already taking all the risk of all 100% with their cyber insurance policies. So right there, just with no other benefit at all, if we did literally nothing else, we would still be just 30% of the risk.

But we have all this stuff we're doing to mitigate it to virtually zero. So let me get a little vulnerability nerdy on you talking about scanning. What does scanning entail at Root Evidence?

Because that's something you do off of your SaaS host somewhere, or it's got to be inside. Am I just scanning your production environment? Can I use Root Evidence in my pipeline, get this stuff before it gets released, which would be nice for a change?

We're external at the moment only. Mm-hmm. Although a lot of our partners really do want us to scan internally, so that is something we're looking into.

But before all that starts, we need to know what to scan. So part of our warranty and part of the way we work is we create an external attack surface management map of your environment for you, and then if you get breached on an asset that we didn't find, then again, millions of dollars back. And the reason that's really important is because that acts as the very first feedback loop we've ever seen to make sure that we're actually doing external attack surface management well, because otherwise, how do you know if you're doing a good job or not?

So we have this feedback loop that allows us to get better and better and better over time, and we have a lot of incentive to make sure it does well, not just okay. It's got to do extremely well to make sure that we are identifying all the assets that can lead to this financial loss that we're talking about. Then we scan them for those vulnerabilities, and ideally daily.

That's what warranty means, that we're daily scanning. And the reason for that is we don't want to have a full month or a couple of months where the adversary has free reign over your environment because you left some vulnerable version of some VPN sitting out there. So we do require scanning daily just by virtue of the fact that we're trying to do the same thing that you're trying to do, reduce your losses.

Right. Well, I think also the infrastructure, the code, we live in a world of continuous development, continuous deployment. Right.

What was there yesterday is not necessarily indicative of what's there tomorrow. Guys, let me play devil's advocate. I've been in security a long time.

I've been in vulnerability management a long time. I remember back in the day, Jeremiah, it might've been you, had a deal with Aon, right? Where back then it was only a million bucks because we didn't have such big losses.

It was a million dollars if you got breached on something when you were using this product. Mm-hmm. But there was so much fine print involved that it was virtually impossible to ever collect.

I don't think one million ever got paid out. What's different? So the first time I did a warranty was at White Hat on web application security.

That was the- Yep ... the first one of its kind, and it did really, really well. And to be perfectly honest, no claim was paid because we never got a single claim.

There was nothing to deny. As long as you fixed it, it worked. Then I took a similar model over to SentinelOne a long time ago, when just before the ransomware wave hit.

" And I got it underwritten by an insurance carrier. Even through WannaCry, there was no payouts because there was no claims. I never even got a chance to test the fine print of it.

And so we've done a lot of those warranties, Robert and I, over the years, and this is the latest iteration, is our warranty is there's no additional requirements, really, over a normal cyber insurance policy. Do some of the basics and fix the vulns that we set. If you decide to not fix the vulns that we say, and there's a loss, can't really blame us for it.

But we want to create a strong incentive to patch the ones that matter, and that's it. Just really simple. Mm-hmm.

And if for some reason you think we get it wrong, like false positive or- Yeah ... or it's dangerous to do it because of some other reason, fine. It's just that we're disclaiming it at that point.

You know what I mean? So it's not even like- Yeah ... we're not like a regulator or an auditor where you have to do something.

It's more like, well, if you want to remove this risk from your portfolio, we recommend removing it, but it's your business. Do what you want. And again, like Robert said earlier, we're not opposed to paying out claims, and neither are insurance carriers.

For us, that's extra actuarial data that we can use to assist the rest of the customers in the ecosystem. The rest of the base, sure. This is the feedback loop all of cybersecurity has been missing.

I complained about it for years. Our industry is a $200 billion garage sale where all sales are final. Vendors should take on risk with their customers for their guidance.

We'll do it. We always have. Yeah.

You know what? Vendors have it. You know that, I know that, right?

It's like the PCI Council who says it's never been a PCI-compliant merchant who's ever been breached. Yeah. Because the fact that they were breached means they weren't PCI compliant.

It's perfect. Yeah. Guys, I got to turn to the business side of things.

So how is this packaged? How is it sold? How people sign up, how do we do it?

It's very simple. You reach out to us, say you want to test the product. We schedule a time.

We'll do an attack surface map and get you scanned. It'll be a SaaS subscription, annual subscriptions, and all those sorts of things that everybody's accustomed to in VM land, but it's a very straightforward process. Since we're not scanning for 350,000 vulns, our scans go real fast.

They're very light. We provided just the right amount of security, just the right amount of scanning to meet or exceed the capabilities of the adversary. No more, no less.

It's designed for that reason. Mm-hmm. Because we think a lot of this tech is just wildly over-engineered.

" And so they're one-upping ship purely on number of checks as if that has somehow anything to do with what the adversary is doing. The adversary has their very specific things that they like doing, and who cares you found one more Joomla exploit that no one's ever going to touch, right? Or some internal vuln that, again, no one's ever going to see.

So. So our product is designed to make it fast, easy, simple, not feature parity with the competitors. There's features that we don't have because they plainly don't improve efficacy.

The only thing that we want is the same thing our customers want: not get breached, not have a loss. That's it. Mm.

Mm-hmm. Excellent. Hey, guys, this is running obviously during Black Hat, but we're going to have a follow-up after Black Hat, and we'll dive more into it, and hopefully, maybe we'll have some feedback from the community by then as well.

But I want to wish you both the best of luck with Root Evidence. Two nicer guys, I don't know. And as I said in the beginning, two smarter guys.

" Mm-hmm. Jeremiah, Robert, thanks for coming up here on Techstrong TV, and we'll continue this conversation at a later date. Thanks for having us.

Thanks for dropping by. Thanks, Robert. All righty.

Oh, you know what we didn't mention. What's the website, guys? com.

There you go. All righty. com, go check it out.

We're going to have more. You're watching Techstrong TV.