Hardened Containers Address AI-Era Vulnerability Risks

Hey everybody, we're back in Minneapolis. We're at the Open Source Summit, and we're having a little chat about containers, runtime, security, because, well, it's a hot topic these days. Alex, welcome to the show.

Yeah. Thank you so much for having me. One of the things that did occur at the show is you guys announced an alliance between Edera and Minimus.

Yes. And as I understand it, that brings hardened containers to a hardened runtime environment. Yeah.

And this is going to be where two and two equals seven or better. Yes. Explain to us exactly the relationships between these things, what's going on, and what's driving this particular agreement.

Yeah. So, there's all the recent Mythos news and all the various bugs being found, but I think it's not just the AI models finding the bugs that's scary to us right now. People are scared of the fact that because these bugs are being found by AI, other people who are just humans looking at the code are also finding bugs.

And so there's just a mass of exploits going around and vulnerabilities that need to be patched. And the scary thing is that right now, all the ones in the recent news, I think there was literally another one yesterday or the day before that is a major privilege escalation, a Linux kernel, and things like that. Those kinds of bugs are just the things that are known.

Right? And those things are solvable through things like patching with Minimus. And so you can patch packages and actually solve those vulnerabilities really quickly if you rebuild your software often.

But the challenge is that what happens when you have a vulnerability that people don't disclose. And so if you stack the fast patching that Minimus provides on top of Edera's hardened runtime, you get sort of the known and unknown vulnerabilities solved in your production environment, which is just super critical for people who really need an environment they can trust and who are kind of scared of the current exploit environment. " Yes.

Is that what's happening here, essentially? And now people are looking at, "Hey, you know what? " I think although it has magnified the crisis, the crisis has always been there.

I'm giving a talk right after this about microVMs and how we're trying to solve the isolation problem in security. And one of the core things that I really think about when I look at the security landscape today is that we have effectively designed ourselves into a corner where the security of our systems is really relaxed by default. So, the problem has always been there, but this is just magnifying the problem and accelerating the risk.

And I think it's actually incredibly interesting that the vulnerabilities that have been found, there's sort of two schools of thought when Mythos came out. Well, maybe three. " I'm more in that camp because I'm pessimistic about exploits and vulnerabilities because there's always a new one.

" But I don't agree with that. And primarily because software is always a moving target. And I think primarily because I focus so much on low-level software, I see the intricacies of everything, and it just scares me.

And then I think there were a certain amount of people who were like, "Oh, this is nonsense," which I think you would be disingenuous to say that it hasn't done anything at all. Right? Mm-hmm.

But I think I saw some report from Google that said that the exploits are showing up like seven days before the patches show up now. Yes. And that rate seems to be getting faster and faster because all this is starting to move at machine speed now, right?

Yes. And things are moving quickly. There are exploits being found in versions that are just released that don't even have time to patch.

Mm-hmm. So the timelines have just shifted, and I think that I recently was in talking about Mythos in an article. I talked about how really what it has done is made the number of people finding exploits exponentially larger because now it's machines and not humans.

And because of that, we're just seeing things accelerate. It's like literally fast-forwarding, right? Like fast-forwarding a videotape.

And so we're seeing that stuff magnify. And the patch kind of gap, effectively, where you have vulnerabilities that aren't getting patched immediately, that's where Edera comes in, right? The vast majority of things, especially with embargoes and things like that, are going to have patches on day one or maybe even, like in some cases, patches will ship before the vulnerability's even announced.

Mm-hmm. Those kinds of things can be solved by Minimus, and that's why this partnership is really critical for us. And then Edera can come on top and solve those things for when there is a gap in patching.

Mm-hmm. And you can trust both. And it seems like the creation of the patch might get faster in the age of AI, but there's always going to be a little gap somewhere.

Yes. And correct me if I'm wrong, but the bad guys are getting smarter about exploiting that gap and better at it. Yes.

I would agree. I think they're finding ways to get creative, and I think one of the things that's really interesting is that patchingThe types of vulnerabilities that we're finding these days are not trivial in some cases, right? Mm-hmm.

It used to be, oh, there's just a use after free or some bug like that, where the change is literally a one-line change. But now we're seeing things where, oh, for example, Log4j back in the day. That vulnerability was very, very severe because the impact and the code change that was required was more significant.

It was a fundamental flaw in the design. I think you're seeing that in Linux as well and other systems. And so the patches are getting more complex, so they take longer to develop.

AI is helping in that regard, and there are companies that are doing things like auto-generating patches for vulnerabilities, but I think it's still early days, and ultimately, humans have to test things because, ultimately, AI can't just trust itself to generate the patch and test it. You have to have-- Especially with some of the packages that have vulnerabilities, they're super critical. Like, for example, a browser- Mm ...

where a vulnerability in a browser is a very big deal. " Right. But if I'm running thousands of these things, and some percentage at any given time are going to have vulnerabilities, the bad guys know this, right?

Yes, they do. And I think although we would like for people to always update everything immediately, the reality is that regulations and different financial frameworks don't allow for that- Mm ... in some of the most critical enterprises.

I think mentioned in the announcement is about how financial institutions are most at risk with this sort of problem, and that's really what we're trying to solve for them. Those kinds of people don't get the ability to update as quickly as they would like to. They have to do change management and things like that, and so deploying a patch takes time.

So Minimus will deliver those patches to you, but it might take a little bit of extra time to actually get that into production. And so that gap when you're running containers is a huge deal. And I think importantly, that containers are sort of-- We conceptually, especially the Kubernetes, KubeCon kind of vibe, they tend to really think in like, oh, containers start up and they shut down quickly.

But actually, in production, fairly often, containers will be long-lived and run for months at a time until there's a patch or a vulnerability. Yeah, and it seems like the containers are getting bigger. Yes.

Yeah. Container images are huge now, and then containers are scaling up. I would say that AI has maybe accidentally made this thing where microservices are slightly less popular now.

More people are building kind of not quite monolithic, but larger microservices or maybe large macroservices. I just call them big-ass microservices. Yeah.

Exactly. They're huge, and so you get a lot of attack surface and updating, and takes time. And so we're not practicing necessarily what we've preached in the last 10 years.

Mm. And so the realities on the ground are different than the ideal state, which is everything's so tiny and small that it's quick to update. Particularly when you're talking about AI, where we had a container image at Adera that we actually had to develop an entire new system for pulling images because it was 20 gigabytes of image to pull.

And that's a very large container image, and you got to think that that thing's updating multiple times a day. So we had to literally go back to the drawing board and think about how we work around this image size. I'm sure you're not alone in that.

I know a lot of folks are having the same conversations. Yes. As you think through, do we need to reevaluate our approach to DevSecOps in general?

Let's start with containers. Should they all be hardened or some be hardened? And then at the end of the day, how do I manage that through a pipeline?

What does that look like? So the key goal of Adera, and I think Minimus as well with the way that they develop their base images, and us with our runtime, is that it should be super easy to adopt. And so my hope is that people, and we will make good enough software that people can adopt it across their entire stack.

Mm. Now, realistically, will people adopt 100%? Probably not.

But everything is now vulnerable. Literally everything you can think of. Anything from a privileged controller running on Kubernetes to your actual application.

All of those things can contribute to the insecurity of the system. So I strongly believe that as we go forward with this new future with Mythos, we have to actually run everything in a hardened environment with a hardened image. Sure.

Now, I was talking to some folks about this, and I'd love to get your opinion, but everybody thinks that we're hiding some super-secret thing in Mythos, and boy, it's a good thing that won't get out yet. 5 looks pretty powerful for finding and creating vulnerabilities. Yes.

And I'm hearing from the open source folks who are creating AI models that they're not too far behind, so- Yes ... this is all going to be out there and for everybody to use in a matter of weeks. Yes.

When Mythos launched, there was a lot of people who were skeptical. Mm. I was not one of those people.

And primarily because I fundamentally believe thatIf this was going to happen, and the first time it happens where a model gets good enough to do this kind of thing, it's just going to be replicated by people. And then you have the good actors and the bad actors. There was all the talk about Project Glasswing and all of the who got access and all of that.

But what scared me the most was that, sure, we can limit access for a time, but that is for a short period of time. What happens when everyone has access, when it's now commoditized to be able to find these vulnerabilities and develop exploits quickly? And so I strongly believe that the risk is only getting worse.

I think we're only at the beginning of it. And I also think there's secondary effects that people haven't considered. I think there's a lot of the LPEs and the local privilege escalations and things that are being found in Linux as of recent.

Some of these vulnerabilities aren't actually being found by AI. They may be AI-assisted, but not wholly so. And I think humans are now being more empowered by the models to do the harder parts of the work.

And so it's accelerating and having second-order effects that I did foresee, but it's actually worse than I thought. I think people are now excited again that vulnerabilities exist, and so vulnerability researchers who aren't even using AI are getting more motivated. Right.

What's your best advice to folks then, as you look at this? Because on the one hand, I feel like we should be more proactive, but it's hard to go down on the business side and explain this and make a reasonable conversation why we need to spend more money on application development when they already feel like they're spending a lot of money on application development. Yes.

" Business is funny, right? I think some businesses will have to have that disaster happen. Mm.

That's just the reality of it, and it's only a matter of time. The businesses who are forward-looking are going to realize that this is a real problem, and I think you're seeing that. We know that banks are really invested in trying to find ways to work around Mythos and that people are actively freaking out effectively about it in some of these organizations.

And I think that is just, who better to listen to than the people that control our financial data, right? Some businesses will lag behind, and I think ultimately, it's just a matter of time until someone gets exploited heavily. Actually yesterday, there was a vulnerability or an announcement that there was a potential exploit into GitHub as an example.

We don't have any information on that because it's brand new, but these kinds of things, you never know how these are going to happen and how these develop. And so we just have to be aware of those things and take things seriously. Ultimately, what my message to business people right now who are dealing with these situations is to take this actually very seriously.

Mm. What is the role of the government in all of this supposed to be? Should the president be on television declaring something that looks like a national business emergency here, or what do you think?

I think that it's complicated. I think that there is a strong argument to be made that the government should have more of an impact on forcing people to actually care about the problem. Unfortunately, there are some people who will always lag behind because they're only motivated by the dollar and not by the dollar they can't see that they're not going to have in the future, right?

It's a common problem. I think that the risk of someone like China or nation states, actors who could maybe potentially attack US and Europe systems with these types of vulnerabilities is of real concern as well. So I think from a national security perspective, it's a serious problem.

Cynically, I don't know that it's going to happen, though. And so we have to be a little bit self-motivated in the industry, I think, to try to solve these problems. It shouldn't take the government, really, to get us to solve these problems, but I see it as a last resort, almost.

All right. Well, folks, you heard it here. A tsunami of vulnerabilities is on the way.

They're probably already here. The only question now is how quickly can we batten down the hatches. Hey, thanks for coming by.

Yeah. Thank you so much. I appreciate it.

All right, and we'll be back in a minute.