Log4jShell Vulnerability Report – Dan Lorenc, Chainguard

This is texturing TV. Hey guys. Thanks for the throw.

We're here with Dan lorenk. Who is the CEO for chain guard? And we're talking about the Cyber review board has a new report out talking about the log-forward J vulnerability.

They're kind of like, you know, the people from the government are coming around and examine the cause of accidents and why did crash or the train crash and it's similar idea and now they're starting to look into some of the major incidents and vulnerabilities. So Dan you looked at this report. What can we learn?

Yeah, this is a that's great analogy. This is like the people, you know coming in after you know, a real bad infrastructure failure and trying to figure out the root causes and write up a report and recommendations for both what happened kind of the factual statements as well as what we should do going forward. The factual stuff here is not to be kind of glossed over.

There's a lot of confusion the week log 4J happen in the patches were coming out and nobody was unsure of exactly which versions were affected all that. So I found the timeline part at the start of this really really useful just kind of seeing it all laid out in the right order now that we've had a couple months to look back on it. I also found the recommendations really useful just like you said is a government kind of report and so it gets to include recommendations that are a little hard to do from the company and it gets to include stuff that the government can do which is a slightly different set of tools.

Is it your sense that a lot of the instances of log4j are actually vulnerable or is it just that we use it so much that everybody kind of panic but when you get into it not a lot of it was actually externally facing and maybe accessible but we learned some lessons or what's your sense of just how big an issue was this looking back at it with some 2020 hindsight? No, this is really a perfect storm of an issue. Pretty much all of the instances were vulnerable and we use a lot of it and it was running almost everywhere.

I think the the report found that I attackers didn't actually get an opportunity to use this too much in real world compromises and I think I'd preface that or it's suffix that with a yet. This is one of those where everybody paying attention kind of didn't get much of a weekend that week and even stretched out into the following week. All the people kind of running secure systems responsibly.

We're on top of this they did a really good job shutting systems down patching them and getting stuff rolled out. But the other analogy they made in that report is they call this an endemic vulnerability. I mean, it's just one we're gonna be living with forever.

There's as long tale of Shadow it systems running that people don't even realize or running and log for today is going to enter every attackers toolkit from now on whenever you're trying to get into, you know, a compromise system. Just check to see if you can find any log for Jay still sitting somewhere. I mean we're going to see some really embarrassing breaches probably in the next couple of years is my prediction.

Into that point some folks. I talked to are still looking for their instances of love. So today they kind of know that they're out there but you don't know exactly where or how they were used.

So what makes this so challenging and pernicious that's it right there, you know in Java the Java ecosystem law for Jay are used everywhere and a lot of these systems don't get rebooted often, right these are you know, Mission critical systems that are left running and supporting, you know, critical workloads and we have a tendency to forget about those and Industry, you know, this was even used on the Mars rover. That's how wide reaching this this vulnerability was and so, you know that this is been pulled into the type of systems that you know, don't get weekly patches that don't have somebody standing at the keyboard making changes to them constantly. So that's the reason it's going to be so widespread and so long term Okay, I ransomware attack on the Mars rover might the very least interesting see how that would be patched, but we'll see how that might go.

I'm When you think about all this it also seems like we learned a couple lessons about what's possible with open source software Security in general. So do you think that we're moving forward with that in the right direction from here? There's been a few reports from the community itself.

But I mean, the truth of the matter is a lot of these projects are run by a handful of people. They don't have really all the time in the world that patch everything and respond every vulnerability. It might take them some time to get to that.

So is there some other way of thinking about all this? Now these are the right questions to be asking and you know, I think this is an example of where log4j was a huge success for the industry, right? This library was maintained, you know, it was maintained by people on a volunteer basis, but it did have professional maintainers that were capable of dealing with security fixes around, you know, I've seen much less severe vulnerabilities and you know much less widely used applications, but I've seen similar ones where the maintainers weren't there anymore Library had been abandoned and the rollout process in the patch process gets much more complicated in that case.

If you don't have somebody that you can reach within a couple hours or a day that still has access to push code to their pository to apply a patch. It turns into a much failure mess as people scramble to figure out how long they should wait for somebody to appear when to actually do that first for how to coordinate the community moving over to that for which Fork to use in the case everybody panics and you get multiple with incompatible fixes and then eventually how do you merge all those back together and who's gonna maintain this thing in the long term? So I think this is an example we're logged for Jay was a huge win.

We got very lucky and let's maintainers were doing a great job and we're able to get that patch rolled out. I think there's an alternate history you could play out where they weren't around and they weren't able to do that and you know would have been weeks or months rather than days to a week. But it did start a bunch of you know, really interesting conversations about open source that you kind of just touched on here, too.

Do you think Enterprise organizations may be a little more circumspect about what open source they're using because of this issue and now they're asking questions that they previously might not have asked at all. I think they should be I think they need to be right open sources everywhere now, right? This isn't the 90s.

This isn't the early 2000s when it was really a fight between open and proprietary software open source is one and I think Enterprises over the last decade or so that it too easy. They mean using open source. They've been taking open source, you know, there's arguments about whether they should give back.

How much do you give back? Where do you give back? But I think the bigger question is the one you just touched on.

You know, what happens when it does break and should they be paying attention to what they're using and should they have a plan for what to do if a vulnerability like this is found and I think you know, it's impossible to go say open source should change open source is no one person. That's no one Community. It's no one code base.

I don't really make any sense. But the way Enterprises use open source has to change and I think you know, this was another wake-up call. We'd seen them in the past with you know, Apache struts or Heartbleed or some of these other really widely used pieces of critical infrastructure.

And I think you know, there's only so many more of these that we can handle before us something bad does happen to an industry. So I hope this is a time where people actually start reevaluating this and figuring out the right way to use it responsibly. And the phrase give back, you know, when a lot of sea level folks here that they're like, well, I don't have coders who are going to write code and contribute all that stuff.

However, there's a lot of other ways to give back whether it's Financial commitments or just helping a review code or for that matter working on the documentation. So do you think people really understand what it means to give back? Yeah, there's no one way to give back right?

I think that that makes it pretty hard some projects, you know are set up to take funding if you give them a couple dollars here and there they can make use of it others aren't right, you know, just the overhead to set up funding and being able to take it especially if you do have another day job or something you're working on this for fun on the weekends can be harder right money isn't the only answer. Sending patches sometimes cheap aren't interested in that either right? There's no one way to do open source, and sometimes these maintainers release code and don't want anybody to bother them about it again.

So there's no one one right answer here. I think, you know being conscious about it being willing to help out where you can documentation updates tutorials. All of that can be useful.

The other thing that seems to occur is that people will take open source code and then they'll add something to it and they're kind of created their own little Fork but they don't give back the thing that they added to it to the community or there's and then eventually, you know, we hope that the forks will come together at some future date, but ultimately too often my Forks have Fork so is another way of thinking about this. No, I mean that's covered in the licensing. Right, you know, when to Fork what you're allowed to do what changes you're allowed to make when you have to share stuff back right and you know.

That's that's rarely the problem here right organizations taking code applying their own patches and refusing to give those back. Happen and that comes with a lot of pain too right inside of the organization because you've got to keep those patches up to date as you try to bring in Nuance, right? It's almost always easier in the long run to get your patches Upstream because then you know, it gets maintained with the rest of the process.

You're not constantly rebasing merging resolving merge conflicts that crazy headache kind of fast moving project. That said that is the best method in a lot of cases, right and it can actually reduce the overhead on maintainers. If you've got some Niche feature and you want to run this code on a platform that nobody else cares about adding that Upstream is really just gonna add a maintenance burden everybody else working on the program.

And sometimes that is the simplest and most friendly way to add a feature that you care about to an open source project. All right, you're a king of the open source company for a day one thing you wish you could fix or what's the one thing that you wish would be changed or make everybody's life better. King of the open source Community man.

This is like bringing order to chaos and Anarchy can't even imagine that. Oh man. I think you know and again this is like a selfish take on.

You know what I think would help everybody more responsibly use open source and make better decisions. Is if everybody would add a kind of statement and some kind of text explaining their expectations for the project right licenses only go so far, right they say, you know, this comes with no liability no warranty, but a lot of people do try to fix buggies and responsible time and have a security policy and want to encourage contributions. People that go the other way aren't doing anything wrong, but I think it would be a lot easier for people to collaborate not productive discussions.

If you stated your expectations for yourself for the code, what type of things you were looking for and actually updated that, you know, when situations change which is a lot to ask you might start out with the intention of maintaining something forever and then get a new job something changes in your family you run out of time. But clearly stating, you know, kind of what you're looking to get out of the project. I think would help everybody make better decisions.

All right, folks here first. We all learned a lot and hopefully nobody got hurt in the process so we can move on from here and things will be better as we go forward the more we know the better off. We are Dan.

Thanks for being on the show. Awesome. Thank you for having me.

All right back to you guys in the studio.