Software Weaponization Raises DevSecOps Stakes

Hey everybody, we're back at the Open Source Summit here in Minneapolis, and we're going to have a little chat about the weaponization of software with Tracy Reagan and Kate Scarsello, who many of you know from the Techstern gang. So you could consider this maybe a mini gang meeting here in Minneapolis. But anyway, good to see you both.

How are you? Good. Good.

I think we're talking more and more about vulnerabilities, but one of the things that's been going on for a long time now is various governments around the world have been collecting vulnerabilities that they don't disclose to everybody else, and they're starting to weaponize these things, or maybe already have, and we just haven't caught them yet. And I wonder, with all this focus on mythos and vulnerabilities, are we going to have a bigger conversation soon about the fact that all this stuff is starting to be weaponized? What do you think?

Well, yes. So China in particular, recently, they basically have a cybersecurity incident recording law that has been in place for many, many years. And it had different regulations about how you handle an incident if you find one.

In November, they strengthened that law, and they basically said if you find a vulnerability, you have to report it within one to four hours, depending on the severity. And instead of reporting it to a public place, now it's being reported to the government's database. Mm-hmm.

So you report it to the government and they manage it. I was at BoneCon a couple of weeks ago and I went to a session on this, and a couple of researchers who have been following it claim, I don't know if it's true, that there are fewer now vulnerabilities being reported out of China. Much fewer.

Now, keep in mind, the Chinese have an amazing community of software engineers. They have many of them. They're highly skilled, highly trained.

The universities there are serious about software. So they were reporting many of the vulnerabilities. So according to these researchers, they claim that it could be about 10,000 vulnerabilities that the Chinese have found over the last six months that they have not reported because their public databases have pretty much flatlined.

Hmm. So we're not seeing what they're finding. Is it possible they're not finding anything and they're doing other things?

Or they're just like us using AI coding tools and starting to find these vulnerabilities and just keeping this to themselves? Haven't found anything. There you go.

It's not possible. So my next question goes to you, though. So we in our Western democracies like to be shocked and appalled when we hear about these things, yet I feel like we have our own three-letter agencies who maybe give as good as they get.

So is all sides doing the same thing now, and is this whole thing going to slide down into a war over who gets what vulnerabilities then hides them the longest? Well, I would say yes, I agree with that statement. However, I think because there's been so much fire in that perhaps we are actually not discovering as much as the Chinese quite crazy.

Mm-hmm. Meaning that yes, with all the agencies that have been decimated. Well, this sounds like our approach to COVID.

If we just stop testing, there won't be as much of it. That's right. Yeah, so I have to bring this up because it just came through this morning.

S. " So basically what they did was they created a text file, which they named private-cisa. And their tokens and passwords all were there in text form in the text file for the last six months.

Now, CISA has seen about a $700 million decrease in their funding. Hmm. Many of their top people have left.

So what they have left are people who probably are not that skilled in this area. I'm sure that they didn't understand what they were doing. They may not even understood it was a public repo.

But the point is, is that we have the Chinese who are now centralizing vulnerabilities to potentially weaponize us. The other shoe did drop. We heard that they are also working on a super hacker LLM.

Mm-hmm. Take the super hacker LLM and vulnerabilities that we don't know about, and you have a nuclear bomb, basically, for critical infrastructure. And smaller companies who are critical to us, but they don't have resources to deal with some of the stuff.

So we are decentralizing our cybersecurity vulnerability process. China is centralizing theirs. Now, we used to do that.

In the 2000s, there's a really good book called "They Tell Me How the World Is Going to End" written by, I think first name is Nicole. I can't think of her last name, but look the book up, and it talks about the zero-day vulnerability of marketplace before we really knew about vulnerabilities. Mm-hmm.

And the individuals who were playing in that and decided that they didn't want to play anymore because the government stopped paying them for vulnerabilities that would take them six months to find, started Black Hat and started saying, "Okay, we're just going to expose the vulnerabilities. " So we used to do that. Now the Chinese are thinking that that's probably a good thing to do.

There was one vulnerability, it was on a HP driver that allowed, if you had an HP printer sitting on your network, it could worm its way through the network and get to root anywhere it wanted. And the US knew about it and didn't tell anybody because they were using it to spy on people. Yeah.

There you have it. So cyber warfare has always been here, it's just changing. We have such a dependency now on open source vulnerabilities.

It's becoming-Critical, like Striker. Right after the Iranian war started, we found a small, not really small, but an ambulance service that connected an ambulance communication to the hospital, and they were pucked. They were bucked down by a vulnerability.

They were targeted. Mm-hmm. Critical infrastructure in a company that's probably resource-constrained, perfect store.

And also the gas stations. There was a gas station bomb business. I did hear that.

Yeah. " And that's been known for a while- Right ... as a person who's worked in power and utilities.

We've seen that vulnerability for a really long time, so it's not new. Right. So.

Well, if there is a hot war, I guess their assumption is they want to disrupt our energy supplies, would probably be one of the first things they'd go after. And it really does go now to the weaponization of software- Mm ... and the supply chain, and...

So what do developers need to do to change their habits or what's the impact of all this? Because I think they'll read it and they'll shake their heads, but it doesn't necessarily correlate to a change in behavior for the developers about what they're doing and how they're going about building software. So what's to be done?

Well, before building, as you know, one of the most important things, and it is on the first slide, is de-bloating and- Reachability ... reachability- Yeah ... is really, really big.

And reachability is not something to be casual about. And that's on SolarWinds, right? Where it's- Yeah ...

also in use. We can say, is the problem reachable? But SolarWinds wasn't reachable, and it was critical.

So de-bloating becomes the next question, and developers are often, they don't trust de-bloaters because they're only about 40% accurate. So you don't want to de-bloat something that's going to cause you a problem. They're known for causing the problems.

I'm full- I am, too ... full of their problems. I think we need to.

I think we really need to disrupt, and I'm just going to use a word that I don't particularly like, but disrupt the way that we think, because we're not winning. Mm. And if we don't start to think differently about how we go about this, we're going to be crushed.

We have just simple, these examples of something that if they really went all out, it would change the way we are working and interacting in a broad span. Mm. Yeah, yesterday Kate laid the law down at the CD Foundation when she did her talk on "Who Let the Dogs Out" around AI security, and we still have DevOps folks who don't want to bother with generating an FSBOB.

Mm-hmm. " She was very kind about it, but I think there was many of those folks in that Lynch about the message. They really did.

In my world, though, we have done a really good job of doing offensive work, and I mean making sure the vulnerabilities that we know of do not make it into a production setting. Right. It can't.

Mm-hmm. For the most part, high risk could approve anything over an eight, most companies are saying if it's found in pre-deployment, fix it. But the real problem that I see is 100% of attacks happen in production.

They don't happen in pre-deployment. Right. They can't, right?

No. So what we're doing with Artillius, which is why we're wearing these giant creatures, an alien on my shirt, is we want to create a way, we're building a digital twin that represents the production environment from the package mount to the end port. So when a new vulnerability that's a high-risk or critical vulnerability is reported, you can find it as fast as possible and fix it as fast as possible.

That's defense. So I would say the reason why Buffalo lost last night is they lost on defense. True.

It was so sad. True, it was sad, but let's go back to reachability for a minute. " But what was not reachable yesterday is most certainly reachable today because somebody changed the code, or added an API, or made something available.

So it's almost impossible to make a case for not doing something on the basis of reachability in my mind. You really need to assume that it's going to be reachable at some point. Well, if it's not reachable, then they should de-bloat it.

Yes. That should be the strategy, right? If it's not reachable, de-bloat it and take the chance of something breaking- Yeah ...

that we broke instead of a hacker breaking, because- Yeah ... it wasn't reachable, but it was a SolarWinds exploitable that had a time bomb in it. But ultimately that comes back to, do I have enough time to maintain my code?

Do I create the patches and all the fixes? I'm so tired of people talking about time. I know.

Well, here's the thing. We all seem to worship, at least in the land of DevOps, it's speed at all costs, and yet we have all these issues which seem to come back to the fact that we're just not spending enough time on some of this stuff. No, exactly.

We are. And the problem is it's going to cost us more time at the end. We have to get this right, and I can't stress it enough.

Time has to become something that we not put totally on the back burner, but we need to take it serious. Why are we pushing out features that you don't even necessarily need? We're not even adapting the new features that we have.

Just- Much less actually using, because- Yes ... all right. So why not just stop?

Why not just say, "Okay, let's work with what we have and make it-That are naked excellent because who knows where it can possibly go from there. Well, because some business executive said, "If we add four more features, I can increase the revenue flow of the subscription," blah, blah, blah, but it never turns out to be that way, right? Well, the thing about it is, we do have organizations that can step up to start working on this.

Everybody uses the same package, so not everybody has to de-build or look for reachability. So maybe our non-profits, Linage Foundation, the OpenSSF, MITRE, these are joint projects that we could be participating in to start making open source safer. Mm-hmm.

Right? But it does require quite a bit of money. Yeah.

Alpha Omega got $12 million, but that's not enough to- Drop in the bucket. It's a drop in the bucket. So it's going to take a commitment from industry to support that kind of effort.

I don't know if we have that interest right now until something big crashes. Yeah. Stripe was a biggie.

Even then. Yeah, and I also think one of the most important things, and I stressed it in the thing of the talk yesterday, is we collectively, as people, have a voice. We need to come together and actually start to be serious and to work on this issue.

I mean, crowdsourcing is real. The bad guys are doing crowdsource. Where's their money coming from?

The good guys need to crowdsource. It's coming from the ransom payments that we gave them. So, that's where it comes- From the Russian government or the Chinese government.

No, I do know that. It is very possible. But it seems like every organization is approaching this issue in much like the way I approach driving, which is, I'll be 10, 15, 20 miles over the speed limit on the assumption that the police are only going to catch two people, three people in an hour, and I might not be one of them.

That's the way the business is thinking. The business is going, "We'll deploy all this software. " That's why I want to focus on post-deployment.

Let's just fix high-risk and critical vulnerabilities as fast as possible. 80 days is too long. It should be less than a day.

If there's a high risk or critical, we should be able to identify it, and auto-remediate it, and test it, and get it back through into the deployed side within 24 hours, but we can't do that right now. " So is it that you deploy all the time, or is it that you can't deploy? Which one is the problem?

Well, if you're deploying all the time, then let's get those dependency files updated. Let's find the breaking changes, and let's get this stuff faced. That's actually impacting and threatening operational endpoints.

That's the real... And the operational endpoints is what's being threatened. That's where we have to put this.

Defensive versus offensive. I am a big believer that compensation drives behavior. So do we need to change the compensation models for the business executives and all the IT folks involved as it relates to safety and the security of the software?

Yes, and I also believe that if we were to actually look at the money behind this, I think it'd be actually more profitable to fix what got broken right now, put my lights on. I think we would get a better customer base. I think businesses would actually get better customers.

I think if you like a certain airline or a certain hotel, don't we frequent it? And wouldn't it be more profitable to have the customer be happy and not be like, "Oh, I'm not convinced. I'm going to switch," which so many of us are doing.

So Alan, who you all know and many of you do as well, frequently will say, "You'll get better security when the customer demands it," but the customer's not demanding it. So what is the issue with the end customers? They don't understand.

I had a discussion with somebody yesterday. He came to learn, and she was confused about SBOMs and what they were in either years. And she thought that what we were saying is that you have to go through your SBOM, and you have to take tools away from software developers.

She didn't understand that these were deep transitive dependencies that the software developer himself don't even know is in there. And who's putting them in there are in the... Why those transitive dependencies are in there is because of hardworking open source contributors who built these open source platforms.

They're not the problem, though. The problem is people who have infiltrated into those communities and are adding pieces that are dangerous. These are hard concepts for people who never compiled code to understand.

Right. So it's hard for, until you have things that are happening to at the consumer level, a bank comes down, electricity grid comes down. It's not painful enough for the consumer to understand or wake up.

Do we need to vet who's contributing what code these days? Because you do hear reports of folks that are participating in open source projects and happily being along, and then the next thing you know, there's malware that somebody dropped in. Yeah.

For me, it's interesting. And it's actually not a bad idea, something I haven't thought about before. As always, you're bringing up very good questions.

You would hate to not encourage open source and development, but maybe we need more humans in the loop. Maybe we need someone who actually knows the code to hit- except instead of just allowing it to go. Which is what happened with YZoodles or XYoodles.

I can't even- Yeah, XY. " So it was a human in the loop that actually stepped out on that. But I do believe with AI, it could help us get out of a problem as much as it's getting us into the problem.

I agree. So I think eventually, Jennifer Mulford did a really good presentation on AI at AS devOp, and she talked about how we have to have the human in the loop until we can understand how to slowly build our interactions, our prompts with AI, and begin trusting it so that we can advance it. And it's a really good suggestion for everybody on how to approach AI.

I believe that AI can start finding our vulnerabilities faster. " All right. You each are going to get this question.

You have been given supreme power over all things software development. What's that one thing you want to fix? Post-deployment vulnerabilities.

I'm proud of today. All right. Kate?

And visibility is at the end of the day, we need to understand what is in there. So that's, I think, visibility. All right.

Folks, you heard it here. It all starts with you, the developer, at the end of the day, and hopefully, there are no spies among us here, but we'll be right back in a minute.