State of Software Security Report 2023 – Chris Eng, Veracode

This is texturing TV. Hey everyone, welcome back to Tech strong TV. I'm really happy to have my friend Chris saying with us for those who don't know Chris he is.

Well, he is one of the guys insecurity. I I've known Chris saying for I don't know 20 years maybe even long. A long time it was that steak and then it was semantic and then very cold and very coded broadcom in varicode.

Not a broadcast the permutations here. Chris is still here. He's still he's still doing security.

He's still one of the sharpest guys in the industry Chris. Welcome to see you. Oh, it's good to talk to you Alan.

So Chris, I mean look I our audience knows it's very cold. But just in case there's some people not familiar. Give it if you don't mind, give us a quick Vari code background story and then we'll jump into it.

Yeah, well error code helps our customers build software Security Programs, basically through a combination of automated tools and services and that's that's basically been our MMOs and since day one of the first companies out there to kind of do this in the cloud and that's what's kind of allowed us to do the reports that that like the one we're releasing this week. Absolutely. Yeah.

I don't want to embarrass Chris but look varicote was one of the leaders kind of inventing the application security space. The whole deficit cops movement they were early on and one of the early movers and shakers and leaders there as well and and Chris has been right in the middle of it. Right?

He's because you lead the director or VP. I don't even remember your time. I leave the research paper organization.

Yeah, that's right. So all the new capabilities that we build out in church of scanning languages and platforms and Frameworks. That's my team that's kind of researching those and and understanding what is it that we're going to stand for.

How are we going to scan for it as well as looking at all those new capabilities that we're bringing on as we kind of Branch out into other markets and other technology. So a lot a lot of cool stuff going on. Absolutely.

All right, I think we've sufficiently embarrassed you. Let's move on. Let's move on to the results of this new survey and now the report that you guys have come out with Sure.

Well, I mean just a little bit of background on it something we do roughly annually and one big difference from a lot of the reports that you read out. There is that this one is not a survey. This one is real world data.

So we take the results from all the applications that we scan through through varicode products through varicose services, and we we analyze that data set. So it's 750,000 applications. Probably the largest data set of this of this kind anywhere in the world.

And so then we look for Trends we look for you know, what's happening. What behaviors are we seeing? What outcomes are we seeing?

And then how can we tie different factors to better outcomes and and give people give readers some actionable takeaways on what they should go do to improve their programs. So that's the idea of this we've been doing it now for 13 years the first time we did it and 1500 applications in it and like I mentioned now we're recorders of million, so we're pretty pretty fun stuff. Just the scale is amazing.

And you're right. I did. I did know it wasn't a survey but I know you people would serve it.

I mean, it's just a tremendous the data set, you know, one of the things Chris I I was writing about so we have this predict virtual event where we talk about, you know, what the year coming up holds and one of the things Mike Rothman is is heading it up this year for Tech strong research. When one of the things we spoke about was that big data is getting bigger, but we're better. At at handling it right then.

We were five ten years ago. So you get a data set like this again used to Boggle the mind how you would find anything actionable in there and you know like needles in Haystacks. But today we we actually can you know peer into that data and see patterns and see yeah, it's true.

I mean you used to be able to do this in an Excel spreadsheet. Yeah that one for that and then at some point yet if you had up you had to go up a level and actually put it into a database and now, you know, you're the point where what we do what we do. Yeah and have done for the past four or five years now, I guess we've we've we've worked with a data science firm scientia Institute and and those guys can really take this this massive amount of data we have and we can start to Spend more time like with the verification spend more time thinking of the questions that we want to answer and less time kind of worry about how are we going to how are we gonna take this data and come out with those answers?

And so, you know like this time we've said well, we spend a lot of time in the past looking at flaw remediation patterns. And what is what are the factors that lead to Better or Worse fixed times for developers and this time we said well, Let's let's even shift further to the left and let's talk about flaw introduction rather than looking at this big pile of of vulnerabilities that we've built up over time and how long it's taking to fix them. How can we actually prevent those from being introduced in the first place?

And what are those patterns look like? And then the data scientists just run with it and we can kind of go back and iterate with them as we find stuff like doing more into this part of the chart or something. So, yeah, it's it's really is it really is amazing?

Not only what you're able to glean from a data set of the size but also like How long it takes to like we start on this probably six months before it comes out because they're just so much there's so many so many questions. And let's face it. Data science wasn't a science right where would you know starting out with this Sunday really is now the data likes so let's jump into it though.

Chris what you know, I always like to know what what's the what's the top three? Yeah kind of findings from this year's report that our audience should be aware of yeah, well, I'll tell you one of those real surprising ones as we looked into that flaw introduction thing that I mentioned is that there is there's not a correlation between Application growth and the rate of flow introduction. So by that what I mean is like even though once you start developing and maintaining an application, it's growing the code base size is growing by about 40% year over year as you would expect you had new features and so on right but the number of flaws that you're introducing does not follow that same pattern.

It doesn't follow the pattern of the size of the code base. What actually happens is that when an application is new you start out the first time you scan it there's about a 30% chance you'll discover some accumulated flaws and then in that first, That first few months you see this drop in terms of flaw introduction teams are introducing less and less. Hopefully they're fixing some as well and then over the first year or year and a half or so it kind of plateaused.

There's this. Yeah. We're calling it in the document just like a honeymoon phase where the fly introduction is.

Just the rate of flow introduction is not really growing. It's kind of flat even though the application is getting bigger doing that time bigger and more complex. After that one and a half to two year mark, the flaw introduction starts to start Rise Again start rising at a gradual Pace, but it's weird.

They're totally disconnected from one another. And so what that means not really sure we're kind of like why is that happening? Certainly when a team went into application is new you have a lot of concentrated knowledge by the developers on that team like how the application works and all the moving parts and over time developers fall off the project.

They move on to something else. It gets more complicated you start building integration, you know everything so there's a lot of number a lot of factors that come into play there that might contribute to that honeymoon period and then and then that changing so that was one one big finding The second thing we try to do was figure out all right. Well, here's what the general shape of the curve looks like for flaw introduction.

How can we change the shape of that? Curve? How can we make that initial drop faster?

How can we make the plateau longer or that the the make that the instead of it increasing over time stay flat or decrease over time? What are the factors that we see teams doing that could influence that rate and so what it comes down to at least at least in the the items that we were able to kind of isolate as scan automation scan Cadence and frequency and developer security training. So these are all things where we when we looked at applications and we kind of separated out the applications that were for example scanning and in their pipeline, we saw that those had a reduced probability of increasing new flaw adding new flaws.

We saw for applications where developers that completed at least. Can interactive security trainings the rate and their volume of flaw introduction was lower? So at least things are additive right?

So you do one good practice. You get a good you get a good outcome from it. You do too you get even better.

I'm sorry. You scan every month, right you get an even better outcome. And so there's actually quantifiable metrics around all these things which you know, I'll let you know like listeners go read the report for that but these these stack on top of each other.

and the third thing that I thought was pretty interesting was we took a look at open source, which we've done before but we rather than looking at the whole universe of open sources out there. Which a number of reports do and that's fine. But to some degree a lot of it's irrelevant, right?

Because there's a lot of stuff on GitHub that maybe has some bad code in it or even malicious code, but it's actually never used by anybody. It's never used in like a major Enterprise application. So what we did was we Took the you know, 750,000 applications that in our data set and we looked at the GitHub repositories that those applications are actually using.

Right. So this is representative of real world. And then we started to look at some characteristics of those repositories because these are the ones that actually matter.

These are the ones that are in use and you know, protecting customer data and so on so we looked at things like how many maintainers does it have a lot of them have one maintainer? How often does it get updated have there been commits happening in the last month the last year and then the most surprising thing to me was we we looked at all of our applications and we said well what percentage of them? Rely on at least one of these fragile libraries in JavaScript.

It was 92% of applications. We're using at least one library that had a single developer. And they hadn't been maintained in over a year.

So not to say those libraries are bad, but it does point a little bit too like the fragility here because Right, right. You don't like what happens if that developer loses interest run over by a bus like how quickly are they going to respond when something happens and like do I really want to decide to take it down? Okay, right, which is which has happened and then half the internet just breaks.

So if you're developing a good business critical application that's built on tops and all top of all these different pieces of Open Source. You have to think about that in terms of your resiliency and fragility and that's something we want to expand on a lot more in the future. But this we barely had a chance to kind of get into it this time we'll tied to vulnerabilities will tie pics rates and things like that, but this open source is so much in the conversation now, to me the apply change security that we haven't paid attention to.

Supplies chain security has strictly been on. Are we using vulnerable components here somewhere and it's usually open source frankly. Right, right.

the whole open ssf around all of that, but I think once we kind of get our hands around that a little bit the next thing we're gonna need to look at is exactly what you've called it the fragility of the supply chain, right? Is that a single Source a single point of failure as we used to call it? Right?

Right, and if we if we don't have that as we don't have that supply of that is their alternative. What is the alternative and I I think robust application development you're going to need option B's and options, you know, you got to look at the fragility of your of your supply chain. Not just the vulnerability of your supply.

and I don't think we're there yet. I don't even I think that that might be a day two thing for supply chain security, right, but it's coming. Yeah, it's yeah, it reminds you that one, you know XKCD cartoon right where there's just a one little pillar and says like this this part of the overall Global software infrastructure is being held together by you know, one person in someone's basement in Nebraska, so I can't remember the caption but you get the idea, right?

Yeah. That's just you know, what? Well what what happens when you knock that knock that piece out?

Yeah something you have to think about. Yes security is absolutely just one one aspect of it. I mean tell the truth when you heard about the FFA thing this morning or the FAA thing this morning.

Yeah, I think was the security incident. I was just having this conversation and it's like maybe somebody was probably somebody is like tripped over a cable. You know somebody yeah, I I infrastructure.

I think if I remember yeah, remember one time another I think was another flight related one not a couple years ago. It was just like some just some piece of Hardware had just died. Because it was an old but yeah and all these yeah aviation industry between Southwest and now what happened with that.

Yeah, you know, a lot of technical debt is is coming up to the top. Someone's gonna have to pay this bill. Yeah, right.

Yeah. No, I I think it's a real real issue. So Chris for people who want to get the report where can we go?

com. We'll be right there on the front page a state of software security is a report and we'll be following this one up with industry kind of industry segment. Yeah, you usually do.

Yeah. Yeah. I mean it's long enough.

It's already over 50 pages. So we'll come back and we'll do a few page slice per industry and just kind of compare them against one another and because ultimately a lot of times that's what people want to know is not just like, how am I doing an absolute standpoint? But how am I how am I doing compared to my peers?

Right? We all want to know that so the peer benchmarking is as important. So we'll we'll get there but a lot of great stuff there's report we barely scratch the surface here.

Very cool go check it out. com Chris another great job man. You guys, you know, I know I know it's six months worth of stuff and I know that much work goes into this thing, but it's well worth it, right and thank you if you told this already I don't have to be the only one but thank you for what you do with it because I think we all benefit from it.

Thanks for helping us spread the word. I do the best we can man say hello, right the other Chris and all my other varicode friends you bet. I was actually out in San Francisco scouting Moscone Center and RSA.

Oh God too soon. Not too soon will be here before. You know it man.

Just too soon Allen. Through food. I will see you out there.

Alright. Take care saying from varicode here on Tech strung TV. We're gonna take a break.

We'll be right back.