The Year of AI-Powered Federated DevOps and Security Data | Predict 2024

Hello everybody, and wow, it's 2024. Welcome to Predict 2024, the the time is flying. Um, I'm gonna talk a little bit today about AI since it is sort of the topic, um, of the century, in my opinion.

Um, AI has been, so, we've been thinking about it for quite a long time. We've been exploring it, we've been discussing it, but now it's becoming a reality. Um, I know that some people are afraid of ai.

I know that there, there's messaging around how AI can be used for bad, but it also can be used for good and it also can be used in DevOps. And I'm gonna talk about how we can start using the data we generate and security tooling and DevOps tooling to create and AI powered federated DevOps system. So, hey DevOps folks, you're great.

You're awesome. Um, we have been working from a DevOps perspective for quite some time, um, to automate the process of moving code from a commit all the way through a production release and really automating it so that we don't have our hands in it every single time. We're not sitting there at a command line trying to do this work.

So great. We have achieved an amazing high level of suc of success in automation through the DevOps pipeline. So congratulations to all of you who have done that.

Security teams are also working behind the scenes to improve, um, the security of our software. And we have been working on this for quite some time. I think it's important to point out that SBOs have been around for the last decade.

Um, SPDX standards were, were developed in 2011. Cyclone DX was 2017. Um, SCA tools came out around the same time.

So tools that were really starting to look at the data around the, how our software is composed. Sonotype and Jfr both were formed and, uh, 2008. So yay CISO teams who have been pushing this technology, encouraging the use of it within their organizations, uh, and are continuing to, uh, build upon the hard work that's been done over the last 10 years.

So we have the data so far, so good. So what is the question? These, this information I pulled from, um, JFR swamp up because I found it to be insightful at, to say the least 45% of enterprises will experience a supply chain attack by 2025.

That's a big number. This one, I couldn't, I, I still can't believe 220 days to containment. Okay?

We've done a great job with the security tooling. We've done a great job with, uh, with DevOps, but it's still taking us this long, 227 days to containment, and 65 to 80% of companies are looking for more visibility into DevOps and security logs. That should give you a key, uh, a clue, I should say, as to why we may need to do more.

So then what is the problem? We've done so, so much work. We've done so much automation, there's scan scanning tools, but we can't seem to solve the problem.

We are still looking at, uh, attacks. So if both DevOps and security teams are producing the data, why do we continue to have the issue? Why is it, does it take over 200 days to contain a, uh, a potential hack or vulnerability?

You know, why do we continue to struggle with knowing where open source packages are running across our environment? Why did we have to ask the question, where is log four J running? That is what we should be asking ourselves on a regular basis?

And I do, uh, I've asked this question now for quite some time. Why was it that it took so long for so many companies to address Log four J? We have all this automation, we have the data.

Why aren't we using it? Mainly because we have created systems where the data is fragmented. Um, SCA scanning results, SOM results, SOM deployment data, the open source inventory, that kind of, those, those kind of key data elements, kind of the metadata that makes up the whole DevOps and security data pool, um, is really fragmented.

We all have our own tools. We all have our own, um, I shouldn't say we, but the IT community in general. We purchase tools that have their own dashboards, their own data stores, but they only look at a particular aspect of it.

SCA tools, look at that, that aspect. SBO m tools provide some deeper insights. Um, our deployment tools tell us where something has a container is, is running, for example.

But if, if the data is fragmented, is it that useful? It's useful for specific siloed, uh, analysis, but it may not be useful for a broader look at an organization's security profile. To be quite honest.

And without having, um, a, a central place to start massaging this data, we don't really have an option to build some AI systems. If we think about chat, GPT, which I love and I use all the time, thank you. You know, for the teams who pull, pulled that together, they had a pretty big, uh, source of data.

They had a, a large language model to to, to build that from. Um, even things like copilot, there's literally millions of open source repos and GitHub that they could look at source code and start building upon. Where is that data for DevOps?

And there's a lot of it. Recently, the, the Jenkins usage report, uh, came out and we're talking somewhere in the ballpark of 70 to 90 million workflows that, uh, that CloudBees still keeps stats on 70 to 90 million workflows a month. There's a lot of data and a lot of insights and a lot of intelligence that could be derived from that data.

If we started using it and pulling it together. Fragmented data, it does little to push us into an AI powered DevOps process. So let's think about why and how we did this, how we broke ourselves in terms of how, why the data is fragmented.

So everybody had every, every container, every object that you're sending through your, your, your, your workflow, whether you're using a a Jenkins or a Circle CI or something else is built, is based on one particular component, one container. Now, that container may be monolithic, and if it is, you have a little better data, but it also could be fragmented even deeper as we build out, uh, decomposed, um, component-based microservice applications. So when we have, we've created this workflow process in the DevOps space that manages specific components through the pipeline.

And when we do that, every pipeline could be doing something a little different. Not every company has standardized pipelines to the point that every single pipeline is doing exactly the same thing. And in fact, you really can't, you know, a a, a visual studio versus a Python pipeline's gonna look very different and have different tooling.

And every single one of these DevOps and security tools, they track their specific data before that particular component. And then what do we do? We sometimes store the critical logs in the build directories.

So an SBO m you generate an sbo m if you do it after, not at the point at the time that you'd, you create your container, but after the fact, you're probably gonna have that, uh, SBO log generated results, stored a build directory. And then again, cloud native, uh, architecture is decoupled. So let's say you have a, uh, microservice application with, let's just make it easy, 25, 25 microservices.

You have 25 SBOs. You have 25 places where SCA data may be stored. You have 25 places where the DA the deployment data is, is stored.

And then what we've done is we've lost the logical application. So to put together an SBO m for an application and a microservices vir, it means consolidating the, the information 4 25 from 25 SBOs. And that's just to produce one report.

So the data is critical. We have to start realizing that data in DevOps. Um, we've always relied on to solve particular problems, but we've never thought about it from a higher federated level where we can start creating, um, some threat models, for example.

Uh, or, and I'll talk a little bit about that, but just think about what we could do if the data was more, um, federated. So consolidating the data is the first step towards building some kind of a predictive AI system for DevOps and security teams. And wouldn't that be an amazing thing to do, is to consolidate that data and, and pull it into a single location so we can start acting upon it.

So AI powered, and I like to call it software supply chain management. Um, once we start federating the information, we really have to start thinking about the fact that what we're doing is we're starting to track all the pieces that make up our software, whether it be monolithic or whether it be decoupled. We're starting to track the pieces of the software and the future of AI and Dev SecOps.

If we pull the two together, really depends on the ability to create historical trends. That's where these models can be, uh, start coming from. So by federating DevOps and security data, um, statistical methods and al algorithms, algorithms can then be trained to make classifications or predictions to uncover really key insights in the DevSecOps pipeline.

This is something that we have never really talked about because we're all so siloed. We have siloed teams, we have feature teams now we have different DevOps tools, um, we have different security tools, and in many cases, the security tools are not even running under our, our, our DevOps pipeline. So as a group, as a community of DevOps professionals, what we have to start thinking about is how to pull our information together so we become more powerful as a whole.

And, and data is power. So let's just think about what we might be able to do with this kind of data, um, and what problems we could solve from a DevSecOps perspective. Let's just take this first use case, the question, are our pipelines compliant with security?

Imagine if we had 90 million a month, uh, workflows being pushed that we were collecting data on. I know that's a big number, but even if we had 30,000, we could start looking at what the pipelines are doing, and we could start determining how compliant they are from a security perspective. We could see if they're running, if they're generating SBOs, we could see if they're, they're gathering SCA information, we could see if they're doing a better job of tracking the pieces of the software that we deliver to end users.

So for example, a Java developer creates a basic pipeline for their service. It's missing code signing and SBOs, an AI powered DevSecOps system could identify the weak pipelines, provide an updated pipeline, and hand it off to via a pr, a, a pull request with all the security, uh, model that our organ, the organization, uh, is requesting. When we think about the challenge alone of updating DevOps pipelines, just to add SBO M generation, it's massive.

We have, there are millions, literally millions of Jenkins pipelines alone, not even looking at the other, uh, all the other tools, GitHub actions, for example. How do we update all these quickly? Because we are behind, um, the eight ball.

We really do need to solve the security problem in the, in the DevOps pipeline. We need to start consuming these, this, uh, these tools at a minimum. Well, if we had that data, if we were tracking it, we could do that.

We could generate those, those workflows. And we all know that a workflow was not horribly complicated to generate. Um, it, it requires a standard model and some, some, uh, basically some changes in environmental settings and, and things like that.

So if we had that ability to look at what everybody was doing from a central perspective, we could potentially generate better, uh, DevOps pipelines. Let's look at use case two. Um, when there is a threat, what do we do?

That is the question most a lot of people are asking. They're putting together policies and procedures to address the threat. But if you go and you look at your CVEs on a regular basis, sometimes you have any idea if the CVE is even impactful.

Now, there's some tools that are coming along that are doing a good job of telling you if, if it's reachable, if in your system how the pieces and parts are put together, if that vulnerability is reachable, that data would be very useful in a central location where we know where that particular package is running in all our production environments. So automating emergency response and then generating threat models based on what we know is another way that AI powered DevOps could make our world better in what we do. So the question is, how do we know if we're, we've been impacted by a CVE?

A vulnerability is identified, but no one knows about it happens. The AI powered model would notify producers and consumers of the threat impact and the mitigation steps. Wow, that is the kind of discussions we should be having from a DevOps perspective and how we can bring in security information, how we can leverage it, and how we can start producing AI of responses to problems.

On this one, we wouldn't even have to have this, this is not a generative ai uh, discussion. It's, it's really machine learning and the data that's we already have could produce this. We just don't pull it into one place.

It's fragmented all over the place. All right, let's do an easier, an easy one. Um, case three, AI software composition policies.

We can automate policies on safe software consumption. So this is a, this is a problem I have seen more than once. Um, the question would be, is my project meeting policy requirements on dependency usage both upstream and downstream?

Now, I just read an article that said that the top four cyber, uh, risk in 2024 includes. Number two is software vulnerabilities running on an older version of software. So let's say a project is using two JSO pars, annoyingly, we both got stuck into the container.

One is secure and the other is not. The AI model would determine that the unsecured parser can be replaced with a secured one and to meet the policies and do it for us. Again, we have within the software supply chain, there are so many moving parts that it's extremely hard for developers or security pros to stop them from coming through.

But we can leverage data, we can leverage historical models to do the job. We can't do it without the data. So yes, I'm gonna talk about an open source project that I would love for everyone to take a look at and consider because it has the ability to federate that information.

So let's just think about AI powered DevSecOps. What, what it would look like. So if we take that fragmented supply chain security and DevOps metadata collected from the pipeline for every single time we release an object now in a de decoupled environment that should be happening all day long in a microservices, uh, or in a monolith environment, we might be doing that all day long too.

We're just building and releasing a bigger, a bigger, uh, more objects in the container, let's just put it that way. But once we do that, we can start tracking the open, open source usage and inventory. We can look from the beginning of the SBO M and see this is the, these are the package that are included in this, and we can follow it all the way through to where it's being deployed.

So if there's a vulnerability in that, that s in that SBO or in that, that container that the SBO m has exposed to us, but we don't know about it 'cause it's getting ran every day, these vulnerability checks, once we do find it, we have the vulnerability blast radius. We know where the object is running in a particular environment. It could create organizational risk scores.

Um, so it could take a look at everything that's running in our, at our, let's just say production environments and determine the CVE risk level of the entire production environment versus just one single object. And from that, we could see what are the highest risk, uh, vulnerabilities that we need to work on, which is a question that everybody is asking. We get it in the open source community.

We have CVEs, do we need to worry about it? But an organization is bigger than an open source project. If you are a, a large enterprise, you have lots of changes moving through, and you have CVEs popping up all day long, what is our organizational risk score?

And what does the runtime look like? Is it safe? How, how, how do we audit the runtime as opposed to one simple container?

Then there's this thing about sharing SBO m information. Um, if you are writing a feature compo, if you're a on a feature team and you're writing a particular microservice or component, how do you share that SBO m information with every application that wants to consume your feature? Let's just make it an easy feature.

You're, you're on the team that writes this, the login routine for everybody else to use. You need to, you need to provide that SBO m to other indi, other comp, other consuming, um, teams. So what about having an AI system that just pulls that information together and your, your SBO M is already aggregated and federated for you.

And then the most important thing, which is historical versions and trends. Every company, every environment, whether it be test or production, every team has different historical trends. So that means an organ, a large enterprise, if they have a hundred application teams, they can have a hundred different problem sets that they're having to deal with each one of those teams, having their own trends and historical data.

Aggregating that up to the organizational level allows for threat modeling because then you can begin seeing which teams are performing quite well, which may not be doing so well. And you can start understanding a threat to one particular aspect. And we can't forget about configuration metadata.

Um, there was recently, uh, a problem with Nginx, um, and Nginx reported that they, it was still a zero day, uh, vulnerability, but Nginx gave it a solution and it was a change in the way Nginx was configured. So let's not forget that while open source packages are important and they create vulnerabilities, there is con potential vulnerabilities in our configuration data. So that information should be a part of this, this common data store.

Now, the big dream, um, I'm a dreamer. Anybody who's ever worked with me knows that I'm I, I'm I'm a dreamer, um, would be that if we had open source communities using a center repository for reporting on their packages and what their packages consume, because in that way, we would have a, we could have producers and we could have consumers, consumers with private domains and producers with public domains. In those public domains.

We're gathering in all kinds of information about what we, what we as open source organizations are providing, whether it be from the Linux Foundation or for, for from from Git on the private side, they could consume that information. So highly reg, highly regulated industries like USAA or Northrop Grumman, they could pull that information easily and integrate it into what they're working on. So now we're taking all the pieces of the software supply chain from the tiniest pieces all the way up to the enterprise level and pulling that information together so that we can have exposed OSS within all of our, uh, with across the organization with artifacts and vulnerabilities, with inventories of where the open source is deployed with aggregated SBOs across both open source packages and, and the the private enterprise.

And the private enterprise. They could start tracking this historical information and doing their trend modeling. Now, a roadmap for, uh, for Orillia, and what we're beginning to work on is this predictive analysis with machine learning, continuous watch and rapid response system and automated policies on safe software consumption.

Those are the three kind of, you know, if we, if we look at Predict for ortel, those are the three top areas that the Ortel community would like to start addressing. We really do wanna build ai, uh, the a data, the large data model, and the large, uh, data lake that's needed for doing real AI around DevOps. It's been gaining momentum.

Um, it's an interesting project. It's incubating at this continuous delivery foundation. It was put in the Continuous Delivery Foundation in 20, I believe it was been the December of 2019, early 2020, um, before the open SSF was around.

So while it addresses security, it has a focus around the DevOps pipeline. So it is sitting on the, it's, it's sitting under the DevOps pipeline and watching what's happening and consuming and pulling up that data to aggregate it and federate it on LinkedIn. We've got about 800 followers.

We have about 120 GitHub, uh, forks and 275 stars. I don't know if anybody looks at Stars anymore, but they're out there. Um, and about 222, uh, in, in 2022, we did about 200 call requests.

All in all, we have about 30,000 person hours of contributed code to, so we're pretty excited about its momentum, and here's what we've gotten done. You should never have to ask, where is Log per j running? Because Orillia now can tell you where it's installed.

That is the key, right? You have, it's not that who consumed it, it is where is it running? Who, where, who, what environments are impacted by this particular problem with Log per she or Log four J TIUs has got those reports working aggregation of the data.

What OSS packages do we depend that depend on? And where are the vulnerabilities? So whether you're a component or an environment or an an application that's a collection of microservices or components, or the ARTERIUS team has provided a way for you to see all the vulnerabilities for that particular object that you're looking at, not just a single container.

And it does the, the SBO m in the same way. So yes, please, we would love to have you sign up to help us build an AI powered open source DevSecOps solution. Um, deploy hub.

Uh, the company I work for is, uh, it hosts a free SaaS version of it, so you can play on it and give us some feedback, and you can find it right here at Deploy hub free SaaS software supply chain management. And we'd love to have your, uh, your, your opinions, your feedback, and have you join the Discord channel. Let the team know where they need to go with it.

io. You can go to our LinkedIn page where it's Orillia dash open source, or you can go to XI should have an X there, Orillia oss. Or you can reach me at Tracy Atlo hub.

And yes, that is my real phone number. Call me if you're interested. And before I forget, let me just tell you who I am.

I, my name again is Tracy Reagan. I am the co-founder of a company called Deploy Hub, who had this idea several years ago to create an open source project that could gather DevOps and security data and track the deployment of it across all the pipelines. I am part of the Continuous Delivery Foundation's Technology Oversight Committee.

I am not new to open source. I helped, uh, start the Eclipse Foundation years and years ago. I was the Lone Woman for about three years on the board.

I have also been on the board of the Open Source Security Foundation, uh, and I love working with the Linux Foundation on all they do. So thank you today. Uh, it's been a, a pleasure.

I know that there's a lot of things on your plate, so thank you for listening to my, um, my discussion and my passion about building DevOps and security data in one place. So it's not a matter of, we got the data so good so far. So what?

Let's make it actionable and let's start solving these really critical security problems together. Thank you. I.