Enhancing Cybersecurity in DevSecOps with Ortelius with Kate Scarcella and Tracy Ragan | Open Source Summit NA 2025

Hey everybody. We're back at the Open Source Summit in Denver with Tracy Reagan and Kate Scarcella, and we're talking about Artelia, which is a project they have underway to kind of help us find neuro vulnerabilities and remediate them. And there's a lot of work to be done within DevSecOps flows and continuous integration CDs platforms.

And so, if you don't mind, let's start with you, but give us an update on this project. 'cause I think we talk about it all the time on Textron Gang, but I'm not sure everybody knows what it is exactly. And I also know you wrote an article about LLMs and security, so put it all together for me.

Right. So I came back from a sabbatical, and to me it was Groundhog's Day, meaning I stuffed into exactly what I left off. And although it was just bigger and faster and I thought we are in serious trouble.

So it was from there that I started thinking about what was so cool is this groove called orus. And the reason why is when I started thinking about the Renaissance and how can we make not only cybersecurity, um, consumable because right now it's not, but also make it interesting for people, make it fun, make it, um, just really fascinating. 0.

0 that was kicked off in 2020. 0. So how do I bring this all together?

Well, this really cool group called orus, which from Abraham orus was one of the first, um, you know, scientists really to bring all these different other, like a geographer and a cartographer and everything else together and really did sharing, which from a cybersecurity perspective, we are horrible at sharing. We do the antithesis of sharing. Um, but the bad guys don't, the threat actors, I mean, they share, there's like, Hey, I have this malware, you know, $3,000, 24 by seven support for us, it's like we, we, we sort of tighten our reins.

We make it really as a cybersecurity architect, we make it super hard for, um, for people to do their jobs and let's, we wanna change that up. We talk about aelius, but we never explain what it is to people. I talk about Aelius, we're gonna send that to you as like one of the things we should talk about in our future Games meetings.

Yes, absolutely. So explain for the folks watching this exactly. What is this thing?

So Arterius is an evidence store that sits on top of the CICD pipeline. And we gather two real, we gather a lot of different types of data, but two very specific pieces that help us solve the problem that Kate's referring to. We, we take an sbo, a software bill of material report.

We version it as soon as it gets created. If it's not there, we generate one using sift. Then we go and we watch the deployment.

And when the deployment occurs, we're gonna pull information from the log files and say, these are the endpoints it's being installed to with that information. Then we constantly synchronize with OSV Dev and we say, Hey developer, you may have found vulnerabilities pre-deployment, but now you have them post-deployment. So we're identifying the threat landscape of post-deployment vulnerabilities.

And what we're beginning to work on is going from that reporting to auto remediation. Where what we do is we say we know exactly where the repo is. 'cause that's one of the things we gather.

And we can have one vulnerability that's impacting 500 repos, which means there's probably 500 package managers that have to have an update to the pinning of the version of the package that they need to consume to remediate. We wanna go out and find all of those packages, remediate them and create a pull request for critical and high risk vulnerabilities running post-deployment. Who is actually approving the remediation in that model.

Because there's been this long running debate and, um, you know, the security people are like, we could just automate the fix ourselves. And when developers are like, no, you're gonna break my app and we are in this infinite loop. So is it the developers that are gonna go back and autumn remediate the situation or who's gonna step up and kind of and own it be responsible?

Yeah, We were just talking about that actually. Well, right now the developers go and open up every single one of those package managers and eventually they have to fix it and then they create a pull request. Yeah.

So we just wanna get rid of that toil. And the other problem is, is noise. We have a lot of noise right now coming through in, in vulnerabilities like tools like depend upon, it'll tell you there's new vulnerabilities.

It may have been in a SBO that you've had and it's gonna report on all of them trucks. There's gonna be 40,000 of those a hundred vulnerabilities a day. And nobody really knows if it's actually running or not.

It's just in an SBO that's in your repo, it may be deployed. So We wanna cut the noise down and just focus on those high risk. So explain to me this little bit though, 'cause you were talking about it and saying, well, the developer may have discovered these vulnerabilities, but then they showed up again in a repo or later in the, in the lifecycle.

Why does that happen? And how do I kinda get in front of that a little bit so that when I do ship left, I'm actually fixing things and it goes all the way through the lifecycle Because they're discovered new every day? Yeah.

And the same exact stuff that you just scan. So it's a difference between dynamics, uh, detection and point in time detection. When it's in the, when it's in the CICD pipeline, you do a, you do a static analysis and oftentimes you'll do a dynamic analysis at testing.

So it's all good. Everything's perfect with the world Tomorrow you deploy today and tomorrow you found that something now new was discovered in that particular package that you just deployed. That's what we wanna focus on because that's where it matters most.

That's where the threat is the most, is when it's actually running and we don't know about it. So These are all new quote unquote zero day vulnerabilities. They Are zero day vulnerabilities.

Got it. But they have a but, but we're gonna go look for the mitigation and we're gonna either create a pull request to push it through, or we're gonna create an issue that says this has got, uh, breaking changes. So you're gonna have to fix your code.

So we'll create an issue to say it's here, it's critical. It's actually running in your environment. Either you have a fixed POM file or you have an issue that says these are the mitigation steps you have to take.

And that comes back to the issue where developers are always telling security people, you know, no, that's not internet facing, or it's not in my code, or it's not running. And then they stop listening to the security people 'cause they're like, It's just too much noise. I don't think that they purposely wanna stop listening.

There's just so many. And you know, most of the policies now say you should have zero vulnerabilities. We're not gonna, we're not gonna achieve it.

Let me come back to you when you're talking about LLMs and I cannot help but wonder if we're, are we as just incapable of learning? 'cause it seems like with LLMs we're making all the same mistakes over again that we made earlier. We are, I mean, the exact same vulnerabilities, right?

I mean, we're poisoning. We can po poison the, the models just like we can poison, uh, the database. And, and we are, and that's what is so concerning.

What I see, and I feel like we really need to, to look at the tools that we have, which is something that we're doing within our sig, um, is actually looking at this, you know, which tools are really showing the vulnerabilities that are there. And at the same time, I almost believe that we need to, to take a pause. And I know that that seems like absolutely crazy.

Like how do we take a pause? But I was really thinking about like, what type of analogy would be good about this? And right now I feel like if we're looking at baseball and a heavy hitter comes to the plate, we are all in the infield.

And you know, and, and we're not even like we're chasing the ball. We should be able to position ourselves when we see a heavy hitter and we may not be able to catch the ball, but we should be able to understand that it's coming. And I would like us to fix that.

And I think that that's one of the things that I'm doing with, with Tracy and the people at ORs, uh, is really looking at trying to get ahead of this threat. Because I mean, they're taking LLMs and making them literally these attack vectors. Can you imagine that?

You know, this, you know, and, And the problem is that that leads to an AI agent and now I can take over the entire AI agent, which is a workflow, right? It's not just like a small little Vulnerability and how do you feel that loop? It's really hard to kill the loop.

Like if it gets in the loop, I mean, to to stop it, it's a hard thing, right? So, you know, we need to really think about this as a person who's already been through this. It's, um, we're almost repeating the same things.

We're we're getting some, some differences. Like when we're thinking about how we're, we're looking at getting rid of the noise so important because I feel like we are chasing, we're just chasing we're we're not being strategic. And I think with this, we're, we're taking a step back saying, okay, let's not chase everything.

Let's go after what's critical and, and address it so that you know that we can be more secure. Right? Do we have it in our heads that the, the thing that the data scientist is building is somehow different than what a software developer does.

And as such, we're creating like yet another gap in our little happy ecosystem. But it actually is the same. It's code at the end of the day.

It is. And it, you look at the threats, it's the same threats that we've had. It's like literally the exact same, same vulnerabilities, same back doors.

You know, we had back doors back, you know, 20 years ago. We still have back doors, you know, so it really is it, to me it is the same thing thing. And I, I really am hoping that we can, you know, first number one, let's invite people to our sig at, you know, that we have happening.

Um, and she's Talking about the CICD cybersecurity sig, not the ALIA open source project. Yes. Which is within the Continuous Delivery Foundation.

But they both work nicely together. They work, yes they do. Yeah.

In tandem. I mean it's really, it, it's So, so follow that up. How do I actually get involved with that?

Because more often than not, I talk to people all the time and they're like, yeah, I'd be interested. But then they have no idea where to begin. And then, you know, they roll over and go back to bed.

So Yeah, we have a, we have a page, um, just Go out, check in the link, go out to the cd CD foundation. I go to the SIGs and you'll see the ci, cyber security sig and you can join the list, which means you'll get all the meeting notifications and eventually we will release the website. We are, the team is working now on the website.

And what we're doing is we're going through all of the secure software development framework, S two DF and we're associating every, we're looking through every task. This is a hard, it's a, it's actually a hard project to do. You go through, we're looking through every task and we're identifying within those tasks if it can be solved through the DevOps pipeline, using an an open source tool.

That's what we're working on. Can we maybe use AI to help cure our problems a little bit here? I mean Well that's what we're, that's what certainly we would love to do.

And you know, on that other question you had, there are now AI SBOs, so AI SBOs, we, we ortel us will need to start consuming those as well. 'cause it has what I think is a very important part of the puzzle, which is the version of the LLM because it's the version of the LLM that's gonna allow us to detect that that version has a vulnerability in it in the same way as an open source package. Right.

So it's all parts is parts. Yeah, parts is parts. And what we do is we assemble the parts together.

We create what Steve likes to call a digital twin in a, in a database environment so that we can, we can, we can report on what's happening in real time on those end points. 0 human machine teaming, we've come a far away, we've come far, we've, we've advanced, we've done well Sure. Do we still have some of the same things happening?

We do, but we're learning, you know, and, and we understand. We literally understand what we see. We've seen it before.

Now I think instead of chasing this, we can step back and say, okay, we've seen this before. We know what's coming now let's actually do this. Right?

So set up a play, we're gonna set up a play for it. So depending on the day the glass is either half full or half empty. Yeah.

And in the sense that it's half empty, it's, oh my God, there's a lot more at risk here for LLMs on the half full side. More people seem to be talking about this and we're having these conversations earlier. Maybe It's, it's phenomenal, right?

We're not putting our heads in the sand. We say, well gosh, you know what? We have the same threats we do, but we have faced these threats before we know them.

So thus we can, we can combat. We, we have the, we have to, you know, we just have to implement them. Smart.

And we also have to take things off. I, I mean, you know, we, everything is not on the developer. Right?

I mean, it shouldn't be, not everything should be on, on, on the, the data scientists. We need to be more collaborative, which is also one of the things that we're trying to do with our Group. I mean, we can't just shift everything left and throw in the face.

No. Because they can't do what they, they're not psychic. Mm-hmm.

This is the problem. Developers are not psychic. So they don't know the vulnerabilities that haven't been reported yet.

This is true. And it looks like the bad guys are spending more time trying to crack these lms. Right.

A lot more research. There's a lot more vulnerabilities popping up. Yeah.

That are new. And differentness somebody talking about a new novel technique almost every other day, 40,000 is predicted for this year alone. Wow.

But I don't feel like they are new and novel. I mean, I almost feel like they're doing the same thing. It's like, it's really like rinse and repeat.

And so we understand it now. We just have to do it right this time. I don't call it new and novel.

No one will pay attention. They'll be like, Hey, I found the same old problem again in an LLM. Yeah.

Or in the same package. Just in a done in a different way with machines Of humans. Yeah.

Well, let me ask you this though. It still feels like it's gonna take some sort of crisis involving an LLM in a security breach and some event to get everybody kinda squirrelly focused on this. And is that just the nature of the game?

And You know, we, you know what we log for. Jay woke us up and we all went back to sleep. Right.

We're gonna still work on it though. And you know, we, it, it just needs to be automated. Some of this stuff just has to be automated through the DevOps pipeline.

It, it, we have to have a better conversation between security and DevOps is why I'm so glad that she's on the, the Orillia team because she's a security professional talking to a bunch of DevOps geeks. But we need to automate it so that it, when we do find ways to fix these pieces, we're making sure that the correct tooling is in the DevOps pipeline. So it's DevSecOps and, you know, I'm all, I'm all about, you know, disruption and I totally believe that we are in a point in time that we need to disrupt how we do CICD so that we can more easily add SBOs, for example.

Uh, less than 50% of companies even have an SBO generation step, which means that they're, we are way far behind the eight ball when it comes to being able to actively find these issues that are running in production if they don't even have an S bomb. So we got, we have a long way to go. And I believe that the way we write CICD pipelines with scripts and you know how much I hate scripts mm-hmm.

Totally stops us from evolving the DevOps pipeline to be a DevSecOps pipeline. She's gonna have a new nickname called Script Killer, but that's, they, uh, that's an entirely different show. Anyway, guys, thanks for coming by.

Hey, we've been talking about bridging the divide between security and application development for as long as I can remember. And you know what? It's happening right here.

Live in front of your eyes. We'll be back in a minute.