Santiago Torres Arias on in-toto’s CNCF Graduation

Hey guys, thanks for the tour. We're here with Santiago Torres, who is a maintainer of the open source in Toto security framework, which just graduated from the CNCF. And I'm gonna let Santiago explain exactly what that is because, well, there's a lot of security frameworks these days, but then again, you can never have too much security, I guess.

So Santiago, welcome to show. My pleasure. Um, yeah, happy to explain, uh, anything about in Toto.

Yeah, so what exactly does this framework do and where does it fit in the spectrum of things because, well, CNC F's got a broad reach these days that goes well beyond Kubernetes. Yeah. So, uh, I just say there, there's never enough security and we even on finding ways in which things can go wrong, uh, in total is really a small, uh, I wouldn't say small, but a very specific, uh, project.

Uh, when it comes to security, it's trying to target a subset of problems that usually have a very high impact on, on the real world. I'm talking about software supply chain attacks. Uh, these attacks are usually somebody poisons the well or the water supply of your open source project or your, or your, uh, closed source project for that matter.

Uh, the way that I usually explain it to people is imagine that somebody breaks into, I don't know, the distribution of your containers breaks into Docker hub and introduces malware, and then you download those, uh, containers, you use them in your infrastructure and or they take over your cloud. Now, the, the real reason why this happens is that you got legitimate software, quote unquote, from a legitimate source, and you did what you do every day. Uh, a lot of the times you don't even realize that, that something has changed.

And when you put them in your infrastructure is when you're actually, uh, getting hacked. The in total project is trying to provide visibility and insurance that nobody who has been working on producing those containers for you. IE uh, working on your software supply chain ha, is able to poison your cloud.

I don't know if this makes sense or was I to, uh, I Get the basic idea, but how do we go about doing that? Because so often we hear about things like container, gelb, bricks, and a lot of software. There's ways to get from the software to the infrastructure.

So how do we prevent that? Yeah, so I think the, the word supply chain is very useful in understanding how in total works, and I'm not talking about only software supply chains, uh, I'm talking about like supply chains in general. When you get say a can of, uh, I don't know of juice, right?

You know, that there's things like a lot number, uh, tamperproof seal around the bottle. There is probably an expiration date. If it's say even medicine or something like that, sometimes they have a hologram to know that this is not afe product.

Uh, and all of those things are used them, and we use them naturally when we're working with physical objects to say, well, do I trust this thing? Am I actually going to drink from this bottle of juice in total? Is that, but for containers, the idea is that as the containers or really any software is being produced and created and, uh, modified and updated and, uh, analyzed and scanned and all these things that we do before we actually put them in the cloud, we want to know exactly what happened to them and that nobody did anything outside of the ordinary so that when we opened a, a bottle of juice, uh, which is our container, we actually know that every everybody did what they were supposed to do, that nothing was missing, and that nobody, uh, put anything malicious in, in our software.

It almost sounds like there's an audit trail for the way the artifact is built, and I can see what happened when and made sure that there wasn't anything unusual. Exactly. And you, you got it exactly right.

That's, that's the, that's the end goal with this audit trail, then you can make sure that everything is, is happening to the t And and you also said, it, it, it also allows you to think a little bit more proactively as well, right? What are the things that I'm doing? Are they the best practices?

Is everybody following them? Maybe, maybe it's not a fully security problem if something happens, but, uh, if something was skipped or missed or not done according to the expected sort of what practices, then you can go back and find exactly where did it happened and who didn't do it. Mm-hmm.

How does it know when that happened? Is there an agent or is there something recording something or how does that all occur? That is, that is where it gets a little, uh, tricky.

But, uh, going back to the metaphor, and I'll tell you the, the very explicit way that it works, uh, in, in a minute, but it's, it's so much different from us real supply chain, right? Imagine somebody working at the, at the Ford, uh, factory in the 1940s or something. They, everybody does a little thing to PR to create a product.

They usually cl usually clock in into the quote unquote factory, and then maybe they rubber stamp every single piece that they actually, uh, worked on so that you know that that step was done on the entire project. Or really for software, we cannot clog into the virtual factory and do this. But instead, what you can do is you can produce a little piece of information called the Software Attest Station.

This software attestation tells you who did what and to what from those, as you said, you can build an audit trail. It's, it's got a bunch of like cryptographic, uh, stuff in it. But really at the core of it is, what did I take in, say I took some source code.

What did I run on it? Say I run a compiler and what did I produce? Say I run, I created a binary.

Uh, with this, we can build effectively an audit trail that lets us walk all the way to the beginning of the, even the design process or, or the requirements if you want. So is this kind of taking the whole concept of a software bill of materials deeper into the software supply chain so I can see what's happening and um, maybe if I want to dive in deeper from my SBO MI can Yes. Uh, actually they, these two technologies work very well together.

You may have, uh, her the metaphor for assault bill of materials as like labeling, right? It tells you what materials are in this, or if there's, I dunno, peanuts on my granola bar or something like this. Uh, you can, you can imagine all of this other steps being in total allowing you to have both more confidence about what's, what's on this software bill of materials, but also, uh, a way to produce the software bill of materials as well.

Right? If you know somebody's using peanuts, again, sorry for all of the metaphors to produce the granola bar, and then the software bill of materials doesn't say that there's peanuts here, then you have a discrepancy and uh, and something may be a little bit off. Mm-hmm.

So in some ways it can verify the sbo 'cause a lot of folks are kind of dubious about how does that SBO actually know what's in there? Because it's basically a bunch of, it's a list of ingredients, but anybody can come up with a list, right? Yes.

Uh, and that, that's another thing that I think in total really does bring to the table that helps, uh, SBOs be more trustworthy. You can, you can play a, like a liar, liar type of game, right? If you have a software at the station that says, here's this, uh, I did this thing, and then you don't see it on the software real materials, then well, what gets right?

Mm-hmm. So is this all gonna get embedded into our DevSecOps workflows or how will it manifest? Well, the good news, and I think this is partly why we, uh, we're celebrating graduation for in total, is that it really is already embedded in a lot of places that you use every day in terms of DevSecOps.

The way that, uh, you can use in total is either you just flip a switch on, say GitHub actions You can produce at the stage today by, by using the GitHub Actions Runner or GitLab or really pretty much any major CI platform. Or you can, uh, check if your tool supports it, say vulnerability scanners already do it. Uh, there is tooling from the total project that can wrap existing operations.

Uh, witness is is probably the, the one that everybody, uh, would start with that. You can tell it, well, I'm, I'm already doing, I don't know, uh, tagging releases on, on Git. Then you run Witness and it produces and software at the station saying, who tagged this?

How did they do it? And what, what is it that they actually did? That kind of brings me to the issue of the day.

We're all obsessed with software supply chain security these days and for good reason. But, and I feel like we have plenty of tools, but it has, it really just become more of a cultural issue and we gotta change the way people think. 'cause every time I turn around, somebody seems to have a new tool for security, and so what are we gonna do to solve this issue?

Yeah. So wearing my academic hat hat for a little bit, um, I think the software supply chain metaphor is useful because it, it helps us understand how things are made, right? Uh, and software supply chains in particular are relatively new.

So a lot of things are in flux, and a lot of things are, are moving around and we're trying to figure out what is the best way to, to approach things. Uh, one fundamental design principle of in total is that it's not meant to change anything that you do. Uh, the only real thing that changes is you want to include in total into your flow to report what you're doing rather than changing what you're doing, uh, and make that secure.

Does this make sense? Mm-hmm. So let's say in the future that the people creating the software are not people at all, but their AI agents.

Can I extend the framework to keep track of what AI agents we're up to and versus what humans are up to and, you know, are they essentially the same thing as far as the framework is concerned? Or are they slightly different? They're exactly the same thing as far as the framework is concerned.

Um, a very interesting sort of thought, thought exercise that we did when we're designing in total early on was can we attest to literally anything? Could we attest for, I don't know, brain wakes about or intent to commit the code? Uh, and it turns out that you can, right?

Something that we're doing, uh, in my lab right now is can code agents attest to what they're producing for you? Right? If you have Codex and Codex is producing source code, then would you, you would want to know which code is actually being produced by Codex, not because it's better or worse than human code, but rather if something happens, you know exactly where those lines came from.

Which model did them, what was the context? What, what time was it? Was it in a grumpy mood?

I, I don't know, like, uh, uh, agents are a little bit tricky to reason about, but, but you, you can't use this information to better understand what is your security posture. So what is it, do you think that, um, that people who get this and do it right, what do they understand going in that you wish maybe more people would think about upfront before they kind of started trying to secure their software supply chains? 'cause I got a feeling that, you know, there, there's a series of steps, but I'm not sure everybody knows which one comes first.

That's a, that's a very good question. I, so I will say in total is a project that started 10 years ago. It's turning 10 years, uh, this year.

And, uh, as time has passed, we have found that a lot of the people that integrated, they usually think that the problem is running the tool or setting up, uh, setting up the runners or collecting the attestations and all of that is actually secondary. Really, really the challenge is what does a, a good supply chain look like? And what is my expectations?

What is my policy? Um, when people start integrating in toto, a lot of the times they look into what they're doing and then they realize there's a lot of steps that people are doing that they don't talk about or things that people do in order to make things work, but they don't report them. And those are never encoded into what are the, the practices that, uh, an organization is doing at that time.

So a lot of the times they set up all of this, uh, sort of like collection endpoint for the software at the stations, and then they start looking at what they collected and they go like, oh, gee, uh, we didn't know we're doing all of these things. Now we need to review, uh, what does this mean for our software supply chain? I would say that as a general piece of advice is, uh, taking it slowly, I think that the supply chain as, as cliche as it sounds, maybe as weak as the weakest link.

So finding that one link and trying to ensure that that step in your supply chain or that dependency acquisition process is, uh, secured will take you farther than trying to boil the ocean and doing a full, like in total deployment with hundreds and hundreds of at stations out all over the place. Do you think that we might someday, or maybe we already are for all I know, but can we take any of the data collected or created by in Toto and analyze that in some way that might surface some insights into the way we build software that can help us out? Yes.

Uh, that is, uh, another project that I'm trying to, to work in my lab. One of the reasons, uh, I as an academic I'm very interested in, in this problem is I think it's, it does include the data science side of things, right? We can, we can learn about how the better sort of teams actually carry out their operations by just looking at this raw data, this, uh, this audit trail.

And I'm even imagining a world in which we can help other teams adopt better practices just by doing parody of attestations, right? What attestation is missing on this deployment that every, the pros are using? Or what is the secret sauce that all of the pros have that uh, that's like represented in their attestations that we can carry off to the rest of, uh, all the teams, Right?

Folks, what you heard it here, there's an old saying that says, you know, you can't manage what you can't see, and when it comes to software, supply chain, attestation is the thing you can't see. Santiago, thanks for being on the show. Thank you.

Alright, Back to you n in studio.