Kate Stewart, The Linux Foundation | OSS North America 2023
Join Mike Vizard in an interview with Kate Stewart, VP of Dependable Embedded Systems at the Linux Foundation, as they explore the success and widespread adoption of the Zephyr Project. The Zephyr Project is a renowned Real-Time Operating System (RTOS) ecosystem built by developers, for developers. It has gained significant traction in the embedded systems industry, powering a wide range of products and devices.
Transcript
This is techstrong tv. Welcome back to the Open Source Summit in beautiful Vancouver. We're here with Kate Stewart, who's from the Lennox Foundation, and we're gonna be talking about SBOs and a release candidate for one of the formats that's been coming up that's been a long time incoming.
And then maybe we'll talk about something called Zephyr and we'll jump into it right now. Kate, welcome the show. Thank you very much.
Great to be here again. So this week there was an announcement about an update that's forthcoming to the standard, which is the S Spdx software package data exchange. Right.
And we announced the release candidate for release three. And to give you some background on spdx, it's been an evolution since 2009, 2009, and we've gone through multiple stages. But as the technologies are changing, we need to adapt the standard to make sure we can handle the cases that are important to track.
And so we put our first release candidate out for three oh and we'll be coming out with a formal three oh release a little bit later in the year. And then as we merge to this, we're, we've got it there now so the tool vendors can go and implement it and tell us where we're wrong before we lock it down. Okay.
Which is kind of what you need to do in these sorts of things. Um, but what it's doing is, you know, four years ago, five years ago when we had the next, the last major real big change, um, you know, AI and ML wasn't a big topic then. True.
Now we have AI and ML and we need to know what's in there. And so what SBOs do is to give you a way of understanding, having some transparency as to what's actually running underneath you. And most of the focus till now has been on, you know, components and the relationship between components and dependencies.
But we're equally dependent on these data sets and these trained models and how this stuff all connects up together. Um, and so we've got an AI profile and we've also got a data set profile. So we start to capture this information and marry it with the code that is being running on these systems.
So we have some clue when things go wrong, where it could be coming from. Mm-hmm. And the things that go wrong can be more than vulnerabilities now because Oh yeah.
They can be corrupted data sets or something like that. Yeah. Or gets into the safety space.
We all know that like, you know, the autonomous cars and there's all these issues with autonomous cars. Well, AI models trained on data sets running with software. Those are the things that are all coming together right now that we really need to work on.
Um, pulling a story, you know, making sure that we can have a line of sight into trying to debug when there's issues or do recalls or say that things have expired and they need to be retrained. Um, so there's different parameters that are important in these areas than we've been looking at traditionally with just software. There's a lot in common, don't get me wrong.
But, um, there's some other properties like, you know, are some, is some of this stuff confidential? Are there some known biases? Things like that.
We wanna start recording too. SBOs have been around for a long time. Mm-hmm.
Just not many people use them. And now it seems like everybody and his brother suddenly wants an sbo. Yeah.
What has that been like? Cuz now you know, you guys are at the center of that conversation so you have all kinds of people throwing up input that you haven't seen. Input Is welcome.
Mm-hmm. Okay. We'll be very much welcome.
If you've got a scenario you can and you wanna figure out how you represent it with an sbo, if we can't tell you how we can do it with the fields we've got today, we'll work with you to figure out what's missing and try to get into the next release of the standard. Um, one of the things we've done with the spdx project is a couple years ago we actually took it to ISO and we made it an international standard. So we went through the full rigor of the international reviews and everything else.
Um, and so getting it to the stage where it can be used internationally as a standard as opposed to just de ad hoc or defacto was important to the project. And so the Linux Foundation helped sponsor us to do that as a community. And so we've got that and then we put out a two three last year to improve interoperability with others.
Um, and so that slowed us down a little bit on three L cuz we were trying to be collaborative mm-hmm As these things go. But then we came up with, so we've, we've come, now it's time for three oh and this is where we can introduce the ai, we can introduce the machine learning. We're introducing some more security hooks because people wanna record certain pieces of information in the specifications.
And you know, security is the reason that you're seeing the SBO being so popular now mm-hmm. Is the cybersecurity problems that are out there. Um, people are recognizing that we need to start doing things differently at scale.
And part of that is the transparency and some of the big drivers on law of this stuff are the regulatory agencies now think and the various, you know, industry groups like automotive, but like the FDA is the only one that um, was able to preserve you must have an S bomb or we're not gonna take anything from you. Which was a direction that the executive war was heading a couple years ago. Mm-hmm.
And so as of I think this month actually may, you cannot put a medical device into the FDA unless you have an sbo. They will, you know, I think it's refused to consider or something like that. There's some term that they're using anyhow.
Um, but they're sort of saying, Hey, if you want, you know, the same way they wanna know all the ingredients in these prescriptions. They wanna know all the ingredients in the software and how it's gonna be used so that they can help assess where they'll be safe for people. And I kind of applaud that myself.
It seems to me at least that people are um, interested in SBOs and that's great, but we don't have a lot of uh, insight into how to operationalize the SBOs. And so is that something the foundation's gonna help people with figuring out? Or how does that Kind of go?
I, I think there, I think there's um, so there's a lot of work going on with the CSAF folk on operationalizing this. Um, it actually has been operationalized it not talked about loudly, um, because of the open source licensing. However, in large organizations the legal team may not talk to the security team and vice versa.
Mm-hmm. Or the OPO team may not talk to the security team. You know, we have these silos happening, but the information is there in parts of the organization and the stuff you keep for the legal folk, which is what are the components and what are the relationships between them, what are the dependencies is the same thing we need for the security people.
And the same thing we need for the safety except the safety main. We need to take you down from the component level down to the source too. Cuz his standards called for that.
There are multiple SBO formats floating around. Will these become interoperable over time or Well That see earlier comment about the two three release. Mm-hmm.
Um, the Spdx community deliberately put that out to help do round tripping between the Cyclone DX and the spdx. Um, it would be nice if the Cyclone DX did a little bit more to be more intra with spdx, but I'm not in that community, therefore they're the ones who are gonna have to change on their side if they wanna do more. How will SBOs stay current?
One of the things that happens is a lot of people are updating software frequently. Mm-hmm Some daily, some hourly. How does that sbam know what's in there?
Every time you update an image, daily, hourly, whatever, you're creating an executable. If we have the SBO M generation in the build flow, every time you're putting out an image you should be putting out an S bm. So you always have a view as to what is currently there and you want it, you don't want to be amending your S bm, you are just be generating it out.
Or in other applications there's reasons to do amendments and things like that. Like when you say, oh this one I'll put a, I'll put out something that says this one has a vulnerability cause I found it later. But every time you spit out a build output you should have an S bum.
But that's not the only type of S bomb out there and that's why people get confused. So in addition to being a build SBOs, which is what the security people are mostly focusing on, there's a source S bombs, which is what the legal folk have been focusing on cuz the terms of use. Right.
Um, and you need both of them for safety. So you need the, the build S bombs and the safety, but then you also potentially need how you've deployed that application. So you need to record that.
So all the configuration information is key for understanding am I vulnerable or not and do I have an issue or not? Do I need to remediate or not? And so these are things that are, you know, where I think we're at the cusp of getting some really good scaling happening, but the operationalizing, as you say, is what we need to be focusing on.
All right. You mentioned safety and I know that Zephyr is part of your world. So where does Zephyr safety and SBOs come together?
Okay, so Zephyr has a um, an ability when you actually do any build, um, you do a pre-com command and then anything you issue out will have an sbo. In fact it has three SBOs. So every build image has three SBOs, one for the application sources, one for the Zephyr sources, and then one build sbo that refers back to the application sources and back to the Zephyr sources so that the dodos from the CS are mapped and the dodos that go into the libraries are explicit.
And the libraries that get combined into that final elf image that you're gonna align, you know, download are all explicit. So you can go from that elf image you're gonna be putting on a system, trace it back through exactly which libraries were formed with which sources and have complete line of sight. And the benefit of that is meaning all those false positives that are out there on you know, the scanners and things like that, you can be authoritative that no you don't have a problem.
So you get rid of a lot of work by taking that level of traceability and hopefully more of the industry will wake up to that. Um, that we need to be able to have that traceability so we can get rid of false positives. You mean the scan I didn't run in the first place because I was afraid there's too many false positives so therefore in that, So you just do it in the build and then you just know exactly what's there and Oh hey, um, you know what brought this lesson home to me and said some other people in Zephyr was when Amnesia 33 came out okay it was a result of F net and some F net code that was widely used in the embedded ecosystem.
And so that F net code, if you looked at the module level, you'd say F net's there in our lts. It wasn't in our current T of tree but it was in our lts. However, when you actually look for the files, the files that had the vulnerability were not there at all.
And we had no real way of signaling it other than writing a blog post. Hmm. But being able to be explicit about saying, Hey, these files are there or not means you have to do work or you don't have to do work and that's gonna save time.
I am assure we have some folks watching this that are going, what's a Zephyr? So, oh sorry. Thank you.
So Zephyr is a real-time operating system. It's been designed to be, um, Linux can doesn't, my Linux is too big. It's designed for that.
And we wanted to make sure we had a safe and secure option that could be used when can't use Linux. And so what it has is it uses a lot of sort best practices from Linux. Mm-hmm.
Things like the, you know, uh, cake and fig a device tree. It has maintainer hierarchies, it has a code looking very familiar, um, but it is available uh, to work with the ecosystem. And so it's like 10 K and up as opposed to this doesn't get smaller than three meg.
Mm-hmm. So on those sensors and actuators, you still want something with potentially some security, but you're not gonna be able to put enough memory for a three meg system or power for a three meg system. And so we get finding in interesting places right now.
So what is the latest on the Zephyr project? Well, uh, latest on the Zephyr project is uh, we're busy um, working towards our next lts release, which is a long term stable, which we'll put out next year. Um, there's new features coming in every day.
Uh, right now with Zephyr we're finding new products every day, which is the fun part for me. Um, one of the ones that I was really excited to learn about cuz I'm like say my hobby is wildlife photography. Okay?
Mm-hmm. And I've been to Africa and things like that. Um, and obviously poaching of rhinos is a big problem in Africa for the horns.
Mm-hmm So one of the things they've done is, uh, the um, the company earn who's one of our members has basically worked with the conservation people and they've designed a little implant and they drill a hole in the rhinos, horns put in the implant and um, you know, patch it up. And so that, you know, you can't detect whether it's there or not, but it's sending a GPS signal and so the battery has to last for two years cuz you're not gonna be changing the battery on that rhino horn. Right.
And so what they do when the battery stops is they'll put another one in, but this gives 'em a way of tracking the rhinos. And so if they suddenly see the rhino stops moving or they see it suddenly starts, you know, moving at different speeds, they can actually have the evidence to trace back and prosecute. Right.
You mean the rhino won't show up for a tune-up? I don't understand. Oh, well It is what it is.
But yeah, Bringing this a little full circle, so these Zephyr use cases are very much in safety applications, medical, automotive, where SBOs are gonna be requirements it seems like. Are we seeing more regulations coming down the pike in that way? Yeah, we are, we're seeing a lot, um, from the FDA in particular from the medical devices, they are insisting now on having the SBOs available, um, for any device that's going through.
They're, they're, you know, they're know their assessments and so forth, they wanna see the SBO information, um, for the, um, Europeans are also starting to expect the SBOs in some of their areas in the iot space in particular as well as some of the re industries. And so I think more and more you'll see, you know, there's been some pushback in the industry cuz oh, they don't wanna do this, it's scary and new, but basically I think the pressure is now there that um, it really does need to be done as an element of hygiene. And so people are working on operationalizing it, making it more efficient.
Um, there's lots of open source options out there and there's lots of commercial options out there. And the commercial options like, um, like Synopsis behind me and you know, some of the other ones here are all working with SBOs and ingesting them as well as exporting them. And as we can get the ingest and the export happening in both directions, we start to make it the knowledge flow through the supply chain, which is what's needed.
Will we get to the point where I'll essentially have like a red light green light system for deploying something based on what's in the sbu? Yeah, I think we are, there's actually working on with the labeling mm-hmm. Um, authorities I was talking to, our researchers was working with them and coming up with effectively a equivalent of a food label but uh, label for security, um, to sort of assess various properties and come up with effectively a red green type of light.
And so I think consumers reports and those guys are interested in this whole area. And so I think this stuff will eventually feed in, there's so many applications this stuff can feed into to use the data to make it better for people. And so it's really exciting to see where it's emerging.
You talked about using, um, SBOs to track the data that's being used to drive the AI model, but can I use AI to drive the deployment and management of SBUs Software for software? Huh? Why not?
Yeah. Uh, I suspect you'll see that someone will think it's a good idea and move it forward and you know, realistically all you really need is a database and you need good querying abilities. And part of what we're doing with the S Spdx three is designing it to go into databases and have schema generated automatically.
So that was part of the restructuring we did on the internal underneath model because with this format we can generate Jasons yammel, xml, RDF spreadsheets, tag value, you know, it's a language effectively, and people were ingesting these documents and setting up their own databases internally and doing it all a little bit differently. So we went, okay, and this is where our work with Mire was, um, to come up with what they needed for a database underneath. And so we basically, the two collab, the two communities really collaborated together to come up with the latest version of the specification so that we could have something that would work for databases as well as export and import.
Mm-hmm. And that's gonna be fun. Some folks are a little concerned that the SBO m will surface their secret sauce for how they built things.
So how do I kind of protect my quote unquote intellectual property or is So, um, you can think of it as a sbam is a roadmap for the defender, not the attacker. The attackers know this stuff anyhow. So, you know, realistically try to give the people who are trying to do the right thing at a fighting chance is what it's happening.
And realistically, you know, you can say how much of it you're sharing or not, you don't have to put everything out in public. Mm-hmm. Okay.
Mm-hmm. You can be applying, you know, NDAs, you can be applying, you know, other things. You can have business relationships like you do with executables today.
So, you know, as you negotiate your contract to get a product, you negotiate at the same time to get the s bum for it, such as if something goes wrong, you have a chance of detecting it and you know, and you, if whatever terms you're putting your executable on, you may wanna put the, you know, the s bump, the s bum under the same terms. What's your best advice then to somebody who's kind of trying to figure out how to operational s bombs and put this together in a workflow? What should they be thinking about what, you know, what's the cart and what's the horse?
Um, I think the first thing to think about here is where you see risk. What risks do you wanna protect against? And then figure out what types of SBOs you'll need in order to figure out, um, what risk you wanna mitigate.
So if the risk are security risks, you'll probably wanna see more build s bombs. Okay? Mm-hmm.
And so if someone's giving you things, asking for the build s bombs and then being able to import it so you can look up and cross check that there's vulnerabilities, there's open source tools out there and there's lots of good commercial tools out there too. And so just working with the people who have the tooling out there, if you don't wanna do it yourself, but if you don't have the budget to work with, you know, the commercial vendors as open source options, like the Spdx projects puts out tools to validate you've gotten a valid S bomb. For instance, someone's giving you sbo, is it real?
Yeah. Is it garbage or is it something useful? You can at least check that it follows the syntax with the online tools, you can at least check that it follows the NT minimum elements, which is what people are generally agreeing on as a definition.
And so these things become, you know, step by step how it gets interesting. Um, so you can sort of, there's things that are online, you can do the quick checks and then you can build up your sophistication over time. So what has taken us so long to get this, this point?
Because on the face of it, it seems like, so we're putting a list of ingredients on our packaging. I mean, you know, we've been doing that with food and everything else for decades, so why are we just getting here with software? I started working on this back in 2010.
Okay. It seemed obvious to me then. Mm-hmm.
I had to go and adhere to the licenses, but I had to figure this out. And it's, there wasn't, I think enough pain in the ecosystem until all these vulnerabilities happened that people have motivated the change. And so I think the scale of, you know, their, and there really wasn't as much open source in the ecosystem either.
And right now people just are, they would write their code, they would chip at their product, they have much more scope of line of control over everything and other pieces are potentially con changing out from under them and they may not know about them. And that's more like the number of components has, you know, gone exp exponentially. The number of part of open source.
You just have to look at some of the surveys coming out and you'll see it. It's pretty prevalent. All right folks, you heard it here.
First off, there are no secrets. Second thing, do the right thing and make it easier for everybody else to secure their software once they buy it from you or download, downloaded from you or however they got it. Kate, thanks for being on the show.
Thank You. All right, good for the interest Folks. We'll be back in a minute.





