Codesee Vulnerability and Securing Software Supply Chains – Alex Ilgayev, Cycode

This is Textron TV. Hey guys. Thanks for the throw.

We're here with Alex ilgev who is a security researcher with SCI code? And we're talking about how they collaborated with the folks at code sea to find some vulnerabilities and remediate them and we're going to get the backstory as they say Alex welcome to the show. Hello, Mike.

Thank you. Thanks for having me here. How did you guys kind of come together and work on this project in the first place?

How are the vulnerabilities discovered? How did they kind of surface themselves and kind of walk us through a little bit about how the two organizations collaborate. Yeah, sure.

So first, let me introduce how modern sensitive works as an introduction to understand the vulnerabilities and the collaboration. So and nowadays almost every organization implements a cicd procedures to automate the development Pipeline and to increase its productivity. So we are seeing a massive Rising the adoption of cicd Technologies.

And with that also see an increase in in risks and and to demonstrate like what is this CST procedure so it could be Compiling your source code the package it maybe to some Docker container to upload it into a artifactory to deploy it in your Cloud environment. It could be a scanning vulnerability in your code scannable abilities in your in open source package that used by your code and Etc like almost everything you can think of so one of the most prominent tools for this SCD automation is GitHub actions. So GitHub is like the most popular tool nowadays to Circle everyone knows githubs to do this cold storage capabilities.

But since 2018 is also a cicd platform gitably provides LCD pipelines to that are hosted on a GitHub service and this is like platform called GitHub actions. So in any user on GitHub and it starts on and GitHub can Define and this pipelines easily by creating a workflow in a designated path in your repository and GitHub will trigger this workflow accordingly to the workflow you You put in that day in this file. And one important fact that kitab also delegates responsibility for using this technology securely to the developers of this specific work.

So during an ongoing research for a vulnerabilities in this open source workflows in GitHub. We found that a large number of vulnerabilities in open source project originated from a third party vendor that it's called the code scene. So a code say is a developer focused startup company that helps visualize codebase and assist in the code reviews.

They offer a freemium plant for open source projects. The thousands of projects are using it on GitHub. So for you as a project to to integrate with code C when you do that, they creating a new GitHub action workflow in your repository that it runs on every pull request.

However is not familiar with how a code contribution in GitHub Works. Anyone can Fork your project and create full request to add a new code to the Repository. So if you integrate that code see into your predatory, so whenever you create a new code contribution this code C workflow helps you to visualize the edit code and to help it review it faster.

So what exactly was the vulnerability that you guys discovered and and how severe might have been? Yeah, so the the issue is is then this workflow that code see integrated in every in every project and ever each over 1000 project that we've found his several configuration issues and together with vulnerability in the packages that were run in that workflow. Yeah.

It's a combine this together disables any user. To run to execute code on the build a pipeline of any repositor that using this workflow. And the the victim list of this positive script contain some of the most popular projects on GitHub including a free code Camp, which is the most popular and most style project on a guitar was using Code c as well.

And so the the potential impact of this. First because we're talking about open source project. There are open to everyone to every user in the internet every user could actually a fork their the project and attribute some code and Trigger this workflow and execute code on the build pipeline of the vulnerable project.

So this could lead into two consequences to severe consequences the first one because the workflow was running with high privileges. As a configuration is one of the configuration issues we found. The potential attacker could contribute code without permission into the project internal into the Repository.

So in the whenever you manage to execute code it could also insert anybody want you to the project. And the potential insert backdoors or malicious trions and and cause the solar winds like scenarios for that project which could lead into millions of victims the victims of this specific project. And the second the severe consequence for this code execution could be to leak and exfiltrate some very secrets and and sensitive insensitive tokens that the repositor is using whether it's a deploying to the cloud or using some jfrog Docker Hub or a yeah, any other asset?

Yeah during for this yesterday pipeline. It would if such this a sensitive token and secrets and excrete. So this is this is the consequences of this of this vulnerabilities.

Right. We're seeing a lot more attacks on the software supply chain. We think at least or at least we're talking about it.

But what's your sense of you know, or the bad guys really zoning in on these software Supply chains and looking for these vulnerabilities or is it just us trying to do an answer prevention before something bad really happens? And I mean that and because we are seeing more and more adoption of cicd and to a Quicken and development. I plan so we see more modern risk on sensitive backgrounds.

I mean and it's our responsibility to find. This this risk and this problem. This issued we have and to fix them.

I mean we are seeing almost some daily basis attacks on the supply chain and All the the statistics and garden of the shoulders arise in supply chain attacks, and we're will continue to see them. So there is a increase in attacks on the supply chain and a lot of the focus for the security Now this are a protecting the code and protecting the the open source dependencies and we are showing that we'll need to protect the pipelines in the in the build pipelines because there are lucrative targets for attackers as well. For example in the solarwinds scenario.

I mean whenever the attacker is managed to get a photo on such a yeah, look at the Target is a build pipeline is a lot of assets a lot of tokens is yes. High privileges in this field pipeline could cause a massive the damage into this project and a rich millions of victims for that. Really?

I'm who's responsible for securing these open source projects in your perspective because there seems to be a debate between whether it belongs to the maintainers or their consumers of the software because the maintainers are like, well we gave it to you for free and we did the best we could and it was only six of us and we can't do everything or you know, or in the consumers need to be more responsible for what they're using what versions and kind of own up to some of these issues. and this is a good question actually and I mean some of the since interesting in fact that some of the wonderful project that we found was actually owned by a or about corporates and Commercial corporates not only by a individuals and maintainers. So and that answer to this question is complicated.

and if the corporate is in charge of this open source, of course we can we can put the responsibility for a securing money. But when we're talking about the individual's maintenance, so the solution is a it's a There is a isn't a good solution because the resources are. A very limited to a but we are trying as a cycle there.

I'm as a researcher to to educate as much as I can they community and me and component similar company that they're doing a software supply chain security trying to help this individual maintainers, which are doing pretty great job in maintaining this open source project to educate them to help them to guide them through this disclosure that we are doing. and into him and to help to make security by default. I mean as I said previously that the GitHub delegates the the security on for this workflows mainly on the developers and it's a And then it causes that we're finding a lot of vulnerable workflows, but we're trying our best and I'm sure that the maintainers.

Of this project are trying their best to secure them. and and we should cooperate together like we did with code C which isn't a security company and to to create the works this workflows and to secure the software supply chain as much as we can match as we can. Do you think as we go along with all of this and people start focusing on this then we might see greater adoption and best devsecups practices and maybe at the end of the day, we'll all be better off for all the focus on the supply chain all of a sudden because we'll mature our development process.

yeah, I think we already were seeing the same this maturity and we were seeing A larger amount of compliances and start standards say out there and there's a South side at the Google pushes the forward which is a software. Yeah supply chain level for software artifacts, which should increase the Integrity level of software artists and the fact or improve the security posture of the software supply chain. and I was a member of social myself.

I do see a more. A maturity in this terms people the the project that I report my vulnerabilities are reacting almost immediately to the to the reports. They understand the severity idea and any single case in which and the my reports were the decline because if any importance there already understand the importance of protecting software supply chain, and I do think that we see a it is a increase in maturity in protecting this because people understand people see this the attacks people people or everybody knows a solar wind everyone knows for Jay.

There's no negligence in this. And yeah, and I see this I see the I see the maturity rather than a few years ago. Do you think as we go along with the relationship between security teams and developers will improve or are we gonna do this in kind of isolation from each other?

But you know, what exactly does the security team add to this equation because it seems like the weight is Shifting left of the developers. So what is the right role or balance between these folks? And this is a good question.

I mean there was all always a tight tension between the developers and security teams. I I wasn't like in in both places. You can say.

Yeah, I wasn't developer once and now I'm more than looking up from security perspective. and I think the like the more the best way to say it like if we cooperate work together, we can achieve great things but not always works like that and but the It's hard. I don't have like a good answer to this.

Yeah, I don't I don't I don't know if there's like a major difference between software supply chain and other security industries for this like it's a tension that happens like in all the security. All the security categories like a more likely Standard Security besides like software supply chain. And yeah, I I think there will be.

this distention will keep going on and and but I do see like more a security orientation for Developers. than before and if mainly because of like because of like the deficiency which like froze more of the responsibility to the developers to secure their workflow, so there is an increase of like security demands security and security perspective for developers more than before but it's not like We still have security teams in place, which should look like from security perspective on entire organization. This will stay like the same.

Yeah. So what's your best advice to Folks at the end of the day? What's that?

One thing you wish that they would all kind of wake up and start doing more regularly to kind of improve this overall situation that we find ourselves in. Yeah, so maybe the best advice I can give is the awareness. Like both organizations and maybe individuals who develop software or a do security.

I mean, you should be aware. It should be aware of the various risk you have you have been using like using modern cicd tooling. and we adopted this Technologies very quickly.

It's a really great technologist and allowing them to the developers faster. But like with every great technology, they it's come with a security risk. So this should be aware from this security risk, and we should handle them properly and this is that the best advice I can give All right, folks while you heard in here.

We need a little Prudence. I guess when it comes to how we're using open source software and check all those versions. Make sure you're not using something that came out of some illegal repository somewhere else that someone's trying to trick you into using malware or whatever but think twice and or as they used to say, you know measure twice and cut once hey Alex.

Thanks for being on the show. Thanks, Mike. Thanks for having me.

All right back to you guys in the studio.