tea.xyz Tackles AI-Era Open Source Risk

Hey guys, thanks for through. We're here with Tim Lewis, who's the founder of T, and we're having a little chat about what's going on with AI and open source and all these vulnerabilities that are being disclosed because, well, it's starting to look like all our friends who are maintainers of those environments are getting overwhelmed. Tim, welcome to the show.

Thank you, Mike. Yeah, it's a fun time to be alive. True that.

Fundamentally, this just seems like there's a massive amount of technical debt that we were not prepared to suddenly pay overnight, and yet we have these tools that are coming out, and despite some best efforts to kind of control who gains access to those things, vulnerabilities are being found in droves. So what is going to happen here, and what does the maintainer community need, and is this just going to be a tsunami that's going to drown everybody? Yeah, that's what I've been describing it as for quite a long time, probably the last year and a half, when I was trying to scream from the minarets about this coming agentic phase that we're going to enter into.

It is a tsunami. And I think this, along with other technologies that are going to continue to evolve at a much faster rate than we've ever felt before, I think this is kind of the cracks in the system when we've had economics that didn't benefit the people that were responsible for the structural requirements. And so we're seeing people, we saw it before, people maintainers, open source developers.

Everyone had been overworked and underpaid for a long time. And now to maintain open source, it evolves beyond, I think, human maintenance to be able to keep up with the agentic, the AI-based maintenance. And so there is an additional real cost to be able to properly maintain community-based software these days.

And ideally, people are going to be reskilling and try to keep up with everything that's going on to participate, but I think we'll see a lot of things break. I think we're seeing that already. And those organizations that aren't funding solutions, and have just extracted value over the last 15 years or so, are going to face a new harsh reality if they don't have the internal teams to help basically take the buckets and try to throw the water out of the boat.

That's what we're going to be left with. But the tools are there, and I think that the open source community is a decentralized community that really can rise to nearly meet any occasion. I feel that within AI, we're the first to understand the systems, going after the vertical market of development is the first real market that the LLM providers have gone after because that is the highest value target that then helps creates the software, that writes all software better, that will allow the other verticals to come.

So, just like with so many other waves and trends in technology, we're at the front lines. And just like with so many things, we evolve. So seeing the evolution is just, I think that the constant of change is something that the people, I think, in the open source community are used to.

And it's just going to be about re-educating, relearning, and then trying to find the resources to make sure that we can battle. It's a battle of inference, the inference of good versus the inference of evil. Do the maintainers have enough of a stake in this conversation?

" And the open source maintainers aren't necessarily paid to go and write all and fix all these bugs. So, what makes everybody think that suddenly they're going to stay up 24 by 7 to go fix this? This has been the problem from before AI, right?

This has been the problem. The billions and trillions of dollars that have been made on the top of the backs of open source maintainers haven't trickled down, haven't been paid to the people that are tirelessly maintain. I think everyone's seen that XKCD comic where the blocks are all sitting on top of a couple of unpaid devs in Nebraska.

We refer to this as the Nebraska problem. I've been acutely aware of this problem. XYZ, with Max Howell, who's the creator of Homebrew.

Max had faced burnout in 2012 after three years of maintaining Homebrew and not being paid for it. And I had been working corporate fintech development jobs most of my career and had gotten in, you'd say, relatively early to a bunch of the Bitcoin, Ethereum experiments that had gone on, right? And I think that there are some things that cryptography can do well, and being able to validate provenance is one of them.

And doing that programmatically could allow people to be effectively paid in a better methodology than we're choosing to now. And doing this systematically might help some of the people that are trying to provide support, rather than trying to figure out if you need to buy somebody coffee or register for their Patreon to support them. So hopefully, in this next turn, as Using cryptography and using systems for validation have been complicated and hard, but I think AI systems, and especially as they're getting structurally more deterministic, although I'd like to say we're aiming for probabilistically deterministic systems within AI.

I think it makes hard things easy. I think it makes impossible things hard, and it makes the improbable possible now. And so you're trying to, at least for me, we've got to fix the infrastructure.

We've got to fix the way that we're going to try to support pay and trade value. Because now, you're going to have these people that are, again, unpaid, unsupported, and have already been fed up, that are going to face really what are distributed denial of service attacks in the form of pull requests. Right?

Mm-hmm. And so this has been a problem, and now it's just exacerbated. And we've been acutely aware of it.

We've been trying to do something about it. It's difficult in the face of an ever-changing ecosystem, an ecosystem that has both sides, has people who have been... I've been around development my whole life.

A lot of people are tired, and they've been learning things for a long time. And the appreciation hasn't necessarily come. But this is hopefully something that we can all do to try to support better provenance as these systems change.

I think development in the open source world has been a large part about creating community. And there's other things that we've gotten from that community. A sense of belonging and a sense of creation that's been fantastic.

But I think as this develops, those communities that had been the strong communities might waver, and there might be other places, because these systems might need other types of infrastructure. I kind of harshly talk about what I think is the current web and refer to it sometimes as the human zoo, or the clicky clicky web. Whereas someone who's...

I've been now fully immersed in agentic AI development for almost three years now. On the agentic side, you had BabyAGI, Auto-GPT, CrewAI, LangChain, AI16Z, and these other systems that have emerged. Now we're getting to points where we have production deployed, semi-production ready, but production deployed AI systems.

And it changes the way that you look at software development. And so, my kind of view on the future of software development and where the role of the developer maintainer goes into is the role of taste, is the role of governance, is the role of making the things that the people don't know that they want, and kind of in a social way, leading people into the right thing because you might get 100 pull requests, but none of them might not be what you want out of the thing that you're trying to build. So it puts everybody in a difficult position.

So the assumption here, though, is that somehow or other, you guys will be using AI tools to respond to all these requests for fixes and whatever else is going on. And that will just take you a couple of minutes. So...

Well, minutes is a lot of dollars. Right? That's the problem, right?

It's the inference cost, right? And it shouldn't be expected that you have this widespread use of inference to create complex solutions to a thing that you might not want even for the direction of your community. But for you to even validate the complexity of whatever it is that you're being delivered to weigh in on whether or not the amount of money that someone spent to go create the generative request is valid or not.

That has a cost to it that's beyond just someone reviewing a commit or two in 20, 50, 100, 1,000 lines of code. You're seeing pull requests with half a million lines of code, request changes and commits that are just so substantial. And it doesn't mean they have no value.

It's just there's not a human in the world that's going to be able to sanely get through that. But there's a true cost in being able to validate if that has any value whatsoever. So we need to think differently about how that value exchange happens.

So let's get into that for a minute, because there's an assumption that vendors who are making money off of this are contributing somehow back to these communities in a meaningful way. There are consortiums that are pulling down millions of dollars for various initiatives, this, that, and the other. 5 million for the entire open source community to go address some of these issues, and then we saw IBM and Red Hat turn around and launch a $5 billion initiative and program, but that was more like a paid service where they were going to come and help do things for customers.

But to your point, if this isn't trickling down to the maintainers, then is this whole system just kind of broken? It's tough to say. I myself, I had formed a non-profit organization.

Again, I get all messed up, and I've been messed up in what is this Web3 world. Which everyone is so, the knee-jerk reaction is rightfully so. Anytime you're creating new financial tools, it brings out the worst of the worst, and everybody's sick of it.

Most of the stuff is absolute garbage. But that was the same case with any of the beginnings of equities markets and other types of tools that get brought into finance. So, I think that as we get closer towards stable rails, which are becoming the reality, and people are seeing benefits, large organizations are seeing benefits to be able to programmatically distribute value through stable assets.

Whether or not you'd consider an asset like the US dollar or the euro a stable asset in the future, time will tell. There's hundreds and thousands of currencies by governments that have failed as well. But, I think that we need to figure out a method of less friction that can distribute value down and be validated on from either a usage standpoint or a governance standpoint.

I believe the governance of these things are super important. I think that's where the value is. I think some of these communities have a tremendous amount of value they've never tapped into because it's all rested on the shoulders of one, two, or three people who want it to rest on many other shoulders.

So we've been trying to go after systems that, for me personally, I love the idea of these decentralized, autonomous organizations, although the final form that's not clunky, that's easy, that people will like, has yet to really emerge. But I think these sorts of things might help with some of that distribution. I brought that up because I distributed over $17 million, 17 million Swiss francs in value to open source projects over a two-year period doing that.

That I'd gotten from foundations, and they were distributed to open source projects, and it was an experiment, but it was too individually led and not decentralized enough. We had maybe 100 or so contributor maintainers that would get involved in voting, and then that becomes too much of a job, right? And so they're just experiments that need to, I think, happen still if this is going to continue to work, and to see.

But for me, the idea of open source has always been sharing knowledge that you have with the world to try to improve the world's ability to create better and improve better software. And I think that that idea will always remain healthy. And whether or not- It almost sounds like you're saying we need a wheel patch for Bitcoin campaign and go from there, but Well, I don't think so.

509 leaf certs, right? So I think that fundamentally, you can't change, or you shouldn't try to change the cryptographies that people use in their workflow, as it's tied into so much of the process and the tooling and also all to their providence, to their history, which I think is there. And I don't want to be in the world where it's always incentive-based only fix, right?

But something has to change. And if I'm going to go view your pull request, and it's going to go cost me money to go run, you better be tying up something to where I know that I'm going to be able to reclaim some sort of value if I just wasted my time and wasted a couple million tokens on validating whether or not this thing makes sense. We need to distribute that cost.

So potentially, the fixes might come like that as well. The corporations that see something that they need to fix, if they're able to easily not look up. The problem with right now with enterprises, I think have is, they're using tens of thousands of packages or more in their environments.

And for them to try to distribute value or to incentivize based on the direction that they need or the bugs that they find is difficult. Trying to have somebody to go and communicate is a very pre-AI methodology, and we need to get to this being able to distribute value and verify in this kind of post-AI world. So hopefully, we can figure out where that's at so they can start doing their part, and then they can potentially earn better service or governance ability within these ecosystems that they actually contribute value towards.

I kind of wonder if we're at the stage now where maybe the glass is half full or half empty, depending on your point of view. But the half empty side of it is it looks like application security is going to get worse before it gets better. But on the upside, maybe if we address all these issues, will application security eventually get better because we're leaning more on AI to solve a lot of these issues before they ever can arise?

I think it depends on where the gates are, right? So the application security will get better in potentially open source, but again, our distribution methods keep on getting more locked down, right? The moves that Google has made with Android.

I like the idea that we need to all sign more. Only about 8% of all open source is signed. Squashing commits has led to kind of series of not actually knowing who wrote what, when, where.

We need to get better on the way that we're contributing to open source projects, so we can identify series of potential threats from certain actors and be able to trust the actors that have actively contributed and been proven not to- Contribute software that's malicious. So I think that we as a community need to lean into the tools that, at least in the pre-quantum era of cryptography, we need to lean into those tools to try to create that province and then try to turn that into the trust across platforms, whether you're on GitHub, or Bitbucket, or Mercurial, or whatever you're using, trying to be able to validate a history. That history should then bring trust, and that should bring either cost or some sort of trust score into what you're contributing.

And that's going to get harder with these AI agents, I think, and we've seen humans that have been inserted into these supply chains and have done malicious things after doing "good work" for months. But it may come down to that there's going to be malicious AI agents that somebody will create. It may come down that somebody may hijack your AI agent and start injecting malicious stuff into that.

So if we don't have signed code, the complexities here just could become mind-boggling. One hundred percent agreed. I don't think it's one day.

I think these things already exist. They have existed for quite a while now. The supply chain activity that's going on at the rate and at the speed that it's going on isn't at the rate of human development.

And as the models themselves get more complex, this is just going to be easier for the more and more novice malicious to go and attack. Everyone knows that this latest Fable release, and it's interesting, has come out, but it's more interesting for me to look at people like Pliny the Liberator, seeing what they're doing with their jailbreaking skills and tools, and trying to offer that in this transparent way globally. And so, the models are there.

People know the jailbreak methods. The talents that people already know the jailbreaks. The existing developers whose time is already precious but also are already exhausted and have been doing so without being paid, now they have to learn this new system, and that's going to be a bit exhausting to get through.

But I think that this will allow the world to have hard, complex things that are more beautiful and more capable. So it's going to be a rough ride through it, but I think that the world will get there together, and we just have to look at this and understand the security side of it. I think us ourselves, when you look at deploying anything, the human gates of even, say, the package manager tools, some of them were human gated, some of them weren't.

The NPM right now is just so ripe for attack because we didn't build it in a world where we thought that there were going to be more malicious packages deployed than packages for usefulness. And so we have to think about ways to do it, and if we're going to stay in some sort of decentralized way, the only way I really know how to do that is through cryptography and some trust scoring. And it won't be perfect, but maybe it'll level up a little bit to where we can continue to build better things while not getting so bombarded with all the bad things.

All right. Hey, folks, you heard it here. There's a lot of common sense missing from this equation.

" Hey, Tim, thanks for being on the show. Thanks for having me, Mike. Have a good day.

All right. Back to you guys in the studio.