The Open Source Agent Revolution: Democratizing AI with OpenHands

Hey, everyone. Welcome back here to Techstrong TV. Let me introduce you to my next guest.

He's Robert Brennan, and Robert is the CEO and co-founder of a company called OpenHands. Robert, welcome to Techstrong TV. It's great to have you here.

Thanks, Al. It's great to be here. Good, good.

So, Robert, you know, I've interviewed a lot of CEO co-founders over the years. They all have one thing in po- in common: passion about what they're doing. Yeah.

Right? Because you got-- if you're not passionate about founding a company, you're not gonna do so good, right? You're not gonna be at it very long because the, you know, the days are long, the hours are longer, and the work is hard.

Um- Yeah ... give us a sense of kinda your, you know, your path in life and where you found your passion for the OpenHands mission. Yeah.

So, I, I started my career in natural language processing. I found I really, I really enjoyed language, working with language and trying to get computers to understand language. So, I got my degree in machine learning with a concentration on linguistics.

Uh, and then I worked at Google for a few years doing things like document summarization and question answering back in the, in the pre-large language model days when those were still hard problems to solve. Um, but then, you know, as I was working in industry, I found I also really enjoyed, building developer tools. I found I really enjoyed, helping other developers be more productive.

I liked giving them tools and seeing what they built with it. That just gives me a ton of energy to see, see what other people build with the tools I give them. Um, so I ended up, shifting gears and working in the REST API ecosystem for a little while, and then working in the Kubernetes ecosystem for a while, all doing, like, open source, developer-centric tooling.

" Like, that is the perfect marriage of my passion for NLP with my passion for dev tools, right? And I never really thought I'd see those two worlds collide this solidly. Um, I'd done some projects at the intersection in the past, but, all of a sudden, they were, like, fully merged.

Um, and so I started immediately hacking on little CLIs, you know, little, you know, the-- what we now think of as an agent, this idea of, like, writing a, a loop on top of the, the ChatGPT API, in order to just, like, get it to, to do what I wanted and to, to be a little bit more autonomous than just, like, copy and pasting into the ChatGPT chat window. Um, and, you know, got-- had some, had some fun building in those early days, but then met my co-founders, Graham Newbig and Xingyao W**g, through an open source project called OpenHands, which was really born- Oh, sorry ... almost exactly two years ago.

Uh, we, you know, we saw that purely software development was going to be changing very drastically, as large language models get better and better. Um, we saw that change, at the time really being pushed, by, you know, sort of stealthy startups, these kind of, like, black box companies, and that didn't feel good. We, we really felt that the developer community needed to have a say in how that change happens if software development is gonna change this drastically.

Um, and so we started an open source project and just gathered a community around us to help push this technology forward. I love it. I love it.

So OpenHands is a open source project. Correct. But the company OpenHands, commercial company.

Exactly, yeah. And OpenHands commercial company manages, maintains the, the open source project. Yes, but with, with the help of a very large community as well.

Well, of course. But we have- Right? That's- Yeah ...

that's open source. Exactly. Yeah.

Yeah, but we, we- I mean, someone has to manage things, but it takes- Yeah ... it takes a village, right? Exactly.

Yeah, it's a, it's a tricky balance. Um, I've been working in commercial open source for, for over a decade now, and it's, it's always, a tricky line to walk, right? Where you need to, you know, make, make sure that the community feels welcome and included and, you know, that they're part of the roadmap, but you're also trying to bring in all this, all this anarchic energy, right?

Everybody wants to pull the project in a hundred different directions. Everybody has their own idea of what should happen, and it's, it's up to us and me as the CEO to really set a direction and say, "Hey, this is what this project is. This is what this project isn't," and, and, you know, really kind of set a tone and, and set a direction for the community so you can guide all that energy, you know, in a, in a productive direction.

Um, there's a- Absolutely ... there's a model from, the early days of open source, it was called, like, the benevolent dictator model. Um, and that's, uh- So I call it the Big Brother era.

Yeah. Yeah. And so I've been, I've been in o-- I've been involved in open source about twenty-five, thirty years myself.

Yeah. You know, when I look at, and I've said this before, when I look at open source, there were kind of three eras. One was, like, the, I call it the cathedral and the bazaar, if you know- Yeah ...

the book, right? Absolutely. Dr.

Stallman and all these people, where it really was kind of the Wild West and down with the man and free and open and free and open. Yeah. And, and, and that was kind of the genesis of this whole open source movement.

I'm not badmouthing it or downplaying it. Sure. But kind of the next era of open source is what I call the Big Brother era, where basically you had a project that was maintained, managed by one company and, and they, you know, they were a bit of a benevolent dictator, if you will, because they were doing things that they thought were both good for the community but also good for the commercial entity that was behind it, right?

And then I think, you know, maybe ten years ago, eight years ago, we moved into what I call the foundational, the foundation- Yeah ... era of open source right? Yeah.

I'm a big Isaac Asimov fan. I love the foundation. Yeah, I love it.

But the foundation era of open source where you know, you had these not-for-profits managing because now it allowed companies that normally wouldn't sit at the table together- Mm-hmm ... to sit at the table together, right? And the- Exactly ...

the thinking is that the rising tide lifted all boats, so everyone had a floor to work on, and then different companies would, you know, scale up or down on that common foundation, if you will. Um, and it's been successful, right? Look, the Linux Foundation's a very successful, the Cloud Native- Sure ...

Computing Foundation part of it, but I, I think that certainly when projects start out, they need, they need someone's full-time love. Do, do you know what I'm saying? It can't be- Yeah ...

one project of two hundred projects that we're, we're, managing or what, what have you. Um- I totally agree. But Robert, you know, it, it, it begs the question of, okay, so here's what the open source project is.

How do we make money? What does the commercial- Yeah ... company do as a, you know, viable business model?

Yeah. So the, the line that we've taken, and we've been very transparent about this with the community, is that we want the open source to be as valuable as possible for an individual developer working on their laptop. And so for instance, all of the research we do about how to make agents work well, how to help them manage their context window, how to manage long-term memory, you know, all the, all the, all the stuff that our research team does ends up in the open source, right?

And every developer can take advantage of that. They can use it, you know, as much as they want without ever paying us a dime. Uh, and that's, that's really important to us, and it's really important to our community, right?

The whole, the whole purpose of this being- Goes to the heart ... an open source project is to bring this- Yep ... bring this technology to every developer.

Uh, but once you've got a team of developers who are working together, who wanna basically like, you know, use the same instance of OpenHands running in the cloud, who wanna connect it to their Slack and their Jira and their Linear instance, that's when our commercial stuff kicks in, right? That stuff is held back because those, those folks generally are operating with a budget, right? They have, they have some organization behind them, and there's, there's some ability to pay there, and so we expect those folks to contribute something back financially to our community.

So, you know, it sounds to me like a-- there's a SaaS version and an enterprise version with enterprise features, so it's like an open core model, right? Exactly. Where- Yeah ...

you're gonna get some of the freemium features for enterprise and teams and, and a hosted version so you don't have to self-host and know everything that that entails. Yep. And- We do, we do have a SaaS.

We actually find most of our customers want to self-host it though. They actually really like the ability- Yeah ... to control all their code and their data.

So I- Yeah. You know, it's funny. I think that pendulum has swung back and forth a few times over the last couple years.

We're seeing it even with agentic AI right now, right? Mm-hmm. The, the guys in my office, the folks in my office, they're all buying Mac Minis- Yeah.

Yeah ... and putting OpenCore on that versus using a hosted, you know, SaaS version or whatever you wanna call it. And it's harder.

No me-- no doubt about it. Putting it on a Mac Mini is like pure open source, right? You gotta- Yeah ...

everything from the packages to, you know, y- And but the beautiful thing is with AI, just say, "Okay, how do I do this? What do I change? " It, you know, it- Yeah ...

it kinda teaches you and walks you, holds your hand. But clearly, the preference seems to be swinging back almost away from the SaaS, you know, polished vers-- vers-- version to this open source, I wanna roll my own. Yeah.

And it, it's, i-it's an interesting pendulum, you know, from what we saw, let's say, five or seven years ago, where I think people were very much into, "Give me the hosted version," right? Yeah. " Uh, so it's interesting to s- Yeah, I'm really excited about that.

Yeah. So now one, one of the things you guys are doing at open source though is you recently launched what they call the OpenHands-- Not open source. I meant OpenHands.

You've recently launched the OpenHands Index. Yeah. Tell us about that, Robert.

Yeah. So, I think one of the trickiest things about, large language models and the agents that use them to write code is that it's very hard to understand how good they are at actually doing their job, right? Um, there's, you know, at least a hundred, plus major large language models at this point.

Um, there's, dozens of different agents that can, you know, plug into those models, and it's really hard to understand, you know, who's the best, which is the best combo. Um, and, you know, every time somebody comes out with a new model, they'll release a benchmark score saying, "Hey, we scored, you know, eighty percent on SWE-bench Verified," meaning they got, you know, eighty percent of the problems in that, in that benchmark correct. Um, but you're kinda trusting that large language model provider to, like, report the right score.

They might use a different methodology than another large language model did. And so you get these, like, apples to oranges comparisons, and it's really hard to tell who's actually the best. Uh, and so with the OpenHands Index, we, did two things.

One is we kind of cut out one of the variables, which is we, we used the OpenHands agent for all of our evaluations and then just swapped out the large language model. So exact same methodology, exact same harness on top of the LLM, and we're just switching the LLM, so we can see exactly how performance changes along a few different dimensions, right? We can see how accuracy changes, we can see how the cost profile changes, and we can see how, like, latency or, like, the duration of problem-solving changes, right?

So you can actually optimize across three different dimensions to figure out, okay, do I really care about speed here and don't care as much about accuracy? Um, do I care about cost? Is that, like, my biggest driver, and I can, like, cut back on accuracy a bit?

Um, and then we also incorporate several different benchmarks from the industry. So it's not just solving software tasks, it's a bunch of different aspects of the software development life cycle to give you a more holistic picture of, is this agent good for a software engineer in general, rather than just like solving GitHub issues. Excellent.

I love it. Um, you mentioned, I mean, there's this-- you're hugging face in all these places and we've got literally thousands, millions of, of models out there now. It-it's enough to get lost in.

As again- Yeah ... I, I was experimenting, I told you. I, I-- So rather than using the OpenClang, I went to Perplexity Computer.

Mm. Which is a hosted version of the agent. Yeah.

Yeah. It come-- Well, and it comes with, you know, ChatGPT and Claude and Opus and Gemini and Nana Banana and everything all kind of built in, right? Mm-hmm.

You, you get a certain amount of credits a month, and, and so be it. Um, but the, the thing about it is i-it's, it's very polished and everything. Don't get me wrong.

I-- And I loved it. Sure. I, I've been diving-- I'm deep down the rabbit hole.

But- Yeah ... it just seems, Robert, there's so much out there right now, and things are-- every day is another development- Yeah ... if you will.

And, and what, what looked like, you know, the, the cutting edge two weeks ago is old hat. Yeah. Yeah, absolutely.

Yeah, it's exhausting, but it's really exciting. So I got to imagine that's the need for OpenHands, right? Yeah.

This, this kind of how do you stay on top of something that's going faster than anything we've ever seen before here. Exactly. " We're, we're giving folks, like a whole SDK.

Uh, so we have the OpenHand software agent SDK, which is, you know, basically all the nuts and bolts you would need to piece together your own agent and to decide how you want to drive that agent forward. So we've seen folks build all sorts of cool stuff, by taking our SDK and like really building the developer experience that they want for themselves or for their company. Um, I've basically built my own OpenClang or Black Perplexity computer using OpenHand, the OpenHands SDK.

I have my own VM running in the cloud that has my own kind of developer setup. It's got a whole like vibe coded interface that I built myself. Um, it's a, it's a great time for to be building custom software right now.

It's a great time to be building the solution that works, you know, particularly well for you rather than getting something that just works out of the box. Uh, and our goal with OpenHands is to really give you the, the raw components, you know, the nuts and bolts you would need to build those things. You know, as I quote some of my friends that I've been talking to, like people who haven't developed in twenty years, right?

Yeah. They were CEOs, all kinds of... They feel like they could build anything right now.

Yeah. Yeah. I could build anything.

Now, let me ask you, and we didn't have this in our notes, but as long as we're talking, what the heck? Yeah. Um, you know, this week, a lot of noise out of the Nvidia GTC conference, right?

What is he calling it? NanoClang or something, right? Mm-hmm.

Is, is, uh-- It's basically like an-- It's an-- It's based on OpenClang, and they're adding some security and stuff in it. You know, it's gonna be optimized for the Nvidia stack. Mm-hmm.

Um, what is, what is something... I mean, are we-- I, I just can't help but think we're gonna go through, Robert, a period now where every-everybody and their mom is gonna come out with an a-- the agent you should use, right? And not just the agent- Yeah ...

but the stack, right? Yeah. 'Cause with people like Nvidia and Microsoft and Google and so forth, they don't want you just to use the agent, 'cause the agent is gonna be basically based on open source.

I don't think anyone- Mm-hmm ... is gonna be able to charge money for the agent per se. But the stack- Right ...

is where the money is. Yeah. Yeah, we've actually worked very closely with Nvidia, with their Nemotron models, and getting- Mm-hmm ...

a full, a full stack basically running on Nvidia hardware, with Nvidia models, and then using OpenHands as the application layer for developers to interface with that. Um, and that, that works really well, and we love it because it's a lot of open source that's getting pushed together, and shipped together so that you can get a fully, a fully self-hosted, you know, fully air-gapped, developer environment that is, you know, top of the line, with, with OpenHands kind of driving the developer experience. Uh, so they've been very fun to work with.

Um, but yeah, I do, I do think kinda everybody's going to... What's nice is a lot of these, components are swappable, right? Like, you can just swap large language models under the hood.

You can take one and replace it with another without real- really much hassle, right? Um, and so, I do think, everybody's going to have their preferred stack, the same way people were building PCs, you know, back in the like early 2000s. Got really excited about, you know, picking the best graphics card and the best monitor and all.

I think you're gonna see that again with, with some of the newest technology. Love it. Um, got one more kinda area I wanna dig in on with you.

So talk to me behind the scenes, how is this evaluation being done? I, I got to assume it's being done agentically, agentically, you know, with AI and... or is it just more of like a crowdsource thing?

The evaluation of like how good these are language models are? The, the various LLMs, yeah, for different tasks. Yeah.

Like what's the right tool for the job? Yeah. So there's, luckily academia has come out with, with a bunch of great, benchmarks for understanding how good these agents are, and there's some really clever solutions for, kind of like automated testing here.

Uh, so like the most famous one is SuiteBench, where they literally curated a bunch of issues from GitHub where there was a pull request that solved that issue. Um, and that pull request also came with a test suite to prove that it solved the issue. Uh, and then we basically give the issue text to the agent, let it make some kind of changes to the code base, and then you throw the test suite at it and see, did the test pass.

And if the test passed, you know the agent solved the problem correctly. If they don't, then it didn't. Um, and seeing, you know, what percentage of those, of those problems actually get solved is, is what we call SuiteBench.

Um, and we're-- we've gone from when OpenHand started, it was like twenty, twenty-five percent success rate on SuiteBench verified, to now we're like at eighty plus success rate. So the, the-- You can really see, you can really feel the, the progress that we've made over the last two years. It's, it's night and day.

I love it. You know what? We-- I don't think we mentioned the website, Robert.

dev. What have people got? dev.

dev. com/openhands. Absolutely.

Yeah. I love it. Hey, Robert, this is great.

What a great time. What an exciting time to be doing this as well, right? Yeah.

Um, congratulations and continued success to you and the whole OpenHands team. Keep us posted, because this is something I think we all... we're obviously very interested and we all wanna watch.

Definitely, yeah. We're having a blast building, and, we're excited for the future. All righty.

Robert Brandon, CEO and co-founder of OpenHands, the people behind the OpenHands project and the OpenHands index here on Techstrong TV. We're gonna take a break. We'll be back in a moment.