Why Open Source AI Coding Tools Are the Future – Insights from Continue CEO Chad Metcalf

Hey guys, thanks for the throw. We're here with Chad Medcalf, newly appointed CEO for continue. They are a provider of a AI coding tool, and we're gonna be talking about what's going on in this market.

Chad, welcome to show. Hey, thanks for having me. Right.

There's already no shortage of AI coding tools. So a what attracted you to continue and from where you sit, what differentiates who these days? Yeah, so I met the continue founders at a meetup in San Francisco in like summer of 23.

And so this was kind of post pandemic meetups. Hadn't really come back to San Francisco. It was one of the first that I had been back to.

And I went to go visit my friends at alama, um, and I know them from Docker days. And there was this startup that was demoing, uh, an AI coding assistant on top of Alama. And I saw it and I had this like little tingle in the back of my neck about like, oh, that's, that's pretty cool.

Um, and so I reached out to some friends and I was like, Hey, have you seen this yet? And then I met Nate and Tai and I started just trying to be helpful however I could. And, uh, that started with some introductions, uh, to some investors.

Then that sort of went on to like, Hey, maybe I could advise or something. And so then a little bit of advising and then a little bit of, uh, consulting and then a little bit more and a little bit more. And at some point they, uh, they asked, Hey, you know, if, if this was to happen, like would you be interested?

And yeah, I, I thought like, well, there's probably only one startup in the world that could probably get me to do it, and this is probably the one. So, uh, my career has been a long time and, uh, open source and in enterprise and in dev tools. And so sort of the, the perfect world.

So, which probably goes in a little bit about differentiation. So I think if you go look at a lot of solutions out in the world, those solutions are all like very vertically integrated, not a lot of choice. Uh, you know, sort of very, this is what you get.

And I think in the beginning that's a great way to get started. But if you look at the trends in our industry over time, eventually the open solution wins. And so, uh, continue is open source.

It partners with model providers. We don't make our own models. Uh, and when it comes to, uh, our developer side, lots of our developers want to use local models or they want to use open models, or they want to use the latest models and they want to tinker and toy with it.

And on the enterprise side, they want all of the same things that enterprises always want. They want choice, they want security, and they want governance. And so for them, where do you want the models to run?

How, uh, what, what models do you want to use? How do you want your developers to use it? And sort of the consistency and control that generally goes with enterprise product.

And that's just not generally available in the tools that exist today outside of continue. So I think that's the, that's sort of the bright line for how we think about ourselves. How disposable are the models gonna get, do you think because they all are accessible via API and I might have different tasks that I'm trying to do at different times, or is all that gonna happen in real time or am I gonna swap them out every couple of months?

How does that work? Also, I, they think if you look at our, like open users, they do swap them out. They, you know, will release a new one.

Somebody will, will add support for a new one and they'll, they'll tinker and toy. But I also think over time models will specialize. And so certain models will be good at, at certain languages or, uh, other tasks.

And one of the things that continue allows you to do is when you specify one of our assistants, you can pick and choose different models and you can have many of those assistants. And so that's something that I think in the long run, like somebody could have a model that's just fantastic at Java or at Python and then be small, fast, cheaper to run. And you could have, this is the way we do Java, this is the way we do Python.

And those things can use different, different pieces or parts. Or speaking of like a llama and local runner, you could run some of that com, some of that compute locally and only go off onto the hosted models for very large things. Like, uh, I want to do reasoning model with, uh, I wanna ask a question of a reasoning model, but for everything else, uh, a local open model is, is fine.

So I think, um, if you look what's happening with like the Quinn family and the deep Seek family, you know, the, the gap between them and the frontier is, is narrowing. And it'll be a really interesting time to see what happens. Where are we on the spectrum of the quality of the code being generated?

'cause I'll talk to some folks and they'll say, well, the machine generates code, right? But it's not tuned for my environment and I have a hard time debugging it 'cause I didn't write it in the first place. So where are we kind of in the maturity cycle of developers using these tools and, and how much further do we need to go?

I mean, I think we're probably at the, uh, I read a a, a spicy blog post about this from the folks at Fly, uh, and they described it as we're at the intern level in terms of like the level of code quality that you would get. And one of those things is, you know, if you, if you just say, you know, Claude, take the wheel, um, you're gonna get what you're gonna get. And so I think, uh, this is still a, this is still a point of, you know, the code that you check in the code that you, you know, you're still responsible for that.

And so, you know, it's not a magic bullet where you're never gonna have to look at curly braces again. It's time to like, you know, you get the machine to, to do something, you review it and just like any other code review, you know, you need, you need to make those small incremental changes so that it's easy to understand what's happening in 20 lines and not be dropped with like a 2000 line change and try to reason about what that, what that change is doing. So I think it's, you know, you see a lot of, like on the vibe coding side, uh, it's great for prototyping, it's great for like getting something going.

It's great for seeing what, what's like possible. And then on the, the side of the house where you've got existing code bases, it's pretty good at doing drudgery. And you know, I, I love writing software.

Matter of fact, you know, I, I stopped writing software when I converted over into the, into the, the go to market side of the house. But These Tools have like knocked the rust off me so that I can go back to writing software again. And I think in that case, like I use them as like a, like a partner in this.

And so like, I'll, I'll direct them for a very specific change. And I think if you use it like that, that's fine. But if you're, if you're just trying to throw at the wall, you're gonna get, you're gonna get what you're gonna get.

I also think that it's really difficult for, um, very large code bases. You see a lot of where, you know, it's great for series A startups that don't have, you know, large code bases and no technical debt yet. And then around, you know, series D when you've got established code bases, things start to come off the rails.

And so it, I think it just depends like any other tool, what are you using it for? Uh, I remember there was an interview early on where they interviewed some woman and she said, you know, I don't want AI to create art. I want AI to clean the kitchen so I can create art.

So, um, there's a lot more that goes into building applications than just writing code. So how will you know, AI kind of evolve to take hand or take care of many of those tasks? Yeah.

So developers can do the part of the thing that they love. Yeah, I think if you, if you look at, it's interesting, if you look at what a lot of open source, or sorry, if you look at a lot of young startups that are in this space, they talk about how fast the tool enables them to write code. And if you look at how Amazon and Microsoft and others describe it, it's a very different story.

Like Amazon talks about the Java eight to Java 11 migration and how many centuries of engineering time, uh, using tools like this saved them to do that migration. Um, there was a, a pretty cool, uh, Airbnb use case where they were updating, uh, a react testing framework and they estimated that that team, it was gonna take them a year and a half, and they wrote an agent for it, uh, spent about four hours doing that, and it got about 80% of the tests. They spent another four days doing that, and it got another 17% of the test.

So in five days of using a tool, they migrated, you know, 97% of those tests, and then they had 3% left, which they just decided to body. And they, they used the team and the team went ahead and just migrated those tests manually. 5 year process, and it's up taking about five weeks.

And I think when you look at that, that's where, at least in my customer base, where I think you're gonna see a huge amount of the early value in this, where if you think about the average enterprise spends, what, over 50% of their time maintaining their existing portfolio of applications versus innovations, and some of that can be much higher than 50%. You can knock a bunch of that loose and have, yeah, migrations and tooling and, and other things that takes the drudgery away of like, oh, I need to update this from Java eight to Java 11, and that's going to be a lot of work. Versus I need to supervise this to update it from Java eight to Java 11.

That's a much more tact, uh, tackable prerogative. I think we all agree that there's gonna be a lot more code and moving through our pipelines and heading into production environments and the code bases might get larger. What are the implications of all this for, say, DevOps teams or the f or, or the folks on the backend who support all these developers?

Yeah, so I was, uh, I was, uh, very, I I used to do operations. Uh, I was at Puppet, uh, before and I used Puppet as a, as a an end user, then joined Puppet. And then after Puppet I went to Docker.

And I felt like Docker really accelerated this, uh, years ago where it was much easier to get, uh, code shipped into production and there was lots of difficult feelings about like developers lobbying stuff over the fence and just saying, Hey, it's ops problems now it's in a container. They can deal with it. Uh, I have a sneaking expectation that that this is going to, uh, make those sorts of problems worse.

Because what's inside of those boxes, if they're, if they're boxed at all, is going to be much more varied. It's gonna have a lot more tech stacks in it. And I think that, you know, you're gonna see one of the things that we see a lot is the folks that come to buy Continu Enterprise are often platform teams and architecture teams that are looking to drive some amount of consistency around the way that software is done in their organizations.

And they're looking at continue, like, can, can we help, can sort of the multiple coding assistant sort of style where you, you specialize an assistant for a specific framework or a specific language, can that help sort of minimize some of these complexities, uh, where operations is just gonna get random tech stacks and just tossed over the fence. So I, I think it, it, it really is gonna be very similar to Docker and like you're gonna want to put some amount of control around how this works, but that'll be an organizational, it will be an organizational challenge. So what is a reasonable expectation for AI and coding?

'cause you'll see these, uh, grandiose statements sometimes made by people that are like, oh, we reduced the number of developers we have and we're cutting them back. And, you know, it almost seems like it's generally aimed at say, wall Street investors, but from your perspective, you know, what does the future of software development teams look like? Well, I think it's interesting if you go look at the companies that are talking about selling products where you can eliminate the developers and you go look at their career pages are quite large as they basically try to hire all the developers they can.

Um, you know, I think, I think there's just a lot of marketing right now and I think, you know, when you, uh, when you look at sort of the naysayers, uh, I'm not convinced that they're necessarily naysayers is that it's, it's just so oversaturated with claims and hype and, and everything else. Uh, you know, there is definitely impacts. Like we, we see it amongst, um, certainly very like entry level or, or fresh out, you know, hires are definitely down.

I think the industry will pay for that I think in some number of years when this all rationalizes and those entry level folks should have been mid-career folks. Uh, and we regret that there's a sort of a pause in that and potentially a, a long live pause that we will feel this, uh, in the years to come. But, you know, I think that a lot of folks are trying to understand it.

It is, I think that what's really interesting is the process of writing software has absolutely changed like that. I don't think that that's up for much discussion, at least in my, in my set of folks. I think though, however, the business of writing software has not changed.

And so, you know, what the company still needs is that it needs to bring an application to a customer and yield a value exchange. And that that's still true Now, you know, uh, whether or not that's gonna have a reckoning whether or not that's gonna have a realignment. Um, but this was, this was coming this, you know, if you think about like even before this rush, there was all of this push from enterprises to try to understand better about how their engineers were working and engineers pushing back and saying, Hey, you can't measure us like sales because it's not as clear as that.

So I think what you're seeing is the bubbling over of tensions that have been building in enterprise software for a long time. Uh, in part probably because things have gotten so much more complicated. Uh, when I first started as a developer, I feel like it was much easier, and this was a long time ago, but I really just needed my language, which in my, that particular case was c or c plus plus, you know, the tool, the, the particular editing environment that I used and then our version control.

We didn't use a ton of external libraries. Uh, we wrote our own network library, our own file system library. We had one big repo.

We didn't use a lot of open source. Uh, we targeted sort of Linux posics sort of APIs, but other than that, we didn't take any dependencies. If you fast forward today, there's many frameworks.

They're rapidly moving. There's Docker, Kubernetes, the clouds, and so like all of this complexity has had a cost. And I think what you're seeing is actually the, the, the business which wants to ship applications to a customer and wants to do that faster is struggling for anything to do that.

And this is a, this is a highly hyped and marketed way where they, they are hoping that this is going to be the, uh, break the log jam that they've had for, you know, a decade. All right, folks, it remains to be seen exactly how much of an impact all these AI coding tools will have, but it will be profound. However, it may not be the silver bullet.

Everybody thinks it is. Hey Chad, thanks for being on the show. Hey, Thank you.

All right. Come back to you guys in the studio.