When AI Starts Building AI

Hey everyone. Welcome back here to Techstrong TV. I've had the good fortune of interviewing a lot of CEO co-founders of fairly new companies this week.

I want to introduce you to another one that I was lucky enough to get introduced to. I want to introduce you to Kunal Bhatia. Kunal is the CEO and co-founder of a company called Hexo Labs.

They've actually been at it for three years, so they didn't just launch, but three years goes quick. But let's find out more about it, and let's welcome Kunal to the show. Kunal, welcome to Techstrong TV.

It's great to have you on here. Hey, Alan. Thanks for having me.

This is- My pleasure ... great to meet you and your audience right here. Absolutely.

So Kunal, let's start with a little bit about you. I mentioned you're the CEO and co-founder at Hexo. What got you here?

What made you go found a company? What's your experience and journey been like? So this is actually my third company that I've started, and third in the AI space itself.

I've been building in AI for over a decade now. First one, we were trying to build an Alexa kind of device you could talk to and it'd control appliances around your house. This was back in 2014 to '16, and then founded another company, which was in education and AI, where we were building an after-school learning app for kids where the idea was, GPT-2 had come out back then.

This is in 2020, and we were super excited, and we thought this is an amazing model, and the idea was to build an everything learning app, that you can learn anything you want, because it was a general purpose model. And now when you look back, GPT-2 seems quite rudimentary, but back then it seemed very promising. Well, I know what you mean.

I've seen this in waves of technology before. And it's amazing. It wasn't really that long ago, where we went from there to there.

So what was the motivation behind Hexo Labs? So actually, it's very interesting. We started off as a very different company.

We started off as a consulting services company. " So we were like, "Hey, the space is moving really fast. " So we took a very contract research kind of approach.

We said that, hey, we did a whole bunch of contracts. In fact, if you go to file a trademark with the government of India today, you're using our algorithm there. They wanted a trademark search, counterfeit image detection model.

For a semiconductor company, we had built some models there. For another dev tools company based in the Bay Area, we built some models. So we're doing a whole variety of projects, very consulting services type.

At some point, our pipeline grew quite a bit, and the question I asked my co-founder was, "Hey, I'm not able to service all these customers because we're manually out there servicing them. " So my co-founder was the researcher, and I said I need an AI version of him, which was like I need an AI that builds AI, which can service all these customers. And that is where really the genesis was of what we're doing today.

And when we started thinking about that, the thought kind of just took over us. " And that's literally what pulled us in. So we were growing a very profitable business.

" I love it. It's an interesting story, but it's a pattern that I've seen before. Starts with a consulting business, because a lot of us just naturally gravitate to that.

But then when you start seeing patterns emerge across your customer base, you realize what's really missing. What's the problem? What is a common problem, and what's that solution look like?

And you wind up productizing it into something. So it's pretty cool. So let me ask you.

I got to tell you the truth, you're the first person I've ever spoken to who was on GPT-2. So when you look back, even over the three years of Hexo, from 2022, 2023 to now, did you think it would come this fast? No, I think it's interesting because...

And this is something I keep saying also, is that every one of us, it's very hard for us to process exponential curves. Our brains don't process it. We think very linearly.

So when we're predicting what's going to happen in the next one year, we look at what happened in the last one year and then try to extrapolate. Mm-hmm. But the thing is that it's kind of accelerating.

So the curve is not like this, it's like that. So I would say no. Of course, we had an inkling that things are going to get very interesting.

But what that looks like, I think you just have to kind of live through it. Agreed. So with Hexo now, you're working with a lot of cutting edge scientists and research institutions, et cetera.

And again, I'm going by my own experience, having founded a few companies and Like, for instance, one of my companies, we did a lot of work with the DoD and some of the three-letter agencies. And it was great, don't get me wrong, it was an amazing experience. But we wound up so specialized that we didn't have a product for the general public, even for the enterprise, because it was so specialized towards that mission for the military and national defense and so forth.

Do you worry about that with Hexo that, hey, you're going real up market here, but do you lose the bigger market? Yeah. So I think it's very interesting that you asked this question, right?

Because the way we look at it is that any kind of progress we're seeing, whether in enterprise or even at the bleeding edge of frontier research, right, it's all happening because of machine learning algorithms. Right? Even if you're training a model for an enterprise, for say, a brand image generation use case, or whether you're training a quantum computing algorithm, there are AI researchers who are actually taking data sets, running training algorithms on it, and evaluating it.

Simple as that, right? Whether you're doing it for use case A or use case B, right? What we're building is actually an AI agent which does that, right?

With infrastructure that can support it, of course, right? To run parallel experiments. So for us, for our agent, it doesn't really matter, right?

Because at the end of the day, these are general-purpose reasoning models that we're building. And they're learning how to reason to solve AI problem itself, right? So if you're building an AI algorithm for something like material discovery, or whether you're building for an enterprise sales BDR agent, it's all the same because it's just reasoning on how to solve a probabilistic, how to build a probabilistic model at the end of the day, right?

And how do you optimize towards an eval? How do you hill climb towards an eval? So we're actually building that general-purpose infrastructure for AI development itself.

How does this AI agent and system build new AI that can be applied in these use cases? And it's interesting that in some of these cutting-edge use cases, frontier use cases which you mentioned, those are very interesting case studies to work on. Of course, there's a whole conversation of where the scalability and revenue come from, and there's a lot of simpler use cases that enterprise might take.

But I think finding a balance between both is what's needed. Excellent. I want to talk to you about, I guess it's called, is it SIA, S-I-A?

Yeah. Is that how you pronounce it? Yeah.

So this is something you guys spun up, open sourced. Yeah. Share with our audience a little, Kunal, what is it about?

Yeah, so the interesting thing about SIA is that it's the first self-improving AI system, and why we've called it self-improving AI, and SIA is basically the short form for self-improving AI, is there's a lot of conversation going on around loop engineering and how do you build self-healing harnesses and self-improving harnesses. And we've been fundamentally working on this for the last year and a half now, is that how do you build an agent that can train models itself, like an AI agent that can build other AI agents? And when we talk about AI agents, we talk about two parts, the harness as well as the weights, right?

This is the first system in which one agent can train not just the harness, but also the weights of another agent simultaneously. And the reason why we call it self-improving AI is that if you make the meta agent which trains the target agent, if you make the target agent the meta agent itself, so it starts improving itself. So, it is towards, in the direction of recursive self-improving, that one agent can improve itself again and again, but you could give it any other target agent itself.

So that's the interesting part of it, that we're not focused only on the harness. We think that you need to be able to move the weights as well. And at the end of the day, if you look at all the conversation that's going on right now, whether it's what Satya Nadella has been talking about for the last few weeks, or Alex Karp spoke about, they're basically saying that enterprises need to own their own intelligent layer, and the intelligent stack, rather, right?

Which means that you're not just giving them the harness and saying, "Hey, you fix this," but how do they own their own weights as well, right? And if you really had to scale it across all enterprises, there's not enough talent out there to be able to do that for all enterprises, right? Would you imagine one of the best AI researchers in the world, let's say someone like Andrej Karpathy, going and working for a FMCG enterprise?

No, right? That's where an AI agent that can perform like Andrej Karpathy would come in and build your AI for you. And that's fundamentally what we've built, and SIA is the first version of it, which we put out there.

This brings up a few things to me, Kunal. First of all, we're already hearing how AI is actively building the next generation of AI, right? And that gets people a little freaked out, but that's what's going on.

Increasingly, we're relying more and more on the software to build the next generation of software. But when we start talking about open weights and, I actually wrote a couple articles this week on both what Satya and Alex Karp said. Actually, it was last week I wrote them.

A lot of their thing was, you're already paying these AI models for their intelligence, but then they're taking your data, and that data really is yours and it shouldn't be taken without your express written consent kind of thing, right? And aren't we better, in that case, to build our own stacks, our own models? Or it doesn't have to build our own models, but to host our own, so that we can have greater control over our own data or our own output.

Not to mention, take the tokenization aspect of this off the table and start valuing AI for the job it does for me, not how many tokens it used. Right? I think that's another side aspect of this same argument.

But interestingly, also this week, we've seen now two of the Chinese models come out with new models this week. It's only Tuesday we recorded this. But two models that claim to be on par pretty much with OpenAI and maybe even just a little below Anthropic, but it's close.

Right? And these are open weight models. They're not small.

They're trillion weight models. But we also saw Cisco release something called Antares for security. I don't know if you saw this one.

Right? Was it half a billion? Yeah, half a billion and one and a half billion models that run locally that it's not quite Mythos, but it gives you tremendous security, vulnerability, capability, right on your desktop, in essence.

So this seems to be the way we're moving, away from that kind of, I almost hate to say it, but away from the American model of this big proprietary model that decides after you put in a prompt, whether it wants to run it on its most recent version or it's going to kick you down to one of the older versions, which is something Satya wrote about. Where do you think this ends up? Look, I think the bigger risk that we see is that concentration of power and intelligence and this is the most important technology we're building.

If there are just two or three people who have access to it and to super intelligence and everybody else is renting it out, I think the bigger risk is there. Right? And we've seen this play out before as well.

If you take the story of e-commerce, there's Amazon, which is a platform for all these sellers to come and sell their products, and then they start looking at the data and say, "Hey, you're selling great beanbags. There's good sales there. Let me launch beanbags as Amazon Basics.

You're selling knives. " And then what's happened is that the same suppliers who are in your platform are getting nuked by the platform itself. And we're seeing that play out all over again, like with Anthropic launching Claude Design, then a specific product for legal, specific product for finance, and then going vertical after vertical.

It's happening. Right? What really would happen is that enterprises are already waking up to the fact that if we're giving away our alpha to these companies, we're in deep trouble.

Because they have these capabilities which we don't. And I think where the gap is really is to have a Shopify version of how in e-commerce Shopify enabled those merchants and saying, "I will give you that infrastructure to own your own storefront and own your own supply chain and all of that stuff. " I think there is an opportunity to actually build that kind of a layer for frontier AI intelligence and say that- But what you're describing is almost, a lot of people say that's how AWS started.

" Whether they're still keeping the data or using that data behind the scenes is speculative. But that's the way it happened. I think what we're seeing, Kunal, and I'm just finishing a book on this that I've read, is that all of these people want to move upstack.

I think they realize that the intelligence layer is going to be commoditized. Right? People aren't going to care, people aren't going to pay a premium for Anthropic Fable or whatever comes after that if one of these open weight models can do the job for me.

I don't need the sledgehammer to kill the fly. I need a fly swatter. And so if the fly swatter's good enough, that's good enough.

And so as that commoditizes out, well, where does the value go? The value goes upstack. What do I build on top of that?

And I think that's what OpenAI and Anthropic realize, and that's where they're racing to. And not just them. Microsoft, Google, Amazon, they all want to move upstack.

Yeah, and as an enterprise, you want to own that capability as well, right? Absolutely. What's going to happen, where at least the trend with this is going is that because of these self-improving loops that are kicking in, where not just the harness, even the weights get trained, and you have different functions from an organization bringing different context, and then the entire system is just constantly evolving.

You would want to have that capability as an enterprise as well because at the end of the day, what you're saying is that the model capabilities start saturating beyond a point. But what starts happening is that this new context starts evolving the system itself. And when I say the system, I mean the model weights as well as the harness that's building the system.

Which you would start seeing as the self-evolving enterprise itself. Where new data is turning new processes, new context is going into marketing copy, new context is going into your customer conversations. That's feeding back into the product development cycles.

All of that starts creating a loop. You want to own that entire loop. And that's why you would need to have infrastructure which can support that, but you don't have access to the talent which builds that infrastructure.

And the only way you can have access to that talent at scale is if AI is the one building it. And that's fundamentally where we see the opportunity, because AI is able to build that self-improving AI for you. Excellent.

Kunal, we're over time already. I apologize. But you know what I didn't realize?

I don't think we mentioned the website. Yeah. Hexo Labs.

How do people reach you? com. Yeah.

Easy enough. Yeah. com.

Kunal, thank you for coming out here on Techstrong TV with us today. I appreciate it. Keep up the great work.

We'd love to have you back on and find out more about what you're seeing on the edge there of AI. Thank you so much, Alan. Thanks for having me.

My pleasure. Kunal Bhatia, CEO and co-founder of Hexo Labs, here on Techstrong TV. We're going to take a break.

We'll be back with more.