The Resilience of AI Stacks – Techstrong AI Podcast EP45

Hello and welcome to the Techstrong AI Podcast. I'm Amanda Razani, and I'm excited today to have Karthik Ranganathan. He is the co-founder and CEO of gigabyte.

How are you doing? I'm doing great, Amanda. Thank you for having me on.

And hi everybody. Yes, glad to have you on the show today. So can you share a little bit about your company gigabyte and what services do you provide?

Absolutely. So at Gigabyte, we are the builders of a cloud native relational database. The database is called Gigabyte tb, right?

The best way to think about what we do, um, increasingly all services, our entire economy is be becoming increasingly digital and all, it's all becoming data driven, right? On the enterprise side. And, uh, all of the architectures are going cloud native.

So we help bridge all of that. We offer enterprises the ability to use something as familiar as Postgres, uh, you know, a really good relational database in cloud native architecture. So it's Postgres for the cloud.

I hear it's all about data. We hear so much about being able to wrangle all that data properly. And as companies are digitally transforming and going more to the cloud, are you seeing some, some IT legacy challenges and can you share what you're seeing?

Yeah, I think there's a, a few different dimensions that we see. So typically as companies modernize, I mean, what does that even mean, right? Like, um, uh, right?

So there's a few different goals that they have in mind. Um, they want to be able to build features quicker so that they can keep up with the demand of their users and really give a good customer experience. Um, they want to build features that can power the new type of asks and usage patterns that their customers have.

So it's satisfying the customer demand, customer experience that they expect. Um, it's doing so in a way that's cost effective and while complying with regulation. So there are these four things that are, you know, acting in tandem.

And when you look at it from the entire stack perspective, I think we come in at the data layer or the database layer where the very things that we talked about those aspects would be required at the database layer. So these are some of the challenges we come in and fix, which is, you know, on one side, you know, our customers can go for a completely new database, but that's too much of a learning curve and a retooling and a rewrite. It's too complicated.

And if you just stick to what's existing and try to stretch what it can do in the cloud, it can only go so far, right? So what we've taken is a novel approach of taking something very familiar Postgres and making it truly cloud native. So we're a distributed Postgres for cloud native applications, right?

So that's really where we sit. Those are the challenges we try to solve. And the specific data layer challenges are around, you know, availability and resilience.

It's around scalability, it's around replicating data across different geographies and making it all operationally simple. And API driven. Of course, now we throw artificial intelligence into the mix.

And so where does AI fit into all this? And what should business leaders be concerned about and aware of when they're implementing AI technology? Absolutely.

So where does AI fit into all this, right? So, uh, we live in the world of building new applications that service users and that expose APIs, right? That's really the part that gigabyte plays in, as opposed to the warehousing world, which is more around data intelligence.

This is more powering applications, right? When you think about AI in the world of powering applications, there's two things that are happening. Applications are getting infused with ai.

So this is AI infused applications where you attach AI components to make it more reactive, more responsive, more valuable to the user to give a better experience. And a second class of applications, which are AI core applications where AI is at the being of the application getting developed. So we see these two things.

We see both of these, and they both present interesting challenges from a database perspective, right? So from a data database perspective, in the first case, which is AI infused, there's an existing application that's running already. And you know, the industry is trying to attach AI to it and make it even more valuable, right?

In this case, it, that means the application footprint will go, will grow, and that means the scale increases. You also have to be, you know, like almost pushing the feature set all the time, like relentlessly, you had to keep iterating and experimenting, right? And what this also means is AI usually makes everything quicker, faster, better from a customer perspective.

That means any outage is now felt even more acutely by the user because it looks like a very advanced stack that you're never fail. But if it does, it matters a lot. So really where you see it there is around, you know, scale and resilience.

Those are the two aspects. Resilience specifically being very important, and the ease of use. Those become very important.

When you look at AI core applications, the thing that becomes important is how can we unlock a wider category of applications with the AI space iterating as far as fast as it is. So you don't want re implementations, you want ability to leverage the latest and greatest in ai. You want to be able to treat AI almost as an extension to building applications, as opposed to having to re like, think about it as a separate piece of architecture.

So those are the two ways it comes in. So let's think about that a little bit. The AI outages hitting harder, as you said, all technology is gonna have glitches here and there, that's just expected.

So what should business leaders do to kind of prepare for these outages and um, and, uh, what, what do you think is a future solution? Yeah, so when you think, I think it's important to think about outages in the right perspective, right? Like people often say, Hey, what's new about outages?

We've solved for outages in the past, right? The main change is it's not just ai, it's also cloud native architecture, both happening at the same time, right? Cloud native architectures are inherently more flexible and more agile, but they are, and they're cheaper, but they run on commodity infrastructure, commodity meaning, you know, off the shelf parts, which can fail more often, right?

And so on one side, the scale is increasing more ai, more applications, but on the other side, you have increased the probability of failures, which means you're going to see a lot more failures, right? And when we talk about failures, people often think about that entire region blew up, an entire data center went down. That's a class of failures.

It's still not as prevalent, although it is. But the important class to think about is also the transient failures. The thing slowed down, it stopped working for 10 seconds.

This network was partitioned for 30 seconds, and they happen multiple times in and out entire day, right? So if we don't take that, that part into account where the incidence of failures is very high, the probability of failures is very high, which is inside a region, and sometimes it's transient, fleeting, it degrades, it doesn't fail outright. I think we'll miss the whole picture, right?

So that's the most important thing to think about. So think about the architecture from what it does as an SLA for the end user. That user customer experience cannot get compromised.

The, you know, validity of data, the integrity of data cannot get compromised because ultimately we are a data driven world right now, right? So, so I think those are the important parts to think about. And one of the ways I feel like this is getting solved for the non-AI parts and which doubles down when AI comes in, is using, you know, data, you know, pieces of the data stack that can lend itself to replicating and ensuring, you know, availability and resilience for the right, kind of the right kind of resilience with the right kind of trade off.

So don't compromise the performance latency too much, keep it good while being able to survive these failures automatically and letting somebody that takes care of a data center service these failures at humane hours as opposed to putting it on the critical path of it needs to get fixed for the users to really get their experience. So that is very important. That distinction starts to play out, and there's a number of pieces of the data stack that a user would need, an enterprise would need.

It's not just this database or that message bus or this queue as a lot of pieces. So it's important to think about this resilience for every layer, every component in the stack, right? And, and that's where we come in for one of the pieces, which is the transactional database, right?

And that's what we do for that. Do you think with, as rapidly as AI is advancing and being implemented into different things, is there still a skills gap in companies? Is that still a problem that they're having?

I think yes, there's a skills gap, and I think there's a bigger problem, which is the culture gap, right? So it's like a, a lot of people that I see, there's one camp that has embraced AI, perhaps a little too much. There is a, a, a second camp that has embraced AI to perhaps the right amount that they're iterating.

They know its place, they feel like, you know, we need to find out how we can use AI and embrace it. And a third camp, which is probably a majority that feel like AI is, you know, too advanced. We're probably sitting it out or waiting, right?

So if there are dangers there, there's probably a little less danger if you're embracing it and moving forward. There's a lot of danger in getting left behind because, you know, there's two things. There's a drafting, the industry is an active part.

Everybody's active responsibility. So, you know, bringing out the needs, understanding what it can do, that unlocks innovation. The second one is because it's moving so fast, AI actually speeds things up, waiting it out for a bit.

It may just look like a couple of years, but the gap can be a lot more to cover because, you know, it's just intensifying everything. So, so yes, there's a huge cultural divide, and that brings with it the skills gap of exactly how and when to use ai, right? And so there's also, because it's such a nascent and emerging market, there's also different ways to solve the problem.

Like when you look at ai, should you go for proprietary models or should you go for open source models? Should you go for AI specific databases? Or should you go for general purpose databases with AI support?

Or should you go for like, you know, certain kinds of environments in which you run ai or should you make AI run in any generic environment? So there's a lot of these, these, um, things that are emerging, and I think it's important to get the right skill, the right mindset to understand what the trade offs are, because that'll also ultimately help influence the direction of where we go as an industry. Hmm.

So if there was one key takeaway you could leave our audience with today, what would that be? So the key takeaway I would say is, you know, um, it's a twofold, it's a balance. So find the right balance between moving forward and understanding the problem space.

So things like performance, scalability, resilience, are already here to stay. And you, you, I mean, like, and it's only gonna get more intensified by ai. So that's on one side, which is the fundamentals.

So you don't want to invest in AI with a weak fundamental only to have a poor experience for the user, because they would assume the stack is very advanced because it has ai. But if it's not able to be able to have the fundamentals and deliver on that, they'd be like, oh my God, this is not really what I expected. So that is one very important thing.

And the second piece of that coin, the second side of that coin is don't sit it out, you know, be involved, even if it's in a small percentage, find the right percentage that works for each business. They have to do that, but, you know, find the right culture, mindset involvement and stay in the gate, right? So it's, it helps a lot in crafting the thought process in evolving the company and guiding the industry.

All right. Well, thank you so much for coming on the show and sharing your insights with our audience today. Absolutely.

Thank you, Amanda. Really appreciate it. Thanks for having me.

Yes. And thank you to our audience. Stay tuned.

There's more.