Redefining Cloud Operations with Cast AI’s Laurent Gil

Hey guys, thanks for the throw. We are here with Lauren Gill, who's president of Cast ai, and they just recorded $108 million in additional finance, and I guess we're gonna dive into what's driving that. But Lauren, welcome to the show.

Welcome. Uh, thank you Mike. Thank you for having me.

Alright. Even in the age of AI where people are getting funding in the multi-billions of kind of era, a hundred million dollars is still nothing to sneeze at. So what you know is the plan for this.

Where are we gonna go from here? What's left to be done? Oh, Mike, we have a lot in store.

So one, uh, what is really cool is the, and the investors who, uh, uh, who, uh, joined that raise are soft, SoftBank and G two G two Venture. Um, SoftBank is very famous to invest a lot in, in the AI space, as I'm sure you know. And G two has done a lot of pioneering investment as well.

They invested in a company called Cruso who happened to be a target partner of soft, of, uh, SoftBank, open AI and Oracle. So there's a lot of, a lot of things that will come from, uh, from this, this, these guys are trusting us to bring, to bring the AI industry to the next space. And I can't wait to, uh, to show that to you.

Mm-hmm. So where are people on this journey right now? I mean, I feel like we've been talking about automation of infrastructure, Kubernetes and all kinds of stuff for a long time now.

Um, how far are people gotten down this path? 'cause I feel like, uh, depending on who I talk to, there's still kind of all over the place. Oh yeah.

It's still all over the place, but there is a few things that happen to us. So, and we, we see a very large amount of the industry. We're probably the kind of dominant player now in the Kubernetes automation space.

So we have a lot of customers of users of the product. We see some very, very strong trends. One of them is that almost all the Fortune 500 that are using cast ai are using cast AI to run GPE workload, not for training, but for inference.

And we start to see the beginning of this, why, why I mentioned this is because inference is what we were all waiting for, you know, training. There is five, six companies in the planet somewhere very close with from our investors, but that's it. There's missile, great company, open ai, fantastic and many others.

What is exciting to me now is to see how these companies like BMW as a client are using those LLM to do some real business. And we see this happening because they run on cast ai. A lot of those workload run on caste.

In fact, one of the largest AI company in the planet is entirely running on cast ai. And so we have, we see this visibility, we see this workload, we see so many opportunity to make those things better, thanks to automation. That's what's happening now.

And to finish on this, we also see a lot more Fortune 500 being inside Kubernetes. So they have moved, or they're still moving during the journey of transition between monolithic VM based application to containers, whether it's on-prem or on the cloud. We see that migration has accelerated in 2024.

But it's interesting now is all of those applications benefit tremendously from efficiency and automation or from what we can see from our, our angle Has the rise of ai, which is dependent upon GPUs. And most of those applications seem to be running on Kubernetes. Kinda made people a little more sensitive to costs in general.

And they're trying to get higher utilization rates. 'cause frankly, you know, we kind of ignored utilization rates for as long as I can remember. Yes.

Uh, the same thing for gpu. So I'll, I'll tell you, uh, what we see now in the market case. So the issue with GPU is that the hype is very strong because the hype is strong.

A lot of companies are rushing to buy GPUs in advance. Sometimes we see companies buying one year or three years in advance. It's called reserve instance on the cloud of, uh, of, uh, GPU expensive GP I'm talking H 100, a 100 on all the three cloud providers.

What we, uh, we start to realize now, and what our users are realizing now is there are much better way to identify GPUs and have them when you need them. That information is public. You can browse the, the hyperscalers.

They will tell you, they will give you a kind of good way of thinking, oh, this region have availability for a 100, oh, this one has some H 100. But because US human, we only maybe do it once a month or once every few months, we're gonna miss entirely the picture of availability. And for GPU, cost and availability goes really together.

You have to find them. And once you have them, you have to use them. That's what we fix.

That's what we solve. Our automation, our AI agents is so aware of where all these machines are, but not just that we know how much they cost. We know how much quantity of computation compute they provide.

So we are able to choose the right type for the right job. And we can even find spot instance on the hardest to find GPUs. And Mike.

Mike, we did a, we published a report a few weeks ago. We made a simulation of the following. If you were agile, agile means you would be able to move inference around based on availability and performance.

Then in 2024, you would've spent between 10 times and three times less in GPU workload as opposed to if you didn't know where they are. And you, if you could not move around. Sometimes US east has the best price, but, um, um, it may happen for a month, but the second months may be US West.

Maybe it's going to be Europe and Frankfurt. Maybe it goes to Asia. You see, the beauty of what we do is we both and engine, an AI agent that mirrors exactly what the human would do, the DevOps engineer would do, except that our agent doesn't do that cycle once or twice a month, does it every second.

So it's able to move things around and adjust and make it more efficient and make sure that what you have is what you need. Nothing less, but not too much above it. Mm-hmm.

And when you do this, when you mirror the DevOps workload, but you make it every second with something that is obsessed with efficiency, then you get this kind of result. And lemme tell you, Mike, we we have a lot of Fortune 500 on the platform when we show these to our users, it's the difference between the science experiment and something in production. When you show this kind of, of efficiency of three x four x five x lower than the average cost on any hyperscaler, then suddenly you see the light bulb with our users saying, oh, I can do it now.

'cause now there is a return of investment to use LLM from a chatbot, oh, for my Gen AI application. That's what we, and that's what SoftBank see, and that's what G two C and that's why they invested in us. I feel like there's a, a cultural aspect to this and kind of comes down and developers, you know, they have a bias towards not getting called at three o'clock in the morning.

So they'll over provision as much as they can. And we haven't really kind of put any guardrails around that whole issue. And the end result is utilization rates are kind of abysmal.

It's one of the dirty secrets of it. And then CFOs scratch their head and say, how come everything's so expensive? But is there something we need to do, or conversation we need to have with the developers who at least theoretically are supposed to be building modern applications that dynamically scale up and down, but don't, You know, you, you really speak up the dilemma.

That's the key, what you said. Uh, it's, it's a debate between do I do it myself and I control it, or do I trust automation to do it? The, the, the way we talk about it with our client is the following, in a few minutes, you are going to see the result, the impact.

It's instant, but look like it's instant because it would be instant if you would do it yourself as well. Except it's not someone, it's a, an entity does it so fast that it turn on every possible aspect of making this more efficient. It's so fast that I call it irresistible.

Like, like think about it. You, you turn on something and your productivity increased by 10%. Say 10%.

I need to do some work for that. It's not worth it when you show 50%, 60% impact, it's irresistible. There's no more debates us, you know what they say, how soon can we extend this everywhere?

It's because the value is so deep, so hard, so impactful and substantial. Why is it substantial? Because it's the difference between you do it yourself and you have something that would've done the same thing you would, but did it so fast using all the possibilities that the arsenal of tools that the hyperscaler will give you.

And that's the, that's the irresistible impact of cast. That's why the, the Fortune 500 are all there. But most of our clients now are these large enterprises.

And you would've argued, well, they're probably the one moving carefully, let's call it this way. But when you show this like we had a stock exchange, the cost of goods will draw by 40% a stock exchange, 40% that that's it. There's no need for, oh, do you wanna keep it or do you wanna go back to trying to do it yourself with the screwdriver?

Mm-hmm. That's, that's the philosophy. So do we do this to your point, at the top end of the food chain where it's some sort of command and control system?

Or do we really just need to kind of put the costs in front of the developers and the data scientists in a way that they can easily see? Because a lot of them would say, I have no idea what the costs are, so I'm just trying to execute the mission. That's a great question, but no, and I tell you why, because we already beyond this, um, we are not, you see, we are not a finops tool.

We are not telling you where you spend money. Of course, when we reduce your cost, we need to show you what you are spending default. So yes, that piece we have of course, but what we do is we give you confidence that whatever you are spending is just right.

It's a right amount. This is the impact of automation. We're not here to do, to give recommendations.

Our agent is actively fixing things. And Mike, one, one thing people believe, oh great. So you will eliminate over provisioning.

That's fantastic. But not always. We made a study last year on, in 2024 and we realized the following.

Look, listen to this. It's mind blowing. 6% of your containers on average are agreeing to run out of memory in the last 24 hours, 6%.

The rest, they are overprovision five times five x, but 6% of those, it's actually not the case. It's the opposite within, it's a, it's statistically relevant, right? This is on millions of, of CPUs, thousands of users, 6% of what you have is going to run out of memory.

You know what happened when they run out of memory, they break. If you check in, you may have to check in again. It breaks the application.

Reboot, restart. You cannot make up memory. You can, you can have something slow at CPU, but you cannot make up memory.

It breaks it, restart. We fixed that. You see how deep that is with automation, with, let's call it a smart engine.

The smart engine is obsessed to be sure that whatever you need is what you have. Sometimes you don't have enough. We fix that.

Most of the time you have too many. We fix that too. That is the beauty of automation.

When you show these to users, when they realize, when our, our partners DevOps publish realized this, there's no more discussion about automation. No automation being in control. It's so, so much better that what you can do yourself.

And then just to finish on, 'cause we worked a lot of the on this, you are always in control. That's the most important for us. We build these tool for the DevOps engineer, for the application owners and the DevOps engineers.

They're in consult. They can say this, I want this one. I don't want this one.

I want to be more aggressive. This one add pad because I'm more comfortable when you do that. So you see as a DevOp decision, you know, you become an analyst, you describe how you want this to be, and then you just watch it.

Have you seen The mood among customers change in the wake of the current, a level of economic uncertainty that we're all seeing? Or are they kind of making some assumptions about maybe not even being able to get additional hardware? So they're trying to think through how do I increase the utilization rates of what I got?

Yeah, we, we see, we see a lot of increase of utilization rate. That's a lot of, uh, discussion we have with, with our, our client. But we see also a lot of expansion in what we do.

So if we go, we recently, we, a very large manufacturer, car manufacturer in Europe went on board and they were, they just put one of their applications, they have thousands of them. They put one or two applications, tiny, tiny things, what we see last month. So we, they, they are customers since December, what we saw in March, early March.

They call us back and they say, Hey, we are ready. We are ready for what, oh, we are ready to put everything we can because now we have pressure. We've seen how efficient you are.

We need everything to be efficient as soon as possible. So we see a lot of acceleration that way. Acceleration of adoption of automation because the impact is so important.

And right now it's so important. So as you kinda work through this, what's your best advice to organization, short of a course, getting your platform, but is there something they should be thinking about how they structure themselves or function in this environment today? Or, 'cause you know, AI is new to everybody.

The economy is different. A lot of the ways I used to think about infrastructure need to change. You know, IIII, I don't like when people say, oh, everything is getting increasingly complex, increasingly expensive, increasingly, increasingly increasing.

No, it's very easy. It's very simple. There's nothing complicated.

Think of what you do once a month or twice a month. Think now that you have a colleague. You tell the colleague what to do.

It's an agent. You tell the colleague. It really is like this by the way.

You, you really do that. You tell a colleague what to do and you ask the colleague, can you please do it for me? Oh, and do it all the time.

In real time. Constantly, continuously. That's, uh, what we give, uh, advice to, uh, to, uh, to teams.

Say, don't throw it. You are always in control. You decide how you want your team to behave.

The same way as you hire someone, you're going to train that person and into joining your team is exactly this. There's no difference for an AI agent, except the AI agent will take no holiday, no, no lunch, no breakfast work, 24 by seven at nights and weekends, because you tell the agent to do it that way. So there is, when, when you explain it this way, there is no fear, there's no, there's no, oh.

But I don't know. No, you, you know, because you are the one teaching, training, asking the agent to help you. 1.

It should be that size and not that size. You know that as a human. I can see it with my eyes.

You, you use Datadog, you use Dynatrace, great, great of Observ product. They will tell you this. But what do you do when you have 10,000 of those?

You say, ah, they're just too many. I'll do the important one. When you use automation AI agent, it will do it for all of them in real time.

Sometimes it will become that, sometimes it will increase to this, right? That's a phone to this. Sometimes it'll go back to the small airport and so on.

Something will disappear when it's not necessary. Sometimes it will ask to run on the GPU because it's required. Oh.

Other times it would take a small piece and move it around because you need to charge it. It really is that way. Mm-hmm.

And when you see it, there's no more question. It's great. Yeah.

Folks, I think you heard it here. I mean, ultimately the workloads, it's not so much complexity, it's just the, you know, entire environment's a little more dynamic than ever. And you're gonna need a little help from some AI to get through that.

But it can be done p Laurent, And the AI is friendly. That's the point. There you go.

Hey Laurent, thanks for being on the show. Thank you. Bye.

All right, I'm back to you guys in the studio.