CloudBolt Brings FinOps Discipline to AI and Kubernetes

Hey everyone. Welcome back here to Techstrong TV. I'm really happy to introduce you to Yasmin Rajabi.

Yasmin is the chief operating officer for CloudBolt. io is the website. We'll get that out of the way.

But let's welcome Yasmin. Yasmin, it's a pleasure to have you on here. Thanks for joining us.

Thank you for having me. Our pleasure. So Yasmin, I mentioned you're the COO over at CloudBolt.

But I always like to give our listeners, watchers, whatever they are, readers in some cases, because this gets transcribed as well, a little bit of background on who they're watching, who they're listening to. So if you don't mind, give us the path you took to become COO at CloudBolt. Yeah.

I'd say it wasn't a standard path to get here, and COO is a lot of things at a lot of different companies. I run product and engineering here at CloudBolt, and that is my background. I've been doing product for probably most of my career.

If you go far back enough, I started out in engineering. And I was very lucky early in my career. " And I had enough people that told me that.

I was very stubborn. I was determined to stay in engineering. I think my engineering team will tell you they're very happy I didn't do that.

But getting into product, especially at Puppet, during a time where infrastructure was changing, and that was a good balance for me of engineering and product. And I got to learn a ton at Puppet. I also got to meet a lot of really great people in that community and at the company that I still work with today, and we have a lot of ex-Puppeteers over here at CloudBolt.

Oh, I didn't know that. Yeah. We've got quite the team here.

What's really funny or just full circle moment for me is when I was at Puppet, I used to come across CloudBolt in competitive articles and listicles and that sort of thing. And then now to kind of be back here and be helping shape what's next for us across our portfolio is super exciting. So did product at Puppet, did a little bit of services and customer success strategy, that sort of thing, but really just always came back to product.

And then joined StormForge, which is a Kubernetes rightsizing company, because infrastructure was changing and I wanted to kind of grow my expertise and kind of grow with the infrastructure change. And led the product team at StormForge before becoming COO there. And then we got acquired by CloudBolt and gave me the opportunity to kind of broaden my role.

So now here at CloudBolt to kind of look across a product portfolio, in terms of product and engineering and some marketing and partnerships and the technical stuff like AWS, that sort of thing. So, sometimes it's like a team of misfits. I'm technically also the CISO, so the role's been- So COO and CISO is not something I see a lot.

I see a lot of CTO and CISO or a lot of CIO and CISO today, right? Yeah. Which never used to be either, but now it's pretty common.

It's not standard, I'd say, COO and CISO, but I think it's just the nature of our team and being a technology company. Our security practices are very intertwined with product and engineering, and it's just the table stakes is what people expect. Especially, we collect a lot of data from our customers.

It's all metadata. We're not collecting any PII, but we need to make sure that we are providing that in the most secure manner, and we have a lot of federal customers. So, having kind of air gap solutions, FedRAMP solutions, that's kind of part of the game, I guess.

Absolutely. And, it talks to how, first of all, cybersecurity's become more important. I'm 25 years myself and more in security now.

And, I started a few companies in that space. It's so much more of a priority. It used to be such a fight to get people to take it seriously.

Now everyone has to take it seriously. And whether, like you say, whether you have PII or not, you are a vector into whatever network you touch, right? And so, it's the weakest link that always somehow they exploit.

So, and that's why it's important, regardless of what data you're keeping, that security's important. But you know what else? You said something at the beginning, you took an unorthodox route, maybe.

I don't think there is a normal route in technology. Life's funny like that, right? And I know a ton of people who started in the engineering coding space and then made the leap to the dark side and became business leaders, and their career, and never looked back.

Right? Their career just kind of took off. So congratulations to you.

And that's a great story. And I loved... Puppet, was Nigel there when you were there still or?

Yeah. Yeah. Nigel- I still talk to him from time to time.

He's awesome ... always enjoyed talking to Nigel. He's great.

Anyway, let's shift over to CloudBolt. It's been around a while because it was competing with you at Puppet. They've done some acquisitions.

I'm sure a lot of people who maybe knew CloudBolt back when you were at Puppet don't really know what the CloudBolt today is all about. So how would you describe it to our audience? Yeah.

So the CloudBolt of today is the infrastructure company that will take you across your cloud journey as that evolves. So it used to be traditional infrastructure, VMs, which we still do a lot of. Like VMware migration is still a very hot topic where people need help on, how do I evaluate the platform I want to go next?

How do I make sure I have leverage and I'm not kind of putting all my eggs in one basket? That's still a challenge people have, whether they're moving VM to VM or they're starting to migrate to Kubernetes, which we see a lot of, and helping people with that transition. Or now we're starting to adopt AI tools, so how do you make sure that...

Before it was like, okay, any developer can go provision infrastructure, and then now you have to make sure that's audited, managed, using the right RBAC. Now it's even easier because it's all natural language, and with MCP tools, you can interact with majority of infrastructure tooling and then go build what you need, do what you need in a way that's not necessarily governed from an enterprise standpoint. So a lot of the conversations I end up having with folks are, "Yes, we are adopting AI tools within our organization, but we still need to govern them.

" And so the problem is still the same. It's how do you manage tools in the environment and the infrastructure that they sit on? But the technology and the application of it has changed, and so just our goal there is, and our responsibility is helping those enterprises as the technology changes, and that's where we're focused on today.

I love it. So Yasmin, a confession. This is going back 10 plus years ago at least, and we were thinking about launching a site just dedicated to, at the time it was containers, it was Docker, right?

And DockerCon was in Austin that year. It was the first year it was in Austin because it used to be only up, I think, in Seattle. But I went to Austin to DockerCon to put my finger on the community, and I came back from that and I said, "I don't know about this container stuff, but everyone there was talking about this Kubernetes thing.

But I wouldn't worry too much about it. I had a demo of it. It's so hard.

" That's why I still have to work and I'm not retired already, because I should've known Kubernetes was going to be what it's become. But you want to know the truth? It's still not easy.

It's still hard. And allocating the cost of Kubernetes, if you will, allocating the aggravation factor, what does it really take, right? The whole platform engineering community basically grew up originally as how to manage the Kubernetes platform, right?

It moved to IDP, and now it's agentic, and it's all that, but it's still at the heart of the platform today. And it's one thing for you as the COO running engineering ops product. Then you got your CFO people, and they keep saying, "But isn't this free and open source?

" Right? " Now that Kubernetes stack's become the AI stack, right? AI applications are running on that stack, but then AI itself is being embedded into that stack to help run.

How do you, as the COO at CloudBolt, but also talking with your customers, how do you reconcile all this? What's the answer? Well, for us, everything we run is on Kubernetes.

So the same problem we help our customers solve, we dogfood or drink our own champagne. Or champagne, whatever. It depends on what day it is.

But- We have the same problem internally. And for us, especially like you mentioned CFO, it's very important to our CFO that he understands how much do customers cost us. And our platform is entirely built on Kubernetes, so on one hand, it's easy for us to have that visibility because we built the tooling.

On the other hand, it's a really hard problem to solve. And most teams, what we see, and this is how we started even, but a lot of enterprises we see have basic showback. " Or even if I take it one step further, I'm going to split it by namespace, for example.

I can tell you how much it costs. " But if you wanted to actually go back to that team and charge them for what they're using, the second you get it wrong by a dollar, then it's like, "No chance. How do I know what you're telling me is accurate?

If you're just showing me that, you're saying, 'Okay, go improve, go reduce waste,' that's one thing. " And I think that's where a lot of the solutions kind of fall over and where we've had to invest for our own needs and, of course, for customer needs of getting that bill accurate. So the same number that the finance team is actually paying for needs to be the number that the engineering team see if they're being held accountable.

So just taking a list cost and then kind of splitting it by usage isn't enough if you don't have the discounts that are applied, the kind of savings plans that are in place, and then the actual per container, per minute usage. If you're off by maybe 10 cents, people will give you some leeway. But once you start to get off by even just whole dollars, we've seen teams be like, "Well, how do I know you're going to get it right next time?

" Mm-hmm. And that's where a lot of the challenge comes. And we see teams kind of get into these political debates within an app team that's like, "Okay, first of all, I didn't know I'm going to have to pay for this," right?

And they get the bill afterwards, and then they don't know where to start to go and optimize it because in their mind, they're not incentivized to make it run cheaper. They're incentivized to deploy to production and add new features. " And so it's really like, yes, it's a technology problem, but it becomes a people problem, and how people communicate and how they build trust to know we're looking at the same data, we're looking at this accurately, and we're building tools that are safe to go and actually do something about it.

Agreed. And I see it. I mentioned something before about it's the AI stack, but it's also AI is being built into this.

How has AI kind of changed the equation at CloudBolt in terms of your Kubernetes platform? Yeah. There's a few different things.

Even just like we run GPUs because we're working on GPU optimization, so we have to run it internally. And I got the bill, and I was like: "Excuse me, what? " And we're a small environment.

Imagine you're a large enterprise. So on one hand, from a core infrastructure standpoint, that's a big shift of where you could use a slice of a CPU. Using a piece of a GPU is a lot harder.

Like, a lot harder. So people just end up giving workloads, entire GPUs, and a lot of waste becomes there. But that same problem I mentioned of accurate chargeback within actually allocating your Kubernetes use, think now with AI tools, it was very easy to pad a resource request in Kubernetes, right?

You just pick a number, aim a little higher, put in a buffer. But how do you know a developer that's building an agent, how do you know they haven't shoved 100,000 tokens of context in every request that they're making? And once they make a change, there's not that direct tie to cost.

It's so spiky. So getting to that granular ability to actually allocate the cost is a hard problem. And we're, again, dogfooding because we use a lot of AI to build as well.

So, pulling in the model cost and being able to get to the granular level using tagging of like, okay, which teams are using which models, at least gives us visibility to say, "Do you really need to be running Fable for this? " But where it gets really hard and kind of the problem we're working on right now is, how do you allocate when, say, you have an agent that fires off 15 sub-agents, and each one of those is using a different model. One of them runs for two seconds, one of them runs for a minute.

Getting that granularity of allocation is what feeds the accurateness of it. And if you're not accurate, people don't believe you, and you can't do proper chargeback. So it's the same problem, just way more complicated.

So where are you getting that granularity? So we pull it in from the focus spec on the public clouds. So think like Bedrock, all of their usage- Okay.

Yeah ... on Azure as well, Google as well, and we pull that in, and then we can map that to our usage of the application itself. Got it.

Let me ask you, and this has nothing to do with CloudBolt per se, but it's my own curiosity. Have you started using any of the open-weight models to try to bring token costs down? So we've considered it.

We've gone back and forth. We're still in the sweet spot where, and I don't know if I should say this, but we're in the sweet spot where we're a small organization, and a lot of our token costs are subsidized. Yeah.

Because we can use a premium- You can do that. Right ... account and get a lot out of it.

So- Right, the enterprise account ... we're lucky in that. When we tip over that employee count and that changes, I think the- That us, too.

We're in the same thing where we've got a Claude enterprise account, we've got Perplexity enterprise account, and it's enough for us because we're small. I don't know for how long because it's like a self-fulfilling prophecy. The more you use it, the more you're going to wind up paying when you hit that threshold.

But what I'm betting on is by the time we do, the equation changes, and whether it's open-weight models or something else, but we go through the same thing. Because the tendency is, oh, I want to use the latest and greatest. I got to use Fable.

I've got to use Five, Six, Sol, or what have you. But the fact of the matter is, it's not like we're doing serious coding here, right? We don't.

We're a media company. We're writing, marketing. That doesn't necessarily require Fable.

Yeah. But I hear you. I feel you.

I live it every day. I know what you're talking about. Yeah.

And our teams are actively looking at, okay, which models are better for writing the code, for reviewing the code? Right. Where can we delegate to cheaper models?

And then it's a lot about enablement, too, of like, I just this morning had a sales rep reach out and be like, "Hey, I need more tokens. " And it's funny, he made a joke. " Yeah, no.

I personally live that. But that is, to a certain extent, we've created this, right? And not just we, the AI companies created it, but the model companies created it is what I mean.

But I do believe that in order for AI to reach what it's capable of, its potential, we've got to be able to solve that. We can't have a sales guy worried, jonesing for when his next fix is going to come, right? We got to be able to figure this out, but I'm confident we will.

Anyway, Yasmin, we're over time, but I want to thank you for coming on here and talking. You know what's interesting? Usually, we get a lot of people who are talking about what they do for their customers, but it's also good to understand how companies themselves are building around this and incorporating it and dealing with it.

It sounds like you've got a great handle on it at CloudBolt. They're lucky to have that. We're excited.

I have a really good team. We're excited about what the next set of technology problems is and how we help our customers solve that, especially by just starting with ourselves. Absolutely.

io. You can get it all there. Yasmin, thanks for being on Techstrong TV.

Hopefully, we'll see you back here soon. Thank you for having me. My pleasure.

We're going to take a break. We'll be back. We got a lot more coming.