How AI Agents Are Driving the Next Wave of FinOps Optimization

Hey guys, thanks for the throwaway here with Rahul Ke cars Chief Product Officer for Digitate. And we're talking about well, cloud computing costs and how maybe AI agents are gonna help us. Finally ring this in.

Raul, welcome the show. Thank you. Thank you, Mike.

I think it's, you know, not much of a secret, but nobody likes to talk about it, but the utilization rates for it infrastructure in the cloud, or for that matter outside the cloud, are not so great and we wind up paying a lot of money and a lot of extra money for things that we're not really using. And I wonder, you know, I mean, the truth of the matter is it's just difficult for humans to do, but with the rise of AI agents, are we now gonna have a tool that can kind of constantly monitor this stuff and maybe bring down those costs? Yeah, absolutely.

Right. I mean, there is this cloud growth paradox, right? So adoption is growing at 20 plus percentage year on year, but at the same time, the wasted spend is a third of that cost.

So extremely inefficient. Majority of enterprises, the unit economics of this do not really add up. Uh, the challenges are very well known, right?

It's very, cloud is very easy to provision. At the same time, it is equally easy to forget about it. This leads to sort of over provisioning sprawl, lack of overall understanding of usage and the cost At the same time, uh, amount of information that is generated for billing as well as metering is huge and extremely siloed.

The providers don't make it easy for anyone, and this composes the problem because of multi-cloud kind of scenarios, right? So it's impossible to analyze this manually, for sure. Uh, traditional approaches, tools, techniques, if you see that they are primarily generating statistical observations, uh, often with siloed data sources, uh, it's like a barrage of observations that gets generated.

Picking the right observation from a haystack is a huge issue, often leads to information fatigue and translating this into actionable recommendations that actually get accrued in terms of savings is another ball game, right? This is the challenge. So we need a new tool sort of tool that typically goes across multi clouds and, uh, does not cause fatigue for the enterprises.

Mm-hmm. Will the cloud service providers get on that program? Because a lot of them seems to me, to your earlier point, first we said we had no insight and then they kind of just made so much data available that we couldn't make any sense of it.

So do they have a vested interest in kinda ensuring that there is gonna be some sort of AI agent that really optimizes consumption and acquire resources? Absolutely. I mean, uh, I mean some of these, uh, bills so-called bills run into like gigabytes every month, and, uh, how does anybody decipher?

This is a challenge, and I wouldn't go that far and say that it's deliberate, but it is just very fine-grained metering information and every provider will provide it in their own manner, in their own format. That, that makes it very interesting. Has the rise of AI kind of changed the game a little bit too, because I think as there's more AI workloads are deployed, there's more concern about optimization by the cloud service providers who are kind of trying to maximize the capacity of every data center they have.

So maybe they now have a greater vested interest in making sure customers do use all this stuff efficiently. Yeah, I mean, uh, on one hand I think there are just too many variables at play here. I mean, there are just variety of subscriptions, models, costing models.

In addition to that, the operational discipline gaps typically in things like tagging and all create further chaos. Uh, that's the issue. And, uh, there are multiple stakeholders involved in this kind of setup, right?

In any, every enterprise, there are business owners who want to govern the spend, govern the return on investment, and budget everything and approve everything, right? The operations teams, on the other hand, they want to monitor, implement cost controls wherever possible and act on the recommendations, right? And then there is this finops teams, which are responsible for actually becoming the glue in this very complicated data that gets generated, right?

So they need to review spend, they need to review, uh, budgets, periodically allocate costs, do the trending, identify savings opportunities. So there are multiple stakeholders with different objectives, and this is where the AI agents kind of bring in unified transparency. They're able to distinctly quantify value, optimize the usage and cost, uh, consistently, and most importantly, prioritize actionable recommendations that actually lead to accrual of savings, right?

It's very, very important, uh, to establish this consistency collaboration amongst different stakeholders so that the overall value that gets delivered as part of a finops program, uh, is consistent. That's really where the AI agents, I believe, are tremendously helping. Um, I often felt that it wasn't necessarily that the developers and software engineers didn't care about the cost of the cloud resources.

It's just they never had any real tools or insight into what was happening there. So they couldn't make a informed decision. And in the absence of that, they kind of defaulted to, well, I just wanna make sure this thing is always available, so I'm gonna provision as much infrastructure as I possibly get.

Um, that's a natural human tendency, but will the AI agent be able to better distinguish between, you know, our availability goals and our performance goals and our cost goals? 'cause that's seems like it takes a fair amount of reasoning. Absolutely.

I I believe so. And we have a, a fair bit of experience in this area, right? Uh, one good thing this machine kind of things AI agents are able to do is they're able to localize areas that need attention very quickly, because as we discussed, there is just too much data in different silos.

Uh, they are able to bring to the table a lot of levers, right? Things like right sizing, right pricing. There is just too many pricing models out there.

Pick the right one that is, uh, able to leverage the variable cost models of cloud, right? Configurations, right placement. Are we consolidating the workloads, right?

Optimizing location strategy and so on and so forth, right? They bring all of these levers to the, uh, table. Um, they also do a great job in forecasting, right?

Because many a times the way these cloud costs are managed today, it is, uh, reactive management, right? So typically people will then allocate a large lump sum budget for unknowns, and then they will subtract from it as, uh, like you said, right? People just want to provision and, uh, forget about it, right?

That costs Subtracts and One fine day you are like, you bridge the threshold and then you start getting notifications. Everybody reacts to that, right? That game needs to change to a more, uh, react, uh, proactive game, uh, that is sort of embedded in day in a life of all of these personas, right?

If it becomes here to another job to be done, then half the game is already lost. So it needs to seamlessly fit into their day to two operations only. Then they will actually really, uh, take advantage of the optimize.

Hmm. Um, how will these AI agents can negotiate with what I'm assuming will be AI agents put out by the cloud service providers who are gonna be their customer service reps, and those AI agents maybe have a keen interest in ensuring highest margins possible, and the AI agents, the IT team has, are gonna be interested in how do we drive down the cost of this as much as possible? Will the two of those negotiate, or will they just kind of beat each other up to the point where they're just gonna call us for help anyway?

Right? No, so I, I think, uh, I mean, negotiation is perhaps a few, few months, few years away, uh, perhaps, uh, the models, the variable cost models of, uh, both cloud providers and the kind of deals that get signed between enterprises and the providers are very dynamic in nature and, uh, at some level they're dynamic from a consumption point of view, but you do have a sort of menu card, right? Where there is a price unit price attached to it.

So for a given period of time, it is a reasonably deterministic problem to solve. The challenge is that menu card keeps on changing very often. As a result, you need the power of AI to sort of become dynamic adaptive so that you are able to solve this optimization problem week on week, month on month on a continuous basis.

Eventually, I believe, uh, we will get into some kind of a marketplace situation where that supply demand will take care of, uh, pricing. Mm-hmm. How dynamic do you think all this will get?

Because theoretically I can move workloads, but a lot of people today don't because, well, it's hard and they've also probably signed some sort of enterprise licensing agreement where they get rewarded for consuming more resources over the course of a year, and so they're reluctant to move just to get a savings on a particular spot instance for, you know, a week or two. So how, how do you see all this playing out? Yeah, so I, I think, I mean, naturally if you, if you look at the enterprise IT distribution, right?

It is not, not just the production business applications, but they do run a lot of dev test integration kind of, uh, workloads, all of that, I believe, uh, within a cloud provider, right? There are obviously, uh, not withstanding the data localization considerations, there are right placement, uh, strategies possible that are able to pick the right, uh, I mean the lowest price point for a resource by changing regions and things like that. So that's, that's one thing, right?

You can move workloads that are movable to those regions, uh, going across cloud providers. I think, uh, with more cloud native stacks that are like containerized, I think it is becoming easier to do that. We have, we have ourselves been able to move across multiple cloud providers, not, uh, like instantaneously, but it is the, that period is coming down to hours and not days and weeks as a result.

I believe as we mature on this path, this is going to be a lot more dynamic in terms of, that's, that's why I was alluding to a marketplace where, uh, if you keep on reducing time taken to migrate or transform, eventually you will get to a point where it's, it's a very valid option for optimization. That's where we are going. I think As you kinda think about these AI agents, or is this gonna be like a capability that I buy from somebody or is it an AI agent that I as an organization need to go build and kind of train and, and optimize?

I mean, where do these AI agents come from? So I believe, uh, buying is the best option at this point of time because, uh, the time to build these, train these and do upkeep is a fair bit of investment for every enterprise to do by themselves, right? So buying these so that you are able to seamlessly put them into action into your enterprise persona day in life, day to day life, uh, so that the conversation becomes easier.

The AI agent effectiveness keeps on improving as it learns more and more about the enterprise. Uh, and then you sort of start leveraging that insight because then people warm up to it. Uh, I think that's the best way to sort of, because this is also going to require a little bit of change management from an organization role point of view, right?

It's not like, uh, it's not a person that you talk to, right? You converse to a machine and it'll take a fair bit of adjustment. So the more seamless, uh, the agents are made so that you don't disrupt the in life of the stakeholders, the better chance they have to simplify their life and realize the savings.

Mm-hmm. I think also one of the challenges people have had historically is just it's hard to figure out how to compare apples to apples sometimes in these cloud environments and different things and things have different names and different price points. So will the AI agent kind of make it easier to normalize all that so I can make an informed decision?

It may not be one that I'm gonna make every day, but it seems like today, uh, I don't even make the attempt because it's just too hard to figure out what's what, Right? No, I absolutely, I think a common ontology, if you will, uh, and it, it's not the first time this is happening, right? If you look at procurement, right?

Uh, procurement gets standardized because people put out these common catalogs so that vendors and, uh, buyers can talk the same language same way. The instances maybe called M1 here, M two there, M three there. We we'll get normalized at some level on this ontology so that there is a common vocabulary because it's absolutely, you are absolutely right.

The chaos we will create, because the same thing is called in 10 different ways, uh, needs to be managed well. So this, there is no choice but to have this common ontology that describes a resource type, resource configuration, and price point. Yeah.

So we've had this notion of finops for a while now it's kind of a set of best practices for consuming cloud resources and hopefully it teams and the finance team are getting together. But, uh, the truth of the matter is, I think it doesn't happen as often as we would like or maybe, uh, have wished for in the past, but are we getting the point there where maybe those finops best practices in the age of AI agents just become ubiquitous and everybody has it, Right? You are right.

And uh, that's where I was sort of, uh, I would say that the nature of how the workloads are consuming cloud resources varies from enterprise to enterprise and within an enterprise it varies on a week to week, month to month, almost day to day basis, uh, even sometimes hour of the day basis, right? So, uh, and the price points because of the contracts also vary. So even, uh, even if the strategy is more or less, I mean, right sizing is right sizing, right?

I mean, there is nothing anybody does right sizing will do right sizing as a strategy. Uh, the ability to, uh, do that effectively in a closed loop manner, and I'm using the term closed loop very carefully here. Not only give out a recommendation or observation, but also give a prescription on how to right size a specific observation that is observed and then possibly automate that right sizing so that you actually accrue the brown dollars, if you will.

I think that is really important. Uh, unless you do that, it's an advisor and it'll have only that much value, uh, unless you sort of, uh, force the issue by going two more steps to give prescriptive recommendation and at a click of a button, the agent is able to actually enact that recommendation. That is going a long way, in my opinion.

All right. So the AI agent needs some sort of automation framework to execute against then that will be as big a part of this conversation as the AI agent itself. Absolutely.

All right, folks. You heard it here. The way we consume cloud is never gonna be the same.

The cloud's not going away anytime soon, but boy, you're not gonna recognize it soon. Rahul, thanks for being on the show. Thank You.

Thank you. And back to you guys in studio.