Flexera Brings FinOps Discipline to AI Tokenomics

Hey guys, thanks for through. We're here with Jay Litkey, who's the senior vice president for cloud and FinOps at Flexera, and also a member of the governing board of the FinOps Foundation, and we're having a little chat about, well, there's a new animal in the zoo. They're called tokens, and they cost a lot, and we're running out of them, and nobody seems to know how to manage these things.

Jay, welcome to the show. Thanks, Mike. Good to see you again.

Good to see you. We've been talking about FinOps for a while, and now people are using the phrase tokenomics, which I guess is a thing, but explain if you would, is tokenomics a real thing, and is it going to be with us for the short term or forever? Tokenomics is a real thing, although I'll say in the circles I play in, it's only a month old.

So it's a brand new baby in the scene. I think it's here to stay, at least for a timescale measured in years. Let's not say forever.

But it really is apt in reaction to this explosion of AI spending and usage. And also, now you're part of the FinOps Foundation, which is an arm of the Linux Foundation, which also launched the Tokenomics Foundation. So what is the relationships between these things?

Are they separate and distinct, or are they extensions of each other? Yeah, I'll say that. They're all under the umbrella of the Linux Foundation.

So as you know, as you mentioned, the FinOps Foundation's been around for quite a while. I'm on the governing board. And last, about a month ago, or a little less than a month ago, JR Storment, who runs the FinOps Foundation, announced the intent to create this new body called the Tokenomics Foundation.

So it'll largely be a sister org to the FinOps Foundation, and there's going to be a lot of connective tissue between the people that work in both, the companies like Flexera that will be involved in both. But again, all under the umbrella of the Linux Foundation, is where it's going to live. So to that point, will there be some crossover among the teams at the end customer who manage this?

Because it feels like the people who are monitoring the tokens may not be the same people driving the FinOps because they're at different levels of the organization. Or from a customer perspective, how do you see them approaching this whole thing? All of the above.

So let me sort of explain here. FinOps, as you know, in recent memory, revamped its mission to focus on technology value as opposed to just cloud. So for quite some time, FinOps has organically grown to span a lot of the traditional technology silos, including cloud and licensing and data center and what have you.

Well, now AI has come like a freight train, seemingly overnight. Hasn't been overnight, but it seems like it has come overnight, and it's permeating into all of those same technology silos. So in organizations where somebody is a more mature FinOps shop and their team has already expanded across all of these technology silos, then I would say, they most likely, or they will be heavily involved in the management of tokens and of AI.

Whereas in a more traditional, or I'll say legacy or less mature FinOps organization in the enterprise who maybe still only owns cloud, then yeah, there's a lot of AI stuff and a lot of token stuff happening outside of the cloud, and so they're going to have to collaborate with other people. So I think the summary is now that AI is a freight train, it's come into all facets of technology. It certainly is relevant to FinOps people, more or less, depending on how many scopes, FinOps scopes they've evolved to embrace.

We hear the phrase token shock, and a lot of people have quickly gone from token maxing, where they're- Yeah ... driving people to use as many tokens as possible, to now questioning, "Well, wait a minute. " So what's happening here from your perspective?

Is this just a natural curve of maturity just happening at an accelerated rate, or... Because it seems like we're bouncing pretty hard between one extreme and the other. I've never seen such a rapid adoption.

I haven't seen all technology waves, but let's just say I've been at it a while, hence the gray hair. I've never seen something adopted so quick. I've never seen something that goes through massive transformational changes itself so rapid.

So you're right. It feels like it's oscillating from one side of the ship to the other. It wasn't that long ago, the token maxing, some people thought it was a good idea to say, "Hey, we want to drive the adoption of AI.

It's this new thing. It can drive value. So let's create a leaderboard.

Let's gamify it. " And so guess what happened is people did do that, and they found ways to do that very well. And so that token maxing didn't stick around too long.

It went from we want you on the leaderboard, and the superheroes are the ones that are on the leaderboard, to seemingly overnight now, we don't want you on the leaderboard of burning the most tokens because you're burning the most money. So it's a whipsaw for sure, and it happened in a very compressed... And not all that token consumption directly related to something that drove a return on that investment.

It seemed like we were essentially incentivizing people to burn tokens, and they just went out and created all kinds of weird things to go do, but not all of them had any immediate value to the business. So do we need to just get smarter about what projects we're kind of launching and starting here? We do, and one thing on token maxing, without naming the guilty I've spoken to developers out there in the world who intentionally wrote code or submitted code in certain ways to maximize that token maxing to get on that leaderboard.

And so that certainly wasn't valuable. But in the broad purview of AI, I think the reality is the ROI of these AI investments is far from proven for most of these investments. So there's a mass rush into using AI, and we are early innings or early days of people being able to speak to the value from those investments.

And to be clear, there's going to be a lot of bad investments made as people experiment and learn what to do. But the majority of people out there cannot definitively draw a straight line from their investments in AI to ROI or value. So as we kind of get our heads around this whole token consumption FinOps model, is the rise of AI and this token conversation pulling us along to revisit the best practices for FinOps in ways that maybe a lot of organizations kind of saluted, but they didn't really implement as deeply because, well, it was hard to get everybody on board.

But now, maybe there's a more compelling use case here. Yeah. So I think there's a lot of talk last week at the FinOps X conference, right?

That's where all the FinOps community comes together, and I was there, around how is AI and tokens different. My view is it's the same but different in that one of the wonderful things that FinOps really did was put practices and words and people's careers around managing consumption-based technology. The first consumption-based technology that FinOps aimed at was the public cloud.

So the roots of FinOps was in public cloud. As I mentioned at the beginning when we started speaking, the mission of the FinOps community has organically grown to not just focus on managing the consumption of cloud, but the managing of consumption of all sorts of different types of technology, including SaaS, data center licensing. So the principles of managing and the practice of managing consumption-based technology is very applicable to AI and tokens.

But when I said it was similar but different, or the same but different, there are new constructs. There was no concept of tokens, of course, in traditional FinOps. There are new kinds of consumers now.

In FinOps, we thought a lot around allocating costs to teams or to groups or to people, users. Well, now we have AI agents coming on like a freight train, and so consumers of AI can be AI agents. And so we need to kind of rethink now the surface area and some of the constructs of what does AI mean, and how does FinOps and its principles, its standards, its practices, how does it need to evolve to encompass AI as well?

So the summary is consumption-based technology was always at the heart of FinOps. AI is very much consumption-based, so we have a great foundation to start from in the FinOps community for managing AI. But the differences are here, and that's really what we have to extend.

And what JR Storment realized was that's different enough to do it through another foundation called the Tokenomics Foundation. But to use some of the same standards, like the Focus specification. The Focus specification is a standard from the FinOps Foundation around cost and usage.

How do we create standards around that? We want to extend those things we've built in FinOps now to manage AI. So it's a lot of mouthful to say.

There's a lot of great connective tissue and learnings we're going to apply to Tokenomics in FinOps. They are related but different. And now the Tokenomics Foundation is going to focus on those differences and in many respects, extending FinOps, right, to satisfy them.

How dynamic will this get? Because I can imagine that the AI agent is smart enough for me to embed a set of best practices for it to follow, and as such, then it will just start looking around at the different LLMs, and based on the task assigned, will pick the LLM that is the most cost-efficient way of doing that, assuming that the mission can be accomplished. Is that a fair assessment of where we might be?

In theory, if everything's working perfect, and if the agent's working on perfect, immaculate, correct data. Garbage in, garbage out. One of the risks here we know is agents in AI, its lifeblood, it feeds off data, and it makes its decisions off data and context.

And so there's real risks of agents doing things we don't want them to, and they can do things very fast in real time. Agents can spark workflows, so consumption and spend can get very high very quickly, right, without a careful eye. But again, if agents are acting on poor context, poor data, they're going to make incorrect decisions.

So we're not going to be in a world where everything always works perfect, even though it's automated. We need to have mechanisms in place. We need to have controls in place for when things don't work like us humans want them to.

Things don't go from us, too bad. Do you think the providers of the LLMs and the cloud service providers might recognize that that mistake was made and then not charge for something that spiraled out of control? Or is it in their interest to make sure that it didn't spiral out of control in the first place?

I look at just history repeating itself as much as we like to think that we'll never repeat past mistakes. If you look at just the whole FinOps movement, if you look at just systems management and vendor community in general. If I look at the public cloud providers, they never solved their own problems for cost optimization.

They were happy to see your bills going up In many respects, and that created an entire ecosystem of vendors and a whole practice called FinOps to help you control your cloud costs. I think if we view the new frontier LLMs, all the AI providers, as similar motivations and a similar pattern as the cloud providers, they will do some things to help you manage your bills and your runaway, but they're incented to see you consume more. And I don't believe today the AI providers are going to fully solve the problem and provide all the capabilities to make sure we, as consumers, only consume the bare minimum of what we need.

Are tokens the right atomic unit for tracking these costs, or is there another way to think about this? They are an atomic unit. Right now, folks are saying they're the best we have right now.

We do need to recognize tokens only account, though, for a portion of AI spend. There's storage, we put our data in storage, there's software licenses. There's a whole bunch of things, labor, that go into managing the AI.

Tokens is the best atomic unit we have for the AI economy. But to be clear, if we're thinking about holistic costs of AI, tokens is just one component. " Or is this going to be like the way we consume cloud resources on a spot basis?

Or- I think all of the above. You're going to see the providers come out with financial instruments to try and get you to make a commitment- Mm ... to say you're going to use a certain amount of their capacity, and they'll offer you discounts for those commitments.

So very similar to how the cloud providers created all of these commitment-based vehicles to get you to commit. I think you're going to see the same thing continue. It's already started.

You're going to see it continue with the AI providers, and this is going to be a very fluid, fast-changing landscape as we're already seeing. So you're already seeing the velocity of new models coming out, already disrupting previous generation models. The time between those is so compressed, it's incredible.

" So I think all of that's going to swirl for a long while before it settles. But yeah, I do think you're going to see more and more of these commitment-based opportunities. It's in the interest of the AI providers to get us, as consumers, locked in.

But on the other hand, we've got to be very careful because of how fast things are changing as a consumer. Some folks are saying that there will be a collapse of the pricing of tokens as more open source models come on, and basically, the cloud service providers will fire up open source software as they always do, and it'll just be an AI model, and they'll charge for tokens per se, but it won't be as expensive as proprietary models. What's your sense of what's going to happen here?

Or we see enterprises already pursuing AI in their existing capacity in their data centers, saying, "I've got capacity. " So we're going to see all of the above. I think you're going to continue to see a need for premium AI models out there for certain use cases.

" And then you're going to see, obviously, some of the especially larger enterprises that already had data center investments, or even individuals running their own AI on their own PC under their desk. We're going to see all of the above as different types of AI to tackle different types of use cases. Do you think as we play with all of this that maybe we just need to get some sort of dashboard in front of the people who are consuming all this in real time so they can see what their actual costs are?

I always felt that one of the issues with app dev was that the developers didn't really know what it was costing them to consume that infrastructure, and I feel like maybe we're about to repeat that mistake in the age of AI. But is part of this issue that we just need to put the information in front of the right people at the right time? You are going back to think about how public cloud got adopted.

We don't know who's using it, we don't know how much they're using it. Let's give them a little visibility. So I think that's exactly where we're at.

We are at a low state of maturity in traditional systems management speak, where we need to give people some basic visibility about what people are using. Once we illuminate that, we did that with cloud, then the next questions they start asking are, are they using too much or too little? The next questions lead to value like we hit on earlier.

Was that good spend or bad spend? A spike in spending does not mean it was bad use. So in the cloud world, we all know if I'm a retailer and I have a spike in my cloud bill in December, well, maybe I'm servicing the holiday rush for the Christmas season in the US.

That might be the best money I ever spent because my e-commerce was going through the roof. So yes, we need the visibility, and we need to start thinking in terms of value and ROI because, again, a spike of usage or a spike of spending, we can't equate that to just being a bad thing. It might be an excellent investment.

So what's your best advice to folks about how to get their arms around this? " But at the same time, nobody wants to be the one explaining to the board where all the money went. Yeah, I always steer people, and I have a bias, the FinOps Foundation, I said I'm on the governing board.

They've done excellent work so far, creating lots of content, even certifications around FinOps for AI. So there's already a base of knowledge, a base of suggestions, frameworks out there for how to think about managing AI. And now I encourage people to get plugged into the brand-new Tokenomics Foundation, because that's where this is really going to take off.

So you've got some of the best brains out there and some of the best companies coming together. I've never seen a standards body from formation attract the caliber of companies that have said they're going to be behind it. " So should I create an office of the FinOps, and who goes into that office?

Is it the CIO folks? Is it the app dev folks or the finance team? Or how do I make this happen short of maybe throwing everybody in the room and locking the door and hoping for the best?

I think what we've learned in FinOps, and I think AI is going to be the same thing, it's a team sport. And so why FinOps works well is it's not just a finance person, or it's not just an engineer sitting owning the FinOps problem. It has to be a team sport.

Because in one respect, you're looking at bits and bytes or GPUs and processors, but you're also looking at dollars and cents and EBITDA and margins. And those are typically different languages. And historically, the two didn't really come together too often.

FinOps established really this idea of a cross-functional sport, knowing for these disciplines to come together. The same thing's needed for AI. This can't just be really managed by a single thread, a single discipline.

So do you think at some point we might equate all this to the total cost of IT in a way that people will understand that every dollar spent on X is a dollar we don't have available for Y, and sometimes the Y is the staff? Yep, we have to get to that. That's where AI has set us back in maturity.

Where FinOps has been getting us to is thinking about technology value and how do we manage that total value. Now with AI, we don't know who's using it, what they're spending. So we've taken a bunch of steps backwards.

FinOps and cloud has traversed this path already. So that's where the journey's going to feel very similar, where first step is just getting your arms around what we have and who's using what. Then we can start getting better around how to manage it, optimize it, and allocate it.

So you're right, fast-forward into the future, this is just all technology. It shouldn't matter whether it's AI, software, hardware. We need to think about now the technology investments we're making and tying that to an ROI or really a value that's outputted.

All right. Well, folks, you heard it here. Tokens, in some ways, are a new type of commodity, and they go up and down in value, and you're going to have to keep track of that if you want to keep control of your AI budget at the end of the day.

Jay, thanks for being on the show. Thanks, Mike. All right.

And back to you guys in the studio.