GDT Maps the Infrastructure Cost of AI Readiness

Hey guys, thanks for the throw. We're here with Erwin Teodoro, who's Senior Vice President for Advisory and Transformation Services for GDT. And we're having a little chat about, well, unless you've been hiding under a rock, you might have noticed that the price of infrastructure is going up across the board, largely because, well, memory's gone up and everything along with it.

Erwin, welcome to the show. Hi, Mike. Thanks for having me today.

So you've seen this trend as much as the rest of us, but what impact is this having on the way people are planning for IT purchases and their infrastructure? Because none of this stuff just magically happens overnight. People are usually trying to figure out what they're going to do weeks, months, sometimes years ahead.

So what are you hearing from folks? Yeah, Mike. So I think with respects to, let's unpack your first opening statement, which is price increases across OEMs.

I think we're seeing a couple things, Mike. Number one, there's just so much more advancement in each of the vendor technologies nowadays, right? If you look at whether it's the compute side, the storage side, data protection, security, networking, what have you, there's an embedded layer of AI that we're seeing in a lot of these technologies.

So I think that's what's happening, is the solutions are more enhanced and our vendors are having to recoup some of that investment through raising their prices to put these new features in, right? I think that's part of it. But I think, the other side of it is, aside from supply chain, and we know that there's definitely some supply chain challenges, but I think it's more than that.

You look at the solutions today regarding AI or modernization or in some cases, security, hybrid cloud, there's much more of a dependency now across the infrastructure. So if you're looking at it from a solution perspective, you can't just buy one piece of technology and solve your overall strategy. What you're having to do is look across the stack.

So today, if somebody's going to invest in a new network modernization, you almost always have to look at, what are they going to do on the compute side to make sure that the compute side stays on par with the more modernized network. And then along with that, Mike, what about the storage side? Okay, then how do you protect it?

And then how do you wrap some security around it? So it's almost like a chain of events that has to happen that I think are causing more of our clients to spend more money, right? It's not just back in the day, I grew up in the storage side, right?

Where somebody would just go and, at the time, just buy a new array or refresh an array because A, it either came off support or is on a refresh cycle. Today, what happens is if you have to replace that storage array, you're forced to look at across the stack, or what else are the dependencies around it. And I think the vendors are aware of that, and they're having to enhance the pricing around it.

Because, A, that embedded technology or those enhancements they're making to their solutions, but knowing across the board that they all have to work with everybody else. Mm-hmm. And there's much more, I would say, Mike, support across the stack now, where, as an example, Cisco has arrangements with NetApp, right?

NetApp has arrangements with some of the data protection providers. They're working in conjunction with security. So there's just a lot more going on, I think, in the industry that's forcing vendors to really look at their pricing models to see how they can recoup some of these costs, I feel, is what's going on, because of the interdependencies.

Do you think that this amounts, therefore, to something that feels like an AI tax that we're all paying? Because regardless of how much AI is in my workload, the cost of the infrastructure is going up anyway. I do.

And regardless of whether you're in an AI initiative today, you might not be, but you're having to at least brace yourself for that, right? I'll give you an example. So our organization, amongst all the networking partners, we do a lot with Cisco, right?

But we know that when a 400 or 800 gig switch is being sold, that we know that at some point it's bracing for some level of an AI workload downstream. And so to your point, is there a tax? Yes, but I think it's also the folks that we're working with are bracing for the inevitable.

There is going to be some level of an AI workload, whether it's today, certainly tomorrow, at some point, they're going to have to be ready for it. So that's why you're seeing a lot of stack being positioned at the same time, because you don't want to have a great network. You don't want to have the most latest storage and have not the right compute in there, as an example.

All right? Or you don't want to have all the bells and whistles ready for AI with storage and not have the ability to data protect it, because now you're protecting more data, and you have to do it at faster pace, right? So I think it's a little bit of both.

There's that tax, but there's also the, you have to have the ability to look forward, to see at some point, you're going to come down the path of AI workload, and you got to brace for it, is the way I would answer you. Has the whole process flipped? Because I think people used to wait as long as possible because they figured that the price was going to drop, and there was always Moore's law.

" I think you're spot on. That's very accurate. You used to-- But I think it kind of coincides with what I was sharing earlier, is that you can't just prepare for one piece of the pie anymore.

You can't just change one piece out and think that because traditionally, that's how you used to buy technology, either when it's no longer supported or when your refresh cycle is up. And you know who does a great job of this? Nvidia has what's called a five-layer cake.

Or maybe a better way to say it, they abide in this AI factory. And the factory just doesn't focus on the GPU itself, it focuses across the stack. And I think that's what you're seeing now, Mike.

And that's what's forcing buying patterns to be different, is that you can't just make one change. You got to change everything. And that goes not just with the hardware, but potentially the software.

And potentially, if you go down the line, perhaps even maybe the folks that are driving or running the environment. You have to look at different skill sets. So it's a whole thing that's happening.

The catalyst definitely started mainly, I feel with maybe cloud and security. And it's really evolving in AI, where everything's just-- If one thing changes, you have to look at the other changes to the environment as well, not just to your point, just changing out a storage array or something that just came up because you normally buy it every three to five years. You got to look at everything now is really what we're seeing.

To your point about that, and I might argue that maybe we all got a little lazy about this, but do we need to get smarter about right-sizing the workloads to the infrastructure because, well, shall we say the costs are much more real? You nailed it. That is exactly it.

I think the cloud taught us that. The cloud taught us that you got to look at really the whole workload, not just the app or just traditional performance requirements. You got to really look at unpacking the dependencies, really, and not just from the infrastructure we talked about, but everything.

It's very cliché, but I think it really fits to what we're doing today, where you might have heard this in this combination, Mike, for years where a lot of the consultancies always talked about people, process, technology. And I think you really got to unpack that workload that just not dependent on infrastructure, but what are the processes to run it? What are the resources you need to manage it?

And you're really having to look at a workload-centric point of view, of which technology is one piece of it. We even go down at GDT, we go down to even the underlying databases. And some of the interdependencies that have to happen from that perspective in order to understand the workload.

But yeah, the whole concept of looking at the workload is really coming back into prominence. I think we learned it from the cloud when we just thought we could just pick up everything and move it. But what we didn't really account for when we did that leap of faith into the cloud, we didn't look at all the complete services that's required to really drive that workload.

And that's what we're doing now. When we do use cases as an example, we obviously identify the use case around AI, but we want to unpack really what does that workload entail. Where are the apps?

Where are the tier one or tier two apps that need to drive it? What is the data required in order to make that workload function and perform? We recently had a client where they jumped headfirst into AI, and they went into one of the hyperscalers, but they didn't realize the impact, as an example, on where the data was.

And in the prototype, it worked great. They looked at some modeling before, obviously, they released it into the general population. But that first week, they overspent on their budget in that first week alone because they didn't account for the data, where the data lived, how much data do they really need on-site versus at the edge, as an example.

And because they didn't take that into consideration as part of the whole workload alignment or analysis, they got a result that they didn't like, which was it cost them almost 10 times more even running that first week than it did in their P&C. So workload, to your point, very instrumental in how we shape AI, especially in infrastructure. Do you think at some point we'll be using AI to help us navigate all these issues that AI is kind of creating?

But it seems to me as this gets more complex, we might need a AI purchasing agent to help us figure out what to do, when, and where. Oh, no doubt. We're already seeing that.

What AI is helping, example, in the cloud is right-sizing subscriptions, so you're not over-subscribed or in some cases, to your point, you're not under-subscribed, but you're appropriately subscribed. And AI's helping that. You look at a lot of the FinOps tools that are out there.

Obviously, one of the things that- Well, when we're with our clients, why clients pull out of the cloud, as an example, is the cost. And there are tools now to help us, right, with that, and AI is a big driver. Whether it's pattern or workload, or looking back into data, and having that historical data accessible, it really helps with shaping what infrastructures we use.

We use a lot of those tools. In fact, the vendors have those already as part of their sizing, especially when it comes time to sizing GPU and those kind of things on the compute side. There's already a bunch of platforms that are out there from each vendor that enable you to come up with the appropriate configuration based on what you're trying to do.

So how long do you think that this is going to be the case? Is this our new everyday reality through the rest of the decade into the next? Is this the new status quo?

What do you think? I do. It's a great question.

Obviously, at some point, there's going to be-- Right now, everybody's getting into it. I think what you're going to see over the next couple of years is more and more organizations coming out of the woodworks, so to speak, that are going to contribute to the whole ecosystem. And then much like the cloud, you're going to see some fall out.

But I would expect this to be the norm. Look, running an AI workload is 10 times everything. So everything from, again, you need more compute, you need more data, you need more data protection.

Your security just went up 10 times greater. You need a faster network. So that's going to be the future.

Because there's just so much data that is being processed here. And at the speed at which you need your outcomes or your output, you're going to see this for quite some time. But I do think that there's going to be less standing five years from now.

I don't think you're going to see the number of people playing in the game. I think you're going to see either folks just not be as successful. You're going to see consolidation, acquisition.

You're going to see the big players just get better at it. But I think to your point, this is going to be the new way. You have to start looking at your roadmaps.

You can't look at it as things like, again, when that storage array is up or when the maintenance up on this, I just renew or refresh. I think you got to look at your whole roadmap as to how you see this technology unveiling or what future workload you see down the road. So it's forcing CIOs and CTOs, along with the business, to be more aligned.

Because you can't get away with in the past, where you're managing multiple support contracts as an example, or so many assets till you can't get anything out of it anymore. I think what you're going to have to do now is set a framework or an architecture around where you think you're going to be and then stick to it and know that there's some flex along the way. I think that's how you're going to have to do it.

It's forcing CIOs and CTOs, to your point, to think way differently. Traditional methods aren't going to work anymore, because of the interdependencies of the technology and also at the pace. This is at a different pace than we've ever seen before.

I think what's the difference here, I've lived through a couple of these booms. Whether it was the cloud, whether it was the dotcom era, whether it was the virtualization era. I think at some point, you got to do things differently.

You're going to have to just to accommodate for the pace that's happening here. But I think to me, the best organizations that are going to deal with are the ones that create that roadmap, that create that framework, create that reference architecture that they want to build and just allow a little bit of flexibility along the way. Because you can't be certain at 100%, but I think if you build that right foundation, those are the orgs that are going to be successful.

All right, folks. Well, you heard it here. Hey, at the end of the day, it all comes down to smart shopping, and we're going to need those skills more than ever.

Erwin, thanks for being on the show. Yeah, you got it. All right, and back to you guys in the studio.