AI Disruption, Big Tech Gains and the $500 Billion Bet on Infrastructure – Infrastructure Matters EP80

Hello and welcome back to Infrastructure Matters, uh, episode 80. Uh, welcome Keith. Um, and I think our, our partner in, uh, crime, uh, Kimberly is, uh, is out this week.

Yeah. So that means I have to ask you, where in the world is dying? Well, I'm still in the Balkans, that's why we're gonna be till mid-June that I'm heading over to Rome, so, oh, and this half year overseas.

Yeah. And then, uh, then we come back. So, uh, lots of fun.

And this, it's getting the weather's real nice here. We're on the beach, uh, on the Adriatic. So good stuff.

How about you, Keith? Where, whereabouts are you? I just left, uh, I actually just left Kimberly.

She is, uh, off doing tech Field day, uh, and learning all out about AI infrastructure from some of the biggest AI infrastructure companies in the world. But I am back here in Chicago, at least for the next couple of weeks. So you'll be in Rome just in time to, to greet the new Pope, so that, that'll be, that'll be good.

Tell 'em I said hi. Yeah, absolutely. Um, and so, um, interesting week, um, you know, there a lot of people are looking at what, uh, how the tech industry is gonna weather all the things happening with, uh, the economy, um, the geopolitical instability, the, the new, um, administration in Washington.

Um, but ServiceNow, S-A-P-H-P-E all turned in very interesting numbers. Uh, uh, good numbers, uh, ServiceNow beat expectations. Um, subscription revenue grew 90% year over year.

Um, the shares went up 10%. SAP, uh, reported Q1 cloud, uh, Q1 cloud revenue rose 27%. Um, their backlog grew 28%.

HPE stock outperformed, even though they had given pretty grim guidance last time around, uh, shares climbed, uh, almost 6%. Um, and so, uh, so there is, the market's been doing good the last couple days. Uh, so it's, it's very interesting to see how, how, how things are, are going.

What, why, what do you think of it? I think you guys move cloud news as well. Um, Keith?

Yeah. So the, uh, just a note on HPE, I'm really surprised. Well, I'm not surprised that Elliot is going after Antonio Neri.

Uh, they won his head on a platter. Uh, I think it's fair to say, and both of us have covered HPE for a really long time. Uh, he is generally really loved, uh, amongst he is, and He's also not, they're, he's not an easy guy to take down either.

We both know Antonio pretty well. He's, uh, he's not be, we, We know him pretty well. He's a tough, he came up from the trenches.

He was, he started there at Support Desk. I think that, I think, uh, Elliot is going to have a nice battle on their hands. But, uh, Google Cloud, uh, and I'm zero in on Google Cloud numbers, but, uh, on the macro Alphabet had outstanding numbers.

2 billion. Uh, but if we look at, uh, Google Cloud, specifically the infrastructure part of Google, they're up 20% year over year. Uh, And that's good.

But it's all going right out the door as far as I can see. They're are, they say they're gonna drop 80 billion on CapEx. 2 billion in income.

2 billion on. And I think, uh, I think one of the things that get hidden in a numbers that while Google Cloud or Google Big is making the investment, and Google Cloud kind of takes the brunt of it, mostly, the most Google services run on Google Cloud. So it is not just a cloud only expense that is spread throughout all of Alphabet, because Google Cloud is the service provider of Google.

So, uh, what I thought was really interesting, uh, we were at Google Cloud actually earlier this week. Me and Kimberly were, and we're living, listening to the product team for their accelerated compute. And the product manager was saying that the demand, I asked them about, uh, Jensen won SWGs, Hey, H one hundreds, H two hundreds, basically, hopper, you won't be able to give them away.

He's, he, he, uh, let out a sign. And he said, well, the product manager for that product, you know, was, you know, having a, uh, a bit of a heart attack. But he said that, but he, he said that they've seen, not, not just reduced demand for it, but increase demand, and they have the latest, uh, Nvidia chips.

They're one of the only cloud providers to offer instances direct directly for the GB 200, I think it is. Mm-hmm. Yeah.

And they're saying they're seeing great demand for all of us. So the, the, I think this is a great good indication that the H one hundreds, H 200, those depreciation schedules for those, uh, for those chips are not going to be at risk. This is a really interesting, uh, uh, seeing this play out.

Well, and I think part of that is there's the, you know, just getting, uh, affordable compute time for AI is the challenge. And if Google prizes them, right, they should have no problem getting, uh, you know, selling hours on those. And that's what the Exactly.

The, uh, product teams kind of reply to us. Not only is it priced right, it is where they need it, where customers need it. Data has gravity.

The gb, the G 200, I'm sorry, the B 200, GB 200 are still relatively, uh, has light demand. They said they're only been able to accomplish, uh, uh, accommodate small clusters of 32,000 GPUs. I thought that was interesting.

32,000 GPUs had been the, the ceiling just a couple of years ago. Now, 32,000 GPUs are, are considered small. Mm-hmm.

But, uh, that the availability of H one hundreds and H two hundreds is so pervasive that this is where the, uh, demand is at, and this is, and it's where the customer's data is at, and it's available in, you know, in every region, et cetera. So it's, it's, it's, it's, it's interesting seeing the, the drive of technology hit supply chain and the reality of enterprise it refresh even at the hypervisor skill. Yep.

Well, and, and, and in my analysis, 'cause I've, I've, I've spoken a lot, uh, to CIOs about Google Cloud and the real challenge that I get, I, I, I, I've received calls from, from, uh, CIOs saying, I really want to go to Google Cloud. They are the most modern cloud. They, you know, they came a little bit later with a lot of their abstractions and a lot of their concepts.

So they were able to, to create more refined, you know, more elegant, um, uh, you know, cloud architectures. Uh, and, and I agree it's one of the best integrated, easiest to use clouds that there is, uh, are, but they just can't get the talent. Uh, there's, uh, because it's the number three cloud, the talent base is not there.

You can get a ton of people who know AWS ton of people who know Azure, um, getting the, uh, from service providers or hiring talent or wi sourcing from any, anywhere. They can't get enough people to do Google Cloud. And that's always been kind of the holdup.

But I think now the real advantage is if you have great, a great AI capability, um, and you've got affordable, uh, cloud services and you actually have available capacity, you're gonna get business now. You know? Yeah.

5 is. It is, uh, the, it is safe to say they've not just caught up for a little while. They were leading until maybe oh three came out from open ai.

So one of, I think this time last year was They weren't getting the leaderboards the last, uh, month or a half or so for sure. I hadn't even seen the new ones yet. Yeah.

And I, I think a year ago I talked to a CTO and I said, Hey, why, why aren't you using Google Cloud? Uh, ai. He laughed.

He was like, Google's AI is a joke. I don't think, I don't think it's a joke. I don't think customers are considering it a joke anymore.

And the numbers bear that. Yeah. No, it's, it's interesting.

So, uh, Google Cloud is firmly in the game, uh, but I think they have their, their work cut out from the, the ever pull out of number three is really gonna be the challenge. And that I don't, and Google's not used to. They don't like being number three.

They're not used to being number three, they just be number one. So, um, but I don't think you have to worry about Google Cloud going into Google grave graveyard anytime soon. So, good, good for them.

Um, in, in other news, so the, uh, management consultancy, AIX Partners just issued a, a major report. They were studying the weakness, the susceptibility of the, of the top, top public software companies, um, against AI disruption. So someone comes in and creates a born ai, you know, AI native version of ERP, for example, or whatever.

I mean, because, because the broader shift is gonna be away from apps and more to agents, agents are gonna do more and more things for you. You're just gonna go to the agent, say, I wanna do this, that, or the other thing. And you'll be using them.

And as versus apps, even though an agent, of course is still just an app, um, just, it's just a different way of, of packaging it. Uh, they are saying that, that the, the top a hundred software firms, or most of them, are vulnerable for imminent disruption by these startups. And for example, I was talking to the founder of, um, hyper mode last night.

Um, they make a enterprise grade agentic framework that you can build, run, and manage tens of thousands of simultaneous agents of every flavor, doing all the interoperability use, using your knowledge graph. Um, and they have all the governing, they, they, they give you the control plane to actually do the, the digital labor management across tens of thousands of agents. This is a, you know, they, they rethought the whole problem from the ground up, and they're not trying to, uh, fit agents into their existing AI frameworks or their existing enterprise suites.

So you've got, you know, Oracle and SAP and ServiceNow and all these companies who, who have gotta make, they, they can't, uh, reinvent everything for AI because they, they can't change all those applications that they have. So the, the born native companies, the born AI companies, uh, the as theory goes, have an, may have an inordinate advantage. And, uh, and so that, that's gonna be interesting to watch that could really change both the public markets, uh, and the tech industry.

If that turn, if that bears out, Yeah, I can, uh, I can s share a firsthand, uh, story of how a hundred year old manufacturing company is actively moving away from SAP to an agent AI model of where they're having the, the, they're having, you know, kind of this immutable service bus that says, oh, once a transaction is, uh, uh, made on the service B bus, every, all my different SAP modules that I had before, instead of having SAP do that functional work or the workflow going through SAP, it becomes a, a gen workflow. So, uh, you know, they have some challenges around, you know, uh, SAP is deterministic and you know, what you're gonna get each and every time. So, uh, finding out what parts of this workflow needs to be, uh, deterministic and what parts of it is, uh, kind of the, this AI problem or the human almost.

I, I, I, you know, both you and I posed this question, I think we're at the same conference, and we asked one of the, uh, one of the major SaaS providers, you know, what's to stop someone from taking a agent, uh, AI solution a, a agent, and replicating what you do? And they were pretty confident that they would not get disrupted by ai. But I don't know, I'm starting to see a little, starting to see a few cracks in that, in that arm.

I'm, I'm seeing ai, uh, really, uh, as we've studied agentic ai, we just did a big market overview of the top platforms. We also did a deep dive on agent force for Salesforce recently. com.

Um, we found that healthcare is one of the top use cases. It's been very resistant to PA and a lot of other automation because there's such a huge body of knowledge you have to train for. Well, these models, AI models, gene AI models are trained on even, they, there are these big health models now that have studied all of it in all the terminology, all the different things in the ins and outs of the industry.

And, um, and they've studied, you know, thousands or millions of, of patient records. Uh, they, it, it, a healthcare is now very easy to, uh, tackle with AI because it can take all the different domain knowledge and all the different, you know, hospital and healthcare management knowledge and, and actually do things and answer questions. And there, and, you know, the, the, the big problem is it's still only about 90% correct.

By 10% of the time it's still wrong 'cause it's a probabilistic scenario, but all that's gonna get fixed with consensus based models that, you know, over sample until you get the right answer and grounding and all that. But yeah, it's fascinating to watch. But speaking of health, And I just, yeah, Sorry, go ahead.

Before We go move on to healthcare, just to back some of this up with data, the door ai, uh, AI code assistant report is out kind of the results of what happens that now that we have these code assistance en mass in the market in the increase or lack thereof of productivity, I think a key vector that's missing from the report is kind of this gap that these assistants can fill. So the experienced teams are seeing only a 2% increase in productivity effectively from using adopting AI assistance. One of the things that I would love to see, and you hinted to that in the healthcare, uh, part of this conversation is what happens when you team these AI assistants with non-practice developers, non-practice, uh, healthcare professionals where it makes sense.

So IE you know, me creating my latest application, I'm not a developer. I don't consider myself a developer, but I'm able to develop because of these AI assistance. How do we begin to measure, uh, productivity increases, uh, from when companies are just an employees are just in capable of doing something?

And now what can someone who's adjacent to these areas of discipline, what happens when you give them the same ag, uh, tools I had Love I See On, on how, how these tools impact productivity. Exactly. Well, we know that, you know, Keith, you and I know, you know, having been developers, um, we know there's a whole body, there's a whole bunch of stuff you have to learn, you know, you know, avoiding, you know, watching Big O notations.

So you, you create computationally same algorithms that, that when you given a bunch of data, they don't, they don't, you know, stall and die. All these things that are I a no code, um, you know, business, uh, analyst developer would not ha have any of that, that background. And they could easily develop something that's not workable in an enterprise environment, works great in the lab, but doesn't actually work in the field.

Um, but these ai, uh, uh, code generators, these agents, they know all those rules, and you just explain in plain language what you want. And in fact, that's what we're seeing with these agent builders now, is just, it's just a prompt. You're just saying, I need an agent that does X, Y, and Z and follows this, this industry set of rules and this policy, which I've just, I've included a policy document in this, build me something that can process documents according to all these rules and these policies, uh, and gimme outputs and drop that into this database.

And, um, it will just do that and following all best practices and the rules. Uh, and, and, uh, so these agents are now proving very easy to build when they say you can build one in a in a couple hours, it's usually not even that long, is what I'm hearing now to harden and then test it. Yes.

That's actually where most of the work is to make sure that you've got determining if a human needs to be in the loop for the before the final answer, to check the final answer or, uh, what you're going to do. Um, but yeah, the future of COPE building is, uh, developing applications and agents is writing prompts. It's very interesting.

Which are it just humans? Yeah. The, it, there's, there's plenty of opportunity for, uh, folks to un who understand observability, folks who understand the logic to, uh, make a big change.

And one of my challenges has been observability and the ability to make sure that the ai a, the AI agent is getting the right return from the LLM, which is not always consistently, it's non-deterministic. So getting expecting their for, uh, uh, uh, uh, a response to be formatted in the way that you expect it to, and you code to, has become like one of my number one challenges. Yes.

And we see, like NVIDIA's, uh, agentic framework, uh, has advanced log fi logging that explains why it did what it did, so that you can go back and say, now you gave us an answer we wouldn't, didn't expect, or we think is wrong, but explain it to us. And, and you can go through its reasoning and see that, I think reasoning AI is gonna be very popular, these new reasoning models, because they can't explain, you know, why they came up with it, you know, so that that's encouraging. But we were talking about healthcare and, um, or Oracle had an unusual data breach.

Um, they're generally considered one of the safest, one of the most reliable, uh, enterprise vendors from a cybersecurity perspective. Uh, it was legacy servers. Oracle says, if you're on, if you're updated on everything, you're fine.

Uh, but many IT departments can't be up to the minute on all patches and, and be everywhere. This was in legacy, but 6 million healthcare records were lost, um, on Oracle infrastructure, legacy servers, um, that, uh, were not patched, but, uh, you know, with mandatory, uh, cybersecurity reporting for, for, for public firms that they had it to disclose. Uh, and that's, that's a, that's a black eye for Oracle for sure.

Um, um, given there are stellar record really, uh, for, for this up, uh, up to date, but, and it's getting quite a bit of attention as a result, even though the breach is relatively small. So was this from their acquisition of Epic, or is this outside of Epic? Um, that is a good question.

Um, it, uh, Oracle Cloud Classic and Oracle Health is what they're saying. So I don't know if Oracle, Yeah, so that's not, I, I don't think that, I don't think that would be considered, uh, the Epic. So obviously with them acquiring Epic, this is big, big news because, uh, they control, uh, or they, they host a good percentage of Percentage direct national Yes.

Um, uh, health records nationwide for sure. Um, but moving, moving on beyond that. So it's something that definitely for CIOs to watch.

Um, the, um, is we're seeing, again, renewed. We, we saw a spate of announcements on, um, uh, AI vendors announcing just truly huge investments to show how serious they are about the space and, uh, and why people should partner with them versus the other ones, because of just the scale of the investments that we're super serious. We have the, the funds to really build, uh, whatever's required to, to create the, the next generation of models that are gonna dominate the industry.

Uh, all the vendors have been trying to prove that. We saw that with, um, project Stargate from, um, OpenAI where they're, they, they said they're gonna spend up to $500 billion, uh, with SoftBank pointing up a lot of that. Uh, well, we had another round of announcements.

Um, uh, uh, the eu, uh, uh, has, which is generally considered very behind in ai. They don't, they're not really in the game at all. Um, and they, and, but they acutely feel it, they just announced, uh, the, um, invest AI initiative, uh, $200 billion, which is a, for Europe, a lot of money because they're, uh, IT in Europe is very conservative.

Um, they, they spend wisely, uh, they wait too long. Uh, but they also have very good ROI numbers compared to the United States, who is, uh, where we're willing to be a lot more speculative about IT investment. They're not.

Um, but here we are with the EU out, out ahead, uh, with our best AI initiative, $200 billion that's, um, for helping companies and building the infrastructure, uh, to, to put, get on the, at least get on the map with ai. But that wasn't the big one at all. The big one was in Nvidia announced, uh, plans for $500 billion in AI infrastructure in the United States.

So a big, big kudos, uh, you know, uh, to the, the Trump administration who's, uh, pushing with tariffs, trying to get Taiwan, uh, based companies to invest more in the United States, uh, which is a critical hedge. If, if China does end up blockading or invading, um, Taiwan, that's gonna have a tremendous impact on, you know, Nvidia, uh, TSMC and a host of other critical companies for the high tech industry. Uh, it's a smart move.

The whole question is, is the regulatory red tape gonna be cleared out to actually do anything with that money? Because right now it takes up to 10 years to build a chip fab here in the United States, and by the time you get it online, it's way outta date. And so that's why it's not done here.

So, well, we'll see what happens if that, the announcements are great, but we're gonna see what really happens. Yeah, and I think the, I'm, I'm really intrigued about this EU investment. We do need strong partners in AI and Different Yes, exactly.

Yeah. And competition. We, we need a different vision for AI that maybe takes this EU centric, privacy centric focus to ai.

And I think if you, if you know, let's compare it to deep seek, if the EU can do with ai what the Chinese did with, uh, deep seek and move forward the, the ball without using our private data in the way that's being used now, I think it's overall good for the industry. Justice deeps seek was. Yeah.

And I think, uh, you said is right. There'll be a, the, the EU will have a different take on ai, one that maybe is more respectful of privacy, uh, more careful about it. We'll see.

Um, and more regulatory friendly. 'cause that's where it really, you know, the EU is one of the, the joke is that it's one of their greatest, um, um, their greatest outputs is regulations. Um, that's so, uh, you know, they're the reg, reg tech heroes.

Uh, let's see what they, they do with ai. So I, I'm encouraged, uh, and there are some AI stories out of Europe, that's, to be honest, they're not big. Um, but this could make them big.

And so I wish them luck for sure. I mean, it's always good for the industry to have to see this, uh, but that's not all AI investment's not the only thing. Um, sounds like, uh, there is interesting things in Dell storage land, uh, Keith, can you catch us up on that?

Yeah. So Dell pre beat briefed us on this in a while, but, uh, there is no secret that companies like Vast Data are taking advantage of their architecture. Vast will tell you that this is by design, their vector services that they have, et cetera, are all aligned to being able to get into better training, et cetera, vast.

I know we're talking about Dale, but, uh, it is a note that Vast and, uh, super Mac Micro this week announced a what this super pod, this this Nvidia type super pod that has eight, uh, uh, uh, eight, uh, gb, two hundreds, along with fast data, the importance of the, the importance to the Dell announcement that the AI data pack pipeline matters. You, you know, both of us will tell you we're no Kimberly Bates when it comes to the, uh, the ones and zeros of actually storing bits onto storage. But we understand the importance of the data pipeline and all these announcements across all of, uh, Dale's power lines from, uh, the power scale all the way up to the power of Max, which is their big iron, uh, nonstop run.

My most important, important workloads are getting AI enhancements from accelerators all the way to, uh, uh, partnerships around data lakes to help organizations organize their data and keep this data ready for ai, which, uh, I think both of us are seeing is consistently a problem for, uh, enterprises, which is making sure they have the right data for ai. There's this really interesting debate that, do I need to prepare my data for ai? You'll get two very different camps who do, do not agree, Sam, who says, bring the data that you have, and others are saying, you need to organize this data.

And we're seeing the industry at large, the storage industry, these deals announcements play into that of, of being able to organize your data, get it ready for AI so that you're, uh, that so that your organization is with it in compliance and, uh, going at the speed that they need need to, at the same time to feed your GPUs and keep 'em, uh, as efficient as possible. Yeah, and there's no question, well, I mean, yeah, there a, a couple issues that organizations have. One is keeping the, the, the GPUs fed.

Anytime a GP U is idle means you're, you're, you're leaving, you're leaving money on the floor. Uh, you're, you, your cost is too high. But, um, then you gotta get, you, you have to have all the data that, the real issue is, is these context windows aren't large enough, um, to handle the, the, all your enterprise knowledge.

So where do you keep that and what format do you keep it in that's optimal to make sure you've got good coverage of it. Even if you have these models that have these large windows, they often don't pay attention to everything that you provide it. Um, and so this is, like I was talking, going back to that hyper mode conversation last night.

You know, they're, they're trying to store knowledge in, in disc based knowledge graphs that can get as arbitrarily large as you want without harming performance, uh, and making sure you're taking everything into account and you're not ignoring, you know, part of it just, uh, to produce a an answer quickly. So yeah, there's a lot of performance issues here we, I think we have to, um, have to worry about. But yeah, storage is, you know, has this thing where it gets, it gets sexy again when we have, we, we enter new revolutions like the ai, you know, the whole Gen ai, um, and, you know, Amazon's famous for having what a theater up to 12 different types, models of databases that their, their cloud supports.

So for e every use case, but, uh, uh, storage is essential. And data gravity is, is still one of the biggest challenges in trying to say I want to build one AI infrastructure. Um, and with a rise of private cloud, we saw, again, a very sustained interest in our CIO survey.

We just got our data in around trying to figure out the best place to, to run workloads. And those workloads need to be powered by data, which is kept on storage. So interesting times.

Uh, everything's kind of in motion right now, uh, in the Gena Revolution. Alright, Anything else to wrap up, Keith? No, it's been a busy week.

Uh, I'm sure, uh, Kimberly's gonna join us with plenty of data from AI Field Day. I mean, AI infrastructure field day two, which for you, tech Field day geeks used to be called Storage Field Day. Now it's called AI Infrastructure Field day two.

And, uh, the range of companies from Google Cloud to Vast, not, sorry, not va, those one, but, uh, Soine and store pool and, uh, the, the list goes on and on. I, Juniper Networks was there. Uh, the, the, the, just the variety of customers, I'm sure she'll come back with more data and more insights than, uh, one person can absorb.

Yep. So CAD, just next week, uh, I also have CIO chat, uh, every Thursday at 2:00 PM Eastern Standard time. Uh, stop by and watch, stop by and contribute, uh, and we will talk to you next week.

Thanks, everyone.