Stargate – The $500 Billion Money Grab – Infrastructure Matters – EP68

Good morning, everyone. Welcome to Infrastructure Matters, episode 68, and I have got all of my buddies here again. Um, we've, uh, coming in from Chicago, um, is there any cold up there, Mr.

Keys? Oh, it's gotta be cold up there. Uh, you know what?

We are, uh, I think yesterday we got up to a balmy 22 degrees, so we're we're, yeah. At that, you know, we broke zero for the first time in a while. So I'm, I'm fine.

And Diane, and you're in Washington DC isn't it warming up, or did you guys, you didn't, you missed the snow. Yeah. Got all the vortex.

We are two degrees yesterday, so Yeah, it was cold. Sweet, sweet. All right guys, we've got quite a week this week.

Um, and there's a lot of things that are going on, especially on the government, politics, all that stuff, or whatever you wanna call it. So, but we're gonna bring it back to kind of what this really means to our infrastructure matters. Cool.

Guys, that, that listen in to us because, you know, this is like, I learned so much from this podcast. I don't know about you guys, but I do. So what we're gonna kick off with here is, um, kind of the big stuff that started happening this week that, um, Diane has been doing a lot of work with, and that is the CEO insights work that he, they are futurum has been doing with, along with Kearney that got released at Davos, um, to great fanfare.

And I guess you've been doing the roll around on all kinds of news, and I, I saw, I saw our numbers all over the place, so very congratulations on some great work. So why don't you head in, tell us what this is all about and, and what the, uh, CEO should be thinking about, and the CIO should be thinking about an infrastructure matters. Absolutely.

Well, thanks Kimberly. Uh, yeah. So we released a, a major knee report that it's titled our CEO's Ready to Seize AI's Potential.

Uh, and so we sought to actually answer that question. Uh, and so, uh, it, it's very interesting. The, um, we surveyed, uh, over 200, uh, top CEOs.

We interviewed, uh, over 20 of them as well, uh, to really get the color about what are they doing. And, uh, c uh, AI is top of mind for the CEO. They, they see it as, as a major competitive issue.

Um, it can dramatically cut costs on, on one side, and it can also create break breakout products and services, um, for their customers on the other side. So they both want to innovate and they want to cut costs. Um, broadly, we saw that they expect their businesses to be highly automated in three years, uh, tied to his, uh, the CEO of one, uh, major audit company.

And he's, he thinks the audit function's gonna be replaced completely. That's not gonna be their core business going forward. They, they're still gonna sell that as the first thing they do, but then once the AI has all that data, then they can be advisors and they can go up the stack, as we say, and do more strategic, uh, work, you know, consulting and, and helping them guide their business.

Uh, you know, that's the kind of AI transformation that we're, we're seeing. And so, uh, uh, we saw that 59% of CEOs directly lead their organization's AI strategy, and we also found out that that's not necessarily a good thing. Uh, they should, um, the, the CEOs that step back after they set the mandate, say, you need, we need to rethink our business in terms of ai, and then they step back, they had a higher success rate, uh, a higher reported success rate.

Uh, And why is that? Why would that, why would you, did you describe, They may not have the expertise to really figure out where the best place to apply a AI is? Uh, uh, so they're, they're not the subject matter experts, uh, even of, even of parts of their business, nevermind the AI piece.

And so the ones that, that got overly involved in that, um, uh, reported less success. So that was very, that was one of the kind of big insights that CEOs absolutely have the biggest voice, and they can make it happen, but they shouldn't go along and keep their hands on it too long. Right?

They, uh, they, they constrain success. So That's the total micromanagement stuff, right? Yeah.

Yeah, exactly right. Yeah. And it is really interesting, you know, Diane, both of us cover, uh, you know, the SAPs and Oracles of the world.

And we see this not just in ai. We've seen this historically when it comes to implementing these company or wide initiatives such as ERP, that the success is not, uh, driven by the CEO, least not managing the project, but typically by them sponsoring the project and giving the, the, that inferred power of change. And that's probably the most indicative.

Did you find any corollaries from kind of traditional enterprise IT projects, which I don't know if I would call AI enterprise IT project? No, that's why, uh, you know, yeah, we, we took the CIO, uh, uh, sorry, the CEO view. Normally we look at the CIO, what they're doing, uh, but we're seeing that, um, the CEO is the one that that's having to really drive the, the, that, that massive change across our organization.

And they have the ear of the board, the CIL often doesn't have. Uh, and yes, this is coming in from a different angle, uh, than we often see it coming from, and it's this enterprise wide transformation. Uh, it's broadly happening.

Uh, there was very few CEOs we talked to that, that aren't planning to do something immediately are do already doing something big immediately. Uh, so that, that was, that was interesting. And, and like you'd expect, uh, the biggest area where we're seeing AI being rolled out right now is in customer experience, some aspect of it.

And it's a lot around customer support and customer service right now, uh, but increasingly it's going into marketing sales, r and d. There was a lot of, uh, CEOs that are planning to formulate product. They want AI to formulate breakthrough products for them.

They think, they don't think it's ready yet, but they're, as soon as it is, they're gonna pounce on that. So it's, it's fascinating. com, um, go to the ai, um, uh, report portal and get yourself a copy.

Well Make Sure to Yeah. I, I can have it up now. Uh, it's been up on my desktop or the past since it was released, and I've just been, you told me that the latter part, once you get past, you know, this is an infrastructure matters podcast, so if you do get access to the report, take Diane's advice that he gave to me, get past a few pages of fluff that US infrastructure folks don't care about, quite frankly, and get to the latter parts.

There's great, there's great graphs, data points that, uh, infrastructure geeks will appreciate, You know, and it's super reflective of the what's ha what just happened this last week, which is the, uh, two different things. And then we're gonna talk about that. One is the Stargate announcement and what that's all about.

And the other thing is this on continuing, you know, discussion or, um, announcements around building more data centers. Um, and it, like the latest one that I saw was Facebook talking about you dropping in like 58 billion or something like that into what, whatever the big number was into, um, or maybe it was 60 something into data centers. And this is like all this money getting dropped into this.

It's just enormous. It's, it's like, it's kind of head spinning if You'll Oh, yeah. Well, and that's where the infrastructure, a large percentage of, uh, infrastructure in the future is, is gonna be out there in the cloud in these massive spend.

So, so SAT Nadella said that he's spending 80 billion this year alone, just in 2025, on, on infrastructure. Uh, you had, you just talked about Zuckerberg's announcement, uh, and of course we have Project Stargate, which we're about to talk about. Uh, and then we heard OpenAI stole Meta's head of, um, infrastructure and, uh, storage, uh, I dunno if you guys have heard about that.

So now they're, you know, they're, they're, they're competing, uh, amongst themselves for people who are capable of building out these, you know, multimillion dollar, uh, sorry, multimillion C uh, GPU compute, uh, cluster, right? The, they train these models. It's fascinating.

Yeah. The, the, the battle for talent is not going to end. And I, I, you know, just harking back a little bit to the CEO, uh, report you talked about in the report the skills gap, and I think this is a indication specifically in our industry, where you're going to see competition for people who know how to not just implement the infrastructure and the technology, but I'll actually put it to work, and this is, it's gonna be a fascinating, uh, Yeah.

I think there's a difference between building a data center that can run a lot of different workloads through building a cluster, a a mega cluster that can train up a frontier model that, that has national and, and international competitive implications, right? So it's very, you know, it's a different level. So it, it sounds like that's, they're gonna start picking out of, uh, Lawrence Livermore, JPL Fermi lab, a few, a few others around the world that they'll probably be picking some people out of, um, because, uh, as well as, you know, the, the guys from the super use from the, um, public cloud, um, major cloud providers.

So I could, would think that that's kind of where the skills base is gonna be coming from. Yeah. And, and, uh, the in, as we get into the big announcement, uh, the kind of these resource restraints, just not around hardware, you know, this talent resource restraint, and the other resource restraint is data.

And we will, and part of the, the big announcement was, uh, Oracle, we'll get into that in detail, but where's the data's gonna come from? We, we've already, I think Elon Musk has been quoted in saying, we've basically already scraped all of the data on the internet for these models. Where's the rest of the data going to come from to make better ai?

And that is, I think, a bigger challenge than, uh, I think that's going to become a bigger challenge than the compute challenge. It It is, although the numbers are supposedly that only about two or 3% of enterprise data has been sucked into these models yet. And so I, you know, I work with a, um, a lot of different, um, professional associations and their CIOs.

So like the American Geophysical Union, they've got an enormous amount of information, which is all behind their, you know, a hundred years of research and, and research papers that are not publicly available, um, that, you know, of course you'd want that scientific knowledge in your, in your, uh, foundation model if you can get it. Uh, but the geophysical union's not gonna just give that up. And so they've, there's a lot of deals now that have to be made to unlock all the rest of the data that's behind these, um, behind these, uh, organizational boundaries.

So, And I think this is part of the macro discussion around AI and AI models and, and the haves and have nots. You'll have the haves who can afford to build the compute to generate lines And buy the data, have both build to build the compute and buy all the data that, that their competitors don't have yet. You know?

And, uh, and then you'll have the have nots that cannot, uh, access the data. So I think we're going to see new service models come up, you know, people are going to want to sell that capability, you know, uh, anthropic and open AI won't be able to go to these private organizations and get their data because they, you know, it's, it is the, the, the keys to their kingdom. I've talked to, uh, fortune 50, uh, companies who just simply said they will not use public cloud services for AI because, uh, the vast majority of their crown jewel data is within their four walls.

And there's no way they're going to allow that data to be used for training for public models. Yeah, absolutely. So, do we wanna skip over to Stargate and talk about what that's all about?

I think that that's the biggest news of the week. Yeah. Who wants to, who wants to break do that one?

So Diane, you wrote, you wrote a whole research note on it, so I think you are the resident expert on all things Stargate, as far as it, it, it goes. Yeah. So let's, let's jump into it.

So the, the big reveal in, um, in AI this week was the announce of announcement of Project Stargate. Uh, it's a, uh, consortium of the top companies or some of the top companies in ai, and that's, that's the rub. It's some, it's a, a, a bunch of companies that have come together.

It includes open ai, obviously, uh, kind of at the top of the, the Apex company. And the one I think the most stand to benefit from all this, um, Microsoft and Oracle, which, uh, you know, from an enterprise standpoint, we're very interested in their involvement. Um, uh, and we also have, on a hardware level, we have Nvidia, uh, and a MD, um, and, you know, from, uh, the hardware infrastructure piece.

And they have with SoftBank, that's the, the Japanese, um, uh, uh, venture capital firm, uh, but come up with a $500 billion fund to create next generation AI frontier models. Uh, and to assure American competitiveness and leadership in AI for the foreseeable future is the stated goal of Project Stargate. Um, and there's a lot of, you know, but a lot of debate about whether that's this is a good thing or a bad thing.

Um, the, uh, they're, they're already starting in the first 10 data centers that are primarily gonna be located down in Texas. Um, and they're, the intent is to build models at a level that no one else can. And, uh, by building, uh, you know, training capabilities and, uh, that can, and train and create all the deeper linkages between knowledge that, uh, that a regular model can't do, like in a year.

Like it take, it takes some of these models all nearly a year to train on all their data, uh, and, and so they can only go to a certain depth. If you have much more power, you can get, you, you should be able to squeeze, model your data. Now, there's a big debate about whether you can do that or not.

If a larger parameter model that goes deeper actually creates better data, they believe that they can. So that's Project Stargate is you got a tremendous amount of attention. I've never had so many immediate inquiries about, uh, a piece of news.

Um, and in a, in a regular climate, um, this might be viewed as, you know, uh, creating a monopoly power, uh, big vendors colluding and keeping the other ones out. So like meta is not inside, um, uh, uh, the consortium, um, neither, um, is anthropic. These are, these are arguably leaders, um, in AI that are now, you know, because they're not part of this.

And, and Musk has grok as well, and he was very, very upset, had very negative things to say about, uh, about the whole project. Um, and so it's, it's very controversial in some circles. Uh, yet it stands to, to, to, to benefit, uh, United States and keep us in leadership, because that's, uh, the gap is closer than we thought.

We also learned, uh, something else, uh, which we'll get to, uh, in terms of there's now international competition in ai. So when we Think about what Stargate is going to be creating here, this sound, it's sounds like it's a government initiative, but it's not because it's not, yeah, it's private money. It's Supported By the government.

You know, this was all done by, you know, great fanfare with Mr. Mr. Trump up there with these guys and saying, we're doing all this 500, you know, billion dollars, whatever.

So they're creating this foundation model that will be able to be, will be allow used by the general public, the general company, all the companies that can license into it. I Know how many models they'll create, who owns them, or how the profits will be shared. Um, but I think those are, things are up in the air right now.

We don't, we, we, we don't know. And, and often, you know, because there's often, uh, there's competing, you know, company and Microsoft and Oracle are competitors, uh, that, that, that factor often makes these types of consortiums come apart or underperformed. So, you know, they, they can't, they don't wanna mix their stuff up with their competitors, so they won't share their best information or best people or best knowledge or whatever.

It's, that's very interesting. Which has me asking the question, you know, uh, who has the most data? Who hosts the most data for customers, Microsoft or Oracle?

It goes back to how these frontier models will be trained. Where is the additional data coming from? Uh, again, the internet is big, but not that big.

It, when it comes to how much private data that exists on tape, how much private data that it exists in, uh, realtime ERP systems, how do, how will in enterprise customers benefit fit from Stargate? And will there be a monopoly in a sense that they don't want to go down the route of VMware vis-a-vis Broadcom and be stuck in obligated to having their most critical business functions held by a monopoly of vendors? It is fascinating times, and the implications are very deep.

We're, we're in a, a very unknown, unknown state of, uh, uh, of affairs and water, But the Oracle data, all that ERP data, the customer data, all that, those items, you know, whether or not you're talking about sale, our data or the company's data that's up in Salesforce company's data that's up in some ERP system that is not owned by Oracle, Microsoft, either of them. No, but there's gonna be tremendous, a tremendous interest in figuring out how to incentivize the owners of that data to share it even anonymously. Um, and so that's be the big debate.

And of course, there'll be temptation to use it, you know, to use data shadows or, or, or digital exhaust as much as possible. Um, I think that, that, um, what we may see is maybe they need some of that, the massive compute to be able to create, uh, private models to say, look, we can go and trend on all your data with our data, but you're gonna have it in an entirely isolated cluster. So what do, Diane, what do you mean by, Do you Know, what do you mean by digital exhaust?

That's a term that we haven't used here, so if you can explain that. Uh, yeah. So all people and all companies have what are called digital exhaust.

They give off, uh, during their daily activities. They have log files and notifications and messages and information flows. All that's collectively, that comes out of a person or organization's activities is called digital exhaust.

And it's actually very valuable because the more current a piece of information is more relevant, it tends to be. And so digital exhaust is the most current information about what a person or what an organization is doing in the digital world. And it's everything that comes outta that, you know, all the events and that that take place, generate that digital exhaust.

A lot of, it's not visible, but a lot of it actually is visible. You can script the public stuff. The question is how do you get the private, you know, digital.

Okay, cool. All right. So related to this topic is, um, this deep seek frontier model from China that, um, we are hearing about, and frankly, it's part of, I think one of the reasons why we do have Stargate is because our concern is China is advancing faster than we are, um, and getting way out there in front because they don't have any of these privacy issues that we have here, uh, as we, we well talked about with the TikTok front.

Um, but, so let's talk about this. The deep seek frontier model from China who wants to take that Well i's do a, a brief, um, you know, uh, overview of it and, and then you guys dive in. Uh, but yeah, this was a shot across the bow in that, uh, we've not seen internationally very many, uh, highly capable models.

There's been some, uh, but they haven't really been exposed to the benchmarks and proven themselves. You have to, uh, to be able to be considered a capable model, you have to, uh, pass the standard benchmarks at a high level. Uh, AI is, um, uh, generative AI research is very advanced, so there's very sophisticated benchmarks and leaderboards now that, that measure all the different dimensions of a model and scores it.

Uh, and you can go to these leaderboards and see how they're doing. And, and China hasn't been there until this week. Deep seek arrives and it goes near the top of the leaderboard.

It's competing with, um, open AI's best models at a, at a, uh, at a benchmark level across all the dimensions. Uh, this is a big surprise. 'cause as that, as, as much as, you know, they, they don't have the privacy concerns.

China also doesn't really want to create models that have all the information. They don't want their people to have all the information. And so, uh, it makes it difficult for 'em to pass the benchmarks when they have to censor large parts of their model.

So, because they don't want to generate that information, well, this model, they've either figured out how to, how to get around that, um, or they don't care anymore. We don't know what, uh, uh, it is brand new. So we're, we're still kicking the tires, but deep seek has put the American AI industry on notice that China is here, has closed the gap.

I don't think it was the reason why, um, project Stargate was announced, but it's taken the edge off the announcement by showing that there is a threat and it's coming, right? So that's, that's where we are. So from a usability use of the deep seek frontier model, is this strictly gonna be a Chinese model that the only China will be using that model, or are some of the other countries going to be renting it much like, you know, we, we are doing with open AI and, and those kind of things to be able to grab onto that model and do training with it?

Uh, I think it's, uh, because the, it has to patch, uh, open benchmarks, I think it's generally available. Uh, it, you know, we haven't seen how accessible enterprises can actually license it, or if it's just used back for researcher use. That's often when the model first comes out is it's only usable by researchers.

But we'll find out, we're Gonna see where, yeah, and I think it highlights that this is not a black and white issue of US regulations, bad Chinese unfiltered regulations. Good. It's complicated, right?

The Chinese have their geopolitical concerns, uh, they have their own privacy concerns on what Chinese citizens are allowed to view. So that in a sense, handicaps the, their ability to create models. It is exceptionally difficult, more so than I understand to control what comes out of a model, right?

Yeah. We've seen the likes of Google, Microsoft Open AI fail at doing what we call responsible ai. And I'm sure the Chinese government with their constraints have much bigger challenges on how they, you know, uh, we know we have this, you call it the digital exhaust challenge.

Uh, I haven't really coined the term for it, but you have the, this law of unintended consequences when it comes to models. You create a whole new DA data source that you didn't, uh, uh, you didn't count on. So what happens when you feed all your organization's data into a model that the users have rights to, you know, from a access control, they have rights to the individual, uh, data points, but they don't have rights necessarily to the derivative.

So when the model can now give them insights that they probably shouldn't have, how do you control access to that data and those insights? So a big, big concern for enterprise is the lessons we'll see from these geopolitical fights. Well, some of that, how you control some of that insight piece of it is also having to do with your data management and, um, how you're allowing people to get access to that data management, which is now finally bubbling up and, and saying, okay, so this is a big thing.

I mean, this is a big thing in terms of how we are man managing and how we're ma either masking the data, allowing people to access that data, use that data to train, um, you know, the, their, their models that they have internally. So Yeah, Diane, maybe me and you can do a deeper dive on what happens when the digital exhaust gets in the wrong hands. Like that.

That can be, you know, if, if I have, if I'm an administrator, I have access to the digital exhaust, and now I feed that to a, a, even if it's a in-house model, if I feed that to an on-prem model, what insights can I get about the organization that I probably shouldn't have is really mm-hmm. What do you start to, I would love, I would let to do that. Well, this where filtration and, and AI safety is gonna be so, so key, and it's becoming really important to make sure models don't emit sort of things that they know.

Because if you can't just use a scaffold and remove information from a model it's deeply trained in, it's not something that you can go and delete. Uh, so you have to filter it. And those are, how that's done is, is getting better, but it's still an art form.

Yeah. Yeah. So before we we're, we're getting close to the half hour.

I know Keith, we were putting on the spot this time. We have a bunch of other stuff we wanna cover, but I don't think we're gonna have time for it. And Keith, we, we wanted you to give your predictions since we both have had our time in this spotlight, but it's all yours.

Yeah. So, uh, predictions and re and kind of, uh, reflection on the year, this was supposed to be the year that Broadcom lost, what, 30% of their VMware customers. It did not happen.

Uh, Kimberly, both me and you, were at a, uh, uh, a backup vendors analyst session. They have pretty good, uh, purview over the VMware landscape, and they, I think they saw maybe a total reduction of maybe 3% of three to 5%, uh, of VMware workloads being backed up. So, uh, Broadcom has won, uh, at least in the short term.

Well, It is a proof point of how hard it is to get off of your s your cloud supplier if you were to deeply into your infrastructure for 15 years now. Exactly. Well, one of the things I had said that if we didn't have this AI craze that's going on, which is an absolute change in the company and in their orchestration of et cetera, you might've taken the time to do a conversion, But you might have, You have ai, you can't afford to put skills on this, And I think you're hitting your, you're, you're the, you're the setup model for my first prediction, which is the comp, the, the relationship between AI and enterprise.

IT is going to be complicated. The AI is not a IT initiative. It is, as Dion's research has proven out, it is a board level initiative that it will play a critical role in.

And there is a obvious skills gap. There is also this tension between where do I put my smart people, the I'm going to need my smartest, most talented, uh, folks, not just a technology perspective, but from a political and leadership perspective in IT to lead some of this AI driven change. Where are the budgets going to come from?

You know, companies are very frustrated with the licensing models from the lights of Microsoft or the AI agents at $35 a pop. This, this, this is a productivity tool that the burden is falling on it to find out where they're gonna get that additional $35 per user. I'm not revealing, I don't think any trade se secrets.

Microsoft isn't discounting their Microsoft Office 365 agent. I mean, not, not even the agents, just the O 65 stuff. So where are budgets going to land?

It's going to be a very difficult, uh, uh, transition for enterprises to figure out where the budget for AI comes from. That's, that's gonna be interesting. So this sounds like Microsoft has taken a page out of Hock Titan's book and said, I've got 'em over a barrel here, so screw you.

Sorry. Yeah. I, and you're not, and not, and I, I've, the, we, uh, as part of our C-I-O-C-T-O forms, me, and both me and Diane have talked to CTOs who have said, we're not buying it.

Like, it's, the options are from Salesforce to SAP to all of these companies, incumbents embedding AI into their base solutions. The question is, where is the value? Claude has a very different, uh, value prop than a, uh, than a, uh, Microsoft office.

And most CIOs and CTOs are taking a measured look, my son's organization that he works for has, you know, uh, given all of their organization access to Microsoft Office 365. And his reaction was, you know, what? He'd rather have the $35 a month to, to go out and get his own AI tools or do whatever he would want to, to increase his productivity.

So that's kind of prediction one. Prediction two is, you know, we're going to see more movement in private cloud. Part of this is driven by ai.

The other part is driven by, uh, public cloud has just gotten old. It, you know, it, it's, it's matured and enterprises have built, uh, applications in the public cloud 10, 15 years ago that they no longer touch. And we could easily run these always on, or sometime on applications, on private infrastructure.

And we're going to see private cloud take a bigger chunk out of maybe not the innovation that happens in the private, in the public cloud, but we are going to see the optimization of cost happen when it comes to some of these older public cloud workloads. Does this mean that bmware Cloud Foundation wins? I don't think so.

Uh, it is complicated, and this is stuff that enterprises can do simply with, with Kubernetes. So those are pretty much my big, uh, focus on, And my, my CIO uh, survey data backs you up. Uh, Keith, uh, 71% at all, all time high watermark by far of CIOs say they're reconsider where they run workloads this year.

Yeah, I saw that data plan, and I, I, I, I could not have shaken my head more vigorously. Yes. Like Yeah, it is, it is.

It's what I've heard. It's what I've seen. And it's, it's, it's, it's a short term and the long term, But at the same time, your other piece of data says they're gonna continue to invest in the public cloud.

Oh, yeah, Absolutely. There's within that, both of those tensions going on. Yeah.

Yeah. Public cloud's not going away at all. It's doesn't gonna keep growing, of course.

Uh, it's just that there's now a new mix in the, in the equation With the, with the, you know, $500 million, $500 billion here, 65 billion, I, I get a billions, half a trillion dollars. The, that's the biggest, biggest spend in infrastructure we've ever seen in the history of, you know, our industry. Yeah.

And what's inter what, what I love about this is that we're seeing this go into maybe the inner parts of our country, right? The, the middle parts of our country. So that expansion, um, we'll see the immigration from California and, and, and the coast into the middle of the country changed things.

Okay. Data center in Tennessee, who knows, You know, I think there already are. And, and you know, with t and all their water That's right, their water power there is gonna be great.

All right guys, it has been a full 30 minutes that we have been chatting. Um, and you know what? We have not even mentioned the word cybersecurity this time.

Can you believe it? Wow. So thank you so very much for joining us, um, and we will see you next week.