What Does CES Have to Do with the Data Center, Plus 2025 Data Infrastructure Predictions – EP66

Good morning everyone. We are back in 2025, and we are at episode number 66 for infrastructure matters. I am here with my buds, Keith Townsend and Dayon Hinchcliffe.

Welcome to the, the New Year. Um, and usually this week is kind of quiet, but it seems like, uh, some people decided to throw some things up in the wall and, you know, and, and, um, and not to mention, you know, in a call out to our friends in California that are dealing with what they're dealing with, um, which is, uh, very, very tragic. Um, but we're gonna kind of pivot over here to the technology side of the house and give you guys a break from watching out what's going on with the fires, et cetera.

So with that, um, anybody gets some skiing in or any fun stuff over the holidays? Just family. Just family.

Yeah. I didn't get any skiing. I did bring in the new year in camping.

We went, uh, we, we, we amped a, a little bit. So, uh, that's not a bad way to bring in the New year Back in t in Tennessee, Back in t, good old ho Wall, Tennessee. Well, I can't say I gotta Work on my exit.

I can't, I can't say I amped. I did the, uh, back did camp and, uh, tent camp and et cetera, biked and hiked in a death valley. If you haven't been there at fabulous place to go, especially at Christmas time when it's weather is good.

Very, very good. Good. So we are going, we are gonna go into, um, this week big show in Vegas, um, right off the New Year is CES Consumer Electronics Show.

Um, and usually this is not part of the data center infrastructure GY of guys, but nowadays it is, um, because so much of what we deliver as services, et cetera, is through these kind of devices, et cetera. So it's now kind of all blending in and then, and Jan's got a whole lot to talk about, especially with what, how Nvidia stole the show there again. That's right.

Well, and while none of us were there, we all, uh, we, we did have, uh, folks who were there, including our CEO, Daniel Newman, um, who met, who met, I believe, with Jensen. Uh, it was, um, amazing show. Uh, he had, uh, a whole rat announcements, announcements.

Some of 'em were consumer, uh, and not really related to this show. Uh, but unusually for CES, uh, there were some absolutely, you know, very, very much enterprise relevant announcements. Uh, the first significant one was Project Digits.

It's a personal AI supercomputer, uh, that has a thousand times the power of a regular laptop. Uh, and it's powered by the new Grace Hopper chip. Um, and it's gonna be, you're gonna see it in data centers.

There was already people stacking 'em up, um, uh, as high as they could go high, they could get 'em before they, you know, started to melt. Uh, the, just the, the, the compute density of this thing, and it's not really rack mountable, uh, but it, it's the fastest way to get a big model running anywhere you want it, anywhere you wanna put it. Uh, almost the biggest model you can think of.

It'll run. And it's about the size of a, of, uh, of the new, um, uh, apple, uh, Mac Mini. So it's very small, small intense compute density.

Um, you'll see it, um, really helping development shops, uh, and people actually operationally run some ai. Uh, it's, it's, uh, and the price point, it's $3,000. So it's for a thousand times more power than you get in a regular, uh, laptop.

So that was pretty cool. But that was not, uh, super enterprisey, although you will see it absolutely, uh, in data centers and in, um, businesses around the world, in my opinion. Uh, but, uh, Jensen came out, um, with a full size wafer, uh, showing that the GB 200, um, the data center super chip that has 72 Blackwell GPUs on it really exists.

It's in production. They, uh, he, he was walking around on stage with this single wafer that have 72 interconnected Blackwell, GPUs, uh, that will form a basis of their, of their massive compute. If you want to go all Nvidia and get the, the biggest configuration that they have, you get stacks of this, the, the, this giant wafer with 72 Blackwell.

So I see wafer that's like this big or whatever it is. And you're saying there's 72 black, That's big. It's like, it's like three feet across the biggest wafer I've ever seen.

Uh, and on it is, uh, is is 72 connected blackwells. They, they were actually, they were made that way on the wafer, on wafer con interconnect. So they're gonna put this in one computer then?

Yeah, like a mainframe. Yeah. Main AI mainframe, I think IBM MI mainframe main.

Yeah, IBM has something to say about, uh, AI mainframes. So, you know, to help give some perspective here, the digits is a, is one, uh, Blackwell chip, basically. Uh, and that one Blackwell chip has, uh, a, uh, pet floop of compute capability up to FP four, whoop the F four standard.

That enables you, so what kind of real work that means that it can comfortably handle a 200 bill, a 200 million parameter model. A intel CPU type deal, uh, goes up to about, uh, I'm sorry, 200 billion parameter model A, Intel CPU will handle about a 75 billion parameter model. So you, there's GP 272 times that raw compute performance.

So the networking, the memory, everything that needs to be done to kind of, to, to scale that out on a wafer. There's some pretty good computer science. So way back when, if we're gonna go talk about the mainframe again, of course, I will.

Um, some analysts made this huge prediction of how many mainframes would be sold. I think it was something less than 20, maybe it's 15 or something. Who the heck is gonna buy this?

But, and I'm not, I'm not gonna say they're only going to sell 15 of them, but, so this is, this will be used for the, the languages, the large languages. Yeah. This is, this is all still just training.

So there's a question, we'll, we'll get to our predictions over the next few weeks, but I, I do have a prediction around kind of where AI is going in the enterprise. Enterprise teams are not going to do massive training. They'll do retraining.

This is for the hyperscalers, the Facebook, the matters of the world, the, uh, all these companies, uh, uh, grok. Uh, so Z And Microsoft has already, uh, announced they have Blackwell in the data center. And you can, you can get special instances.

You can, you can specially select that. And this trip is intended for those massive installations. Microsoft just this week announced at $80 billion in CapEx.

They're gonna invest this year in 2025 in their data centers. And of course, a lot of that spend will be exactly on this particular chip. So it does bring up questions around that we've talked about over the past 2024 around heating cooling, so power cooling, and how do you keep these things cool.

That that is going to be one of the biggest problems. And of course, how are you gonna power them? We, we don't have enough power now.

It's going to take, you know, four to five years of bring on new nuclear reactors. Uh, companies are buying, like Microsoft are buying coal reactors and putting their data centers next to these coal reacts and taking up all the power from those reactors. It, it's, it's fascinating arms race around ai.

Wow. But I mean, just as kind of like, even, I, I'm, I'll need to go online and take a look at it stuff because I missed it kind of this week. And like, I wanna see this chip, or you can't even call it chip.

Can you really call that thing a chip if it's like, It's up three feet wide wafer a chip. It's a wafer. It's a big, huge, massive wafer.

Okay, what else? Oh, and it finally, they announced and we announced, uh, uh, cosmos. It's our new AI friendly world model.

Um, what, what, uh, a lot of these large language models, they don't really have a good model of the world. Um, they can make a lot of inferences that, that are trained up on a lot of incidental information about how the world works. But there's, uh, uh, uh, consequently a real need for world models.

And it's supposed to be the most enterprise ready, capable way. So you can build true AI powered digital twins that understand all the physics and all the geometries and everything all about the world. Uh, they had, they had some pretty cool demonstrations of, uh, of ais wandering around inside Cosmos, um, interacting with that world as if they were in the real world.

It was pretty cool. I, I don't see that with Elastic. We'll see that in a lot of simulation, r and d manufacturing, healthcare, things like that.

Logistics. And when we get to those world models, I'm assuming that we're gonna need these, uh, wafers. Yes.

Yeah. All they consume massive amounts of power. Well, yes.

Yeah. It, it, it, it's, uh, leading credence that maybe we are in a simulation. 'cause we're building equipment that can run simulations.

So it is hilarious. This is, this is amazing time to be in technology. It really is.

It's amazing. Well, on the flip side of all this technology that we're building and the cre, the things that NVIDIA's doing, there are some new, and we're, you're calling them AI export rules that are potentially coming out or seeing pretty strong coming out from the Biden administration, um, even before we pass over the baton to the next administration. So, um, Dionne, you wanna kind of walk through what's going on, and I understand that there is a massive amount of conversation on this because it's giving some huge angst to companies.

It, it really is. So the Biden administration has proposed something called the export control framework for artificial intelligence diffusion. And the intent is, is ostensibly a good one, which is to prevent very powerful AI to getting into the hands of our enemies and being used against us.

Um, and the, um, uh, however, uh, companies like Oracle, um, uh, have come out very publicly, uh, against it, uh, saying, uh, I quote, uh, it is one of the most destructive, uh, rules to ever hit US technology industry. And, uh, they warned a few weeks ago, uh, but now it looks like it, it is gone through the interim final rule stage. Um, it is not public.

I cannot, cannot get access to this because of how, how it's supposed to be enforced is highly secret. Um, and so it's one to watch, uh, the, the, um, the Semiconductor Industry Association ha had didn't come out swinging so hard as Oracle, but is very much against it. Uh, and the prediction is, is that it would reduce, uh, GPU sales in the United States about the 80%.

Um, and you know, I think some of that is debatable. Uh, the, the thing is we, it's just because we, it's hard to tell. It's hard to understand what is in, in it and how it works.

It's hard to judge it, but it's, it's basically what the government did with cryptography way back in the days so that our enemies couldn't, you know, keep their secrets from us. Um, uh, and, and this is, this is something similar. This is, we, this keeping very powerful AI out of the hands of our A enemies.

And so, um, it's unclear if what will happen if, uh, you know, if it will make it to, to actual official enforcement, uh, before the new administration comes in, who very, very well, you know, uh, you know, get rid of it after that. But anyway, it was the very much, the topic of, uh, discussion this week in semiconductor and in AI circles, uh, is it's kind of, you know, it's on the fast track and came out of nowhere. I mean, no one's really, we haven't been tracking this and this is kind of what we do.

Um, so it's very interesting to see what will happen. It's, it's something that, that certainly our infrastructure friends in the industry will have to watch very closely and try to influence to make sure it, it does not, cause it does not, does not overreach in it. Is this specifically on GPUs or is it on the large language models?

Or is it, what, what does it cover? The um, so it, it's targeted high risk uses and it restricts users of, uh, very high volume users of GPUs. And, um, so we're, I'm still going through everything that it's, it's trying to, trying to cons control the quantities of GPUs that are, that come outta the United States.

Um, and it's, it's, it's a little bit less on the models because what they're really trying to do is, is control the ability to even run the models. Yeah. So this is, uh, been highly debated for quite some time, right?

There's g there's already GPU controls on the, on how, who, who we can export GPUs out of. And China came out with, uh, I forget the name of the model, but that hit the news, uh, last week, what was, uh, a model that worked four times faster than I think it was LAMA or the model that it, that mimics. So the, I think the CAT is already out of the bag with, with some of this.

It's a complex question because there's this, um, there's this debate on what controls Congress and the government wants to pass because I think, uh, there's probably pretty good, if I was to guess, looking at the news or, you know, at TikTok there's probably pretty good bipartisan support for something like this. Uh, the, the government and tech and tech leaders are, are, seem, seem to be at odds at what's best for technology versus what's best for us, uh, national interest security interest. Well, and, and what identifies 20, what are called artificial intelligence authorized countries.

And they're the ones that can get hardware that can run frontier models. They're the ones that will be able to develop ai. 'cause the concern is, even though if we keep our models away from them, uh, the the bad actors, they can still build their own.

But what if we take that power away too? And so the, there's only 20 countries that are on that authorized list, and everyone else has very sharply reduced access to any powerful GPUs using this export role. And this is coming out of the Department of Commerce?

Correct. Because they're the import export folks that are, and, um, those, those organizations, It is a very complicated, um, there's a lot of agencies involved in it. It obviously would've to be Congress, but, uh, well, there's also the Department of Homeland Security, which is sponsor of IT and things like that.

So this will probably, This, this is to watch even from change from administration to administration. The, the, the folks in congress are, are not, uh, the friends of technologists right now. Yeah, right.

Well, it depends Story and everyone should be watching the tech business. So coming off of the, the current administration area, which I'm gonna try to try, try to stay away from that and everything that's going on, we did have, uh, in my world, the data infrastructure world. There's this company, and a lot of people don't know who they are, but DDN data direct, um, networks folks, they've been privately held.

They, um, by Alex and Paul, um, very, very, uh, Alex is a very flamboyant Frenchman, I believe he's French. Uh, and, um, and they, they've been very darling's in the HPC market. They have traditionally only focused on HPC market or the r and d market.

And with AI coming on board, they have just taken off like crazy. And, and I know from talking to some folks that all they've been doing is pouring money into just ship boxes, which means that's all the inventory and everything else that you're moving. So they've just taken a $300 million investment from Blackstone, which is a private equity, actually publicly traded company, if you wanna go look at them.

Um, but that brings our valuation to 5 billion, which I think is kind of seems like that would be a little low, um, when think about comparison to some of the other guys that have gone up. But, um, we'll see. I mean, I'm sure what's happening here is that the, the co-owners are, are holding, um, their ownership, um, and their, the power of the company because it's just been these two guys that have owned it and it's extremely profitable, et cetera.

So it's very exciting to see another, yet again, another hardware vendor and software vendor that has taken on, um, some big dollars into this market. So it's not just about software anymore, it's definitely the, Oh, hardware is back big time. And of course, but that's a, that's a capital intensive game.

Um, and you, you need to be very well funded for the long term to even play these days. Yeah. And they've been, um, very, very close with Nvidia all along because this has been their market.

I mean, this is all they've done. And yes, they bought a company called Tint Tree, uh, means five, six years ago, um, that had was, and they were, looked like they were starting to try to expand into traditional data center space, but with AI taking off, there's really no need for them to do that, uh, in terms of their valuation or who the company is. They just exploded on that side.

So it's been very, very cool and a big congratulations to these guys out there. So, Yes. Yeah, and that kind of goes along with, and I guess the other thing we were gonna do, um, we were going to take this, this session and really kind of go through our predictions for 2025, but with all the things that were going on, we said we didn't have time.

So what we're gonna do is split it over the next few weeks for each of us to take a section and talk about it. Um, so I had mine ready to go, which is of course the, you know, my first two issues there. And there, there are more granular in terms of predictions because there's, you know, yes, the market's gonna grow.

Yes, data's gonna grow. Yes, all these things are gonna happen, but it's kind of like, what are the specific things that we're gonna start to see that probably weren't part of on our venue in the past. And when I sat down and looked at that, I said, one of the biggest things I think is gonna happen is the rise of scale out file and their understanding what we need, the need for parallel file systems and requirements for ai.

Hmm. And to this time, it's always been a secondary market. It's been a market for r and d, it's been a market for the lang, you know, the, the big labs, et cetera.

That's who was, because these were so difficult, parallel file systems aren't not the easiest things to manage to, um, you had, you know, the ones that I'm thinking about would be luster would be GPFS or IBM scale would be BGFS. You know, some of the guys are out there and you know, it's just not something that somebody wants to go play on. But as we're seeing each and every one of the, the vendors are bringing out something to say parallel scale out, and not just scale out, but parallel file systems.

Dell is working on something, you're, you're hearing vast kind of move in some of those areas. So they have those relationships, some of them have those relationships with tho those systems already. But we're also seeing like hammer spaces being put on top of some of these file systems in order to bring a parallel like activity that's out there.

And basically what that means is that the enterprise that's never paid attention to this guy, unless you are a big, big r and d facility, is all of a sudden saying, I can't get my very high speed file system such as power scale from Dell or NetApp to perform at the level I've got to make it perform at for ai. Mm-hmm. What else?

Those file systems you designed for that? I mean, yeah, we're, we're dealing in, in, in entirely new levels, petabytes of trading data and things like that. It's crazy.

So this next year I think that conversation is gonna rise up. I think the enterprise is gonna start saying, okay, so what is this? Tell me more about it.

What is the difference between A and B? Why can't I do this with this? Um, so that to me is the big rise of that.

The second piece of that is the understanding and the tuning in of where objects and file belong in that data pipeline for the ai, um, initiative where that's going. So we've got the data lake that needs to exist, how is that gonna exist in the data center? What's that gonna look like?

How am I gonna afford that? Because we're hearing from the enterprise folks that the, there's a huge sticker shock in terms of what this is gonna look like potentially. So we've gotta look at different ways for storing that, because if this, uh, this side is too expensive, I can't justify the ROI here, but can I not justify the ROI here?

So somewhere along the line, there is invention and opportunity in here to understand what's gonna have to happen from these big object file data lights. Well, I, I think there's a whole ROI problem in general. I'm looking at, you know, all the, uh, the, the venture capital firms are putting the graphs together of the expenditure on AI and the expected profits over the next 10 years, and they don't add up.

Um, and you, and they, you have to do it, but, uh, you have to invest in ai or you're just, you're outta the game all altogether. But even $80 billion Microsoft is spending on infrastructure, uh, which is gonna involve all those things you're talking about, you know, massive file systems of even greater amounts of trading data and all of these things. Um, where's the, the end game?

Are we gonna see exponential growth in, in, in profits taken from all these investments? We don't see that. I, I'm not sure it's, it's, it seems like the industry's not heading in a sustainable direction.

So it's interesting. Yeah, I think in, uh, my experience with foul systems in the enterprise, it is very, very, very difficult to get enterprises to change file systems when you're talking about where they're stored, uh, changing user habits. Uh, and in this case, we're probably talking about data scientists.

We're probably talking about researchers, uh, in the enterprise. And it's, uh, and people who are looking to get the data into AI systems. The challenge is that these, these, the data, you know, exist on Fowlers and NAS systems and it's distributed, it's in the public cloud and trying to get a, uh, without interrupting users', workflows, getting this data into distributed, uh, into some type of parallel file system that that's usable and that can use rag, et cetera, et cetera.

I think that will be the challenge of 2025. The need for performance will obviously see, but, uh, uh, smart folks coming up with solutions that make this as, as seamless as a transition as possible is, is the challenge. And, and then throw in two, two of the technologies that are part of this, part of this entire AI capability.

One is streaming data. If I'm gonna do real time AI with streaming data, et cetera, all of a sudden it brings more complexity to what we're doing to the pipeline. And, you know, we hear, you know, the different folks about how do I adopt a streaming data that's coming in for those decisions?

And the second piece of it, which is already part of the environment, is a vector databases. And as I take, you know, as the vector databases get indexed, they grow, they grow significantly. And so how do I deal with that?

So there's probably invention to be had in, in those areas from an eng from an engineering standpoint. Um, as we move forward at this, these next, next few years in addressing, as you were saying, um, the, the ROI, et cetera, um, the last two that I had on my list is that from the enterprise standpoint, we're gonna see continuing decrease of any stop, uh, uh, hard drives there and moving to QLC that's been driving revenue for these guys, um, and is going to continue to drive revenue, um, for the, the major vendors, um, as they roll out. So we'll see less and less of those blended boxes that are in there as they come of age.

And that will fuel the, the revenue in those spaces. And, and the fourth, fourth area that I'll bring up is the area of data protection and, uh, cybersecurity that does not leave. Um, Diana, as you well pointed out in your CIO insights, it is still number one on the list.

And, um, we will still continue to look at what do we need to change for data protection in order to improve the recoverability and the speed of recoverability. So there's two pieces there. It's not just making sure that we're safe, but now what they're looking at is how fast can I restore, um, and when I get hit, um, and then the other piece of that is how do I protect not the secondary data, but the primary data that's been going on.

So we'll see the increase of technology and that will become a competitive advantage for those vendors that have built into their primary storage, those capabilities to protect those devices. Yeah. And that the, and combining kind of two ideas that you are looking forward to, how do you do this with ai, you know, with AI and streaming data, and now when ai, when this secondary data now becomes production data and, uh, the need to protect that from ransomware, from cyber threats in real time and be able to recover in real, real time, I think we're gonna see, again, your, your premise that, you know, we're gonna see the reduction of hard drives.

We're gonna need much better io so much better. And denser io, so 27 terabytes, I, I don't know if I'd ever say this, A 27 terabyte hard drive just is not enough. Uh, and it is not enough, and it's too slow compared to the a hundred and, uh, the 122 terabytes we're seeing out of the major vendors.

There's talk of being, uh, uh, SSDs, I'm sorry, and, uh, going to 256, uh, terabytes of SSDs this year and the need to just have faster and bigger io it's, uh, it's a, it's an amazing set of challenges going into the new year. The GPUs are so they can, they can take so much data is how do you get the networks and the hard drives to deliver information on the process. That that's the, that's the, the core problem right there is you basically have to match the, the data throughput of the GPUs across the entire data center.

And that's, that's amazing. And they're gonna try to do it, you know? Yeah.

And we, so, you know, obviously we're gonna see opportunities for folks like Cisco, the tech field day folks will be at Cisco Live eu, and, uh, uh, I think that's next month or when we're recording this, and obviously HPE with its acquisition of Juniper and the ability to have these tightly integrated stacks. Uh, Dell has its network stack and this ability to build these Nvidia blessed stacks, which Nvidia has competitive networking. So just as you start to tear this apart, apart, and we start talking about vendors, vendor relationships, the ecosystem, it makes for a really interesting year of, you know, how do we get these engineered systems that we want from our favorite vendors that are blessed by the people who are controlling ai, which is basically Nvidia, And then the US are our US manufacturer.

All of our US customers should consider themselves blessed because they won't fall underneath these export control issues. So if we're gonna tie all of those pieces together, they'll have the freedom to acquire whatever they want to acquire, um, right. And, and be able to build on those systems.

So. Wow. Well, that brings us almost to the end of the hour.

Any other final comments coming out of any kind of cool stuff you guys got and we'll wrap up here. No, I just wanna give a tip to our fans at kaza. They got, uh, they've been horrifying.

Luke and Matt, uh, have been hard, hard plugging, working with our folks in the Signal 65 lab, doing some really interesting ai. The, they received them $11 million in funding from some Seattle based venture capitalist. This will enable them to continue what the venture capitalist is calling the docker of ai.

So, uh, stay tuned from some, uh, some insights from our signal, 65 labs from some of the work we've u uh, used, uh, KAA to help us produce some really interesting research. So, are they, did you say they're located in my backyard? They're located in your backyard?

Uh, I think, uh, Luke's home overlooks the sisters. So the, the, the, he is a, i I I, I saw it as he was a rehab, then he worked for a different, uh, data based, uh, data enterprise data company. So, yeah.

Okay. Well, I'll have to look him up then. Very cool.

Alrighty guys. Thank you very much for tuning in. Don't, don't forget to like, share all those good things that we ask you to do, um, as we continue to bring you infrastructure matters and probably the most interesting podcast that you will listen to all week long.

All right. Have a great week.