AI goes nuclear, x86 gets a group, plus IBM, Lenovo and BMC events

All right. Welcome to episode 59 of the Infrastructure Matters podcast. A podcast in which we count episode numbers because we're all pretty much geeks.

Welcome back. Uh, I, I have with me my good friends and colleagues, Steven Dickens, Diana iff, and Kimberly Bass. Folks, this is the season, right?

We're, we're all somewhere. I'm, I think I'm the only one that's not somewhere this week because I tapped out. I have to tell you, uh, I, I started the week off and I was supposed to go to Hashi Comp, and I just did.

I, I did not go. So to my friends at Hashi Corp, I'll, I'll see you next time, but everyone has been somewhere and, uh, we have a lot of news news to get to. So let's get to it.

And Diane, we're gonna start with you because you were, uh, at a, I think it, it, it, analyst events are special, but you are at a special, special analyst event That's right. For IBM. Talk to us about it.

Uh, while the setting was New York City, and they, uh, they brought people from all over the world, um, they, uh, I IBM's really on an upswing, uh, you know, they're, uh, they're, uh, in their CEO of the last couple years, uh, uh, Arvind Krishna has, has done an outstanding job kind of making IBM relevant in the enterprise again, uh, really, uh, reclaiming, you know, the, kind of the lost glory, what they had with ai. They used to be an AI leader, as you remember, with Watson. Uh, and they really wanna reclaim that space.

And they wanna be the enterprise AI company. Uh, and they had lots of things on show including, uh, large language models. Uh, their, um, their granite large language model can, uh, perform as well as all the, you know, the, the well-known large language models, but at a fraction of the cost.

And AI is expensive. And so this is, you know, this is their value proposition saying you may not have heard of it. We not, uh, granite may not be famous, uh, but it's gonna work for you because it's very inexpensive to run compared to all the other ones.

So they have some very compelling models. Uh, and, and, uh, Arvind was very impressive himself. Uh, he set, I set an hour to come in front of the analysts.

Uh, there was no slides. He spoke extemporaneously. It was an ask anything, so analysts could ask him anything for an hour.

And we did, uh, you know, our, uh, our CEO, uh, uh, Daniel Neman got a good one in, uh, and so, uh, and he helped, helped, he really defended the company and how they are coming back. Um, and they're resurging and the stock prices has been doing well. Uh, so they had lots of announcements, which are all under embargo, and I can't tell you about them yet.

Uh, we maybe, uh, can, uh, uh, uh, we can tip the hat next week at, uh, on infrastructure matters. Uh, but I can tell you they're mostly around ai, uh, and the things they're doing. Uh, but they did, uh, give a big update on quantum.

IBM's convinced that, uh, Quantum's about to hit our data centers and be coupled, just like GPUs are coupled with CPUs, they showed their heron chip with 156 qubits. Uh, it's just about ready to go into the data centers. Uh, and, and 156 qubits is, uh, especially when chained together, uh, with multiple heron chips is enough to get real work done before the, uh, quantum computers were really just toys.

Uh, they could just simplify, you know, simple, basic algorithms really, really fast. Of course. Um, uh, but they are, IBM's really demonstrated their lead in quantum.

The Heron chip is very impressive, very low noise, very high accuracy compared to previous generations of, of Quantum. And so, I, I'm actually gonna, I, I was pretty sure we're not gonna see Quantum for at least another five to eight years. We're gonna see that in the data center, you know, in two to three.

It's pretty interesting. So, Dion, that is very impactful because for a couple different reasons. One, the number one place the technology goes before it hits the public is nefarious applications.

Yes, that's right. So either it's either porn or it's cybersecurity. Honestly, that's where we see you.

Like the videos, DVDs, all that kind of stuff. That's where, that's where that went first. And when we were at Lenovo Tech world, we had there, um, in, I tr Doug, I can't remember his last name.

Yes. We for about an hour. Yeah.

Fascinating, fascinating stuff. And the, you know, the big thing they're talking about is, okay, so, you know, big thing, here's the date that the government is saying we have to have quantum, you know, encryption capabilities, et cetera. And if this is coming online now, we are missing that date party.

You had to have the implementation. And we are they, if that's true, we are behind what we need to do for cybersecurity To be New. This, They've been talking about quantum safe encryption for four or five years.

Yeah. So IBM's been, uh, maybe they knew that they were gonna be out first with Quantum from a technology point of view, but I think it was, yeah, it's four years ago that we, IBM started talking about quantum safe encryption. That's a big deal.

And the Chinese announced that they've, they've broken part of RSA using the D-Wave, um, quantum computers this week. That was the other big news. So, uh, we're now entering this where we, you know, making quantum safe.

That was one of the big messages that IBM had is for that exact reason, is that, uh, the e-commerce on the, the internet, uh, cryptocurrency and a whole bunch of other things that we do private public key signing with are all at risk with, uh, with, with these quantum chips. And so it's gonna be, you know, CIOs are on notice that we gotta start to dealing with this right now. Wow.

Well, there's four new Nest algorithms going through, and I think IBM's been involved In Two of them and is actively involved in two of them. And then one of the others was done by an IBM. So they've been at this for a while.

So as we're thinking about, as we give advice to CIOs, CTOs about infrastructure and protection around quantum, what, give me three things that they should probably pay attention to preparing for quantum, not just from a quantum chip perspective, but from infrastructure readiness. I'm putting you, I'm putting you three on the spot. Yeah.

So I'll take the first one. I think thinking about quantum safe algorithms get up to speed on what ni NIST is doing on those four algorithms that it's updating. I think it's thinking about encrypting data, knowing it's gonna be able to be hacked.

So it's a kind of harvest the data now, decrypt it later type approach that a lot of these nefarious actors are gonna take. So just be thoughtful. Get up to speed on those nest algorithms.

Be starting to ask your vendors what they're doing around quantum safe encryption. That would be the first one I, okay, so the anomaly detection becomes critical on the dataset. Um, and if you don't have anomaly detection going on in your pri, not not just the data protection side of the, this, which is where you recover from, but the primary storage, you're in trouble, um, potentially in trouble.

So, I mean, we, we talked last week about pot, you know, maybe what the, the B of A issue was, you know, was that, was that an attack that then nobody's talking about? Um, but that means that they're gonna be going after, and they're gonna be going after where the biggest, biggest ones are, which is the mainframe and the transaction areas. Um, and that's, so every mainframe needs to button down IBM, um, so, you know, they're bringing it out in the market.

IBM needs to resolve the problems with the, the Z mm-Hmm. And that's why they've been talking about, that's specifically where they've been talking about quantum safe encryption for the last two circles of the box. So as primary storage has got to have anomaly detection, if, and I will go, I'll use another time to talk about encryption, um, because I had a great, great briefing from, um, Broadcom about what they're gonna be coming out with, um, on some encryption stuff that's gonna be going on.

But, um, this, I'll stop there. And I would just say CIOs need to start, uh, creating a, a list of, of areas they have to remediate in their, across their IT infrastructure. Uh, they need to know clearly where they're, they, they need to quantum re reinforce their encryption and reinforce, um, their algorithms for quantum safety.

And they need to get that list and then, you know, work it backwards based on technical debt, uh, cost to, you know, uh, benefit in terms of where the biggest bang for the buck is and, and get started. Yeah. And I'll give a bonus one, processes.

This is going to impact impact processes. You know, do I have the processes to do new algorithms and everything that you folks just mentioned? Let's move on the conversation to BMC.

Uh, Steven, you're, you're big mainframe guy. You know, mainframes are big. You're a big mainframe guy, and you attended, uh, BMCs, uh, event.

What, what went on there? So they'd come out the week before their big conference announcing that they were gonna split the business into two. So I've been tracking BMC, I wrote a Forbes article mid last year.

This is a roughly $3 billion software company, um, 40 years old as a startup, if you will. Private equity owned by KKR. So they've announced that they're gonna split the business into two.

Uh, one of these names is gonna have to change BMC and BMC Helix. That's just gonna get confusing for people. But the announced the split, the mainframe and, um, control M business is gonna go into the BMC business and their, um, IT service management and observability business is gonna be the BMC Helix business.

I think you, you took you for exactly the reasons of why you described Keith old BMC is a mainframe company, BM C'S more than a mainframe company. It's about 40% of their revenue is mainframe related. But I think given their history and their past, they're seen as a mainframe company.

So I see this a trajectory for, um, KKR to be able to realize value from the business that they own, spin out that observability and AIOps business and where it goes from there once they're separate businesses is gonna be interesting. Then I was obviously out at their conference this week, recorded a lot of video with their team. I think the key takeaway there is ai, lots of focus on agen AI rolling that out across their product portfolios, either from the observability perspective and the IT service management in the Helix portfolio, what they're doing in Control M with the, uh, scheduling and automation piece.

And then also rolling out ai, you know, bring your own mo large language model to their AI assistance for the mainframe. It, it was a very on message on 2024 AI type announcement, and it was just on the mainframe. So I think, you know, all the announcements and all the events I've been to this year, it was exactly on message with where they're going, code assistance, code explanation, bringing operate, um, large language models and natural language interfaces to operational tasks.

You know, you could have been sitting in a Dynatrace briefing, you could have been sitting in a, you know, HashiCorp brief. You could have been sitting in any one of the industry vendors we all talk to. It wasn't very mainframe specific, which I think is a net net and good news.

So yeah, so Lot's going on with BMC right now. Alright. And again, we have a lot of news to get to.

So, uh, Kimberly, talk to us about some, some liquid cooling. Good news. I can mention the, well, let me go over to le the Lenovo tech world because we were at, uh, Lenovo had had had a, you know, public event and then they had a day and a half of analyst event.

Um, and, um, why, why, and I am not gonna try to pronounce his name because I would insult him, I think. Um, but why, why, who is the CEO kick that off? And we had a plethora of executives that were, you know, CEOs that were there on stage with him.

Um, and including that Gelsinger and, um, the CEO EO from a MD that was there and talking about the Intel A MDX 86 group that's going to further the purpose of the 80 x 86 and its, um, standards that are out there, which was really, really, really cool. Um, but one of those things that there was announced was some liquid cooling, um, new, the new, uh, Neptune device that is a vertically installed, um, ser you know, server system that is pretty, pretty cool. And it's a hundred percent, you know, non heat.

I mean, it's just that you don't, and you use standard power for this thing. So may maybe you wanna talk take, take that a little bit, um, Steve, because you were Yeah, Liquid callings got cool this week. Um, I wasn't on the show last week.

I was coming back from, uh, HP's AI day. They took us a bunch of us out to their facility in Wisconsin to talk to us about their direct liquid co direct direct fan, less liquid cooling cray systems and high-end, um, AI and high performance computing systems This week. We've had super micro make some announcements as well call it, has made some announcements.

And then obviously there was the, um, Lenovo announcements we're talking about, uh, Ron Westfall and I have taken all these enhancements and synthesized it into a, into a analyst coverage that's gonna be, uh, available on it, uh, futureum intelligence for those subscribers. So, 'cause there's a lot going on in this space. I think the key takeaways for me, you cannot run GPUs in a data center at scale and not worry about how you call them.

These things are power hungry beasts. You're gonna be pushing over 70 kilowatts in a rack, and once you get past 70 kilowatts, you're gonna need to worry about how you call it. And that's where you're gonna need to be looking at liquid calling options and te different approaches.

You know, is it liquidly, glyco? Is it water cooled? How do, how do you do it?

You know, you then get into the physics and the, and the sort of properties of glycol versus water, the connectors, these things obviously don't wanna leak. You know, is it copper wiring? Is it plastic cables?

You know, there's a whole bunch of stuff going on in this space, but it all comes down to that if we're gonna roll GPUs at out at scale, we're gonna have to have liquid cord. I, I I've heard a lot of, uh, um, tier two data centers. In other words, not, uh, uh, data centers not run by the hyperscalers and they're all the new ones are designed to assume that entire building is gonna consume water to, to cool it.

It's very interesting. And the, one of the things that was very clear with, with them presenting, um, and, um, they understand that we have new data centers that are being built and data centers that we have to retrofit. Yeah.

And so they're looking at ways to, and so they have just, just like we talk about, you know, the stages of VMware or whatever, which you do with VMware potentially. And they're talking about the same way of saying, okay, so I've got all this here and I need to accommodate for it. How do I do that?

So they are rolling, you know, strategies out for customers to be able to do, whether or not it's air cooled or water cooled, or systems that are partially water cooled and partially air cooled. Um, you know, depending upon what you know, folks have. And this was really because I asked them the question about where's the break even on water cooling?

It's figuring that is a big, huge, massive data center because my old brain about big frames and everything else. And they said, no, they have a college that is setting up a GPU closet essentially. It's not a data center.

It's more or less like a closet that they're putting in liquid cooling in there. Well, there was Interesting Yeah. Data center.

Yeah, exactly. It's interesting To me. The HPE and lenova both independently said 70 kilowatts per rack is the cutover point.

Once you get Yeah. Sold past 70 kilowatts, you've gotta go liquid Corning. Apparently that's the, that's the break even point.

Well, and there is an r, the ROI that is there as well. I mean, one of the things they talked about with this, the they federal and they, they quoted, um, I don't know, it's probably acknowledged, but one of the, um, and big movie theater, um, folks, uh, not theater, um, designers, they have liquid cooling GPU environment. They can run it 40% faster.

I mean, you, you think about what that means because they're not burning the mach, you know, machine up. You know, when we run, run hot, it burns up. So, I mean, not only are you saving on the cost of the energy, but if you can fast, you can push those machines that much faster.

Wow. And they'll last longer. You might get a few more, a couple more years out of 'em.

So that, that's Good. Yeah. Our colleague guy Cory is at, uh, open compute, uh, platform.

Their summit this week, which is, I'm a little jealous if I'm going to go anywhere and geek out as OCP. 'cause you see hardware on the floor, they're talking not just 70 watt cabinets, which is a lot for a typical enterprise, uh, data center. But right now some of the hyperscalers have 250 watt, 500, uh, kilowatt and 500 ki uh, kilowatt.

And now they're talking about one megawatt racks. We're seeing that density get smaller and smaller. So we're not talking about floor space.

Uh, Diane, I know you're old enough as me to remember when data centers, uh, colos charged on the number of towels that you took up. That's right. Now it's not about towels at all.

And it's, they'll give you as much floor space as you want. And it's about power and power commitment. So this is something to continue to watch liquid cool cooling.

How are you gonna cool the stuff? How are you going to, uh, accommodate denser cabinets? I would love to see how this university is just running the physical power cabling structure to get to the closet, to, to provide enough power safely, and then kind of the just, uh, the, the environmentals around, uh, having that much power and the, and the fire risk that goes along with running that much power in the build building not designed for it.

Speaking about power and buildings not dis designed for it. Nuclear has become, uh, uh, the in vogue again, we're, we're looking at nuclear as the only clean option for, uh, providing all of the power needed for these one megawatt rags. Yeah.

That, that, that's, and the two were absolutely related. We were on the right trajectory, I think with ESG goals. The, the systems were becoming more efficient.

We were drawing less power in the data center. And then Nvidia starts putting GPUs in everybody's boxes and with gold completely back the other way and worse. So specifically the hyperscalers, you know, you look at the sort of four or five big companies that are buying GPUs at scale, it's the three hyperscalers, it's meta and it's Tesla.

They're probably hoovering up 80% plus of the GPUs that NVIDIA's selling right now. So it, those guys are just purely struggling with power supply. You know, we saw small And the Ohio net zero commitments.

I mean, uh, yeah. You know, yeah. CIOs are depending on their cloud providers to help them reach net zero.

Now the, the hyperscalers are having to spend way more on power than they ever expected. And so I think this is why nuclear is in the equation that Google just announced they're gonna get, you know, they're gonna buy six to seven micro nuclear reactors to, to power. Yeah.

It's very More general reactors. I mean, just the, the flip on nuclear, the speed at which it's happened. We've seen Microsoft working with, um, a power company to Recommission three Mile Island.

We've got Google and AWS making announcements this week about looking to bring in small modular reactors. I think you've, this is a trend that's come from nowhere in the last six months. I think they dipped their toe in the wind water.

Somebody broke, you know, Microsoft kind of broke cover with this, with the three Mile Island announcement. And then everybody has jumped on this bandwagon because what they thought They have, are they gonna be, they're gonna be painted the bad guys? Uh, yeah.

And, and exactly. And they're not gonna be able to get their AI dividends. They're hoping that AI is gonna really pay off for them, but if they're being, you know, uh, uh, forced to curtail it because of the, the massive amounts of energy and, and, and, you know, having to pull coal fired plants or, or that could be a serious problem that this heads it all off at the past, potentially.

We'll see. Yeah, I thought it was, uh, I thought it was a few weeks ago, but it was actually back in March. Amazon actually bought a nuclear power plant from Talen, and they, I think they're contracted to take 50% of the, uh, set 960 megawatt, uh, power.

Uh, so this is a trend that we're going to continue to see. Uh, I'm, I'm sorry I cut you off there. Kimberly, you were about to mention something.

No, I was, uh, what I am looking from a geopolitical kind of situation, you know, we've had, Japan has always had continued to have nuclear. France is heavily, you know, invested in nuclear. Um, it's, and, and, you know, United States is kind of like, oh, it's evil.

It's evil, it's, and I'm sorry. So not, And well germany's out completely. Yep.

Yeah. And so Yeah, the, So it's all of a sudden. But, but we will have to deal with the, the deposit Itself.

Yeah. We just can't create enough clean energy in other areas to accommodate this much power. It's a little more nuanced than that as well, Keith.

It's around the, the regularity of that production of, you know, wind is great, but the wind does blow at the same free, you know, the same amount every day. Solar, it's sunny. Some days it's not others.

You need the, if you are running a data center, you need it consistently. So that's the other thing thing that nuclear brings to this equation. Just on the geopolitical piece, I've been looking into this a little bit.

China is building 200 nuclear powers. Yeah. We, we, we aren't there.

Battery storage technology isn't going to fix this problem. We have to be able to consistently maintain, uh, systems. Moving on the conversation, we hit on it earlier, A MD and Intel announced a new advisory group.

X 86 is X 86, right? Well, not so much. If you've managed data centers, if you've managed any kind of IT infrastructure at scale, you know that A MD leaps intel, Intel leaps A-M-A-M-D, and ideally you'd like to mix and max your distributor, your, uh, providers and buy from both.

The nuance is that, you know, just taking a look at the Dell portfolio, Le Novo, uh, portfolio, you'll see a box like the 98 60. It's the big AI box that, uh, Dell has been, you know, making hay off of a little dorky nomenclature. 90, the Dell ends the last number of their servers with O for Intel and five for a MD.

HP does The slate. The number of They, uh, they also do the same. And the number of options you have for old servers versus five servers is massive.

Intel rules. The row when it comes to the number of SKUs that, uh, a customer can get from them based on Intel versus a MD, which means that you're going to end up, even if you're an a MD shop, if you want a 98 60, it's going to be an Intel chip. Uh, and so if you want, you're, you're big on AI and you want the biggest, baddest boxes that Dell presents, they're gonna be Intel chip and interoperability is a problem.

Uh, you know, if you've ever done VMware, being able to live migrate one VM to another requires it to be on the same processor. And these have, this has created problems for enterprises as they refresh and buy from separate vendors. And the goal of this group is to normalize some of that stuff is to be able for, uh, X 86 to be X 86.

My question to you three is will this impact Intel's ability to compete with a MD? Are, are they giving up, uh, kind of ruling, uh, X 86? I mean, there's two ways to look at that.

I think there's, it's, first off, it is an acknowledgement that a MD is not the noisy small little competitor that it used to be 10 years ago. Um, you could argue that they should have done this 10 years ago. Um, it was interesting watching some of the side, uh, the risk five community kind of, um, promoting this and say, well, is we've fixed this with Rick's risk five, um, that's that open source community that, that, that does all of what this 6 86 group is planning to do.

And they've been doing it across multiple different chip providers. We WeChat to the guys at scifi, probably the biggest sort of chip designer and provider in that space. And some of, some of their social posts this week were quite interesting.

But it's, I mean, it's certainly an interesting move by Intel to acknowledge a MD at this level. Um, argue it's good for the, I think it's just gonna push, uh, you workloads up, you know, up the stack in the, the container model Mm-Hmm. Where you don't have that, that type of issue and, and you have much more flexibility on, on, on where that workload can run.

Um, and so, I mean, virtualization's obviously not going away, uh, but I think we're gonna see it more, especially business workloads be much more containerized, uh, to avoid, you know, any of the splintering or any of the issues around that. So I don't play that much into the CPU environment, but my outside looking in, it seemed that this was a way to compete potentially against Nvidia and long game. We talk about the CPU or the X 86 being a viable option from Chase.

And if you know, versus, you know, having a, yeah. The partner or whatever, you know, that come back to competition that's going on and how much they're, you know, driving the market right now. This is a way to catch up.

Yeah, I, I, I agree. Some of it is, I think some of it is ai, pat and Lisa mentioned in their interview with Pat and Dan that, uh, which Was an exclusive, by the way, the, Which wasn't exclusive by the way. Very cool.

The, that, you know, they mentioned the AI extensions A MX and uh, uh, AMD's version, but I think a lot of this even goes down to, to the consumer level. All of these extensions are compatible, but at the software level, uh, it matters. The, uh, that interface, if we, you know, talking about Cuda and the competition to cuda the, it matters how you are able to extract, extract and get to what Diane is talking about, that if I'm buying X 86, I expect the, a consistent interface from a software perspective.

And when I don't get that consistent interface, you're disrupting my operations. And now I'm going to look to a risk vibe group. I'm gonna look to a arm group who, I may not necessarily get the performance and the, uh, price value, but I am going to have operational co.

So I'm looking at this as a operational consistent, uh, play. Alright, we're out of time. I think we could talk about all of these topics for a half an hour.

And if you wanna get more of the half an hour, subscribe to the podcast. Tell a friend, I like to say, you know what, if your grandmother isn't into tech, this is a great podcast for her to still not be into tech Until next week, uh, by on behalf of all my.