Sustainable Computing in the AI Era: The Path to a More Energy Efficient Data Center | The Six Five Summit

I we're joined right now by Scott Tees. He's with Lenovo as the general manager of High performance Computing and ai, uh, infrastructure. And we're glad to have you to talk about this issue of sustainability.

Um, it's so, you know, data centers Scott, are like the center of the world right now. Um, and there's so many issues around, uh, building them. Uh, sustainability doesn't seem to be always be the first thing we think of.

Yeah. You know, this, this huge amount of AI that's being installed out there in these, in high performance computing is, uh, is using a lot of power. You know, the, the things that the researchers are doing with that, it is pretty incredible.

It's game changing research, it's game changing capabilities, but it is consuming a lot of power. And it's, it's making the data center a really hot commodity right now. And, and I mean, hot in a couple of different ways.

One, it's hard to get into. It's hot. Yeah.

Uh, it is odd, you know, the, uh, I've seen some studies out there that say that, that, um, for every, a piece of data that uses for every sort of wad of power, a regular piece of data uses, once it's interacting with AI and, and models and vectorization of large language models, suddenly you're looking at 10 x power consumption for, with the same piece of data. Yeah. I mean, you know, we're, we're gathering so much data together to, to get the insight outta these, you know, out of this data.

It's, it's truly incredible. Um, we, we couldn't even contemplate doing some of the things we're doing today, you know, even just five years ago. So, you know, while, while we sometimes give AI a hard time for the amount of power it's consuming, the things we're able to do now with it, and the insight we're able to gather is stuff that we just weren't even physically capable of doing just a few years ago.

It does happen to, at this point in time, to be pretty power intensive. We're kind of brute forcing, you know, getting answers outta the ai. It'll get smarter, it'll get, we'll get more finessed to it, and it'll get, uh, smarter over time.

But, you know, right now it's, it's pretty, a pretty heavy power lift to, to do these tasks. Yeah. I've been thinking lately a lot about just the actual physical size of the semiconductors we're talking about when, when Jensen, uh, young, uh, comes out and, and shows us on stage a chip that looks like a, like a tortilla.

You're just, you're in a different world. Yeah. It's, it's really amazing.

You know, what's even more amazing is when I look at, you know, I, I own two parts of the business for Lenovo. I own high performance computing and ai. The two are pretty similar.

Um, but what used to take us hundreds of server racks, I can now do it a single compute rack, uh, thanks to technologies like what NVIDIA's giving us to work with. Uh, so you're packaging a great deal of technology in a very, very small amount of space. Um, it's generating a lot of, or it's using a lot of power and it's generating a lot of heat.

Um, and that to us, that's the bigger problem for the data center is, you know, how do you package all this stuff together, keep it nice and tight and concise, and then then deal with the heat that's, that's coming off of these systems. That's the biggest challenge these days for the, the data center operator. Well, and I think that there's a parallel between, you know, how we use ai, like AI might, uh, uh, get rid of some of our, our annoying tasks or make things go faster.

It doesn't mean we're gonna leave work at at noon. It means we're gonna get more done and work just as many hours, if not more. And the data center quite the same, right?

It doesn't mean, yeah. If you're using one 10th of the, the size of a rack to get a task done, it means you're gonna fill the rest of the rack and get 10 times more tasks done. Exactly.

Yeah. That it's a, it's a never ending consumption of IT capability. Um, this is not a question of like, you know, I've gotta run a certain amount of workload, what it do I need for it?

It's almost like, what does my budget allow us to get? And then I'll figure out the, the maximum amount of research I can do with that, the maximum over models I can create simultaneously. What have you, um, you know, see on your point on whether AI is gonna allow us to, you know, to take off at 12 o'clock every day.

Um, you know, a lot of, a lot of people talking about whether AI is gonna replace people, replace jobs. Our firm opinion is, um, AI's not gonna replace jobs. In fact, we'll likely create a lot more high skilled jobs what we have today.

Uh, but one thing we're also confident of is that a person that's applying AI to their role might replace somebody that's not applying AI to their role. So again, a doctor applying AI is more powerful than a doctor not applying ai. Same with a civil engineer, an architect, whatever.

Uh, it's, it's, it's, again, the application of that technology is gonna make us better at everything that we do. That that's what's, so I would like to Yeah, I'd like to believe for obvious selfish reasons that people who know how to ask the right questions will be more valuable Exactly. In a world where we can get those answers.

Yeah. Um, but let's talk, let's get, let's get into the weeds here a little bit about this, this issue of heat and cooling. Maybe, um, it might be illustrative to talk about how what a data center looked like 10 years ago, uh, in terms of cooling and what it's gonna look like, maybe power consumption or what it's gonna look like 10 years from now.

Yeah. Oh man. Massive changes.

So 10 years ago, um, you were mostly installing CPUs, uh, not, not graphics processors, but CPUs. And if you were to really do some, you know, really good work, you could build a rack that might have, might have consumed about 25 kilowatts, and that was doing something that was really pushing the envelope. Most of our enterprise, uh, users were something in the eight to 12 kilowatts per rack, uh, load load on their systems back a decade ago.

And Before we get off that, uh, what in the rack was using most of the power? Yeah, the most of the power was the CPU is probably 60% of the power was the CPU itself. But all the other components, the memory, the, you know, the, the, uh, the networking cards, all those kinds of things, uh, work together to consume all that power.

Um, so, you know, back in the day you were looking at an eight to 12 kilowatt rack on average. Um, today, quite easily, you're 40 to 50 kilowatts in a normal environment. Wow.

Some of these AI racks, uh, later this year, they're gonna be approaching a hundred kilowatts per rack. So we used to, you know, measure data. We still measure data centers in the concept of a megawatt, how many megawatts is your data center?

When you got a hundred kilowatt rack? That megawatt does not go very far getting you like a large number of racks that used to buck that used to get you. Again, it could be as many as a hundred racks in a megawatt.

Now we're talking 10 in racks. 10, yeah. Really amazing.

And so the cooling of those racks when it was, uh, 10 years ago was done. How, and, and I, I'm sure we're gonna get to liquid cooling in a minute, but what we're looking at 10 years ago for cooling Sweats, yeah. 10, 10 years ago was nearly all air cooling.

We, we, Lenovo we're, we're starting to do water cooling for our densest most high performance users, our HPC clients. We were doing water cooling for them, uh, to try to unlock the most performance possible, but the vast majority, 99% of all that it was cooled by air fans inside of servers, moving air outta the server, and then air conditioning and air handlers kind of dealing with that heat once it entered into the data center room itself. So, alright, so let's go in the future here.

When we look, when we look 10 years forward, what do you think we're looking at? Yeah, So today the push, the drive towards liquid cooling is, is, is really been an amazing journey to watch happen. And it's happening all over the world.

One of the thing that people have realized is that movement of air, when you're talking about very high power devices that have to be kept at a very cool temperature, the amount of air that you've gotta move is, is a huge volume of air. And it's actually pretty power intensive to move air fans take a lot of power. Air handlers in the data center take a lot of power.

So, you know, you could be seeing 35 to 40% of your power at a data center level quite easily. Not going to the it, but going to the air conditioning and the air handlers themselves and you got a hundred kilowatt rack, the thought of burning 40 to 50 kilowatts just to do air conditioning and air movement. Man, it, it's just not the, the economics are not gonna work for that.

That's the push towards liquid cooling, which, which allows us to do that much, much, uh, more efficiently. But there's a, there's an environmental impact there too, if you're using water or, we'll, we will get beyond that, but just, just the use of the water. Talk to me about how that, uh, is unfolding and how the technology's evolving.

Yeah. So the way that we, um, we are, what we're doing at Lenovo is we're actually bringing liquid directly to the components. So we're, we're, uh, putting a, a manifold on the rack and we're putting a basically pipes through the systems themselves, and we're bringing liquid right over the top of CPU, the memory, the networking, you know, the, the, the SSD driver, the NVME device, we're pulling that heat away from the device directly and putting it into the water loop.

So our designs, they really don't need any fans. All the heat is being transferred into that water loop itself. And instead of having a loud system with all these fans blowing air around, what we've got is a very small flow of liquid being pumped through the server.

It's very, very quiet and all that heat is being taken away. Our goal is to, to achieve as close to a hundred percent, uh, transfer of heat into that liquid loop. So the data center has, has no need for any kind of specialized air conditioning.

It's gonna save on power costs. Can we try to describe Yeah, well, with, with, with the absence of animations and graphics, uh, let's try to, to describe the loop. Describe to how, yeah, what is the loop, what's the loop where, Go think of it like this.

We, we use pure water. So inside of that loop is water. If I ever have a spill, I can mop it up.

We treat water the same all over the world. Um, you may not be able to drink water in every country, but if you spill it, you can mop it up and put it in the trash. Uh, so we recycle that loop over and over again, and what's gonna happen is is we're gonna have a, a small device called A CDU, which is a coolant distribution unit.

That distribution unit is gonna pump the liquid through our servers, taking the heat away as it goes. Um, it's a, it's a small number of liters per minute per device, uh, but it's enough to, to pull all the heat away from that server. Once we get the water, the heat into that water loop, then we've gotta do something with it.

Um, a lot of times the data centers just send it up to the roof, they send it through a dry cooler, they take about five to 10 degrees of heat out of it, and they send it back through again. So we're recycling that same loop of water all, you know, for months and months and months without it having to be changed. Some of our more progressive users are looking at ways to take the heat that has been transferred into that water loop and make use of it.

There's actually a lot of stored energy in that water loop that we can unlock. And some of our really forward thinking clients are trying to find ways to do that now for heating buildings, supplementing hot water, um, you know, running Yeah. Phy physics reactions to create cold water outta hot water.

So it's, it's pretty amazing stuff. It's super interesting, uh, preparing for an interview. I, uh, I, I'll tell you about my process.

I used AI and I went to, uh, I think Chad GPT or Claude or something and asked it to co co to find metaphors for the use of, of liquid cooling in a data center. And the metaphor that I found, the, with the help of ai, interestingly, will probably burn more heat than this conversation good work. But the idea was, uh, the metaphor was, uh, you're on a, you got a stove and you've got this really hot flame in the stove, so you put a pot of water on it, and if it gets so hot, the water starts to boil it, it hits the steam reactor on the top, which takes the heat out of the water, which allows the water to cool.

But you're sort of twice removed from that, that hot flame on the stove. Yeah, that's a, that's interesting. I've not really thought about it like that, but again, if you, if you look at how well, uh, water or liquid transfers heat versus how, how air does it, um, water's like 5,000 times better at transferring heat.

Uh, to move a small amount of heat with air, you need a pretty big volume of air to get that heat away from the device. With liquid, very, very small amounts of liquid and very small amounts of movement allow us to get that heat away from the parts. And that is the goal.

Um, you know, all the vendors, Intel, A MD Nvidia, what they want us to do is they want us to package in these devices into the server, build 'em really densely so they don't take up a lot of room. But one of the key things is we've gotta get the heat away from the part, um, before it overheats the part in every wat of energy that that part consumes is gonna end up in that server, um, in the form of a watt of heat. It's the law of conservation of energy that wa of electricity gets converted to a water of heat.

I've gotta move the heat away from the part before it overheats and causes like a thermal damage. And the liquid is just like, it's beautiful at, at, at, at its ability to pull that heat away quickly and efficiently and Using that. Well, it's, it's, it's, it's a complex problem because you don't know where the heat's gonna happen.

It's not the whole semiconductor gets hot. There are little hotspots within the semiconductor as it's doing different types of processing and you don't know where they're gonna be. But it's, it's that that dissipation of the heat is so very important.

Yeah. So, you know, it's, it's, it's interesting. So, you know, in a server we can predict what the high power parts are.

It's, you know, it's the CPU, the GPU, the memory, the networking adapter, uh, things like that. Those are pretty easy for us to predict. Um, what's complicating matters is in addition to the power going up on the device, the devices are getting smaller, smaller than they ever were before.

You know, going from 15 nanometer to 10 nanometers, seven down to four. Uh, so the parts are getting smaller. More heat means we're, we're gonna have to dissipate even more heat in a smaller space ever before.

And that means if you're doing air cooling, that means a lot more heat, a lot more, uh, air movement to get rid of that heat. Whereas with water, I, I just turn up the flow a tiny little bit and I'm able to take care of that heat. So, you know, the problem's getting worse as power goes up.

There's just a little bit of time left. I, I wonder if we can also talk about liquid nitrogen, what we might see in the future beyond water. Oh man.

I hope we don't spread liquid nitrogen. Actually, I hope I retired by the time we see liquid nitrogen. So I hope, I hope, um, you know, there are a lot of different technologies that we're looking at, um, that could take us, you know, beyond what we can do today with liquid, but liquid just as it is single phase liquid, like a water or something like that.

It's got a lot of longevity and it's gonna take us a pretty far distance into the future with today's current technologies. As you go past that, we might be looking at things like multiple phase liquids that once they get in contact with the heat, they change from a liquid to a gas. And that transformation from liquid to a gas actually carries the heat away really, really efficiently.

It's a little bit harder to manage, you know, that transaction, that transition from the gas, the liquid to the gas. But it's incredibly efficient at heat removal. So we may be looking at that sort of stuff in the future, uh, to, So you think we stick where we're, we're still in the world of water for a very long time.

I think there's a lot of customers that are gonna try their best to stay in an aircooled environment and we do all we can to optimize air cooling the systems. Um, more and more customers, they're looking at moving to liquid for the very first time. Um, we like to remind 'em that we've been doing that for over 10 years.

We put our first liquid cooled supercomputer install. We installed it in 2012. It was 9,700 servers back in the day.

Uh, we installed it at LRZ in Munich, Germany. It was the first warm water cooled liquid, uh, liquid cooled supercomputer. And we've been doing it ever since.

Uh, so as customers move to water, we like to remind 'em. We've been doing it a very long time and have a very good handle on what it takes to do it. Right.

Fascinating stuff. Alright, Scott t is the Vice President GM of High Performance Computing and AI infrastructure Solutions at Lenovo. Thank you for your Time.

Hey, thanks. Great being with you today. Great conversation.