AMD: Advancing AI from Cloud to Edge to Endpoints | The Six Five Summit

Welcome back to the six five Summit 2024. We are talking cloud infrastructure. And as you probably know, the theme of this year's summit is all about AI and how enterprises and consumers are taking advantage of that.

And of course, we're still building out the infrastructure, uh, as we, uh, as we move along here. And it is my distinct pleasure to introduce Robert Hormo from a MD Robert. Man, it's great to see you.

What have we known each other for like 15 years or something? At least 15, if not more, Patrick. Yeah, it's great to see you and, uh, great to be on the show.

Yeah, it, it, it's been, uh, it's been great and Robert, like this whole data center AI thing has been absolutely amazing. But, but what's as important is the technology is kind of dialing out on the strategy side, right? Uh, what, um, enterprises are doing and, and why they're doing it.

And so, a pretty broad crest question to start here. Uh, we believe that AI is, is really a journey, not a destination. It's not like, okay, we're here.

Um, and I'm just curious 'cause you talked to a lot of enterprises, where do you see them, uh, positioned along this AI timeline? Yeah, no, it's a great question because I, I, you know, you're absolutely right. AI is a journey, and if we kinda look back at all the historic technology waves from the mainframe through, you know, the pc, all these waves have come and they've all been bigger than all last.

And, you know, there's a, there's a strong belief that this is gonna be the biggest one yet ever. Yeah. Which has always been the case.

But I do think we're like super early on the journey because I'd look at it and go, well, kind of what are we using AI for today? Um, and if you look at it today, um, it's mostly human influence. It's by click watch, ad sell, um, very human influence based.

We're moving into kind of the human q and a, you know, we're starting to move into with office, you know, with co-pilot, we're starting to move into human productivity and with, you know, with some of the video and things like that are being, that we're actually getting into reasonable content creation other than a PowerPoint summary, right? Um, but where we're moving to is, what I like to think of is that the far end of human influence is true human assistance. You know, if you're old enough, like me and I grew up watching the Jetsons and Rosie the Robot, you know, that's where we're going.

Which By the way, I did. I am Gen X Robert, so I'm, uh, I'm, I'm there with you. Yeah.

So I think that, you know, that true human assistance is gonna just be huge. And, and it's hard to say who's gonna be the winners, who are gonna be the losers, but I don't think it's, it's gonna be a very big field. And I think that the enterprises are just, you know, they're looking at today where can they get the best return to impact their business, whether it be, you know, fraud detection in banking or, you know, new user experiences for helping consumers on their platform, you know, with, with enhanced chat bots and things.

So, but I think we're really early on that journey towards that end state of really truly helping humans live better, healthier, you know, longer lives and doing useful stuff for us. So I'm excited. Yeah.

Robert, uh, I remember, you know, I started my career when, uh, there were still many computers, but, you know, we were moving to Unix servers, not Linux servers, but Unix servers and, and, and PCs. And it, it feels, it's interesting. This definitely feels like something is bigger than I, I, dare I say, even, even the internet.

I'm getting that, yeah. That feeling right now and that build out, um, feels like there's even more reality on the benefit behind it, but Mm-Hmm. Um, yeah, so AI definitely continues to evolve in advance.

And I was at a, a bunch of conferences this weekend. Uh, there were some people that are using this whole notion of AI time, right? Like AI time, uh, like pre AI time.

It was a year, right? And, and AI time, it's like a quarter or three months or, or or a month, right? Mm-Hmm.

And it's gotta be hard for, for enterprise it to know, like, how do I, how do I get best positioned to be able to implement, uh, ai? What kind of recommendations do, do you have for them? Yeah, so for, for enterprises, you know, what we've been talking a lot about is for them to get ready.

There's, there's kind of, you know, four or five things, six things they need to really think about in terms of doing and or partnering. And one of the biggest ones right now, um, you know, is if, if you kind of look at the power infrastructure landscape of data centers today, um, we're at a historic low vacancy rate. 7% vacancy rate on data centers, right?

So there is no space at the end. Um, and if you look at the data centers in construction, 80, something like 84, 80 5% of the new construction that hasn't finished is already leased. So, you know, we have this power capacity space problem brewing, um, that enterprises have to really figure out.

And so one of the biggest things that, you know, they can do to make space and create capacity for AI is deeper consolidation. So drive, you know, performance and efficiency with like Epic, you know, you can get five to one or eight to one consolidation of those 5-year-old servers. Um, I kind of did the math and it's, it's estimated, there's about a hundred million 5-year-old servers out there.

And, you know, that is about 43 ish. So gigawatts of power, and if we consolidate it all that down to Epic, you know, that cuts it from that a hundred million to about 21 million servers at about, uh, you know, 14 gigawatts. So there's 30 gigawatts right there that we can go create space power to go do ai.

So I think that's step one. Um, you know, the, the second step well, and, and kind of goes hand in hand, you know, efficiency and performance. You, you can't have efficiency and, and sacrifice performance, so you've gotta do those two together, you know, and for us, part of that journey too with Enterprise is trying to make it really easy, right?

How to make, you know, easy consolidation or easy AI adoption. And so we're addressing that both on the epic side and the instinct side. You know, we put out a tool with VMware to help ease the migration across clusters called Vamp, VMware Architecture Migration Tool.

So we help automate that consolidation. Um, and then, you know, on the instinct side, we're trying to make it really easy to just work out of the box at the AI framework level. So if you're operating at, you know, at the, the framework level, we just work right outta the box, right?

And so, you know, so we're trying to make it easy. And then the, uh, kind of the, the last part of this, uh, you know, we talk about is, you know, picking a partner that's really driving the innovation curve with packaging, um, advanced packaging, advanced semiconductor, and that can execute, right? Um, and, you know, execution has been a hallmark for A and b, the last, you know, five, I mean, we've executed four generations of Epic on time.

The fifth is about out of the house, the next one is, is coming soon. And so I was, I was joking at the, uh, adult tech world, you know, we have a word for that in Texas. Uh, you know, uh, strategy without execution is all hat and o cattle.

So we type in cattle and ad we're, we have a strategy and we're executing to It. So, no, I like that. So, you know, you brought up a really unique point that I don't hear a lot, which was that, hey, we're outta room and out of power consolidate maybe your non AI AI servers to make room for, uh, the ai ai part of it.

I just, I, it's, I'm surprised we don't, we don't hear that a, a, a lot more. And, and Robert, you know, just to put an exclamation point on the power, I mean, in Reston, Virginia, uh, there literally is no more power on the grid, okay? Yeah.

Uh, there's no more power for more EVs, uh, more homes, uh, more data centers. They are tapped out, and it takes, what, five years best case to get a new power station, uh, uh, up and running. So I really appreciated framing that.

And it sounds like, uh, when it comes to, uh, uh, instinct, right? Your play is, uh, the easy button, or was that epic Now? Well, it's actually, we're trying to make it easy for us at a MD we view it as we wanna make it easy for customers to adopt a and b technology, right?

With Epic, it's, you know, in the epic world, how do we make consolidation easy and be easy in the X 86 ecosystem of software and all that? So that, that's great for X 86, but we still have to make the consolidation easy, um, to go drive that goodness. And then on the instinct side, it's about, you know, being a, supporting the open source, um, supporting the, the frameworks out of the box.

It's about having the same libraries at the bottom. Um, you know, it's about having a, a rock 'em, you know, if, if people are operating at the Triton level, um, you know, we have a rock 'em Triton back end. And so that again, where we're just right out of the box.

And so we have to, we have to be easy on both, both sides, um, to do that. No, that's great. So let's talk about differentiation, Robert.

Maybe we, we take, uh, the next few minutes, talk, talk through that. How is a MD distinctive as it relates to AI infrastructure? Your thought?

I think if you, if you kind of look at the journey that we were on with Epic, we tackled some of the biggest challenges in terms of, you know, driving core count up and driving massive, um, differentiation and core count IO count and memory bandwidth. And that's been a really good recipe for Epic to, to drive a, a clear performance and per for watt leadership perspective. And if you think about where we're, where we're going with instinct, we get the advantages of all those years of advanced packaging that we've been doing.

And now we get to do it over on Epic, on, on instinct, sorry. 6 x the ATM bandwidth compared to the, the volume competitor right now. And so what that translates to from A TCO, it's really simple.

You know, if you need eight of theirs to fit a model, right, you can do it in three and a half of hours. I mean, that's, that's pretty easy math to go see the, the memory advantage. So we're gonna keep pushing that memory advantage, that memory bandwidth.

And I'd say the other thing that is part of our strategy on instinct that is, uh, you know, we started a, a while back, which was addressed in the open, A MB is always about open industry, open source, open standards. So we started the alter alter ethernet, um, you know, associate our consortia with a bunch of our friends to really go drive ethernet at scale, have an open choice ecosystem of nicks and switches. And that journey is going great, Patrick.

I mean, the, the, the amount of participation in the industry Yeah. And the leaders in there, um, everybody's in, uh, if you, you remember Jimmy Pike and, uh, you know, this is Oh yeah. Money one of Jimmy isms, you know, never bet against ethernet, Right?

You're not betting against ethernet at all for scale out. Yeah. And I'd say I, um, I don't bet against ethernet either.

Um, there are some technologies and high performance computing that make sense to, to, to connect, but it's, it's amazing every time you count ethernet out, ethernet finds a way of, you know, lowering latency, uh, increasing performance, uh, reducing power, uh, in, in many different ways because it's, it's almost an, I don't wanna call it an open source, uh, uh, technology, but the community comes together to make that happen. And whether it's the hyperscalers or the, uh, big enterprise data centers, either, that makes a lot of sets. Yeah.

And then on the, the other part of the open, you know, so, you know, ultra ethernet great for the scale out networking. Um, if we wind the clock back to December of last year when we announced the 300 x, um, we stood up on stage and said we would open up part of infinity fabric for collaborators and strategic innovators. Yes.

And, you know, I'll, I'll, there'll be more of that to come soon. Um, but I'm happy to say, you know, we're, we're, we are fulfilling that promise to go address, um, to go do that, to help on that scale up, um, to create bigger pods of GPUs, you know, within a pod. And so, you know, we're gonna go address it with ecosystem partners and go move, advance that in the industry.

Well, it's great stuff. And, and by the way, some people might say, well, hey, you know, Robert, what gives you the permission to do this? And it's like, by the way, congratulations.

Uh, you now have eclipsed with Epic, the market share, uh, that when, when I was at a MD with, uh, with Opteron, uh, I think it was 32%. So, uh, congratulations, uh, on that. And quite frankly, your mi uh, product line is on an absolute, uh, rocket ship.

And it was really interesting at a conference that, uh, I was, uh, covering, uh, Satya said that, uh, the MI 300 X was the, uh, best, uh, price performance, uh, GBU instance that, that they offer. And that's quite a, first of all, quite a testimonial. It's not theory, it's, it's an action.

Uh, but also when I look at the starting point of, you know, how long that product line has been out, that's like a rocket ship man. Uh, it's been, it's been a, it's been fun to watch. Yeah.

And, and I think a lot of that goes to the testament of, you know, what Lisa and Mark and Forst put in place in terms of the architectural choices, the advanced packaging, the being on the forefront of chips Yeah. Gives up a lot more late binding decisions and optionality as as we go. And, you know, getting back to, you know, well, what gives us the right, yeah.

Why back at, you know, look at Frontier, right? I mean, Frontier's been the number one supercomputer for, what, three years now? Yeah.

Um, and, uh, you know, the, there's a, there's another one on the top five lists now, but it's, uh, you know, if you've looked at it's, uh, first per watt, it's not very attractive. Um, and you know, we, there was a, you know, in this year is at the ISC, you know, there was an, a small run of the 308, right? Which will be the next number one.

Um, you know, that's the basic com, combining the epic cores and the instinct cores all on package. Um, and a true a PU and a true unified shared memory. And I am really looking forward to seeing how El Capitan, so we have fair amount of experience building really big machines.

Yeah. Solving really complex problems for some very demanding customers, Right? On Both, you know, for the US government.

So, um, so we do have, I think we've earned a right, um, to play in this field given our, what we've been investing in and, and tackling some of the largest problems in the world with like, uh, frontier. To me, every time I see the stats on Frontier, I'm just amazed at the, the power, the density, the cabling, the miles of this and that. It's just like, holy cow, we built that, or We helped.

Um, yeah. Well, Robert, uh, you and your team, you know, a lot of people, uh, just cranking out at, at, at a MD it looks like a, a fun, fun, fun thing to be doing and, and changing the industry. And hey, Robert, I really want to thank you for coming on the show and helping us kick off, uh, this, uh, cloud infrastructure, uh, section here.

I really appreciate that. Yeah, absolutely. Uh, complete.

Uh, it is a pleasure, Patrick. Always a pleasure to talk. Sounds good.

This is Pat Moorhead, uh, and Robert from a MD signing off here, AI infrastructure. Man, it is cool everybody's talking about it. It is a core piece of building out all of these amazing user experiences that wind up, uh, powering and improving enterprises.

And as Robert said, improving people's life on the consumer side. Take care and check out all of the cloud infrastructure highlights, uh, from this track. Take care.