The Driving Force Behind AI: Accelerated Infrastructure | The Six Five Summit

Hey everyone. Welcome back to the six five Summit. Daniel Newman here.

I'm in beautiful Santa Clara, California, sitting down with Chris Koopman's, COO of Marvell Technologies, a keynote from last year's event, and he's joining us once again, and we're gonna be talking all about what's going on in ai. We're gonna talk about accelerated infrastructure, we're gonna talk about custom silicon and compute, and so much more. Chris, thanks for joining again.

It's great to have you. Absolutely glad to be here, and thanks for having us on again. Yeah.

You know, it's been a year since we sat down at the last summit. I think you and I probably sat down once more on camera, and we've had somewhat regular interactions as this has all been moving, especially around ai. Marvell came out early, um, really put a meaningful number and metric behind the growth of ai.

Um, I was very impressed by the fact that you were one of the first out there to really get that story and, you know, give that clarity to the market. But here we are a year later. Give me a little bit of your perspective on where things are at.

How fast is this AI opportunity moving? Thanks. Yeah, it, it, it's moving incredibly fast.

It's actually, it's, it's the most exciting time in my career to be in semiconductors. Um, the, the rate at which things are moving, the way you mentioned the, the number that we put out, I think it was about a year ago that we said that we'd do around $400 million in AI related revenue last year, and that would double, or more than double this year to 800 million just a month or so ago. We actually said in our, said publicly that it'd be over a billion and a half dollars this year, so about double of what we thought even just a year ago.

Um, so it's moving incredibly fast and that really goes for, um, pretty much everything, including the connectivity as well as the custom silicon that I'm sure we'll talk some more about. Yeah, absolutely. And by the way, it's never bad when you double your double, uh, so you should be really pleased with that.

I mean, what we found early on, Chris, was there was sort of this bifurcation of companies that had a very clear understanding of what the AI opportunity would be, and then we had companies that were kind of trying to AI wash a lot of things, right? Um, so the ones that got the were in the first bucket did a lot better. And I think Marvell definitely fit into that category.

But then the second part was we were all a little bit in this space of trying to guess how quick it could move, right? And what you're telling me from that is it was moving really fast, but even based on your estimates, you didn't fully appreciate, no one did just how fast this was actually gonna go. So congratulations, because that's, that's a great result.

No one ever is like sad, like, hey, we have to double that number again. But, you know, one of the interesting things I've always found about Marvell is, is you play in a lot of different spaces. And when it comes to ai, there's two that I really like to zero in on with you, you know, the first is accelerated infrastructure, and we can come back to some of the custom compute opportunities, but they have numbers like 2025, I've heard as high as 30% depending on, on, on the data sources that, uh, of the AI opportunity will be related to networking.

Meaning. So, you know, every time we hear about a dollar of custom or of GPU compute, there's gonna be 25, 30 cents of, of networking, huge opportunity. What are you seeing there?

How is that, how is that materializing and growing and, uh, in terms of Marvell, what are you kind of hearing on the street Sure. In that space? Yeah, it was great.

Great point. So, so first of all, I would say that, uh, Marvell, Marvell got here on purpose, right? Not, not by accident.

We didn't happen into this position. We set our strategy in 2017, um, to, for data infrastructure. And so what we focused on is the data and where's the data going.

And of course, AI is the most data hungry application in the world has ever seen. And so our data infrastructure strategy is what led us towards ai. And you, you asked about networking specifically.

If you, if you think about the way data centers operate, it got pretty boring there for a couple of decades of standard general purpose computing, where effectively a processor or a server got powerful enough to process pretty much any workload we could come up with. In fact, it got so powerful that we had to virtualize the processors and, you know, share out fractions of them in order to be able to make it efficient. AI turned that on its head.

AI takes hundreds or thousands of accelerated processors in order to be able to to, to perform. And what that means is that they need to be connected together because they're running one computation on a whole host of separate processors, and they need to communicate with one another during that calculation, which means that the, the network or the connectivity amongst those XPOs is as important to the computation as the processors themselves. And so it's just exploded.

I mean, and of course it's correlated, by the way. I mean, you said, could we guess? Well, no, it was hard to guess because the number of, you know, GPUs in this case that we're shipping every quarter is sort of continuing to upside.

And ultimately our, all of our connectivity, marvel's by far the leader in connectivity for all of these, uh, models, uh, our connectivity is correlated to that. And so it's been growing off the charts And, and we're in a kind of what I would call like a great reset for the whole technology market, Chris, because I mean, look, you know, with compute kind of went generation to generation, it was all sort of iterative, right? Like a lot of the same architecture work for network, for storage, for compute AI has created just this whole new mass scale opportunity.

Every company, whether you're an ISV, an infrastructure provider, an OEMA chip maker has had the chance to sort of reset their whole business strategy and attack and approach a whole new market. And it's really exciting, but at the same time, it creates a lot of, it creates risk, it creates a new entrance into the space. And of course, it opened up a door, I think for, for you, 'cause you've been in the custom silicon business for some time, you've been building custom compute, but custom AI chips, this is something that everyone out there is asking about right now.

You know, we are hearing, you know, and, and we won't speculate here, or people can do their own reading, but you hear whether it's meta, whether it's Google, whether it's Apple, you know, and so many others, they're either, they're making their own, they're gonna make their own, um, they're doing some, um, you recently put out a filing or you shared in a, in a, in a, in a recent filing, about 25% is an expectation. Talk to me about the XPU space or the custom ai compute space, Chris, and, and how that's progressing here at Marvell. Yeah, and, and by the way, everything you just described, that's what I mean by it's never been more exciting.

It is exciting. It reminds me of sort of the, the late nineties in terms of, um, servers where there was all, you were always looking to see what's gonna come next, right? Or even remember when you wanted a new pc, every time a new PC came out, you couldn't wait to get your hands on it 'cause it would be faster and so much better.

And then at some point you couldn't tell the difference between one and the other, and it's all slowed down and you didn't care anymore. That's where we are right now with ai and you're constantly waiting for what's the next thing. And it's, and, and, and the reason is because the applications we're trying to run run that much better every time something new comes out.

Um, and that's why you see so much differentiation. That's why you see so many companies building their own now because it's not sort of one size fits all. Ultimately, there are so many different types of applications, so many different types of specialization, whether you're trying to train on consumer related data sets or whether it's enterprise related data sets.

There's totally different types of information out there. And so, yeah, I mean, what you're seeing is anybody with massive workloads, um, and specifically it, it really comes down to the hyperscale data center operators. By the way.

It's not so easy to just build a data center, right? You have to have the space. You have to have the power.

Yeah. I mean, these hyperscale data center operators are contracting power years in the future. So even if you suddenly found yourself with tons of GPUs, where are you gonna put 'em?

Right? Yeah. So ultimately we see the hyperscalers as all wanting to build their own custom, uh, ac ai accelerators.

Um, and they're doing that not to replace what they have in terms of GPUs, but to augment and to add and create new value for new types of applications that they can operate even better. Yeah. Well, there's, there's always efficiencies in, in, in design.

And I'm glad you, you talked about power, uh, let's just use that as a for instance, right? I mean, GPUs are really good at, at doing AI and, and giving a lot of flexibility. We've also seen that there's some real value in creating custom chips that fo you know, asic you know, and, and, and, and maybe they're asic maybe there's an ASIC with some programmable logic.

There's variance of how this will end up, right? Being delivered to market, Chris. But in the end, I mean, just the power use alone, like if you can do it and it's a say it's a recommendation engine and you're a company that does something that requires it using that particular architecture and type of silicon you can get, you can get more performance, lower power use.

8% of Northern Virginia's power grid is committed. Okay. Basically they can't stand up one more rack of GPUs.

So, you know, I have to imagine part of your vision of Marvell is that part of the opportunity is that finding that very specific compute so that you can help, right? With the sustainability issue. I know it was a little, it was really hot to talk about, then it got a little less hot to talk about.

And now I think we've come full circle because AI is the hottest thing on the planet right now, and it's also the most power consuming thing on the planet, and we're gonna have to address that mix. Yeah, I think that's right. And I think that is one of the drivers, you know, I mean, generally speaking, um, when you talk about building any type of these AI accelerated compute, the key metrics are performance per watt Yeah.

Is number one. And performance per dollar is probably number two because watts drive dollars as well. So performance per watt is one of the more important metrics that you can possibly have.

And so being able to build an optimized piece of silicon to solve specific use cases is critical, I think, going forward. Yeah. And, and, and that's in your number, so where you said 25%, and I think, you know, we've talked about this, but I think speculation is it could be pretty substantially above 25%.

And by the way, this, like, I wanna reiterate this isn't to say that the GPUs that we're hearing about and all the exuberance is around right now isn't real. It is real. The point is that $400 billion tam that we've talked about with, with, with infrastructure, with accelerated infrastructure, with compute and GPS together could, could go well above a trillion.

That's right. There's opportunity for Everyone. Yeah.

I mean, so just to give you the numbers, you know, at our recent analyst day, we, we shared an analyst estimates. So, um, um, not our estimates per se, but analyst estimates showing that they thought the data center compute would grow to about $200 billion going forward in the next four or five years annually. Yeah.

Now some people have said it's 400 billion, so That's the range I've seen. It could be, you know, and, and it's gonna keep growing. Yeah.

So whatever the number is, so what we said was that about a quarter of that was gonna go custom. So somewhere in the 40 to $50 billion range, we was gonna go custom. You're right.

I think if, if everybody's ambitions came true, it would be higher than that. And right now it's probably in the sort of 15% range last year. Yeah.

So I mean, 25% is actually not a big jump. I mean, the ambitions would be to go even larger than that. But, you know, ultimately the bottom line is whatever the numbers are, they're huge.

Yeah. The demand is huge. And ultimately there's really very few companies in the world that can partner with these hyperscale data center operators to help them realize their dreams and build, build these types of complex custom silicon.

Well, and Chris, that's a great segue because you know, our audience out there here at the six five summit, you know, we focus on trying to really tie together, uh, these transformational technologies. And basically Marvell is part of this story. Marvell is one of these companies.

Talk a little bit about sort of what enables Marvell and what is the differentiation, because you're not the only company, right? There's companies in all parts of the world, and there's other companies here in the US that do custom chips that do accelerated infrastructure. But why are you winning?

Why are you growing? Why are you doubling your double? Sure.

Um, so there's a couple of things. I think first piece is that, um, high speed connectivity is very hard. Um, it's, it's not the same thing as digital logic.

It's it's analog mix signal technology, and the number of analog mix signal engineers that can do this type of high speed networking in the world is very low. And they tend to work at a couple of companies. And Marvell has been fortunate enough, both organically as well as through our acquisition of nfi, who was the leader in high speed connectivity inside the, um, hyperscale data center operators, um, really built, uh, a team that is second to none in the world.

And ultimately in order to keep up with this, you know, you said doubling your double, um, as the opportunity is growing, it's also expanding. You can't just build one chip. So what we're specifically talking about is the, the digital signal processor or the DSP that goes inside these optical cables that connect all of these, uh, servers together.

And you can't just build one. There used to be that you might be able to just build one and, you know, double the speed every few years. Now we're having to put out a host of different chips and different solutions for every single niche in the marketplace.

Every single customer, every single speed, every single distance and optimized link because it's grown so fast. And so we've greatly increased the number of engineers working on these projects. And ultimately the number of chips we're putting out and maintaining that market share is, is critical to us.

And we've been able, we've been able to continue to do that. Yeah. Congratulations on all the success.

Uh, if I, if I would love to end on something just to get your sort of big picture viewpoint. Sure. Chris, someone that's leading one of the more important companies in the silicon space and the AI space, and it's working very closely with some of the companies that all the consumers out there, you know, you're helping to enable some of these technologies for them.

There's a bit of a debate right now and, and I'll call it the iPhone android debate about a, about ai. And you're kind of in the middle of this because you build all the infrastructure, you build custom chips. But we've also talked about kind of GPUs and we've talked about kind of, you know, analog digital networking, you know, where do you think it lands?

Where do you think in the end, I mean, is there, you know, when I say the, the Android is sort of the mixed ecosystem, you know, we're gonna use different GPUs, different CPUs, different network providers, we're gonna use different open source software, and then there's, you know, kind of like the, we have everything top to bottom company that kinda looks more like Apple. Like how does this evolve? Do you think it, it ends up being just a massive trillion dollar opportunity, it ends up splitting a little bit like Apple and Android, or kind of, how do you see this growing with the, the community in the community of ai?

Yeah, it's a, it's a great question. So I think, I think first of all, there's sort of different segments, right? There's the hyperscale data center operators who have massive software teams, silicon teams and capabilities and, and their own massive internal workloads to be able to drive the need for custom silicon.

And then there's a longer tail of enterprises, for example, that just wants something to work. And so I do think that ultimately both will be winners. I think that ultimately, um, you know, the integrated solution that just is turnkey and you turn it on and it works and you can just train your models and you don't have to go and become experts on programming things all the way down to the hardware level, great.

They're gonna do great. I think ultimately, um, the cloud models are gonna get better. Um, I think that, you know, the one, one of the ways to think about it now is, is that this is really a cloud first world.

Yeah. You know, for the longest time we were talking now about moving workloads from on-prem into the cloud. That's been the cloud model for the past decade or so.

This is a cloud first. If you're about to stand something up in ai, what's the first thing you're gonna do? Buy a data center and a bunch of servers?

Probably not, right? Just Few companies out there, right? Yeah.

I mean, of course some will. Yeah, there are some that will do that. But most are gonna say, just put it in the cloud.

And ultimately I think they're gonna have great success as well. Um, and that's one of the reasons why you're seeing the need for customs is that they want to differentiate. If you're simply deploying the same hardware as an enterprise could buy themselves, or as all of your competitors in the cloud world, then your ability to differentiate is fairly small.

But if you can then build your value proposition around that, right? Where you're saying, no, no, my, my offering is better in the following ways, right? It has this type of arm, CPU, it has this type of accelerator with this hitting this particular price and cost, uh, TCO envelope.

Um, and that'll provide a much bigger opportunity for them to differentiate amongst themselves. And so you're this, this, this idea of doing custom that you asked about a moment ago is not because they're just trying to save money, they're differentiating their service for the next wave of innovation in the world. And so it's never been more critical for these data center operators to have the most optimized silicon.

Really appreciate you taking the time and, and, and taking all my questions, including the most challenging ones. Uh, Chris, Chris Koopmans, COO Marvell Technologies. Thanks for joining me again this year at the six five Summit.

Thanks, Dan. Great to see you. Alright, back to you in the studio.