Reinventing Semiconductors for the AI Era | The Six Five Summit

Welcome to day two, the six five summit. It's our fifth year, you know, after the AI build out in 2023 of a lot of infrastructure and tools. This year's summit is all about expanding the AI footprint into the data center, into the edge, and delivering real value across enterprises and consumers and customers.

So semiconductors have really been the tip of the spear for the AI discussion, and Daniel and I have joked that semiconductors are eating the world versus software that was the meme last decade. Uh, they really are. And whether it's compute, memory, networking, and storage, AI is pressing the limits of all these chips.

And with that said, it is my distinct pleasure to introduce our semiconductor track opening speaker and friend of the six five Jin man Han, president of Samsung Semiconductor. Welcome back to the six five. Oh, thank you.

Thank you, pat, for having me. I'd like to start by saying that we have to remind ourselves that, uh, about 60 years ago, uh, RA Stanford, uh, scientist, he said that, uh, you know, the impact of technology is overestimated in the short term, right? But underestimated in the long run.

I think we really have to think about that. But whether we are now in a era of overestimating ai, we are not so sure. But in the long run, I'm pretty sure this will be the start of AI era, right?

Yeah. It is amazing how far it's come. And in fact, uh, in the 1960s, the first AI algorithms were created.

Uh, we didn't do much with it, uh, for about 30 to 40 years. And then through advances in semiconductor, uh, technology, uh, programming, expertise, and a whole other confluence, here we are in the world of generative ai. Can you talk about, from your point of view, as literally one of the largest semiconductor companies on the planet?

How is it, how is generative AI redefining your space? I'm not really wanna talk. I don't really wanna talk about the market situation because, you know, AI is one of the very few bright spots that we are seeing.

Huge demand right from the market. But, uh, from the semiconductor point of view, I believe that, uh, this AI will, we define our manufacturing in addition to design processes in a whole different way. I, I believe this is a kind of virtual cycle because we are now providing semiconductors, both memory and logic to our, to the industry.

But this will create new application or use cases for ai. But at the same time, this will help us reoptimize our design process and manufacturing process, which is working kind of a virtual cycle. So it's gonna be very unpredictable what this interaction will bring in the future, but it's gonna be pretty good and pretty exciting.

Yeah, it's interesting. Uh, as an industry analyst, I really have to work hard to call, uh, something, a fad or a trend, and I'll typically go to the end user and the research that we do, uh, to, to, to determine that and then go out through the entire value chain. And it's absolutely a trend.

I mean, I met, uh, with some end users on the enterprise side that we're seeing a 50% increase in efficiency in their creativity process. And we're seeing this in multiple verticals, in multiple spaces. And we haven't even touched the consumer, uh, side yet a little bit.

But on PCs, uh, and smartphones, uh, we believe that the best, uh, is yet to come. So, uh, Samsung as a, as a foundry and a designer and IDM, right? You have to make huge bets years in advance on, on what's gonna accompany what is gonna be driving a demand.

And where, and I'm curious, so you have to be very closely tuned into industry trends. What are some of the industry trends that you see right now changing in the semiconductor market? Okay, so one thing that we are noticing right now is that suddenly the demand for high speed SSD is going up.

It was not like this last year, because last year everyone was talking about compute and the memory. So everybody was wondering, you know, what's, what's happening with, uh, storage, you know, area? And suddenly, all from all this year, we are starting to see huge enterprise, enterprise grade access demand going up.

And this is all because of this high performance GPU and increased number of parameters for AI models. And also, at the same time, you also have to make sure that your trained data has to be, you know, saved every hour. So you really need very fast SSSD technologies.

And this is really interesting. And right now we are seeing which one is making limitation on this whole system from end to end. So sometime you see memory sometimes G-P-U-C-P-U and the storage, it's just just fighting for each other.

So, so that it cannot be a bottleneck for this whole system. That's the thing I'm seeing right now. Yeah.

Yeah. I'm so glad you brought that up. And I like to call that, uh, the infrastructure quadrangle, right?

Right. And I said it in, in, in the runup, which is every big inflection point we've seen really required a balanced approach to the compute, memory, storage, and networking. And if one of those four is not in alignment, uh, you probably won't get the results that, that you want.

And that's where, I mean, everything I've seen, uh, from, I mean, my gosh, all the way to the smartphone, to the A IPC, to what's going on inside of the data center, uh, memory, uh, and storage, is it, it has become a bottleneck and therefore faster and more is gonna be what's gonna happen this year, uh, and and beyond. So I think that should be very good for Samsung Semiconductor. Um, yeah, so No, please go ahead.

Yeah. So, uh, but behind all these, uh, technological requirements, we keep hearing from our customers about power reduction, power efficiency, because, you know, you know, this powerful model requires more demand from GPU and even memory and storage. And this power issue is getting critical.

Oh, yes. Right now, I believe that, uh, in United States, about 4% of electricity consumption is caused by data center. 4% is not big number, but when you think about the concentration of data center in a very specific region is a huge demand for the region.

So power efficient chips requirement is getting higher and higher, although we know that more efficient chips will stimulate more demand. So people might, you know, buy more chips, but eventually we're gonna hit the wall. But at least it's, it can allow us to have more function, uh, using the same power.

So it's good for all of us. And you know what, pat, this week is earth weak. So we have to think about this low power, you know, green energy, you know, issues.

Yeah, I'm glad you brought that up. Uh, sometimes, you know, we throw around performance capacity, and we don't hit on, I mean, literally within a year, uh, we're supposed to double the amount of power draw from, uh, data centers. And a lot of that has to do with accelerators, and that's real CapEx that, that, that people are, are looking at.

And, um, anything that can be done on the foundry side, the memory side, and even the storage side, means that, I like to call it power sloshing. You can provide that power savings either to the compute, the GPU or the processor, uh, or you can just reduce your overall, uh, footprint. And it, it, it, you know, for all those who are listening out there, it is really hard to deliver high performance, increased capacity and lower power all at the same time.

But welcome to, uh, gen mom's world here. So, no, it's, it, it's a huge challenge, Right? It's a huge challenge.

Uh, we try to produce, develop, uh, power efficient chips. That's, that's, you know, everybody knows about that, that's for sure. But also, I, I believe that we also have to think about new process technologies, which can endure high temperature in the harsh work in the workload environments.

You, otherwise, we have to invest too much CapEx on cooling system. I know a lot of companies are, uh, working on, you know, next generation cooling system, liquid cooling, but, uh, the semiconductor is not conducive to heat. So we also have to think about how to, uh, develop new process so that our transit can be more resistant to any, uh, heat, uh, caused by this, uh, demanding workload.

And also we gonna develop, uh, power efficient architecture so that we can reduce, uh, power consumption in the first place. So that two track approach might be needed in the future from semiconductor industry, including, Yeah. One of the, one of the key trends too is supply chain, uh, diversification.

I'm, I'm curious what your thoughts are about, I mean, I, I, I, I'm here in Austin, Texas and Taylor, Texas, you're making a multi tens of billions. I think the overall commitment was a hundred billion. The recent commitment was 40 billion.

So you are definitely one who's diversifying, uh, your manufacturing, uh, in, in multiple parts of the globe. Can can you talk a little bit about that? Oh, yeah.

We had a very nice ceremony last week, uh, acknowledging the ACK funding from US government is, uh, most expensive money. I talked before, uh, because us just us taxpayer's money, right? We have to spend very wisely.

And, um, so our main goal is to have a, a big presence in United States so that we can not only produce chips, but also we wanna, uh, do, uh, r and d as well as advanced packaging. And eventually we really want to promote, uh, raising this workforce in semiconductor ecosystem. So that's, that's a huge, uh, goal for us.

But, uh, the last week was the first, first step. We got the funding from US government. We really appreciate that.

And, uh, you said, you know, we really have to also think about not only geopolitical in diversification, but we also have to think about geographical diversification, right? Right. So every, all different things we didn't expect before it's happening.

It's like blocks swan event. And we believe that something has to be very resilient in terms of supply chain by having multiple locations as a provider for semiconductor, you know, including Korea and us, of course. Um, also, we really want to, uh, emphasize, uh, you know, not just big companies like Samsung, but we also have to think about collaboration between big players and small players like setup so that we can, you know, right, how many recognize value of this data diversification, and especially for this, uh, data center, AI solutions.

I think there are a lot of things we can do together, not just as a supplier or, you know, cus customer, but as a ecosystem builder, I think in that, I believe something can play a significant role in the United States. Yeah. There's one, uh, one thing, digital twin, uh, technology.

I'm curious if that's intersecting, uh, your world. I mean, the sophistication of your manufacturing, the design you have to do, I would expect that, that you must be looking into this. Oh, yeah.

So before, you know, this digital simulation is, is really bread and butter for our development process, right? We have to simulate our chips even though we don't have any process ready. And then we also have to simulate all the devices, how it's gonna work, when it process the chip.

And right now, because of this great technological advancement, we believe we'll be able to simulate the whole chip, a whole fat, I mean, we can simulate the whole fab so we can see what's gonna happen to the process, how we can optimize our, uh, you know, all these, you know, equipment placement or how can you, how many wafers can we produce by changing these processes, right? It's gonna be amazing, right? So, uh, even though we don't have any fat ready in the next two or three years, we might be building our fat in the virtual environment.

We may be able to run the simulations. You see what's gonna happen. It's gonna be amazing, right?

Yeah, it really is. I mean, the, the, the tools particularly, I mean for design validation are, are, are, are pretty amazing. And interestingly enough, similar to other areas in, in terms of performance improvement, even on, you know, even creativity tools, uh, you know, using Adobe Firefly and, and things like that, or some of the AI that Salesforce has.

So it, it's super impressive. So with all of this new technology coming out, right, some of the new popular things are, are chips mm-Hmm. Which, you know, is revolutionizing, uh, may allowing smaller companies to, to, to innovate and be brought to market, uh, in a big company.

Foundry, custom, HBM, uh, advanced packaging. Uh, these all have the ability to speed up the innovation process. And, and I'm curious, how, how are you pulling, uh, elements like that together to increase time to market and enhance, uh, innovation?

Yes, pat, uh, that's one of the reasons why, uh, our customers are so excited about Samsung's. Uh, so-called One Stop Solution because we have memory, logic, process boundary business, as well as advanced packaging. So we can, we believe we can combine all these technologies in a much more chemical way so that we can provide, you know, holistic solution to our customers, specific to their needs, right?

I think there are not many companies in the world can provide their solution. Yeah. And then this, uh, this ship land approach is very flexible.

It's kind of a mix and match approach, like a Lego block, but I think it just started, I think it just tip of the iceberg. 5 D, 3D, but it's all be all between, uh, you know, memories and logics, you know, small chim lists. But when you really think about it harder, there are a lot of areas that we can put, add more dimensions.

Maybe we can add, uh, IP children, maybe we can have, uh, 3D interconnections, right? So there are a lot of room for us to grow into in the future. So this is just the starting, and I believe, you know, uh, there are a lot of, uh, players in this ecosystem, but Samsung is in unique position to provide a very specific tailored bespoke approach to each individual customers.

Yeah. And I really like that you're an IDM and you're also a foundry as well, right? So you have experience on what the end product would look like.

And if AI has shown us, uh, uh, anything, it, it's that, uh, a very tight connection, tighter connections between the compute, other types of ip, right? Uh, high bandwidth memory, and then all of the solution put together in advanced, uh, packaging is, is really a requirement, right? That right.

You know, with Moore's Law slowing down, we, we had to find ways, uh, to, to deliver the end performance exactly. That we needed. And it's so funny, uh, when I worked for a semiconductor manufacturer, uh, years ago, uh, packaging was signed kind of a sidelight, and now, like it's a, it's a first order citizen along there with the design.

I think that's super, uh, exciting. And also an opportunity for, uh, Samsung Semiconductor, Right? We are very excited to, uh, have this advanced packaging business unit.

We just created a team last year, and then it's, it's, it's getting bigger and bigger, but, uh, there are lots of, lots of different requirements from our customers. So, uh, I think our engineers should work harder to meet all those demand, but it's a, it's a, you know, area that we can, that can lift us out of this end of the most law, you know, wall. I think, uh, we'll see a lot of great, uh, breakthroughs in this area.

Yeah, I'm super excited, uh, on all this. I mean, chips to me, I mean, I, I'd love to see small companies who get paired up with foundries, like Samsung Semiconductor capability where they can have a design, have it qualified and ready for the final product, uh, to be, to, to be brought, uh, together. And by the way, these new packages blow blow my mind because typically packages added power and it decreased performance in, in connecting chips.

And now we have Chip, we actually have an architecture with chips that through packaging, we keep the performance and we keep the power low. So yeah, packaging is like this first, uh, order, uh, first order citizens, so, right. We've talked a lot about, uh, you know, your thoughts on the industry today, kind of how we got here with ai, some of the unique parts that Samsung Semiconductor brings to the table.

But let's talk about, uh, the future. What types of technology and markets do you see driving the future? I mean, you have to make $50 billion commitments upfront and beyond.

Be before you even know what the market is, three to four years in advance of the market. E even being there, it still blows my mind, even, even though I work for a semiconductor company for over a decade, Pat, uh, you know, I do pre predictions a lot, you know, as other people do. But problem is accuracy.

And, you know, you know, even two years ago we didn't expect this open AI champion thing to, you know, Right. Be like this. But in three years we may see great things.

I heard that there are lots of, you know, disruptive AI models being developed at the lab. I don't know what that really are, but, uh, I'm very, uh, excited, excited to see what those models will be. And, um, so in semiconductor, it's fairly comfortable for us to predict, predict one or two quarters ahead because you, you have to put your wave in the fast so you know what's gonna happen after two quarters.

It's very hard to predict. But we believe that in the future, this, uh, quantum computing might be one of the areas that, uh, will, uh, give us huge growth as well as healthcare as use cases. That's, that's for sure.

I think everybody's agreed upon on that. But I'm also thinking about this robot technologies, right? 7, right?

1 to maintain our population. I don't think this is unique to South Korea. It's not, I think except for some continents like Africa, the population is not growing as fast as before.

So we really need to compensate for that. Like automotive, uh, you know, self-driving car, they're all robots. And with this great vr ar technology and robots, and this ai, these are all dots.

When you connect all together, we may see great, uh, way of compensating for this, uh, population, you know, decrease. Also, at the same time, this will allow us to have higher productivity, right? To make this economy going forward.

So I don't know what's gonna happen within the next three or four years, but it's gonna be really great Technology. Yeah, it really is. And I'm convinced too that, you know, with all the action that we're seeing, uh, in the data center, and, and, you know, my firm has, has been calling the start of a PC Supercycle, uh, starting, uh, in the middle of this year and really ramping in the first half, 25.

I also believe that, that if we do what we need to do on the smartphone side and with on-device AI, with, by the way, more memory, more storage, more NPU compute, we're gonna be driving a lot of cool, um, use cases, uh, to drive that. And one thing we haven't talked about yet, that I'm excited about is, uh, and you hit it with robotics, is generative AI in the industrial iot. I think what we found is that, uh, if, um, you process data on the edge, let's say it's smart factory, smart transportation, uh, it makes the investment of replacing what's on the edge, uh, a lot more meaningful.

'cause it basically, uh, pays for itself. And I know people aren't talking a lot about that now, but watch this space as AI goes from the data center to PC to smartphones, and then to the, the industrial. It land like places, like you said, uh, in robotics, Right?

Exactly. And behind all of these great technological advancement, we need semiconductor. Oh, I mean, it starts everything, you know?

Yeah. Uh, there are people that said, uh, software is eating the world. And I always, I always like to say, yeah, but software can't run on air.

And, and, you know, jinman, the last 40 years of technological information, um, semiconductors and hardware has leaped over each other, right? It's been this virtual cycle, which has been, you know, the, the, the hardware and the semiconductors do this big build out. Mm-Hmm.

The developers have a lot more to work out, uh, work with. They drive a ton of software development, and then they're like, I need more. Right?

Right. So it's fun to see, uh, the, uh, the, the overlap here. And Samsung is, is taking a huge role in, in driving this generation of, uh, uh, generative eyes.

So I wanna thank you for coming on, Jinman. I really appreciate that. Thank you for having me.

Always a pleasure. Yeah, absolutely. And so, you, you heard, you heard it here, the kickoff for the six five Summit 2024 semiconductor track.

Appreciate you tuning in today. And hopefully you see the value of memory, uh, and storage in the quadrangle of, of ai, uh, and how foundry and foundry services make this all happen. And essentially delivering the AI economy.

'cause without these semiconductors, it's not gonna happen. So stay tuned for all the day two coverage today, and make sure you check out day one, 'cause you know it's uns simulcast. And of course, check in for day three.

Thanks for checking in. Appreciate that.