UT08x06: HPC Technology Transfer with Los Alamos National Laboratory

Much of what we take for granted in the IT industry was seeded from HPC in the National Labs. This episode of utilizing tech features Gary Greer, HPC division leader at Los Alamos Labs, discussing leading edge technology with Scott Shaley of solid ime and myself. Welcome to Utilizing Tech, the podcast about emerging technology from Tech Field Day, part of the Future Group.

This season is presented by soy and focuses on new applications like AI and the Edge and other related technologies. I'm your host, Stephen Foskett, organizer of the Tech Field Day event series. Joining me from Soy as my co-host this week is my good old friend, Scott Shaley.

Welcome to the show, Scott. Hey, Steven. Great to be here.

Glad to, uh, continue the wonderful series of conversations we're having with some of these great folks in the industry. So, Yeah, you know, that's been the fun part of this, uh, whole season, is that we've been able to invite in, uh, people from companies that are doing some cool stuff, of course, but also people who are actually pr uh, practitioners, doing very cool things out in the world. And, uh, I appreciate you opening up your Rolodex and inviting on some of your friends.

Absolutely. That's one of the cool things about today, I think is really exciting is when we think about aspects of what's going on in this industry, we don't really realize that how much of the, uh, labs part of it comes into play. 'cause there's a bunch of labs around the country that, you know, you hear about from time to time.

But, uh, in this particular instance, we're bringing along, uh, a very interesting story, uh, from one of those labs. So, Absolutely. So, um, Scott has invited Gary Greer of, uh, the HPC division over at Los Alamos National Labs to join us today to talk about the ways that HPC has driven compute, ai, edge, all sorts of things forward.

Uh, welcome to the show, Gary, tell us a little bit about yourself. What are you doing over there? Well, I'm the division leader for high performance computing at the lab, and I've been at the lab since 89, so I've been there for a while.

Um, our division basically is responsible for all the high performance computing machines and buildings they, they live in and, um, all the way up to the application. The applications are spread across the laboratory, so, um, our, our portfolio is pretty large. We got, you know, many, many tens of megawatts of computing power and, um, lots of machines and networks and storage.

So, And I think that, as I said a second ago, it, it does seem in many ways, at least to me, that the HPC space has driven the state of the art forward in ways that I think people don't quite understand. It's reminds me a little about the space race and they talk about Tang and Velcro and pressurized pens and things like that. The same thing happens in HPC.

In fact, um, you know, your, your phone or your laptop these days has pneumo technology that in my mind came from the HPC world. And the same is true of so many other elements. You're on the inside of that.

Do you see that? Do you see things that are coming and how they're going to affect the rest of computing? Well, we certainly see things we think are coming.

Um, and we, uh, we certainly can look backwards and say that there's an awful lot of what's in computing today, at least scalable computing that came from, from Department of Energy Labs and, and other scientific organizations. Um, and oftentimes we are wrong. Sometimes we think something that's gonna happen, you know, it doesn't, but more usually it happens, it just doesn't happen when we think it's going to.

Um, you know, a good example of that was, you know, back in the early two thousands, we were working on, um, high performance shutters for, for, uh, for optics because you couldn't flash a laser fast enough, so you have to put shutters on it to make it go really fast. And we were funding a lot of work in that area and we thought, oh, everybody's gonna be watching movies at home. And so they're, you know, this, the world is gonna need all this bandwidth and so we need to get high, you know, high bandwidth fiber going and, um, funded a lot of work in the area and we thought it'll be instantly used by everybody that wants to watch a movie.

And then it didn't happen for a decade. And then a decade later, everybody is watching movies, but they did it in a completely different way. They decided to cash all this stuff out locally to you and all kinds of fancy things.

And so, um, in some ways some technology that we've been working on, still sitting, waiting. Uh, another example is Microchannel Cooling. 5 D integration going on in this silicon industry, and it's headed towards 3D integration.

And the question is, if you have 3D chips, how do you cool them? And long ago, back in the late nineties, we funded a bunch of work called Microchannel Cooling, which is drilling micro channels through silicon and cooling it, you know, from the inside, which is kind of the only way, once it gets thick enough and that TE technology is still sitting on the shelf waiting for, uh, you know, the economy to catch up to it and, uh, the need for 3D to actually occur. And once the economics, you know, is right, I assume that technology will be used.

So I could go on and on about the stuff we've funded and some stuff has made it and some stuff hasn't. Yeah, I, I find that very interesting, Gary, and I appreciate kind of that history lesson of, of things like that. One of the reasons that I thought would be fun to bring you into this particular conversation was we got engaged through one of the initiatives that you're working on that's active today in some of the space that ties directly into what we're doing.

And it, um, it'd be great to hear a little bit about, uh, EMC three and what, what you're doing there and what it's bringing to the market that we're dealing with today. And you know, I, I've participated in it across a couple of organizations now, so it's kind of cool to see that that's still moving forward and you've got some really unique innovations that are relevant to the AI era right now that have come out of that. Yeah.

Um, that's true. So efficient mission centric computing consortium is what EM C3 stands for. And really it was a way for us to, to sort of, um, pseudo formalize our partnerships with industry and with other, you know, using organizations, um, for high performance computing and AI oriented technologies.

And, and the, and where it came from really was the fact that we, you know, oftentimes science sites, um, and that doesn't just the national labs, that's also the energy companies and you know, the aerospace companies and people that do science on computers, um, and not just information science. They, um, we often aren't served well because, um, the larger community, the cloud community, the big AI factories and things like that are, you know, the big dogs and they, they sort of decide what technology's going to be for us. But, so it's, it was really sort of banding together a bunch of organizations that their needs aren't completely covered very well by the mass, um, you know, large sales and, and try to present a market to a bunch of sympathetic, uh, industry partners that might partner with us to, you know, make some of that stuff happen.

And if, and a good example of that is sparse memory access. So, um, you know, if you think about how GPUs work, they gulp in a whole bunch of vectors and then they gulp in a whole bunch of other vectors and then they do a mass multiply of all those, you know, all the elements of those vectors in parallel. And, and if your problem maps to that kind of an, I know that kind of a solution, then a GPU is really, really nifty.

Um, however, if your problem doesn't map to that, you're kind of left in the cold because there's not really a lot of good ways to access the memory. And so, um, you know, one of our projects ongoing for the last many years, um, has been, um, to work on trying to push look up tables down to close to the memory subsystem so that, you know, you, you don't waste cash line, you know, bandwidth and you don't wor waste cash area in the process, or, you know, storing indices to look ups and things like that, that you don't want to do. And, and, and it's interesting because, you know, we're not the only people that see this.

A good example of a, um, somebody that sees it is, is Amazon. So Amazon, when they, when you log onto Amazon, you get a bunch of ads in front of your face, right? And the way they decide what ads to put in front of your face is they take, you know, all the products that they sell and all the stuff that you've bought, and they put it on two dimensions of a graph and it ends up being a super sparse data structure.

And they want to compare that sparse data structure with other people that have similar sparse data structures. And so that whole thing is really a sparse workload. And another really simple example is just a, a table join.

So you take Oracle and they have a, a dense table and a sparse table, and they want to join it together on a common key. And the way you do that is through a lookup table. And so there's all kinds of examples of sparse access to memory in the world, and it's not being served very well by industry today.

And so we've been working on trying to make that happen and probably the other big project that's going on that we realize needs to, you know, have some work and that's pushing, you know, processing down near storage. So just like sparse access is pushing access down close to memory, pushing stuff down close to storage is also important. And, you know, the naysayers out there might say, well, that's been tried a few times and it's certainly true.

I've certainly tried it a few times and myself and funded a lot of work in the area. But the where we really see it coming to head, you know, in, in real need is, um, if you have a lot of, of holdings, you know, at my site, maybe we have three or four or five exabytes of data that we hold, um, and it maybe represents, I don't know, maybe a trillion files and a hundred billion directories or something. So it's a, you know, large mass of data and it's, you know, text and images and output from simulations and all kinds of things.

And you, you couldn't ever train on on four exabytes of data. You know, today people train on petabytes of data, you know, so it's three overs of magnitude off. So how are you ever going to do training or better yet, how are you ever gonna do inference against all of that data or majors of parts of it?

And so the only way is to is to have, you know, rich indexes and be able to subset that data very quickly so that you can cap, you know, just get the vectors that you need and, you know, build yourself a training set or build yourself more likely a rag that, um, your, you know, your, your model looks at to, to enhance it, uh, you know, answering capabilities. And, and so if, if we get to the point to where you're ans you're asking questions of something that large, then you need to push the, the lookups, the similarity, you know, lookups as close to the devices as possible so that you're not moving data back and forth. 'cause that's the only way you'll get low latency to get an answer to, you know, to a question you're asking, you know, of your, your holdings.

And so we see a time when this is gonna really actually be necessary, otherwise you won't be able to accomplish what you want to do. So that's really sort of the long game. There's certainly a lot of shorter term wins you get out of it, but longer term, if you don't have a way to get something from all your data, then why the hell are you keeping it?

And and finally there's mechanisms for doing that. So we need to enable that and pushing the, you know, the, the, the compute near the storage, at least some parts of it reductions at least, um, may be necessary. I mean, that, that's exactly, I mean, it's a great example of how being able to think outside of the market box, right?

So one, the one beautiful thing about the work that you're doing and and enabling through the work that you're doing is it's a, it's somewhat ignoring the, the big guys, right? Because they do have their, their bowl and their whip and their, you know, whatever they're doing to make us, you know, carrot and stick type of stuff. But it's really interesting to see that there can be a lot that can be accomplished if you take a side and look at it from a truly what does it need versus what does this person think they want.

And that's one reason I really like the, the ability to be engaged with the, the work that you're doing there and things like that. So, I mean, it, it's really fun to think about, um, what's been going on at that lab for so long. Like you said, you've been there, uh, quite a number of years, which is awesome and we appreciate all that, uh, work that you're doing there.

But I recently went to, uh, the chip summit in January where it was mentioned that they're now in the process of trying to migrate all of the cool nuclear blast data that was generated at the lab into a form of consumption to do exactly what you wanted to do, which is take petabytes of information you can't recreate and be able to train and use that for useful, useful future data. Yeah. And it's not just, uh, it's not just the, you know, the nuclear test data, but it's all the subrate tests and uh, you know, there's tons and surveillance on the weapons over the years, right?

We tear 'em apart and we try to figure out what's wrong with 'em. And um, you know, they're old, they've been rolled around on trucks, they've been rattling around in subs for decades, right? So, um, yeah, there's a ton.

There's a ton of data and of all kinds. Yeah. So one thing Steven, you may not know about, uh, Gary as I've talked with him over the years is I have been at GCC and I actually met a few of, uh, Gary's uh, coworkers there because he has done a great job of launching a few people out of the lab into the industry because of the hard work and effort that they put in there.

So not only is he helping drive the market, he's actually helping the careers of quite a few individuals as well. Yeah, we have an enormous student program, it's really cool. We have 1200 summer students a year at the lab, um, which is a pretty large program, 300 postdocs, and about three fourths of the laboratory staff comes from its student programs.

And of course, we don't hang on to all those students and or people. So they, they disappear into the, you know, the ether and come back eventually. And, um, we notice that there's some, at some company helping us, uh, vicariously It, it is interesting, isn't it, that, uh, so much of what Gary's talking about, both in terms of technology, but also as you said in terms of skills is, um, increasingly applicable outside the lab and HPC environment because, you know, I guess we're all probably old enough to remember when terabytes was a lot of data and now, you know, you've got a terabyte size SD card, you know, when, um, if, if Scott, if you had said, I've, I'm gonna be, you know, handing you a 100 plus terabyte SSD even just 10 years ago, I would've laughed at you because that would've been, seemed like a ludicrous amount of data.

And yet that's a real con, you know, not consumer, but a real product, a real thing that you can buy. And the same is true of some of these other things. Gary, you know, it's, it, it strikes me that, you know, as you were alluding to in your conversation about, for example, the sparse memory and computational storage, the scale of that data sounds absolutely incredible.

But when we're looking at what's happening in the AI space today, we are seeing people saying, Hey, how can we, uh, train a model with, with maybe not quite that large a data set, but a very large data set. And we're gonna have to get to that point probably eventually. And, and as you said as well, it's on the inferencing side as well.

You know, if we want AI to be able to process, uh, all the whatever data, then we need to be able to build systems that will be able to scale and allow that AI to efficiently access that kind of data. So what you're doing really is what's going to be happening probably five years from now, right? Would you, would you agree to that?

I think so, yeah. I mean, we aren't used to working only five years ahead, that, that's sort of the last decade in my career kind of a thing. It used to be feel like more like 10 or 15 years ahead and then now it feels more like five or or less.

It is kind of hilarious when I go to, to AI conferences and they talk about the woes of having to checkpoint their, you know, the state of their, their training job. 'cause it runs for a long, long time. And I have to laugh because we had to do checkpointing 25 years ago and have been checkpointing ever since.

And, um, and actually still at scales bigger than they are, um, from a synchronous point of view. And so, um, it's kind of interesting how a lot of the things we've, we've done, they're using or had, you know, have to use all the interconnects that, you know, the fin band and, and ultra ethernet and all that kind of stuff has its heritage and parallel computing, you know, and the early two thousands, um, and in fact we, we were heavily involved in the invention of, in Infin band, it was a, a consortia between a couple of DOE labs, um, bank of America and Oracle. Isn't that an odd set of people to work together?

But, um, you know, Oracle, Oracle was trying to do this thing called a parallel database, you know, and at the time, and, and Bank of America, well, they were trying to buy a parallel database at the time, and we all said, well, we need this interconnect. And in the nineties we had funded a bunch of proprietary interconnects for every vendor known to mankind, and everybody was tired of having proprietary interconnects and special APIs to access them for all the functions they had. And so Ban was sort of our attempt at trying to get a common, you know, high performance interconnect that had a common, you know, know now it's called ED layer that lives in Linux that you, you know, use to, to do this large scale computing.

And so, yeah, many, many of the things that we did in the past are, you know, come around again and are used by others. And what's really nifty right now is in the data space that's happening. So, you know, tools like Luster, which I went to DOE to get the money to build and PNFS, which we started in 2002 at the University of Michigan.

And there's all these things that we did for par, you know, for parallel storage. And it's really cool to see, uh, you know, something other than a HPC site starting to use those, those tools. Yeah, I, I think it's interesting, Gary, to your point, 'cause you know, we've, we mentioned that this series is focused on kind of today's AI and you know, where it's going with the edge and things like that.

And we've been talking high performance and you've given us some, some really cool things. I I think it's one of those, there's always the next shiny object. And from, from my point of view, you're working on not even today's shiny object or next shiny object, you're on the the shiny, shiny object, right?

So when we all think we're squirreling to solve today's problems, you've already solved them in ways that you're waiting for the rest of the world to catch up on. So it's kind of a unique ability to have a perspective like that. And I truly appreciate the fact that you're, you and your team and the work that you're doing there has given us as consumers, uh, and or partners in that space, the opportunity to, to work through and catch up, if you will, in some aspects of it.

And it just shows the value of having this kind of forward looking aspect of it. 'cause when you look at a company like Solid, we have an RD budget and we're thinking, you know, we have a three year roadmap, we have a five year roadmap, and we don't tend to think much beyond that. And when something pops up, it's like, oh, but you're like, to your point, you're laughing about Checkpoint.

And it started to make me laugh. It's like I've heard from, I have an older brother who works at a DOE lab as well in Idaho where, you know, nuclear started as far as energy consumption. And I hear stories from him all the time, and it's like, this is kind of cool and innovative way to think about it.

And I really do appreciate getting the chance to have you on here to talk about some of it. So what are your thoughts, Steven? Yeah, I'm with you, man.

Um, it is, it is really cool. And that's why, you know, Gary, I really want to kind of put it to you. Um, what are you working on now that, that, that is in the back of your mind and you're looking at that and you're saying, you know, that right there is gonna be important pretty soon.

What are the, what are the tools, the technologies, the concepts that you think are gonna be driving computing in the future? The only air term, it's, you know, it's much what I've talked about, but, um, thinking, thinking bigger and longer. 5 D is everywhere.

Um, and when you get to 3D you know, once you solve all the problems that we've solved some of already, but not all for sure, um, you're outta Ds. And so, um, what do you do next? Right?

And so what that means, which is an ugly way to think about it, but what it means is you no longer get any of these winds on reductions of size or anything. The only way to get more computing is to buy more computing. And, um, which is a different situation than we've been in, in a really, really long time, right?

We've always been able to shrink and add more. And, and when that stops, when when, when you're at the end of that road, what do you do? And so there's a fair amount of effort going on to try to figure out, okay, well what do you do when, when you run outta Ds and you're done, right?

You don't have any more shrinks you can do because you're at atomic scale and you don't, and you've already used up all three dimensions and all you're doing now is just buying more and more, you know, covering the planet with, you know, 3D silicon, what do you do? And, and so there's a fair amount of effort going on to try to figure out what that is. And, you know, we've, we had an early quantum system at the lab, um, and it was interesting, the infrastructure costs three times as much as the machine, but, um, the, you know, that that's not exactly an answer.

It's part of an answer. Um, there's, there's analog that looks pretty, you know, pretty interesting. Um, and so we have a fair amount of effort going on looking into analog and what would that get for us.

Um, and it's really mostly about, you know, pika Jule per something, right? Because not only will we cover the planet with silicon, but we'll also use up all the energy at the pace we're going at, right? If you run out of ease.

And so, um, somebody has to stem the tide there and it, it feels like analog or biological computing is where we have to go. So we have a fair amount of effort going on in both of those areas. Our, our, uh, center for Nano Technologies is looking at interfaces between silicon and bio, you know, circuitry, um, which, you know, may, may be part of the answer.

Um, so that's the problem I think for all of us, you know, at some point is what happens when we run outta Ds. That that's a very unique perspective. And I, I think it's interesting 'cause you know, there's things like Nia with the DNA efforts that are going there to use DNA to do archival storage and stuff like that, but what do you do to get it back and all kind of thing.

So I really do appreciate that perspective. 'cause uh, I am very curious what the next D is. 'cause I've, I've been on the same train with you about, you know, we don't have anything really in the market to replace NAND and DRAM the way we have used them in the past to replace other products.

So you start getting beyond that end of the quantum space, is that really where you're gonna get solutions, but that only solves one piece of that big problem, right? So really do appreciate those insights. If, if I could jump in on that too, just to be clear for our audience.

So, um, it it, Scott, you, maybe you're taking this, um, I, I don't know, maybe you're ahead of the time, but memory and, and flash, you know, NDA is ahead in my opinion, of compute, of general compute in terms of implementing 3D chip architectures. I mean, you guys, um, especially in the storage space, you're stacking, you're stacking 'em tall. I, I don't know if people know this, but I mean, we're talking about skyscraper style, you know, of chips with, you know, over a hundred levels.

Um, that is not what we're seeing yet in the compute space. Gary, do you, do you think, is that what you're talking about when you're talking about 3D? Do you think that we're gonna have, um, you know, processors scaling like that?

Yes, it's happening. And, and that's a big, you know, so that as you say, that that's something that hasn't yet come to computing, but it is, and it's going to, and, and like I said, I mean, memory and flash are certainly the, the fields that are setting the stage for that. Yeah, it's interesting.

The one, one question I usually get is, what about photonics on chip? And we, we've done work in photonics on chip for, I don't know, 20 years and, and it's, it's ready. It's just that the economy, it's it's economy isn't there yet.

It's just like this microchannel cooling stuff, right? It's technology that's sitting there waiting for a problem that it, it can uniquely solve. And it could be that that's one step between us and full 3D because if you can get the kind of bandwidths that that promises and you can move things apart by inches instead of, you know, millimeters, um, one could imagine spreading a workload out using.

So it could be that finally that technology will actually take off and have an economic reason to exist because it's still too early for microchannel cooling in full 3D but both are probably gonna happen. And, you know, the, the, you know, stock market question of course is when, right. And Well, I mean, this has been truly insightful.

I've, I've actually learned even more just by having a few minutes of chatting time with you. So it looks like next time we, uh, see each other in another event as we continue to cross paths, I'm gonna have to definitely sit down with you for some more conversation and, and things like that. And yeah, Steven, to your point, yeah, when we start stacking the processing chips, when we can get 'em cooled properly and getting, you know, through silicon via versus wire bond, that's one step.

But then we have to get beyond that too. So I'll hand it back over to you, Steven. Yeah, absolutely.

It's, um, you know, because those are some of the challenges just just to, to translate this into, uh, plain nerd from deep nerd, um, yeah, when you start cha stacking, uh, chips on top of each other, you have to figure out ways of powering those chips. You have to figure out ways of, um, you know, of basically distributing power on a bus, kind of, uh, throughout that stack, you have to figure out ways of addressing and, uh, communicating with those. I mean, it literally is, so my background is in urban planning, it literally is the same as making skyscrapers.

You have to think about elevator, um, sky lobbies. You have to think about, um, emergency exits and uh, and water and fire and electricity and all those things. When you make a tall skyscraper, it's the same with chips.

You have to think about how, how am I gonna power the, the ones on top? How am I gonna get the data in and out? And, and for something as uniform as a flash chip, that has been a little, not easy, but a little more, uh, doable.

Uh, and then for compute, it's just wow, uh, let's see what happens there. And um, and then Gary, the other things that you mentioned too, I mean, I've seen some very early, uh, uh, analog, uh, AI processors. For example, I was talking to a company that's doing an analog AI processor.

It mostly kind of works, but it's pretty cool. Um, you know, that's for sure. And if, if they can get it to really work, that would be awesome.

Um, you know, so, so, so much to think about. Gary. Um, is there someplace that people can continue this conversation, can learn more about your work and, and about, uh, EM C3?

Uh, best place probably is to just go to LinkedIn. You can, you can find me on LinkedIn for sure. Excellent.

And, um, thank you so much for joining us. I will say that there is actually on the Los Alamos, uh, uh, website also there is a little bit more about the efficient mission centric computing consortium, if I can read it off properly. Uh, and so people could, could do that.

Is this something as well that, uh, you mentioned students coming in, is this something that people can get involved in? Uh, sure. It's mostly for, you know, other using sites and, um, and industry partners.

But like I said, we have a huge student program, the largest student program of all the weapon, all the DOE labs by factor four. It's really big. And so we hire tons of students every year.

So we really encourage computer science and computer engineering and electrical engineering and mechanical engineering and physics and material science and all that kind of stuff. But we hire an awful lot of students. So, and, and actually just not in Los Alamos, but the DOE labs that the student programs across them, you know, we're talking about tens of thousands of students a year.

So, uh, you know, I I highly encourage people if they wanna work on interesting problems to, to come as students. Yeah, absolutely. I, my my oldest, uh, studied computer science and, uh, some of their, uh, friends actually went to work for the National labs.

Uh, others went to work for fricking Facebook and stuff, but, you know, I mean, you know, to each their own. Um, and, and so it's pretty cool that, uh, people can get involved in some of this cutting edge research. Well, thank you so much for joining us.

Um, Scott, uh, what's going on, uh, with, with you and, and with Solid lately? You know, we're still having fun. com/ai.

Uh, some really cool innovations were introduced at gtc, so go take a look at that if you haven't already. Uh, and we're continuing to just, uh, put things forward, keeping a nice solid focus on that. Uh, me personally, I'm having a lot of fun having these kinds of conversations.

Uh, if you feel like checking, uh, following me around, uh, Scott Shaley on LinkedIn or SM Shaley on both, uh, former, uh, Twitter and now Blue Sky, so Excellent. And as for me, you'll find me at s Foskett on most social medias, including, uh, LinkedIn, as Gary and Scott both said, uh, I'd love to hear from you and, uh, we recently had our AI Infrastructure Field Day event, so if you're interested in, uh, sort of the infrastructure underneath ai, maybe check that out. Thank you for listening to this episode of the Utilizing Tech podcast series.

You can find this podcast in your favorite applications as well as on YouTube. If you enjoyed this discussion, please do give us a rating or a nice review. It really does help people to find us.

This podcast is brought to you by Soy and by Tech Field Day, part of the Futurum Group. com, or find us on x Twitter, blue sky, and mastodon at utilizing Tech. Thanks for listening, and we will see you next week.