Evolving IT Infrastructure for AI with Lenovo’s Mike Leach

Hello, along the latest edition of the Techstrong AI video series. I'm your host, Mike Vira. Today we're with Mike Leach, who senior manager for workstations and horizontal solutions at Lenovo, and we're talking about the impact AI is gonna have on all those workstations and endpoints and everything else that we use every day.

Hey, Mike, welcome to the show. How you doing, Mike? Yeah, great to be here.

What exactly are we looking for these days in a workstation in the age of ai? Will every workstation need to have some sort of additional class of processors now to run these AI systems? Or is it in specific use cases?

I think a lot of people are, um, confused about exactly what an AI workstation is and by extension what an AI PC is. Yeah. And I think, you know, it's a, it's a, it's a great place to be in the tech industry right now.

Um, you know, AI is gonna be truly transformative. It has been for the last five years, and the trajectory is, and the velocity of what's happening now is, uh, I guess stressful and impressive to see from my side of the defense at the same time. Um, when it comes to ai, it's all about the use case and it's all about the data.

So, you know, your question on what type of compute power is needed, and, you know, it really depends, but what we have seen is this unprecedented demand for performance because, you know, AI is one of the largest computational lifts of our time. You know, if you look at things like the internet or metaverse and, and the other kind of, you know, phases of the IT industry from, from years gone by, they didn't necessarily have a huge impact or a huge demand and, and drive for performance. Um, you know, we're still in the dial-up internet days of AI at the moment.

Um, but if you want to see the value, you know, you've gotta bring real world scenarios and real world business outcomes that can happen fast. So you need performance, therefore you need our full high performance components. Typically, things like nvidia, GPUs, um, you know, they bring the most computational, you know, compute power to the table, and it's what's at heart of all of our Lenovo workstations.

Will it always be a GPU that I need, or as time goes on, will there be other classes of processors and different ones will lend themselves to different use cases to your earlier point? Yeah, ultimately, I think the, the easy answer is yes. I mean, what we've seen in the last 12 months, the birth of the A IPC, um, it's the addition of a new processing unit to the table.

So we've forever been working in ward of CPU central processing units. Um, and then GPUs were derived from, you know, gaming and graphics, and they're now very powerful co-processor for ai, but it's the birth of this NPU, the neural processing unit that's gonna be really key today. 0 of those types of processes.

So they are not a performance accelerator, but they are a very efficient process of ai. So as AI becomes something that just runs in the background, and that might be a, you know, a constant background blur, that might be some smart antivirus, that might be some smart security tools that are running in the background, you need a processor that's almost custom built to drive those inference, those AI inference workloads and things like the NPU are gonna be critical there. Um, I think people are still confused to this day, and it's been a long running conversation about, well, where is that line between what we call a PC in a workstation these days?

And is it changing even when we add these NPUs? Well, ultimately a workstation is a pc. Um, it's a high performance, it's the highest performing pc.

Um, you know, PC for us is a personal device. Um, whether that runs Windows or a Linux operating system, it's used by a user. You know, for us, workstations are the highest performing variance of that, you know, more memory, more CPU, um, more graphical capabilities, more expandability, but they're built to run those high-end workloads.

They're validated to run those high-end workloads constantly 24 7. So if you do a nine to five job, you're still using a workstation. But if the computes then running in the background and you are doing things, you know, in the hours where maybe you're not behind the desk, then, you know, a workstation has always been designed to give that, that that resilience and that robustness to those more all important workloads.

I think people can make a correlation in their head to, you know, a workstation and a high-end gaming rig, or kind of similar in people's minds, but, Um, What exactly will be the tipping point that would require me to use a workstation to run, uh, an inference engine per se, versus it's pretty clear that the people who are building AI models probably need the biggest machine that they can find. But, um, what do I need to run these AI models successfully and what will determine whether or not that's a workstation? Yeah, it's a fair question.

So if you look at the workflow, you know, AI today in the marketing lens is all about consumption. It's consuming intelligence has been created elsewhere, and I think many people overlook the fact, well, someone's gotta create that AI in the first place. If you look through the lens of like an open AI and chat GPT, they spent, you know, a large number of dollars creating the AI and actually training this master AI model.

Um, and then us as just mere consumers we're just inferencing and pulling from that locally on our devices. Now you can run those models locally. You don't need to pull them from cloud, you don't need to pull them from service systems.

And you know, as an organization, if you are working off very complex data or very unique data, think healthcare, think legal, think government type workflows, where you need to be able to do it securely, you have to have the performance to be able to run that locally. Um, you know, a model, a large language model doesn't have to be large. It could be small.

So everything from three seven to 70 billion parameters as an example, not all of those can run on your standard laptop. You know, anything over 3 billion parameters realistically you need to drive on on a more high performance device. And anything more than 7 billion parameters has to really be a GPU accelerator device, which, which is a workstation.

Um, so it depends on that workflow back to the data and the workflow and the use case. Um, but at the moment today, you know, you need AI to be performing fast, otherwise it's quicker. Just do it yourself the old school way, if you want to have that real time interaction and AI be that truly transformative element of the business workflow and it needs to be accelerated and, and that really happens on a workstation.

Are we getting to the point too where people are appreciating the latency involved in a lot of these inference engine applications because, well, if the inference engine is running in a data center somewhere, or in or up in the cloud, I I create a latency issue where I'm not really having that, uh, engaging real time experience. So, um, will that factor into our heads about what kind of machines Where, yeah, you need to take, you know, do you take your data to the compute or do you take your compute to the data? Um, you know, I'm, I'm a big avid fan of using AI tools.

It makes me a lot smarter and efficient. I think it's my superpower, don't tell my boss, but I think it's a superpower. I can get stuff done quickly.

Um, but I recently flew back from Europe on an airplane and, and kind of, I felt foul of the fact of like, I didn't have a fast enough internet connection for any of my AI tools to work. So I was, I was superman without his superpowers, and I didn't get done what I needed to get done. So I think for me, latency is gonna be the killer, that that's why we have to have this hybrid AI kind of, you know, ecosystem where you do on the device what you need to do on the device, be that security latency or just the size of the data.

And then you take that to maybe, you know, a, a remote desktop or a server or a cloud instance only when it's needed, and then you move that data around or that workflow around, depending on who's using it. Um, because you know, these AI tools at the moment, they're still in their infancy, but as they're widely adopted, I think, yeah, edge computing and doing AI at the edge where the data is located is gonna be the fastest and most efficient way to use it before it's then maybe bypassed and, and moved into the cloud. You use your metaphor, therefore latency is AI kryptonite, right?

Yeah, yeah. I mean that's, yeah, I, I, I felt very, very weak on that flight back. 'cause I, I, I, I missed deadlines.

I couldn't do what I needed to do because I was reliant too much on my, um, my cloud connected ai. So for the IT leaders out there who are trying to figure out how to plan infrastructure investments in the age of ai, what should they be thinking about today? Because they're, many of them are trying to sort out, well, do I need to buy these things this year or should I wait till the bulk of next year when we're operationalizing ai?

What's kind of the, the curve of where the software and the hardware comes together? I, I mean, yeah, it, I wish I had the easy answer for that. You know, if you look at, you know, the average corporate customer who's, you know, they're, they're up for a refresh on their devices and do I bite the AI PC bullet or not?

And you, the answer is, you, you have to at some point, but do you wait, as of all new technology next year, there's always gonna be a faster version of something around. Um, but you need to start somewhere. So this kind of AI adoption route, we are seeing a massive adoption now of these broader IPCs, but the tools that are available to you as an IT administrator are vast, they're complicated, they're unique to you and your workflow.

But AI is what we are seeing and what Lenovo's doing is providing a lot of these AI enhanced IT administrative tools that make deployment of these new devices better, faster, easier, cheaper, so that you can adopt the technology when it's hot off the press, rather than wait for things to maybe mature because you know, you're always on this kind of rolling refresh. Um, and that you have to give the right device to the right people at the right time. Um, and everybody across your organization isn't on a level playing field of who's working with AI or who can benefit from ai.

You know, we've got legal teams and customers that in those particular fields that are light years ahead of those working in maybe healthcare and life sciences, those in the financial industry of, you know, they're building their own AI models 'cause they won't take things from the cloud. So these domain specific use cases, you know, this whole user persona now for AI is changing. Um, and the good thing that where we come in from a device standpoint is we've addressed that with the widest portfolio of devices so that customers can have that choice without the burden then of worrying like, well, should I wait for this year's refresh or next year?

You know, when do I go? Because, you know, ultimately, you know, it's, it's an open playing field for that. There are of course a lot of options when it comes to workstations these days.

What are the factors that organizations should be looking for when they decide ultimately to partner with the provider of a workstation or any other platform? Yeah, I mean they, they're complicated devices. You know, I'll burst that bubble now that I think we've got three plus million possible configurations of each workstation device.

So, you know, the op, the complexity police when it comes to configurability are real. But if we look at, you know, the use cases, what we've done, we've built out these, um, curated hardware configurations and these recipe cards for success based on user workflows and user persona. So if you're an, if you're an IT developer or you're an IT administrator, if you're a, you know, you're a working with manufacturing and you work with computational fluid dynamics, then we've got all of these pre-configured, these recommended kind of good, better, best scenarios so that you can then see what the recommended platforms are.

And then we've got data to then help back up. Well, look, this model is 32% faster than that model. So, you know, you know, when you base it on the hourly rate of the employee that's gonna be using that, you know, you can see that actually how much value are you gonna get from those new platforms.

So we make sure we bring the hardware itself, but then help you kind of cut through the trees a little bit to find the golden nuggets then of, of real value, which we know those individual users are gonna obtain. Um, what's the one thing that kind of makes you shake your head these days when you listen to people talking about AI and workstations and infrastructure, that kind of thing where you just go, folks, we're missing the point. It's, again, I I phrase this one, but it, AI is not magic.

What you can do with it is magic. Um, but it's putting some of that magic into a, into a format that is valuable for your business. So stable effusion, I can create a picture of Mickey Mouse on Everest drinking a milkshake.

Awesome. What's, how's that valuable to me and my workflow? So you, you know, looking away at some of the gimmicky features of it, it's the ability, this superpower thing of you can naturally now obtain real world value from these vast amounts of data that your business has been, you know, sitting on and obviously building.

Um, data is not valuable unless you can extract something from it, learn from your mistakes, better insights to predict the future, that sort of thing. Um, but if you can give your employees that superpower, and again, your superpower, my superpower, everyone's superpower is gonna be slightly different. What's your biggest pain point?

I'm sat on a thousand unread emails this morning and I'm like, cool, can someone pre-read those for me? Um, and tell me which ones are the highest priority. So I don't fall into an inbox, you know, of hell, I can just block out what's needed.

But if I'm a, you know, if I'm a, you know, working in a law firm or if I'm trying to work through, you know, seven different documents on some, you know, the legislation that's come in, then I need, I need a tool that's a little bit different. So people don't appreciate that as good as chat. GPT is, and it was the, you know, the, the light bulb moment for AI for many of us is that, you know, everyone's use case is different.

Um, so don't look at AI as just one thing. It's a vast array of, of, of opportunities for, uh, efficiency. I also think at the end of the day, people are also trying to figure out, well, are the devices gonna be more expensive in the future?

'cause we've added more components and that's just gonna be the nature of the particular beast. Or will these systems follow the traditional price performance curve and will basically be paying roughly the equivalent for an AI workstation that we paid for a regular workstation a couple of years ago? I mean, it is, yeah, relative.

Um, I always tend to shy away from pricing conversation mainly because it's, it's almost irrelevant if you look at it at the context of, you know, employees aren't getting any cheaper. And if you look at what you are here to do, you know, I, I, I'm a big Formula One fan, and if I look at, you know, someone like a Lewis Hamilton of the world, if you put him in the cheapest race car, then you are wasting money paying him his huge salary if he hasn't got the right hardware or the right car behind him, but more importantly, the right team to go support him. So put it in context of the user, the software applications they're running, you know, the tens of thousands of dollars that, you know, they would be hemorrhaging every month if they were sat there waiting for the hourglass.

For me, putting a performance workstation into the hands of every employee will deliver more commercial value to the business. Yes, the device at a device price point is more expensive, but it's delivering so much better return on investment and a lower total cost of ownership, which ultimately you're drinking less coffee, you're smoking less cigarettes or whatever. So, you know, for me it's cheaper, but it's an upfront addition to CapEx, whereas maybe operationally it's cheaper to to, to go that route.

Alright, folks, you heard it here. Even in the age of ai, you can still be Pennywise and a pound foolish. Hey Mike, thanks for being on the show.

No, appreciate it. Thanks for your time. All right, and thank you all for watching the latest episode of the Techstrong AI video series.

You can watch this episode, others' on our website. We invite you to check them all out. Until then, we'll see you next time.