Your turnkey AI Factory for Rapid Development with Hewlett Packard Enterprise

Hey everyone. Uh, my name is Mark Seiter. I'm a solutions architect, uh, with Hewlett Packard Enterprise, and I represent PC ai.

We're private cloud for ai. Um, and this is an offering that I'm gonna tell you about today. We're calling it, and I think this is, is very true and it's, it's really being received well.

We're calling it kind of our turnkey AI factory. And so any further ado? So, yeah, my name's Mark Ser.

Uh, I've been doing this now for, uh, about six and a half years, a little over six years, uh, with Hewlett Packard Enterprise. And so this product that we have, I'm gonna tell you about, has been through many iterations, right? I came on six and a half years ago in the entirety of my duration with this, uh, with HPE, has been with this suite of software products that are represented as a component inside of the larger pc, ai or private cloud for ai.

And so this is not something that we just threw together. This is something that we've been working at and developing for a long time. This has come through internal development as well as acquisition.

And we've been lending different components that we've acquired along the way or built ourselves, uh, to create something that's really compelling and really unique in the market. Um, my background is in, uh, I would say horizontal ai, uh, and data management. And so I started my career, uh, as a technical writer, technical trainer, um, and worked with a company based in Austin that was trying to attack the democratization of AI problem 10 years ago, um, long before the advent of all of these LLMs and these things that have changed our world.

Also, one of our HPE AI ambassadors, uh, and a dad to three girls, five and under, which is my greatest joy in life. Also, my greatest frustration. Welcome to the club.

So I like to start off, and this is a hot take, uh, and I understand that I used to always use this slide and say, AI doesn't exist, right? At least not in the way that people talk about it commonly. Uh, and I'm a storyteller.

I like to weave in pieces of my life, right? To kind of make points. And I have a friend, he is my best friend's, been my best friend since we were 12 years old, and he's a nervous person.

He's a doctor. And he calls me up and he says, probably once every year or so, maybe once or twice a year, and he's like, did you see this headline AI's gonna take over the world? And, and it's, it's hilarious.

'cause I, and he knows I work in this space, and so I always have to talk him off the ledge and say, Hey, man, like, I, I totally understand it, but what you're thinking about, what a lot of us think about, right? The whole alien with, you know, the, the humanoid kind of robots, how 9,000 matrix, right? That type of AI is not even close.

It's not even on the horizon, right? And so I think people conflate a lot of times these headlines, these kind of fantastic headlines that we, we read. And people will talk through it an LLM and it'll kind of seem like it's giving them, you know, answers about questions that it might not supposed to have, might not be supposed to have like, insight into, right?

Or it's got plans. And that's, it's simply not the reality of the situation. The reality is that we've gotten really good at telling machines what to do, and they do it really fast.

They do it a lot faster than we can really comprehend. Um, and they do it nonstop, right? And so that creates this interesting place.

'cause this was always, I've been saying this for five years now, then in 22 when chat GPT came out, it kind of hit so hard. And so throughout the last couple of years of learning the technology, understanding, you know, on, on a deeper level, I now have amended this slide two. Now, AI has reached a tipping point.

I won't say that AI doesn't exist. And I'll tell you why. It's because of the outcomes.

The outcomes of the AI that we have are so close to being the types of outcomes that you would want from actual ai that it's kind of indistinguishable practically. It doesn't matter to make this delineation, right? It's still something that we can talk about, about how AI is not gonna take over the world.

It's not self thinking, right? But the outcomes are so good that we would be fools to bury our head in the sand and say, you know, not understand that this has the potential to radically shift the way that we live, the way that we work. The applications are practically limitless.

And if I didn't mention before, I love the questions, and I, I know this is a, you know, a, a forum and, uh, I'm excited. There's A lot of disagreement on where AI and a GI is today and, and how quickly we we'll see something with the potential for serious existential risk versus whatever. I mean is, you know, the thing go on and on that, that sort of subject has a no ending conversation, but Yeah, for sure.

Yeah. I, I think the risk is presenting what we have today as what we think is going to be in the future. Yeah, I think the things that, I mean, if you really boil it down, like in the same way as you might say the internet is just ones and zeros if you really boil it down.

And LLM is just matrix multiplication, right? If we take all the architecture and the actual implementation out of the way, that's what it is at the, at its core, right? Um, and that's what it is.

We told machines how to take number rep representations of, of tokens or bits of words. And we taught them how to make a lot of correlations between those things. And then when we query them, we're just saying, what is the next best string of words complete based on my input, right?

So that's the function, the whole, you know, something that could be dangerous. Yeah, we could go on for a long time arguing about that, but I feel pretty comfortable right now with where we are. And so as part of that, I like to talk about the, all of the applications that we have of ai.

And this is the landscape that I'm, I'm involved in every single day with my customers, with new customers, uh, existing customers, talking about ways that they can either adopt technology or continue to expand that technology footprint inside of their organization. And these things go, obviously there's a horizontal component here, right? So the horizontal use cases, the ones, has anyone ever not used the chatbot?

Yeah. It's a part of our daily lives. And so naturally as something that we've adopted into our lives, work follows suit, right?

Maybe sometimes work is the leader where we wanna figure out how we can adopt things like chat bots, right? Or content summarization tools, right? Why do I have to read through a big long document if something can summarize the salient points for me and, you know, a paragraph rather than 10 pages.

Mm-hmm. Right? I'm able to accelerate my time.

Um, task and workflow automation are across every industry. There are things that you can automate, right? And there's a bunch of different techniques to do that that we'll talk more about, and I'll skip the content and all that stuff.

The ones that I think are personally interesting is retail, right? With personalized product recommendations, because that is the specific space that I worked in. Um, previously with that horizontal AI company.

We built, you know, kind of personalized profiles and we were doing this with graph databases and really sophisticated algorithms 10 years ago. Um, so I think the development in that space is really interesting. And how do we create representations of people that we can use to be able to understand maybe their preferences or their interests.

Um, so I think that's a very interesting space. And then edge computer vision is one of the hot buttons for me because I think it's one of the most interesting spaces. I've talked to all kinds of, uh, defense and, you know, level one nuke sites, and, you know, how do we do a better job of using camera vision or computer vision to identify threats, potentially anomalies in the signal data?

Because currently, I can tell you in some conversations that I've had, they have a flip, the flip book has pictures. There's a guy with boots on walking around with a flip book. And if he sees something that he thinks might look a little odd, he gets his handy dandy flip book out and looks at it and compares and says, okay, this is what it's supposed to look like.

Does this look like that? Right? That task is something that could be so well addressed, right?

By computer vision or by those types of technologies. So I think that a lot of these are, it shows the breadth of it, right? And it goes across horizontally, it goes vertically, it goes very deep, right?

Health and life sciences is also a huge one. There's all kinds of drug discovery, uh, and patient classification. There's a bunch of administrative stuff around hospitals too that I've, I've talked with a ton over the last two or three years.

And so I think realizing the breadth of this space is something that it's easy to think that, you know, oh, the applications could be kind of limited, right? Because it's the an LLM, but if you really start to look at it, it's everywhere, right? And its applications are literally everywhere.

So one of the things that I like to think about is that a lot of people are afraid that AI's coming for their jobs. And I understand that, right? There was, you know, some, another large technology company with their HR department, and all of that happened somewhat recently, right?

And introducing chatbots and things into that. Interestingly with our customers, and we've done a lot of research in this and surveys, that is not where people are most excited about leveraging ai, right? They're not looking to reduce staff or reduce head count or, you know, everyone would like to save money where they can, but that's not really their interest.

Their interest is more in how do I make my employees better, faster, more accurate in doing their jobs? And then how do I improve my customer experience, right? How do I delight my customers?

So those are the places that we're seeing most of our conversations trend toward. It's not so much let's get rid of people because we're gonna replace 'em with robots. It's now let's figure out how we can be better at what we do.

So I think that's very interesting. Oh, sorry. Where are these anticipated benefits?

Where did you guys, where did you get this information? So some of it was, uh, an, a survey that we did internally. And then I think some of it also came from, uh, an external, I don't remember the exact quote, but I can get to it afterwards.

It was maybe IDC or someone I'd be interested in seeing what they mean by employee productivity gain. 'cause I think that's what everyone says, particularly up at the IT executives employ predictivity gain. Yeah.

I see it as easy examples for me. Where I have seen that conversation come up are in those things where you would typically feel like you want to employ a chat bot, right? Or a rag application.

Because if you give somebody, and I have an example of this that I'll show you in my, um, in the demo that I'm gonna do later, I'm gonna show you guys a real, real live working system. Um, but if I have with rag, right? If I have whole boatload of documents in a SharePoint, right?

I don't know if in your lives you've had to sift through SharePoint. Um, I have. It's not my most favorite function.

Um, not my most favorite. I'll just, I'll leave it at that. The idea of being able to feed that right into a rag implementation or retrieval augmented generation, right?

So how do I understand and reason over a private corp? But I, Because, but I don't really see actual results being generated or talked about. 'cause 'cause I know myself, I can see all sorts of ways of doing it.

I know we had a tiny team before we got in all these, uh, great data, 6,000 surveys brought in. We, this is a long time ago. There was no way our little team could go through it.

I'm thinking, wow, something like this could take the three of us and provide great benefit. And yet I don't, I'm not hearing so much about this. I've heard a lot of this all seems to be intuitive, but I'm hearing more about things not working.

And, um, yeah. I'm just curious. Say like the SharePoint example, are you seeing that a lot?

I think a lot of this is intuitive, but I'm not, I'm seeing those types of so rag implementations everywhere, almost every customer that I talk to, and it makes sense, right? There's a, I'll reference them later. There's a, a basketball team that I've worked with, a professional basketball team, and that was one of their main things.

They said that we have, uh, the CBA, the NBA collective bargaining agreement. Every time a law changes in California, we have to hire a team of lawyers to go and look at the collective bargaining agreement, understand the law, right? And then tell us if there's any, you know, implications with our player contracts.

And so they're like, we have that and a whole bunch of other documents, player contracts, all of these legal documents. It's very, very difficult for us with our staff, which surprisingly you would think that an NBA team would have unlimited resources. And to me, at least they don't.

So it's not like I can just hire more bodies to throw at the problem, right? We need the staff that we have, but we need to make them better because they're just maxed out having to do this very kind of rote process over and over every day. So that's an interesting Anecdote.

Yeah. I think these are anecdotal, but I'm just thinking just from reading and that's why I'm interested 'cause you're actually out there talking to customers. I don't see a lot of the benefits being touted that you would intuitively think would give you, make small teams able to do much bigger things.

Do you see it because you just haven't read about successful implementations of it? Is that, yeah, I think all these things you hear about we can do this, less people to do, et cetera, et cetera. But then I don't see any documented use cases or, um, in fact, I see, uh, sort of a fatigue all these things were promised.

And yet I don't hear people then saying, you know, we actually took our headcount down by this and our productivity up by this. Um, I haven't seen a lot of articles that have talked about productivity gains, concrete productivity gains being delivered, but that's just what I'm seeing in the Yeah. And your plan right into my hand, I have to tell you because the part of my conversation, there's the second section that we have is about the gap, right?

From pilot to production. That gap is kind of what you're talking about. Everyone can see intuitively, you can see the interesting capabilities there and how it could be applied.

There was a, a report, and I won't reference it too hard, but there was a report that came out that said, you know, 95% failed to achieve ROI. And so we'll talk a little bit about at least some of my opinions, some of my colleagues' opinions on why this is happening and perhaps a way around that because I agree for all of the promise that's there and what can be done, I am not seeing as much success broadly in the market as I initially thought I would say a year and a half ago. So I think that's a very real, very real comment.

That's table. Yeah. Table.

Anybody else before I move on? Okay, we'll leave the other ones for now. So yes, it's everywhere.

Yes. Everyone's trying to leverage it. What's stopping people from getting there?

And so those challenges and some of the things that we talked about, which are creating that situation where we're not seeing as many, like, validated reports of the value being realized. I think it ties back to a lot of these things. It started, does this laser work?

Oh yeah. It started there on the left, right? Where initially, and still some, I would say, I don't wanna say laggards, but right.

Some customers who maybe are laggards still don't really have an enterprise AI strategy. A lot of customers that I've talked to, and there's one that I'll reference as kind of an example. They are a technology, uh, manufacturer, a big one.

And I talked to them and they said, we are trying to figure out how right. We just created kind of like our AI council, we'll call it. And we're trying to figure out how best to implement this because what we have is, we currently have this guy and he built something really, really cool.

It was a chat bot, right? That could do document search and it, and so then he turned it on to a couple of people and they loved it. And so then other people in the Slack channel for it started requesting to join, and then all of a sudden we had 40 or 60 users, and then the hardware started failing because it was on some kind of borrowed old generation hardware.

And so how do we go from that, which is something cool that somebody built, and I see that all the time. Somebody built something cool on their laptop or on some servers they had under their desk, right on this little half rack that's sitting there. Or somebody built something really compelling in the cloud, right?

How do we scale that? How do we bring that to production? You're still in that space where some people are still trying to find how are we as an organization going to do this?

Are we going to develop on-prem, deploy in the cloud? Are we gonna develop in the cloud and deploy on-prem? Are we gonna do it all on-prem, all on the cloud, right?

How do we do governance of all of this, right? What's our data management strategy? How do we, you know, manage all of these assets and make sure that's nothing's getting leaked places where it shouldn't be?

So you have some of those who are still figuring out, you have a lot though, who have started to really kind of put concrete foundations for this is how we as an enterprise want to do this, right? So that's kind of like the first challenge that I see. The second one about things only going only 10% going into production that I think is directly caused by two things, the data privacy and control and accessibility.

I'll kind of just lump those as one and then the deployment methodology, or how are we going to run this in a cost effective manner? Right? Well, but you're saying go into production, you mean production Production, You mean?

Production Production, Yeah. So there's so much experimentation in AI right now. I I think 10% could be a good thing.

10% is the, the, the ones that make it through all the gauntlets and are worth going into production. You Know, That's at least a third factor. Sure.

Yeah. Yeah. I hadn't thought about it in that way.

Um, I would say as an organization, right? If you said to me, I want you to invest $5 million, let's say I'm an appropriately sized organization where that's meaningful, but not, you know, overly taxing that I want you to invest $5 million and we've got a hundred use cases, right? That could provide value to the organization and 10% of those will actually, like, we're gonna develop all of them or try to develop all of them, right?

But only 10% of those are gonna make it. And there's really no judgment on whether or not that's, is that the most valuable 10% that are gonna make it? I I would be dissatisfied with that.

And I think that I definitely see your point, but I would be dissatisfied with it and I would want a more rigorous and controlled process to make sure that the ones that I want to get to production, that I have a path to get them there. I think so I think though that, um, the, the ultimate purse holders, the boards and the business leaders and such, they, they want to see movement. Um, and just the sense I'm getting from the customers I've talked to is that's the game.

The game is movement. They want to message it, they want to talk the board level mandate. Mm-hmm.

Well, in corporate level and what have you. Yeah. Right?

The, uh, the ultimate stakeholders, they, they all want to, to be able to communicate an AI strategy with some, you know, some meat behind it. These are the specific things that we're doing. And if you don't bring up the question, nobody notices is it being successful?

What's the ROI? What's the, the way the, the way I've been talking about it is that it reminds me of the original PC revolution. I've said this in more than one forum, so pardon me for repeating myself, where for 10, 15, 20 years people bought desktops and eventually notebooks hand over fist without any really clear ma micro macroeconomic, you know, uh, productivity benefit being shown.

But the, the value was evident, the value was obvious and evident. Eventually productivity ca gains did come. I think this is the same thing only happening faster.

And I'm not, I'm not quibbling or sorry, arguing with, um, the overall premise and the idea that there are things holding back productivity and out of AI investment. Um, it's just, uh, uh, we have to acknowledge that a lot of the, like, there's a lot of activity that's gonna be ultimately beneficial, but the, the, the standard enterprise metrics not only don't apply in fact, but may not all apply or should not all apply at this stage. Yeah.

Yeah. It's very, it's a new space And you know, it is when people started going out and buying computers at first, uh, you know, one of the biggest differences you saw was typewriter ribbon stop being sold and toner cartridges started being sold. And I was gonna comment on this as well.

An awful lot of enterprise, um, companies, HP included, uh, there's probably numerous projects that get started, and I would guess number of them that actually get to the point of being product or organizational, um, projects is probably not a whole lot higher than that 10%. Mm-hmm. Yeah, I can agree with that.

I think that there's, at any company, that's definitely a thing. I think the one wrinkle here, or maybe more than one wrinkle, the wrinkles that I see are that the investments that are being made in AI are so high a cost that it's almost unforgivable to not have some realized gain out of it. That to me, you're gonna make a significant investment.

And you're right, like that board level mandate or company-wide level mandate, we need to figure out an AI strategy. We need to leverage it has come and it has come on strong and people are throwing a lot of money behind it. And so I, I think the positioning for me, and it'll hopefully help me throughout the, throughout this presentation, is that a lot of times I've seen that go in the direction of people throwing money at the cloud and telling their people to just go do something, right?

And maybe they go spin up some EC2 instances with some really expensive GPUs, or maybe they're going to open AI and they're just, you know, leveraging an endpoint and just flaming it with no real concept of what's The goal, What's the goal, what are we doing here? That is a really good color on this discussion because we all lived through that. We saw it, saw it coming, we predicted it, bloat came over, provisioning came, and then eventually this whole side industry around cost control got developed beneficially to everybody.

And, and it, yeah, go ahead. And, and the other comment I was gonna make along those lines is even if you a, have an AI product that makes it to production, uh, it's, you know, an awful lot of the, the delegates here kind of argue the only way that you monetize AI is be selling AI hardware Stands for additional infrastructure. Yes.

I've never heard that. And I work in this industry, so I'm a little bit ashamed, but I'm also amused. I'm, I'm interested in, you talk about the production, because I don't know exactly the, the connotation, but I would say that chatbots and AI being used as tools, I'm wondering when they're being brought in process and when do they scale and when do they standardize?

And I don't know if that's, that's akin to what you're talking about going through production. So like the example you used with the basketball team, is that something that now they use for all their distilling of big information and that's now throughout the, the group? Or is it just one group found it to be very helpful and they were able to do it?

So still an active conversation, uh, with that basketball team. Um, there was some, some turnover at the top level of the organization, so still having conversations there, um, generally what I'll say is that for me, production is use case driven, right? And sometimes that scope, I would love if AI and I also, I have to fight this pretty frequently where people are like, yeah, let's throw AI at it.

And currently there's nothing that's like, I can just throw a bunch of documents into anywhere, like some kind of repository and then just like type in like, you know, a couple of sentences to be like, I want you to figure out every bit of information about these documents. Any question that I ask, I want you to tell me. Right?

A lot of people think AI works like that. Like just, yeah, you throw AI at it, it does whatever you want it to do. It doesn't, um, you have to really strongly define the workflow that you're trying to affect.

You have to strongly define the use case, the end users of it, what the anticipated benefits are so that you can kind of architect and march towards that, right? The outcomes of it. And so a lot of times production for me is tied to an implemented use case that fits the requirements that the customer set forward, right?

And a lot of times, maybe that's limited to a line of business, a lot of times is where I see like a particular line of business has a function that they'd really like to inject a, you know, smarter process into using these technologies. As far as like going enterprise wide with it, I've seen it, I've also seen, you know, where it just kind of stays as a function, that line of business because it's something that's kind of core and related to what they do as compared to the rest of the business. But I've also seen chatbots, particularly like with HR documents, a great example.

I've seen this a lot of times where great, we are going to have an HR chat bot function, right? And so if someone wants to come in and the questions that they want to ask are things that we can use like rag against, then we're gonna have a rag agent chat bot that we're gonna implement company wide. And it'll do, you know, any question this someone wants to ask, Hey, what are my, you know, PT O balance, blah, blah, blah.

I've seen that too. I think There's a lot of shadow use of AI that's going on. I mean, developers and stuff like that using various tools to, uh, code projects and, um, documentation or, or you know, content productivity being used, you know, things of that nature.

It may not be quote unquote production or, or enterprise, uh, enabled or, uh, authorized, but it's, it's happening under the covers to some extent in an organization. I Don't think you came here to discuss the, the value of ai. No.

So let's, let's actually move on to what you came here to discuss. Alright, I can do that. We'll table it.

I've kind of belabored this a little bit. Uh, so I'm not gonna spend a ton of time, maybe just a minute, right? Two places I've, I think realistically you can start, you can start in the public cloud or you can start on prem, right?

Or private cloud. And there's a number of different ways to do private cloud, right? You could build it all yourself.

You could buy an engineered solution, right? You can, there's a couple of different ways that you can approach it. Our position here and what we've come out with is an answer to that.

And so this is an older slide note. I put it here for a reason, just because I wanna show you the evolution of it. What we came out with was, is private cloud for ai, right?

PC ai. PC AI is an appliance, right? Your AI factory turnkey AI factory and it is a combination.

And so it comes in t-shirt sizes, right? Developers, small, medium, large, extra large, all of them are kind of rack based except for the developer system. And so throughout these t-shirt sizes, you obviously grow in these two, you can see that the types, I'm gonna use my pointer and not step off the X.

Uh, you can see that the GPU types changed and this was our generation one of this product. We came out with it a year and a half, two years ago. And it is meant, and we initially scoped these particular types of use cases, right?

So a small might be appropriate for inferencing or uh, a medium might be appropriate for inferencing and rag application, right? And a large might be appropriate for fine tuning and inferencing and rag, right? And maybe we did ourselves a little bit of a disservice there because what we started to see was somebody, I had had customers that come up to me and say, well, yeah, I wanna take a, you know, 70 billion parameter model and run it on a small because all we're gonna do is inference on it.

It's like, well still those GPUs don't have enough memory to even hold the weights for that model unless you quantize it to, you know, a non-usable state essentially, right? You will degrade accuracy once you quantize it too far. And so what that showed us is that there's more latitude and we really have to be good about how we work with our customers in understanding their needs and scoping to that.

And I like to scope to my customers first 18 months, right? I'm not gonna, you know, advocate that you purchase something that once you do your first use case or first two use cases in the first couple of months, then you know, you gotta buy something else. I wanna scope this to where you're anticipating that you wanna be, how many people do you want on the system doing active development?

What are the size of models that you want? What is all the details around throughput and quantization and like all of that stuff there. Uh, you know, how many use cases do you have that you want in production by that point?

How many do you want in current development? And that's how I start to scope it. What we, that developer system came out just a little bit later 'cause we thought we were missing a segment of the market where we wanna prove value, right?

The board wants to see value or you know, the company wants to see value before we make a huge investment, whether it be in hardware or in the cloud. And so we came out with this developer system. Developer system shares all of the exact same software components.

Everything is the same other than just doesn't have as many nodes. It doesn't have, you know, all the networking and the external file system, right? It's integrated storage on it.

It's got H one hundreds, which are actually very, very good chips. Um, very powerful chips. What's the os you don't list the OS across the line there.

Do you have various different like Ubuntu and Fedora and uh, There's a couple of sore and um, a couple of other flavors of Linux that we've validated, But what is, what do most people run? I would assume it would be something like, uh, Linux of some sort. It's, it's, it's Linux.

Uh, it's only Linux. It's not any, it's no windows or anything. Um, we're seeing lately.

We just made a, an OS shift recently. Um, I wonder if it's, I think it's rocky. Yeah.

Mm-hmm. Stick because I've got the stinks work. Um, and I'm gonna kind of tie this back in right after.

'cause this product makes sense when you see it, right? Like this is a product, it's a thing, right? I build it and integrated it at the factory.

I install the software on it and I come and drop it off to you. You give us some network up links and so an IP ranges and we hook it up, turn it on and you are ready. What are you ready with?

And that's the piece that I'm gonna tie back in, in just a second. Is this, Is this NVAE or something else? NVAI enterprise or Is it?

Yeah, so N-V-A-I-E, which is my least favorite acronym in this entire space. 'cause it trips me up all the time. It is a component.

We have included that on top of our platform, our software platform that I mentioned that I've been working with for six plus years. Okay? That software platform is what we call AI essentials.

And it's had a life, right, for six and a half years or so, and it's gone through iterations, but it's the same platform, mostly in the same experience. We've included Nvidia, N-V-A-I-E on top of that as component inside of there that you run on top of our platform. It's a Kubernetes based platform.

And so that's how we bring things from Nvidia. And I have some slides on that that I'll, I'll tie in. We came out with our G two very recently.

I think we announced a discover or shortly before or after, um, we upgraded all the GPUs. We also, except for the developer system, we'll leave that one aside, but we upgraded to the RTX Pro 6,000. Those are the blackwells.

And uh, NVIDIA's made some interesting announcements if anyone's seen it around the B three hundreds and not offering that in A-P-C-I-E form factor, right? 'cause these are so good. And so I, that's I guess my opinion on that.

And then we've gone to the H two hundreds in the larger configuration. We've also right sized the storage, right? The storage is down.

And the reason behind this is we included a lot of storage with it originally and that that incurred a lot of costs. We realized through seeing people actually use this system is that this is not meant to house all of your enterprise data. We have data connectors to file shares, to object stores to structured data.

Right? We want you to be able to leave your data where it is and connect to that data. The storage that's here is mainly representative system storage, right.

For al work development that you're doing, Are you targeting these for training inferencing, both other AI activities? What's sort of the Goal? The only place where I would not target it is at full blown training.

Okay. So you of very large models, Right? But so you, you, you're looking at these as an en enterprise would stand up one, maybe two racks here of this to build their inference and cluster for their production environment.

But the slide says fine tuning is the, for the, for the fine. This is, that's, that's the large, not the extra large, extra super duper coming. Yeah, I said that here, but they're large currently in market with, was that 32 up 16 or 32 up, right?

16. Correct. With with the racks.

Yeah. So it says fine tuning on there. There's no, uh, I mean, you know, you can always train SLMs or what, what have you.

Yeah. You're not really looking to more than that. Probably up to, I have a, a great example of this and I was gonna include it in my kind of cloud costs, right?

I had a customer who wanted to train, and I'll reference back to it later, but I had a customer who wanted to train a 70 billion parameter model from scratch, right? Because they were in a kind of a nuanced domain and they didn't really feel like anything was appropriate or accurate enough based on their testing and so that you can do it. Yeah.

Right? Because ultimately what they would need is essentially, let's call it 64 H 100, or they were actually a one hundreds was the instance type that they had, um, that was just what they standardized on in their cloud provider. And they were gonna need, um, their dual uh, GPU instances, they were gonna need 64 GPUs.

So 32 instances running at full bore for 21 days to train that model. Right? Their instance cost is 10 to $30 per hour for each of those in instances, if you do the math of the 32 instances times 24 hours a day times, you know, 20, it's $330,000 for three weeks of training to train that model, right?

This is where this becomes very competitive because these are things that you own this, you can pipe whatever you want through. The only cost you're gonna incur is a little bit of extra power, right? And these things don't consume anywhere near enough power.

The AI stacks, they need to consume way more power. Is that a No, we we've talked about racks consuming 120 kilowatts. Yeah.

And more. Well that's because these don't have that, that's what I was getting at is these are designed to be a standalone unit with no optics going out of here to another stack, not part of a, a huge leaf spine network and therefore it's not consuming that much power for all the io Right? I understand They're not also not fully populated, right?

Yeah. Like not every slot is populated. There are a defined number of nodes that are inside of here, right?

With two eight way boxes essentially for the large. So yeah, the large gangs can gang two or no, I don't remember Two Two racks or no, three. Three.

Yeah. The large can be expanded up to 3 48, 64, 3, 3 Plus one. It would be 16 plus the the three which would give Four.

Okay. Yeah. Alright, so it's four racks.

Yeah. There's networking across the racks. There's four gig networking.

Mm-hmm. At that point, once, once you put in, once you put in a network switch in there, that's gonna be a lot more than 17 k Per rack per Rack because then, you know, it depends on if you can go four racks on copper or four racks on optical. Yeah.

Yeah. As much as I like to masquerade, like I'm a hardware nerd, I'm a software software, software person. Great.

Let's get to the software. I like that. Thanks.

So wait, what makes it a developer system? It's smaller. Apologies.

The developer system is very specifically designed, is on your Desk. It does not have, if you notice like you have so close three control plane nos and all the other form factors you've got external, you've, or sorry, you've got dedicated nodes for a file system, right? You've got the networking, the switches included, ER doesn't have any of that.

It's got one control play node, one worker node. There's 32 terabytes of integrated storage in it. That's it.

It's not meant for production. It is meant for you to test and develop your use cases and you get to use the exact same software set here that you do here. And so now workloads become very easy to transition as you grow up the stack.

That's the purpose of the developer. I'll skip the air gap, right? We did just recently come out with air gap for, you know, heavily regulated industries where you can be completely disconnected, right?

If you know you wanna do updates or something, maybe you have trusted mirrors or floppy disks, right? Kidding About the floppy disk. Um, we're awake.

You got us. So what's in, what's in here, right? I've talked a little bit about it.

This is where I start to get into my favorite part of this, the software side of it. So what's represented inside of here, it's four things. At its core, it's hardware from HPE, that's the servers, right?

It's hardware from Nvidia, that's gonna be the GPUs and the switches, and then it's software from HPE, which is AI essentials. And then it's N-V-A-I-E software from Nvidia. Four things that represent what's inside of here.

And there's other stuff, right? The control plane on the right side, very interesting. Um, you know, it's Kubernetes, there's virtualization.

So this becomes a very interesting kind of like set of how we manage skills and then things that we put onto it like a service mesh, right? Things to help manage all of this and keep things communicating clearly and applying policy. And we've got a lot of very interesting stuff inside of here, but I will keep it there.

The capabilities that it gives you, and this is, I swear I'm getting to the cool stuff mm-hmm. Capabilities that it gives you is that when I drop this off to you, rather than you having to go and say, great, if you wanna build this yourself, what's it gonna take? Well, you're gonna have to sit down and you're gonna have to go, okay, what are we gonna do?

Well, all right, let's get some hardware first. What hardware are we gonna get? What GPUs?

Is it all homogenous? Or we have multiple racks. Can we do heterogeneous GPU deployments?

Can we do any of that? Right? What tools are we gonna use?

I don't know if you've, everybody's seen that big old ugly slide with like, you know, the million AI vendors, right? There's been like a decade now where we've seen different versions of that slide. Go pick what you want to use, right?

And then how are you gonna integrate all that together, right? And thinking, okay, what about drivers for your gps, you have the appropriate drivers, right? All your versions and everything, all this.

So you gotta make all those decisions first and then you gotta actually build it, right? And then you gotta set up the network and you gotta do all of these little things, right? Rather than do that, this gives you the opportunity that like literally at the end of day one, someone can go, a data scientist can go and can have fingers on keyboard doing, right?

That's huge because it's so much time and undifferentiated work that we're taking out off the table, right? The differentiated work is what you really want, where people are doing things that meaningfully impact your business rather than just setting up hardware and software.