Real-World Customer Journey with VMware Private AI from Broadcom

So thank you guys, uh, for letting me present to you today. Um, you've heard from my previous colleague here, Justin, a little bit about the architecture and what we're building. Uh, there's obviously more presentations out there.

Uh, we'd love to get in front of you guys and, and talk more too. But what I really want to talk to you today about is a little bit more of that customer journey. What we're interacting with customers, what we're seeing, and how we can get that information, uh, out to you guys and, and disseminate it.

So one of the things that we see here on a regular basis, everything's focused on that data scientist, that AI engineer, but what we're not seeing a lot is that there's a lot of infrastructure behind the scenes in order to get them to be functional and get them to a happy place. So let's just talk about that flow a little bit. This is something that's not unique, I think, to one customer.

This is what we're seeing at every customer conversation I have. So I should step back for a second here. I didn't introduce myself.

So I'm Alex Nu. I'm a staff architect on our VCF Advanced Services Division. We're focused on bringing the new solutions out to market, working with customers and being the interchange between our engineering PMs field, uh, resources and customers.

So when we're doing this, I talk to customers on a regular basis, interact with them, and more importantly, take that feedback so that way we can feed it back into it. So teams like Tasha's team can make better products for our customers in the future. When we talk to customers right now, this is something we see on a regular basis.

Senior leadership wants us to do something with ai. They don't really care about the details, but this is the buzzwords we need to do ai. So for this example, they've asked us to create a chat app.

So we're gonna do this so that we can interact with our data. First thing here is we're gonna talk about, we're going to use the, the buzz is all around rag, retrieve, augmented generation. We've already brought it up a couple of times today.

So we start talking about this. What is it? So the first thing here, we'll talk more in a second, is getting our data.

We're gonna get our data, pull it into the system. From here, we're going to ask a question of, uh, this app. We're gonna have a front end app that we're interacting with from this.

It's going to go search our database database is gonna go grab all the data that's relevant. Doing that search. It's going to come back and come back to our app.

When we're in this app, it's going to create an enriched prompt. It's gonna take that data, the PDFs, maybe it's Confluence pages websites, and it's gonna give it back, uh, to the LLM with our question so that LLM can ask it a question and pass it forward. So, high level there, we can obviously get into a lot of details on this.

Um, for this one though, first step is getting data in here so that way we can, um, curate it and get it through. You guys know this adage, garbage in, garbage out. You don't want to just throw everything at this.

You want to have a curated list, do some kind of cleanup. A lot of customers, when they're starting to get used to this, they start thinking about low hanging fruit, HR policies, HR apps, things like that. So you don't have to go search a 500 page, uh, employee document.

You can search, ask a question, get that in back, and it's not really getting into private data yet. It's a way to scratch the surface, get in there and slowly take steps up. This gets loaded into a vector database.

So vector databases. Why do we do that? Um, that's, so that way we can, A Vector database is a, a special type of database that's really easy to do.

Similarity searches, pull the information out. And this is something that we can provide through data services manager, um, through the VCF core platform. So let's go start doing this.

This is the console for VMware private ai, private AI foundation with Nvidia. We have a few templates that are up here. We're gonna choose on one of the RAG templates.

We're gonna do the AI workstation with RAG and we're gonna deploy this. This is what it looks like out of the box. Top right.

We have a knowledge base icon. We can click on that load documents into it. Towards the bottom left, we have a knowledge base.

We can check that knowledge base and we can use that when we're asking a question and we can go there and have some data back and forth. We're conversing with our data. So this has been deployed for us.

There's multiple containers being deployed, all wrapped up in one vm. Here's a view for us. Bottom left, we have this application, the front end that we're interacting with.

We're going to ask it a question. It's going to then, uh, embed that. It's gonna go through the vector search, come back through a ranking model.

Pretty straightforward. It's gonna, uh, rank those, um, questions and, and documents and then come back to the LM and give us an answer. This is a little bit of the back end here of how we can edit this.

A UI on the left side for the big chunks, right, is infrastructure as code. We can come in here and modify this. If you, depending on the resolution and everything, you can see the model that we're using.

You can see different settings in here. You can make this as simple or as complex as you want in previous life. I've had templates here where we've had 20 or 30 VMs deployed all at once, depending on how the size of the app and the application that we're doing.

So our developers start working with this. They start using it and one of the things that they want to do is create their own customer front end. So they swap out that front end that we gave them that was a little bit maybe generic.

They put in this one here. And now it looks a little bit more of what people are used. This is open Web ui.

Uh, it's an open source tool that we're seeing a lot of customers use. Um, a lot of interest around it. And it's a way to, again, hit the ground running, get up to speed here.

Very similar top left. There's a workspace icon you can click on that you can load documents and then ask your questions. In this scenario, you can actually do different chunks and, and group them together.

So see if this comes out in here, in this view. We're going back to this view. We're gonna revisit this a couple of different times.

I have a purple box around different chunks to show that we've customized these. These are now ours. The open web ui, the the front end that's now purple 'cause that's our code, that's ours.

The database is data services manager. That's a bigger deployment and there's a specific database in front of it that we use that's the Vector db. But again, we're now making it this our own.

We're getting good feedback. Um, but one of the things that we're getting feedback is every question is a unique question. So now there's no history.

Doesn't know what I asked about before. Um, there's also another team inside the company that's using a different LLM and they want us to use their LLM instead of the generic one that we're using. So lemme take a side step here and talk about LLMs for a second.

Here's an example of an LLM and a question I ask it. What is the average temp in Massachusetts in September? It's a fairly long answer.

I mean, Massachusetts is somewhat of a wide state. There's mountains on one side, ocean on the other. Um, so it's a good understanding breakdown.

But if you look here, and this is very simple here on the right side is tell me the average temperature, little bit of prompt engineering at the very basics here. It gets a very specific answer back. Now here's the same family, but a different model size.

The the long answer is shorter. The short answer is longer. Um, but this is some of the things your data scientists, your AI engineers are doing with these models.

They're testing it, they're understanding what the answer is or they're getting back. And more importantly, when they see a new model, they have a banker questions, they're gonna go back through, ask the same questions and say, am I getting better results? The same results are worse.

And I'm gonna judge that and go forward To what my colleague earlier said. Um, the size of the model, who's gonna dictate how many GPU resources we have? These are things that we care about.

So we need to start working with customers to learn. What questions do you ask those AI engineers in order to figure out the size in here? So here we put conversational history and we put that inside the front, uh, app for performance.

It's running in memory in that front end. We have our LLM now as a purple box around it and harbor's there 'cause they have it in harbor. We can pull it down locally.

It's really nice and really simple. On your prior slide, you had those, you were showing the iterative development, the AI analyst developer choosing like this model versus that model or different tuning. Uh, are you, are you, are you in any of these, uh, use cases contemplating like what's the cost, the unit cost of that feature implementation A versus the unit cost?

You know, of feature B in this AB test. Is, is that something that you're contemplating? We don't talk about, I don't have it in this talk track.

We do have those conversations with customers. Okay. Um, and what I've done with some customers when we do workshops and we get into, into it not the install configure, like, okay, let's actually deploy something.

Let's understand your use case. And one of the things we come up is I've started doing graphical, um, uh, interpretations of that. Okay.

Part of that is one of the customers, a senior person and a customer really wanted a 70 billion model. Yes. For whatever reason, I need 70 billion.

Like, okay, that's good. So we showed the server has four GPUs in it. You need these three to run that one model, or you run 13 billion, which the rest of your team wants to run and you can run probably, you know, multiple of them.

So they're running both of them and they're actively testing right now to see like, are we getting the same results? My theory is when you're pairing it with your data mm-hmm. You're more concerned about it interpreting your data than it necessarily knowing all the information itself.

So, so you're, You're not necessarily in the tool, um, facilitating a, a business trade-off conversation, but you're providing enough data for someone to have a business trade-off conversation. Yes. Okay.

Thank You. So I mean, you talking about the conversation history as an evaluation tool, are you using specific valuation tools? Most customers are gonna be using that data in order to test their future.

Now Most customers are using evaluation tools like Arise Galileo. They're using specific evaluation tools for Those at the levels that we're talking to right now. It's more of the collecting information baby steps, uh, in that direction.

Yeah. Um, for future, not necessarily something tied into that. I'm aware of every Client I've visited last year has already got a strategic evaluation tool Arise Galileo or a tool that is built on taking care of that problem.

That's great. I'll have to look into it. I I have, I don't have hands-on experience with that right now.

Okay. Doesn't the LLM maintain a certain level of, uh, conversational history or context over time as you keep prompting It when, uh, when you're doing this in this format, the rag at the front end is, is in charge of how much data it's sending it. There's also a buffer on the LLM of how much it will accept.

Um, and you can limit it on the LLM to only give me so many tokens. But you can also on the front end say, I'm not sending any information or I am sending so much. But I mean, when I typically interact with an LM today, I mean it's, it's starting a, a conversation.

I keep prompting. It's, it's using the information from the prior prompts and the prior replies to understand what the next prompt next reply is. That's Stuff all my experience.

And, and, um, just to be clear here, I could be wrong. All my experience is that's being passed to it from the front end or whatever that chat is. If you were working directly with the LLM itself, you'd be sending an API and if you don't send that history, the LL m's not gonna know it.

The LL m's. Very, um, simple. Uh, it's only gonna say, what are you telling me?

And, and go from there. I gotcha. But we put something in front of it to make it more user friendly.

One of the things I'm trying to get with this story here is that we talk with customers that get, and frankly it's analysis paralysis. They want to know everything before they take one step. And that really slows things down.

So in my opinion, this is a iterative approach. Start with something. We give you something, you click one button, you have something working, you see what's happening.

You figure out what you want to improve. I want to improve this aspect of this great. Go work on that aspect.

You wanna work on that aspect, go do that. You also start breaking off into teams, a data scientist team for an LLM uh, developers for our front end. And this keeps on iterating over.

So we do a scale test, and in that scale test, we have some more people from the company hit it. And what we do to test is we shut down one of the, we shut down, um, we start hitting some performance issues, getting slower responses. So we look at our GPUs and we see that there's some of the GPUs are starting to get heavily used.

So in this scenario, this fourth GPU down, you can see I'm at 99%, but you can also see there are other GPUs that are free. So when I look at this, the team starts doing some research and they start saying, well, we have two options. Build a bigger vm, put more resources into that vm or put, do multiple nodes and have it break out behind it, um, in different things.

Um, so in this scenario, the team decides to go forward, um, and put in, sorry, the team decides to go forward. They do multiple nodes behind a load balancer. Now, a lot of this conversation is also probably the journey from an individual VM to a Kubernetes cluster when you're doing desired state and things like that.

But when we build this out now, when we look at this, you can see we have load balancers in place. Um, and they're controlling access to key points. In this scenario, how Much of this is product and how much of this is professional service?

Um, they guess the comfort level of the customer, um, out of the box. The original one works just out, just as, uh, stated. Nothing special there.

When you start getting into these, it's, um, you can drag and drop load balancers onto a blueprint and then connect them. Or professional services will gladly help you do that kind of things and scale. Um, one of the things with like a Kubernetes cluster out of the box, you deploy it with a single, uh, node, single worker node, single control node inside the ui.

You can choose a quick thing and say, I want to add it to four. It'll then scale up and do that for you. That's just outta the box.

Are you expanding DRS at all to uh, take down GPU load? Yes. DRS Yeah, we are.

Um, it can do it now. So we can balance it on placement, it can move things around for you. Um, and then you can change some of the settings so that way you can be more aggressive.

So one of the things you can do is you can do patching a cluster, um, and you can actually put a host into, uh, maintenance mode, patch that host. It'll move everything off automatically do the normal stuff with the one asterisk, a Triton inference, uh, uh, inference server. Um, they want you to put in a UVM setting, which prevents DRS settings.

So that's the one. Gotcha. While they're bigger servers, they're also, I think, in my opinion, a smaller subset of the AI workloads that are being deployed.

Okay. So this is all post data, so like mm-hmm. So this is all post, uh, data.

Yes. Right. So do you guys have any, and I understand why this is the infrastructure for, you know, receiving the data and doing cool stuff with it, but do you have any recommendations, especially as you start to see the performance of, of what can be done to the data to improve performance?

And is that worked on before you actually get in the middle of a project? So there's, um, there's things that we're working on in future versions will help with the, uh, data ingestion indexing. Um, in this scenario, in data services manager, if you start it off, like most customers with a single node, you can increase that to, uh, three nodes in order to get like a better scale and capability.

So there's some built into there. Um, there's not to my knowledge right now, uh, uh, quick, like, oh, here's an alert, this is what you're hitting, click this button to fix it. But we give you the tools where you'll be able to figure out where the bottleneck is and what the next point is.

I got a question. I, I'm just not understanding why there's load balances on the ingress of a question and the sort of egress from the ranking model. I've not seen that example any.

So in this scenario, What example would you have to have The question the customers on, um, before this front end and we put multiple front ends in order to handle multiple users hitting. Oh, so it's, it Also helps for redundancy. It's the front end.

Okay. It's not really the question, it's the front end. Correct.

Um, yeah, that should, I guess should be a person acting question. And then This is coming out of this, coming into the LLM, so you can have multiple l LMS to handle the bigger load. Um, when we start sizing l LLMs, we're talking about the size of the model, concurrent connections.

Um, and, um, there's something else that I can't remember right now. So when in order to increase the number of GPUs on one VM or you put multiple VMs, that can break the most People using LLMs at that layer to decision layer to decide what LLM to go to. Like mixture of experts.

Yep. You could do that too. And you can add it in.

All right. Okay, Alex? Um, generally speaking, what we're seeing is you start off the simplest way, then you'll go through.

So that portion of this story will just be a few, a few versions now, okay, What's the ranking model? In this Ranking model? I don't know the specifics of it, but it's a basic, uh, ranking model that'll take the data coming outta the Vector db.

So it prioritizes it when it goes to the L it's A K vm. So it's actually taking the replies of the Vector DB and deciding which one to pass onto to the L on You. For instance, versions, like if you has five versions of the same manual, it's gonna meet the most recent, You rank you, you basically just question it ranks.

It says I want 10 back with the best answers, and then you re rank that anyway. That's okay. Models ranking the replies are two different things.

Yeah. So if I come back to this, I, I guarantee it's code. I know.

I mean, it is just, it's just in a Jupyter notebook and code that says K equals 10. I understand. Um, so in this scenario, we have monitoring at the top.

Um, we do another scale test. The scale test, we shut down one of the front servers and when we say do the front servers, we um, we get feedback that the answers are still good, but that we lose our history beforehand. So we research this a little bit and we move the conversational history from being in memory in the front end.

We move it to a database, a shared database. Um, we, and we start working towards that. We also, because we're using this more, um, we're scaling towards putting into production, we add backups in the top right there.

So again, purple for data services manager. This could be the same deployment in data services manager. Um, but they're different databases.

Um, or they could be different deployments depending on which one, The same DSM that comes with VCF. Correct. But in this, so what you're, but you're not bringing the DSM until, I dunno if it's like a time-based build for your slide or you're just trying to break up the, the conversation.

So, um, you started with the lower portion before you added DSM at the top. Correct. Now you're adding DSM because you're at what point in the project where you now we want to do version control of the database.

This is where we're deck database. So two aspects in this talk track. One is just we want to customize it to our own and make it DSM.

Okay. The top part is at introducing the conversational history into a database. Got it.

Instead of in memory. Okay. So before DSM was being used specifically for the vector DB deployment.

Correct. In this instance, we're now comfortable, we wanna now say our conversation history is valuable. Now we bring in DSM again and apply it.

Correct. Okay. And now we detach it from the front end.

So the front ends can be shut down, turned on. They're, they're inconsequential at the end of the day. Got it.

Um, and we start building out that. So the next step here comes to the conversation. We want to get this to production.

What does that look like? So we start having conversations about security. We have conversations about auditing, um, all those different things.

Role-based access control. So this is where we start talking about, at the end of the day, this isn't like some of the other solutions out there that are being built specifically for ai, specifically built for something today. This is the same infrastructure we've been running for two decades.

You have teams that know how to work this. So we have, we work with customers and one of them put us up against a different competitor that's out there. The reason why they went with us is because if they went with us, they didn't feel that they needed to hire new employees or retrain their existing employees to know a different hardware vendor, uh, solution or a different, uh, solution at scale.

They know VCF, they know VMware. They can build and leverage on it. So the delta that we're talking about here is the delta between core VCF that's been running everywhere in your environment, in this AI app that you're running on it, not this entire box that we're now bringing in in order to run this AI app.

Um, now, other, other teams, and this goes back to a little bit of the previous conversation from Justin Murray. Other teams have seen what we've done and they like it. Now they wanna reproduce it.

So what we do is we take this front end and we take the databases, we package it up into a template, and then we provide that to other teams inside the organization. So in this scenario, we now have two different front end rag apps. The RAG app is what's controlling the data that it has access to, and it has its own chat history.

Now, depending on your system, you also have your unique chat history per person, all that kind of stuff. But now we have two different ones here. And I apologize, I think those colors are a little too similar in this, uh, screen.

So let me ask a question. So a rag app, is this like, uh, is this an agent, is this a chat app? Like what is, This is a chat app that's specifically made to interact with your own data and then agents would be injected into this over time in order to do certain things.

So this is just testing the data I have Interacting with the data you have in order to get information back out of it that's up to date. So the LLM that we have in there is only trained to a certain, uh, point in time and maybe not your own data. By doing the rag, we can pull the info from the data without having to retrain the LLM or fine tune it with your custom data.

Okay. So in this scenario though, we have two different teams using their own front end. They can customize it, they want, they can make it look like their own.

But the thing here is they're hitting the same LLM in the back end. That LLM is probably still behind a load balancer. Um, that LLM, that inference server can run multiple models or you have different groups of models, um, running, there's a lot of flexibility here, but we actually have a customer that's doing this for different units and different locations in their company.

They're deploying 16, 17 different front ends, all hitting the same LLM and inference on the back end. What you're Not showing is the, the loading of the rag vector database and those sorts of things, which are kind of backend, uh, aspects of this. And, and here you're saying the the vector database would definitely be different databases versus the same and Correct.

Yeah. Yeah. So in this scenario, they would each go through their own ingestion process for their own data.

So think about it, role-based access control, maybe one's for top secret. Um, you don't want to have full access to it. Yeah.

The other one's public. You can go through that process. And now, because we can't do it independently in the database itself, we separate those out when we are functionally using it at this layer.

So would it be fair, like, so the, so the starting with your first build, we're we're gonna build this simple rag, uh, could, could be like, you know, asking a question of every PDF that's ever been published by a product team. Correct. Now we've progressed, we're now backing up conversations people have had 'cause we saw value in that.

Now we're abstracting even further. And we're actually gonna split it to where, okay, that worked great for this product group A. Now there's another product group they wanna do the same thing.

Is potentially a different group that's represented by this different, uh, second here where it's still gonna interact, but it may have maybe a different vector database that it's attacking? Or is it the same vector database? In this scenario, data services manager could be the same DDSM platform.

Okay. But there are different databases in this scenario. There's multiple ways to do this and you can carve it up a lot of different ways.

Yep. But in this scenario, um, different vector DEB deployed in order to have that separation there. Okay.

Thank you. This is, like I said, we do have conversations with government entities that have fully separated systems. You know, they sit in different rooms, different networks, everything.

This is one of those ways. Now in that scenario that they're not sharing an LLM of the back end. Normal customers though different groups inside, you can share that LLM and kind of build out And you're providing portability.

I think once you've got one and done, you can export that effectively to another organization Or group. Correct. We talked to another, we've talked to a company, super smart people.

They built their own AI system. They love it. They will not give up no matter what.

But it's for a small niche group of people. Because you have to be a little bit of a techie. You have to be a little bit of a, a dork to know these commands to get it to use it.

They don't see it scaling. They're actively working with us right now to look at our system. So when they start rolling out the key aspects that come out of that system, we can roll it out at scale using a very basic front end that has self-service portal, um, um, has quotas, it has policies that has all that built into it that they don't want to put the time and energy and building into it.

So the a view here, everything coming together, everything here is VCF, um, that's sitting inside vCenter. We have our control plane supervisor, namespace, and then we have a bunch of services that are running. And in this scenario, you see Harbor in the bottom.

Right. And now services is where we also dictate the VM classes. Um, VM classes being that the resources we're giving these when we're spinning them up and then those get deployed, deployed into a namespace and that just get built out over time.

I don't see The NVIDIA stack here at All. It's so ingrained in the product. It's just underneath.

No, sorry. Um, yeah, this is all building on that. It's all obviously part of the solution, but these slides here are higher level.

When we start talking about those models that we're running and the, the different parts of that, those could be NIMS easily, uh, running on top of that or something like in the box that we have here. DSMD is gonna be the component that's running from us. Um, a model runtime we could run from Nvidia and NIM and the Triton or A-V-L-L-M open source.

I, I think with this slide, let, let, let me throw out something provocative. Yeah. That y Laun, who's the chief scientist of of Meta said last week, Davos.

Um, and he said, I think the shelf life of the current LLM paradigm is fairly short, probably three to five years. I think within five years nobody in their right mind would use them anymore, at least not as a central component of an AI system. Now we can take that with a grain of salt as being provocative and things like this.

Um, but you have the machinery to help your users evolve. I mean we've, we've seen in the last few weeks changing architectures, right. Divide and conquer things like this.

Can you talk a little bit more just, uh, your vision as things change, as the AI architectures changed, how are you set up to protect and, and uh, shepherd your users through these changes over the next few years? Yeah. I think one of the things that we've made in our decision here is we're the same we've been for the last two decades and the way we've done other things, we are a platform that we're building on and giving people functional options to do things.

Harbor is a huge option for us of being able to have that capability built in to store these models, um, and swap them out quickly. You can easily stand up multiple inference servers, swap out what they're hitting so you can test multiple models at the same time. Go through that, um, process.

Um, both in testing, but also then you start rolling that out, um, to the field. We talked to, um, one customer. This is, um, was surprising to me.

It shouldn't be 'cause I have the experience and I have the knowledge, but I talk to a financial services, a large financial services company. And when they're going through this process, when they get a model that they start using to get answers from, they store that for seven years after everything's done and done. So they then create another project in Harbor when, uh, put the model in there.

When they're done, that project is kinda like locked stays there. So that way in the future they can come back to it. So it's laying out the pieces that all these things are interchangeable.

New model today. Go test it. Now.

Do I personally think you should roll out the model that was announced to, uh, yesterday, um, for your production workloads? No, but we should you be spinning it up and testing it so that you can see what the differences are and see how it benefits you. Yes.

Um, at the end of the day, I'm a pragmatic, you have to have a reason why you're doing what you're doing. So test that model. If it doesn't provide a benefit, move on to keep what you're already using.

Get to the next model that you test. You figure out what those going through. We believe we give you the capabilities here.

That one, take something that's working, make it a template so you can roll it out at scale. Um, and then also have the capabilities of working in changes over time. The story that I just kind of walked you through is only the beginning of the story.

We start getting to agents and we could talk for hours just on all the agents and how you interject those. Um, the, we could probably also go backwards and it's like Star Wars and we do a prequel and sue the whole entire data ingestion service Talk for well, the prequel was VDI. Yeah, yeah.

Prequel was the way that you used BCF to do VI. Correct. I was gonna say, I was gonna comment to, uh, to Bob that it's just a framework and you, you put in, put in the pieces you need in the framework to make it do what you want.

And, uh, the, the VMware cloud or VMware foundation has always been that. And it's a matter of you, you make it do what you need it to do. Yep.

And at the end of the day, VCF automation, if you can script it, do an API call or a plugin, um, you can do anything from this platform. So there's nothing stopping you from growing and expanding out. It's more about setting yourself up for success and choosing a platform that can mer can grow with you and change with you and evolve with you over time versus pigeonholing you into, this is all I want.

I talked to a healthcare company, um, based out of Florida. Um, you know, they have a really great story and they do, um, hospice and they're using some AI tools so that way when they get their files, they can send it to the um, uh, visual LLM scan it to get like the patient name, the the date of birth, all that kinda stuff. And then populate a patient file.

This is significantly faster for them than having a, an actual person get a fax or get something else and scan through it. So it shows them improving the time that they have. Um, for them it's key because their average stay for one of their patients is two weeks.

You don't wanna waste a day doing paperwork and all that other stuff. You want to hit it accelerate going through, they will actively test against four or five different models when they're doing this testing. And then they're getting set onto one that they wanna start rolling out to everybody.

And it just shows that kind of iteration of testing, figuring out what it is, figuring out what brings value to you so that way you can roll it into the, uh, solution. One of the things that they didn't like when they were looking at other pre box solutions was some of these models were very set into, here's the settings. You can't change the temperature, you can't change the model, you can't change this.

And they felt very pigeonholed. And our solution here, it's up to you. You know, we'll give you enough rope, you can uh, you know, cause problems, but at the end of the day, we're trying to make it so it's super flexible for you to get in there, um, make it something you want, and then grow and iterate over time.

So where's the Interface to hugging face? So hugging face is going to be, you're gonna pull those down into a deep learning VM through like, uh, docker commands, GitHub commands, things like that. Um, and then what we see long term, um, for multiple customers, you're gonna pull those down on the side and upload those into Harbor.

And what we're starting to see some customers do is blocking access, um, to those sites directly. And you have to go through Harbor internally. So you have to put in a request for something.

This is where we start talking about, um, there's a reason for, um, there's a reason for cloud. So we start talking about cloud. Um, some cus companies are testing new models in cloud.

When they're happy with them, then they reproduce it on site instead of having to go through that whole entire process of bringing 'em on. Go back to the DSM for a minute. Yeah.

Right. So the database or databases that are inside those DSMs, those are primarily for private corpus? Is that the idea that You, you can use Data Services Manager as the platform for anything that requires the database.

Okay. In this scenario it's usually for hosting data that is, um, related to one of those AI apps that's tends to one, be private and be local. Um, but you could go out and spin up a, a, a database for any reason you have out there for having a database.

Yeah. And The one of my limited to Postgres, it looks like, Uh, Postgres, um, there I can, there's another one out there too. And then we have a few more in the pipeline that are coming down.

But those are generally open source databases then. Okay. And part of it is the pairing.

We're taking the open source databases, adding some enterprise tooling and capabilities to level up level the open source ones versus comparing it to some of the commercially available ones that have a lot of bells and whistles, but then it's hard to come. Yeah. You had Acquired sql, fire, gym, fire, all those things.

Going back to the right. Early Pivotal Kansas.