Building the Sovereign AI Stack: SUSE AI Factory with NVIDIA

Hey everyone, we're back here at SUSECON after lunch, day one. It was a great lunch, by the way, I should mention that. But the place is full with people.

They said there's about 1,500 people here. Every bit of it, I would say, if not more. But let me introduce you to my next guest.

His name is Sanjit Singh. He is with SUSE and I'm going to let him introduce himself. Sanjit, first of all, welcome, and thanks for coming on.

Secondly, if you wouldn't mind, tell the people, I gave them your name, and I said you're with SUSE, but let's talk about your role and what do you do? Of course. Hi everybody.

My name is Sanjit Singh, and I am the senior director for our AI product partnerships. I work with pretty much all our main partners, NVIDIA, AMD, Dell, HPE, and a number of other strategic partners as we are building a new set of solutions targeted at the AI market. In fact, one of the things that we launched this morning was the SUSE AI Factory with NVIDIA.

That was something that we had been working with our partners at NVIDIA for some time now, and it came to fruition this morning. Fantastic. Most of the people who are watching this are not here.

That is true. Because they'd be here instead of watching it. So they may not know about the announcement and the particulars of it.

If you wouldn't mind, Sanjit, in your own words, describe what this partnership's about. Yeah. Of course, we wish all of you who could've been here, because Prague is an amazing city, and it's been a great conference.

So let me talk a little bit about this new announcement. This morning we announced our SUSE AI Factory with NVIDIA as one of the prime solutions targeted at digital sovereignty for deploying AI solutions. This solution is targeted for enterprises that are building their own sovereign AI stack.

Think of an enterprise customer that wants to deploy a private AI where they want to keep their data within their own control. This is the solution that brings the best of both the partners, us and with NVIDIA, into a single sovereign stack. What we have done is we have embedded NVIDIA AI Enterprise as a part of our solution, and we give it out to our customers.

And the key thing is that we are bringing this to our customer base. We will be providing support for them. We will be providing that sovereign stack that the customer needs for their AI solutions.

I love it. Here's a thing. Sovereignty is obviously a big theme here at SUSECON.

Mm-hmm. People think of digital sovereignty, I think a lot of people initially think of, "Well, I want my own cloud platform. " But the sovereignty issue extends beyond your cloud platform, as is the case here.

Yes. Where we're talking about sovereign AI. And now we think about that, we have our frontier models, right?

But basically, it's kind of dominated by US and/or Chinese models. And I think part of what we'll see coming out of this sovereignty movement is new models. Yeah.

I don't know if you call them LLMs, SLMs. I don't care what you call them, quite frankly. Yeah.

But they'll be based locally. They may be private to an individual company. How far off are we from seeing that really become mainstream?

So let's first go back and understand sovereignty in the AI space. In the AI space, it's actually not just even about country. It gets even more than- Granular ...

more than that. More granular than that. Exactly.

Imagine a large bank based out of the US. They do not want their data to leak out to any other bank or any other customer out there. Now, all these major frontier labs, they provide guarantees that we will not take your data and leak it out.

But what about the fingerprint of the data? Will they go ahead and learn from your data and then apply that information somewhere else? Well, that's AI.

AI is based on that, right? So pretty much all of our large enterprise customers have started looking at this and saying, "You know what? " So sovereignty in the AI space has taken on a new definition where it's not just about region, not about locality, but also about keeping the data under control.

So that was point number one, the definition of sovereignty. But when you go to the next level, customers are also worried about just the supply chain of software. We are seeing some of the latest headlines.

Mythos is the new AI model, there are claims that it can hack through and find CVEs in any kind of platform out there today. So customers are starting to get more concerned about making sure that the software supply chain is very well validated. They understand who's writing the software, where it is coming from.

Is it from a trusted source or not? We are seeing that not only in some of the more secure, sovereign type of use cases, but across even other customers. A public organization, like a defense organization of a country, they definitely want to make sure that they are not exposed to any kind of hacks like this.

So this is becoming a very important consideration for them. As a part of SUSE AI Factory, what we are launching is this concept ofMaking a factory out of the software. What we are trying to do, we are trying to bridge two or three different gaps that enterprises have for their AI solutions.

Gap number one is probably around persona gap. You have many different personas that touch a full end-to-end AI production, AI application production, all the way from a developer to a AI engineer that wants to deploy something in a click ops model, to a scale-out engineer who wants to, using GitOps methods, be able to deploy a large-scale production AI environment. So that was gap number one that we tried to solve.

Gap number two was around operations. Think about a telco which has a data center with a couple of thousand nodes, and a few hundred or maybe even 1,000 telco edge nodes, which are deployed across the board. How do they operationalize their AI infrastructure across the board?

So the operations gap was the second one. And then, of course, the whole sovereignty and keeping data under control was the third thing that we tried to solve for in this entire solution. Excellent.

Now, our audience out here, Sanjeet, is saying NVIDIA made another partnership. There's a lot of partnerships. What's unique about this?

So, there are two or three different things that we decided to work together with our friends at NVIDIA. We have been in partnership with them for a long time, trying to build that stack, getting the drivers in there, getting all the underlying hardware enablement done. But with this new partnership, what we are doing is we are bringing NVIDIA AI Enterprise, the best of their software, down to our customers.

In fact, we are reselling, we are embedding NVIDIA AI Enterprise as a part of our solution and taking it to our customer base. What that means is that behind the scenes, we are taking care of all the validation, we are taking care of all the issue resolution before customer actually sees it. Enterprises are struggling to turn AI into production grade.

In fact, one of the recent statistics that I read was over half of the deployments fail to go from POC to a production because of these types of reasons. Sure. So working with NVIDIA, we are trying to overcome the gap of getting things into production.

That is the key reason why we decided to do this. So what's unique about this is the fact that we, working with NVIDIA, are able to cover everything from the developer up to the scale out production that a large enterprise might require. I love it.

Oh, just I thought we were getting a message there. We weren't. When is this available?

So this was announced this morning, and we aim to have it available within this calendar quarter. So our goal is to launch this within 60 days. By June.

I was going to say by the end of June. By end of June, early July is what we are targeting. That would be great.

Yes. That would be great. Now, let's talk last kind of area I wanted to delve into, the open source nature of it, right?

SUSE is all about open infrastructure, open source. NVIDIA has also, maybe more recently than SUSE, which is in your DNA, but NVIDIA's embraced open standards, open source with CUDA and with some of the things they're doing. To me, that's the real distinguishing factor here, too, is the openness of this.

Talk to us about that. Yeah. Well said, right.

NVIDIA has been a very strong open source partner for us for the longest time. We have been working with NVIDIA, open sourcing their drivers since the past six, seven years that I have at least been in this space. But yes, you're right that NVIDIA AI Enterprise's entire components are open source because they, just like us, we understand the power of open source.

We enable our customers to deploy open source solutions for AI use cases, right? Because we want customers to be able to trust us, understand what exactly is their stack that is delivering. And if you look at the AI space, there are two separate distinctions that have to be really called out for open source.

There is the open source software, which allows for secure supply chain, trusted supply chain, customers being able to sort of understand what is happening within their infrastructure. But it also it's about open source models. And NVIDIA works with us to create these open source models.

Open source models are also very important in this space because you can then, to be able to deploy in a private environment, you truly know that you can keep your data under control. You can lock it down. We have customers that are deploying these open source models with open source software in an air gap solution.

They know for a fact that their data will never leak out, their information will never be given out to another competitor. So us working with NVIDIA, we have sort of fully brought these end-to-end open source solutions to the market, to our enterprise customers who are now deploying them in their own data centers and their own private clouds, if you will. Yeah.

I think we'll see more of that. Absolutely. Sanjeet, we're about out of time.

This 15-minute interview- Yes ... they go quick. But congratulations.

Look, NVIDIA is certainly the leader of the pack in open AI, right? A lot of it flows from them. And this is, I think, a great feather in SUSE's cap to have this relationship, this partnership at the forefront here, especially in light of the whole sovereign kind of movement that we're seeing really front and center here.

All right. Perfect. Thank you so much.

Appreciate the time. Keep up the great work. All right.

Sanjeet Singh from SUSE here talking about the NVIDIA partnership announced at SUSECON today. ai on this if you'd like to go check it out. We'll be back.