Mirantis Solution Approach: GPU Cloud in a Box with Shaun O’Meara

I'm Sean Amara, the CTO of Mirandas. With me is, Angelica is gonna help me do demos this morning. So we're gonna run through the product.

We're gonna try and start by giving you a full overview and then breaking it down into the components. Mm-hmm. Um, we'll interleave the demos in between that so you can kind of see what we're talking about at each stage.

Um, let's please dig in. I will try and answer every technical question you can throw at me this morning. Um, Kevin will support me if there's anything else we need.

Right. Alright. So the solution, as we call it, we think of this as GPU cloud in a box.

So you were asking earlier about how do we think about the packaging for our GPU customers. The reality is they need certain extra capabilities that maybe are not necessary with the enterprise. And so we've built a stack of components that we pre integrate, pre validate, um, and this starts in the top left hand corner with what we call a customer service portal.

And it's more than just a customer portal, it's the entry point for the cloud service providers customers into the services that they offer. And we'll go into a lot of depth here, but the thinking is providing those cloud service providers with a set of tools that allows 'em to get really, to get business in the door fast and then change over time. Because the reality is this industry is moving so fast, the big cloud players out there are constantly adding new services.

And we want to help our customers keep up. We have our cordon control plane. Now, this is sort of the key, the meta control plane that runs the entire solution.

It looks after all the infrastructure management provides for the composability layer that gives us this capability that we've been talking about. We've got all of the infrastructure management components, and these are all pluggable composable components that allow us to manage the systems at scale. And I'll go into a bit of depth there.

And then finally, we have a set of pre-canned services that we offer through the customer services portal. The idea is that once again, we go into a cloud service provider, we set up the infrastructure and we support them to have services that they can sell straight away. 'cause you know, we're talking about boxes that cost $450,000.

You know, a single scaling unit is $10 million. With networking, you really want to get that monetized fast. Is uh, is the operator strictly customer side or could that be, uh, morant?

Excellent question. So we don't offer it today as a SaaS service. The control plane solution sits down on the customer infrastructure.

It could live in the cloud. For example, we have one customer who has their entire control plane in cloud managing, managing systems on prints. Sure.

Um, for us it's just part of the composable nature of the solution. I'm, I might, I'm talking about the personnel though. You have a person there as an operator or as operators.

Yeah. I'm wondering if that's part of your PS or, oh, it could be service or what have You. The intention there is that it's actually the customer's operator, but we offer a managed service offering that we can do that on behalf of the customer as well.

It's a choice. The end of the day, as Kevin was saying, depends on how many people you have, their skill base and How much money you have And how much money you have. 'cause you know, yeah.

Alright. Mm-hmm. So let's talk a little bit about the cordon architecture.

And I think this is where we want to get into this whole idea of composability. We're talking about a case here for a new approach to looking at infrastructure management. You know, the traditional approach has been this static, heavily vertical, the integrated stack, um, you know, you have to take my os my management layer, um, my application layer, my PAs layer, and we vertically integrate that.

And the challenge with that type of approach is it's not flexible for the customer. They can't integrate to their existing systems. They can't change that over time.

And it ends up being a lock-in. It ends up being very, very static and quite frankly, brittle. Because every time you need a new feature, you've gotta wait for a vendor for a long period of time to potentially get you that feature.

It's not how any cloud service provider wants to be in a more dynamic environment. We are opting for a more modern approach. So the cordon framework gives us this ability to create operators, to extend the platform in real time and for either us to own those extensions or the customer's technical people to own those extensions.

Um, a really good example of that is we have a customer today who has a custom OpenStack deployment with a custom, um, API interface to that. None of the common platforms out there can talk to it. We were able to work with them to build a custom provider that allows them to orchestrate on top of that platform.

So we are able to plug in a different layers of the stack. What's key for this as well is you can start to leverage what you have within the environment. So it's not coming in and saying you have to start from scratch, rip everything else out.

We can integrate to what you have. Oh, for instance, the model catalog. I mean, there's at least four or five different versions of that today, hugging space, et cetera, that that exists.

I mean, you talk about ML flow, but there's at least five or six different versions mm-hmm. Of that cube flow, et cetera, et cetera. Yep.

I mean, are all, all things that, uh, your customers can plug into Cordon? Is that how you see that working? I mean, Yeah.

So we, we have this idea, and we'll get into the details of, we have this idea of a service template. So we have the whole, well I'll get into the detail in a moment, but the whole system is built up around this idea of templates. And those templates are layered on to each other to create the end service.

So we could have available a cube flow. In fact, we do, we'll talk about that in a moment. Uh, we could have animal flow service, you could have run ai, depends on what you're trying to achieve within your Environment.

Data, the, the, all those server, All of those things can be layered up. Now we have our own opinionated secure registry. We're into secure registry built on Harbor, but hardened, packaged and supported by us.

But if you want to use Artifactory, it's fine. We'll help you package and deploy Artifactory in the environment. Um, again, we want to work with the best of breed vendors in the industry to provide the end customers with the services they need.

Not trying dictate a narrow approach. A lot of time, those sorts of solutions are, you know, almost stacked wide, right? They're not like point solutions at a particular level, but more Nvidia.

I mean, Nvidia spans the whole damn stack For sure. But Nvidia also doesn't do multi-tenancy. IAS if you look at their NCP architecture, there's a big gray box in the middle that says vendor's problem, you know, cloud providers problem.

Um, I, I guess what I'm saying is that it, it's whether you go with ML flow or cube flow is a major discussion. It is. It's not a se it's not a a trivial template oriented, uh, choice.

In my mind. It's The choice of which one to go with upfront is not a trivial question. I agree with you.

But managing that, once you have made a decision, the lifecycle of that at scale repeatedly is a hard problem to solve without having the tooling in place. And that's how we're looking at it. Does that make more sense for the way we're approaching this?

Okay. So what is accordant? Accordant is ultimately a common control plane.

We sometimes refer to it as the meta control plane for infrastructure. It is Kubernetes based. There was a question earlier.

So we're leveraging Kubernetes essentially as an agent that allows us to run and manage infrastructure because Kubernetes workers run pretty much absolutely everywhere. Mm-hmm. Um, the operator model that is inherent to Kubernetes allows us to create this operator provider model.

So we essentially create a set of contracts of expectations for the infrastructure, and then we plug in providers to actually deliver on those solutions. So I can come into an environment that has Nvidia cumulus switches and maybe some Cisco switches, and I can plug in separate providers to manage both that networking layer, both of those networking layers. But from a consistent end user, API, it's open source at its heart, it says core there, but at its heart, it's open source.

The entire cordon project is open source. The only thing that we don't publish is the templates. Mm-hmm.

You know, because those are unique to each customer, each deployment, but everything around them is open source. And we have to harden that open source. We have to manage that open source, and we have to package that in a way that covers all of those FedRAMP and governance requirements.

So we also offer this packaged version of it. Um, as part of our offering, Accordant also supports a regional deployment model. So essentially you have what we call the mothership controller.

We'll touch on that in a moment. But then I can deploy regional controllers in different network segments at different sites in different clouds to extend the reach of the control plane. So not everything, all your eggs in one basket scenario, we focus heavily on the enterprise grade.

There was a question about RAC earlier. It's very easy to take a collection of open source things and make them work. Where the real skill comes in is adding the enterprise grade RAC or proper separations of duties, understanding how do I make it multi-tenant?

How do I support multi-tenancy in the environment? So what we've done is we've built a model as part of accordance that leverages the Kubernetes ecosystem and the RAC model so that we can create multiple role-based controls. We obviously ship them outta the box, but they're customizable, um, to control what tenants can do as well as what operators can do because it's quite important to control what operators can do because you don't want everybody to have full admin rights because of the finger trouble.

Um, air gap deployments, we spoke about that. We support extensive air gap deployments of both accordant core and a selected number of things outta the service catalog. So our customers can actually get the whole service catalog downloaded to a local site, run it locally, and never have to go outside to get access to the service catalog.

The challenge with doing that the right way, which is when we've solved for, is how do we handle upgrades, updates and lifecycle management of all those components. Mm-hmm. That's a critical part of this.

And Kevin spoke about all the work we're doing with Stig FIPs certified governance components, uh, working with customers who are FedRAMP certified. Obviously we're FedRAMP ready, but that's the no certification for that. We have to certify the FedRAMP certification in the deployments.

And so we work with them to do that. But we have the code ready to do that. We've spoken a bit about this, but we have a number of extensive integrations.

We support bare metal deployment. So this is kinda getting to the heart of how this works. We can deploy from empty network undeployed, no operating systems on the systems, all the way up the stack through the virtualization layer and Kubernetes on top.

And when we talk about multi-tenancy, I, I'll put up an eye chart that explains where all the tenancy layers are in a moment. And I've spoken about the easy extensibility. We have a virtualization layer.

Um, and then the last thing I really want to talk about is this idea of entertained automation. Kubernetes is key thing is this whole idea of reconciliation. Mm-hmm.

So we're leveraging that idea of reconciliation to provide higher resiliency within the running environments because we're able to constantly reconcile environments. If we have a failure of a node, we can leverage the solution to actually auto recover from failures. And then we built that all of that capability into the templating model as well.

So we're starting to look at a cloud and compute environment more as a declarative solution. You declare what you want, the reconciliation then manages that, and we use the observability and the capabilities of that to verify the environment as we go. Constant verification.

So even if you have a bad actor who comes in and makes a change on an edge component, we're gonna catch that change and force it back to the expected state or alert that it can't. Yeah. So I have a question about how you actually, uh, document that and archive that.

So, uh, you, you have a configuration for a particular, um, cluster or, uh, you know, end user machine or whatever. Um, how do you actually, uh, document the, the inventory of what's in there and do you actually have like a source code control system Yes. That will allow you to version?

So I'll touch on that in a second when I describe the whole templating and, um, the model of the way the deployments work, but essentially the cluster templates are held as immutable template objects in a source repository in a get in A OCI repository. They're then pulled into the system for the deployments. The actual deployment is held to CRD object within Kubernetes.

So we can then watch that C CR object that's in part of A CRD within Kubernetes, and we can watch that for change. Um, and then we have hooks that'll alert us if one of those change, because that's actually how the system operates. What do you do?

You also version it. So, so like if you upgrade some utilities that are in there, you will have a new version of the template and you can potentially roll back at any point as well. Yeah.

So I, I'm not gonna go into that level of detail here today, but we have a concept called template chaining, which is one of the functions of the system. You can't actually upgrade a template or replace a template unless you've been given permission within the chain to make that change. Um, it it at a, you know, at a command line level, it's relatively complex, but we simplify that through the ui, um, to make the user experience a little bit easier.

But the real power of this comes in when you start really getting to scale and you start adding on third party flow management systems or outflow management solution, you interact with those CR objects, which gives you that scaling capability and you could use a GI UPS model for it. Um, but we, we do that through our flow engine. Um, the idea that we track and trace that using Kubernetes native tools.

So a question on the, the templating process. I think a, a minute ago you said, um, bring your, bring your own templates, right? Uh, what language is preferential yaml?

It's all yaml. It's all Yaml, right? All Yaml, all yaml all the way down, which has its own complexities, right?

Um, we are gonna be bringing more tools to market actually AI tools to the solution to do a lot of that work, help generate on your behalf and all the validation on our behalf. We actually have some tools in progress right now, which are showing a lot of promise. Mm-hmm.

Uh, the real challenge is the way models work and, you know, hallucinations and things. Um, but we are gonna be bringing those to market fairly soon. Okay.

In the meantime, you need a certain skill level of understanding of the system to create those. Mm-hmm. But we have it all documented and there's a lot of great examples available.

Um, and we have a library of community offered, um, templates that people are providing to, to provide to, well If, if you're coming from existing hail management or something like that, then You're, I mean, if you have some basic Kubernetes knowledge, you can work your way through this. Yeah. And then, you know, Kevin mentioned lens, earlier lens just makes that whole experience a whole lot easier, um, for the Kubernetes operator.

Alright. So very, very, Can I, sorry, of course. Um, the, the, on the previous slide you talked about, um, extensive integration.

What's the integration paradigm? How are you, how are you doing this? Because clearly there could be, There are thousands of ways.

So There are thousands of ways. And, and and each customer brings their own Yeah. And bring their own particular, So there, the, the integration points are done through that operator provider model.

So we have these operators. So one of the operators we leverage is an open source thing called capi, A cluster, API, and that's how we do a lot of our cluster management. CAPI essentially has this idea of contracts.

So we create a contract for the resources we want to create. We then have providers that plug into the CAPI ecosystem that then deliver on those. Hold on.

You said we have providers. We tis have, we have provider partners. We, if youre not a TI provider partner, A lot of them are open source.

So we leverage a lot of the open source capabilities. So we have, let's use some, a couple of examples. AWS uh, Azure, Google, uh, currently we support on-premise for OpenStack.

Of course, in fact, we kind of are very heavily contributing to that. Um, VMware, um, we have others coming which are being worked on, but our customers are also creating their own custom providers. Um, I mentioned an example earlier of a large banking customer that we've supported to create their own custom provider.

And the system allows me to plug in those providers in a very controlled way. Um, networking is the same. We have an operator for networking.

We allow security. Security is is a effort of the overall system. Yes.

So this idea of having a standardized set of contracts is incredibly important so that when you're creating, you're declaring what you want, you're not declaring how you're going to get it. Mm-hmm. The providers, the plugins are the how.

And so that's very easy to manage multiple providers and we can work with partners or we can create on behalf of the customer, additional providers. We're covering quite a broad ecosystem already and we're have plans to cover even more over time. So you can't really eliminate the fussing that has to go on, uh, um, in a lot of cases, especially what you described earlier as these broad, very diverse states, You know?

No, I mean, you, you have to at some point do some level of integration. Mm-hmm. But you've also, there is a payoff.

Anything like this, there's a payoff. And I know you're going into deep technical ery with me, but ultimately the payoff is, does the automation and the simplicity of automation at scale outweigh maybe not having access to every possible minutia application? Well, I could see this driving towards a certain amount of consistency and standardization on the customer sizes Point.

But the beauty of the provider models, because each of these providers is an open source element. The whole community can contribute to them and as part of a broader standard community already. Um, so that's, that's what we're driving forward to here.

Um, and we have a lot of those skills in house. We're partnering with a lot of people, a lot of similar organizations that are trying this and our customers. So it really helps us to move this thing along quite fast.

Um, and we'll talk more about open source, but open source is one of the key things that really allows us to move fast in this industry. So very quickly, the key building blocks of what accordant is today is from the bottom up on the left, we've got our cluster manager. This is the operator and tool set that really controls the whole templating delivery and security model for accordant.

We've got our state manager, this is what we use for doing service management, um, and state management of the eventual clusters. So if you think about cluster managers being infrastructure up to the Kubernetes, API layer state management is API layer up. Um, and we've got some capabilities coming as well, which will even be able to deploy virtual machines, not just Kubernetes clusters.

Um, and then we've got our observability in fops layer, uh, which is an integrated layer that does full observability and fops reporting within the fully deployed environments. Um, all fully integrated. Now I'm gonna move a little faster because, um, let's talk about that templating architecture very, very quickly and then we'll move on.

So I kind of spoken about this already, but essentially what we have is we have libraries of templates today we have three primary templates. Again, I'll move from the bottom up. The provider templates are the templates that determine the configuration of accordant itself.

And this is how we add in customizations, how we, uh, extend the system and create extra capabilities. Okay. The services templates is what we were talking about earlier.

This is how we deliver capabilities for the end users that are no longer infrastructure, but actual applications. We call them the beachhead services. For a Kubernetes environment.

You know, Kubernetes is wonderful, but on its own, it's a glorified scheduler. It becomes useful when I add C-N-I-C-S-I and other capabilities on top of it. This is what we use the service templates for.

And then the cluster templates are actually defining the infrastructure. Those combine with those CR objects I mentioned, which is the config, and then the operators pick those up and actually go and deploy that infrastructure. That's it in a nutshell.

Um, any questions on that before I can step forward into this? Alright, let's talk about the CSP experience. So the CSP experiences, how do, as I said earlier, how do I enable service providers to sell a service fast?

So the key thing here is we have three parts to this. We have an end user portal. This is how the end user will interact with this environment.

It's the entry point. The key thing here is we're providing a framework with the expectation that the cloud service provider is going to wrap that in their own CSS, their own look and feel. But it's essentially an easy way for the end user to come along, click a button and get a service.

We've got our product builder. Product builder is a flow engine that allows us to create very complex product skews. Maybe if you wanted to look at them that way.

And this is a combination of yes, a piece of infrastructure. So a request for a piece of infrastructure may be an email confirmation, may be a connection to a billing system or an inventory system. But essentially it's a complex set of flows that are needed to actually create a service that's delivered.

'cause services are not just a piece of infrastructure. There's a whole lot more that goes into that within an environment. And then we've got a framework that allows us to create these plugins that allow us to build this, um, environment.

So just kind of putting that into a picture, if you can imagine each of these products is built up of a set selection of building blocks. Those building blocks then create a ultimate product, which presents on the customer portal, and the customer comes along and clicks a button, puts in a few inputs of information, and they get a service outta that at the end of the day. Um, one of the examples which we'll demonstrate shortly is a cluster customer coming in, clicking a button to get a Kubernetes cluster and the whole life cycle of that.

Mm-hmm. All right. So Angelica's gonna walk us through this demo.

Hi everyone, my name is Angelica and I'm with ti. So a challenge that we discussed that CSPs often face while developing a cloud console is the need of a more streamlined way of efficiently designing and launching new services at the drop of a hat, while also making these services easily accessible to their customers through a shared multi-tenant portal. During this demo, we'll go over two components of our solution.

First, as Sean discussed, we have the customer portal, which would serve as the entry point for the CSPs customers with the aim to deliver a simple intuitive experience for ordering and deploying services such as GPU enabled Kubernetes clusters. By using the portal, customers can white label the products by integrating their own branding. Secondly, the product builder, a tool for service providers to quickly create new products by combining reusable components.

Similar to building blocks all from an extensible catalog. We'll start with what the customer experience looks like. So first, here's the product builder.

The goal is that customers can use the product builder to quickly add new services without having to write the code. Let's demonstrate how we can build a product with the product builder. Okay, back to this.

We're going to define an example AWS cluster product and select the AP south region. And then the next step would be to configure our cluster parameters. So as you can see, cluster customers can easily slide to set the number of control, plane nodes and worker notes.

And then once all of our requirements are met, deploying your service is as simple as clicking a button. Once you deploy, this popup will appear. So customers contract deployment status, and then we'll let that fast forward.

Alright. Once your deployment is complete, customers are offered the cube config. With the cube config available, your cluster product is ready for use.

And let's see what it looks like in the marketplace. And just like that, we have our example product available under active deployments. Now that we've covered the customer's experience building products, let's take a look at the operator's experience.

So in this demo we're going to create another example, AWS cluster product and take later to take a look at two views available. We have the card view and we have the graph view. So to start off, we have basic product details, an internal name for the product, and a display name for the customers.

Additional configs can be implemented such as how the component gets its input values, whether that be customer input or a fixed value. And then for greater flexibility, we offer the graph view, which enables better visualization of the product we're building. Graph view depicts each component with inputs and outputs like nodes in a network.

This view allows operators to define dependencies and interactions. Components can be easily added and connected. For example, I'll take this email notification component by dragging it to the canvas and connect it to our cluster component.

By connecting these two nodes, I'm adding an email notification to be sent out once the cluster is ready. And then now I'll add the Grafana dashboard registration component to our product as well. This is dependent on our user input and it will register the cluster name in Grafana recap.

So we have on screen, we have our node, which is our AWS cluster product that is being set to be dependent on user input. This node allows customers to set the name and number of worker nodes, but region and control plane nodes are fixed values. Once the cluster is ready, the email notification component knows to send out the message.

And then lastly, we have a node to register with Grafana. This node enables Grafana to run immediately after registration. So once we have our components set, we can now just click save product.

And just like that, we have successfully created a product, the customer portal and product builder empowers CSPs to design, launch, and deliver services. This solves the challenge of getting new services to market fast. Thank you, Angelica.

So do you really think your composability is a differentiator? I mean, for the most part, any of the past people we've talked to in the past all talk about, you know, services that can be deployed upon their system to that extent. I'm, I'm just trying to understand where composability, uh, is different than any of the other paths suppliers that exist out there.

It's a great question. So I think it's the way we're doing it more than the fact that it is there. The fact that it's configurable, it's all templated, it's all in template code.

Even this, um, product builder portal that you've had a look at is made up of a number of components that can then be added and, and changed over time. A lot of, a lot of the rest of the industry is doing this directly in code. Right.

Um, it's hard coded code. I mean, to, well, Well to to add on to that. I mean, it looks like you're building a graph, But we are building a graph.

It is a DAG based solution underneath the hood. Correct. Gotcha.

That's what I wanted to get to. Yeah. I was like, obviously you're doing ized yaml, we get that, but you're tying that to a relational Graph.

Yeah. So what, this is the top, very top layer of the stack mm-hmm. That we've just shown you.

Now, this is the, the cloud service provider tooling that's actually separate from the infrastructure components and it's made up. So, so I'll come back to what it's made up of. It's made up of a set of plugins that the graph directs, um, those plugins, those interfaces and those plugins actually using MCP.

Mm-hmm. You know, we're, we've selected to use the MCP standard as the, as the interface between those plugins. Um, so you could also start to plug in intelligent modules as part of that.

Over time, once you've created those graphs, they can be adjusted for your particular business' needs. Now a lot of the, a lot of the solutions that are on the market today have flows great, but they're in code. Mm-hmm.

To change them, you have to go and change the code. This is actually designed so we don't even need someone who understands the code to go in and make those changes. So we can have business people making the product change and talking to the infrastructure.

You can start to create much more complex services. And I think the key thing to understand here is if I'm a cloud service provider and I'm sitting here and I'm selling a service today, my service is not just spin up a Kubernetes cluster. It, that's a really critical point.

It's not just spin up a Kubernetes cluster or, you know, in the AI world, take a single scaling unit, stick a cluster of CHOP across that stick slum on top of that. I have to do this all dynamically. It is registering with the inventory, checking the inventory, understanding is that, does that user have quota, um, across multiple diverse different systems potentially.

And if you try and create this intense integration at the very low layers of the stack, you're creating so much complexity from a management point of view. But if we move it up the stack and start to have it as distinct building blocks that create a service at the end, hence the DAG model. Now we're providing something that we believe is incredibly unique to the service provider.

There are tools out there, there are big players out there that do some of this, but again, it's very much forcing you into a pattern. We're doing this in an open source way. And so do you have a way, I guess in your templatization of, of what's happening underneath to expose different edges to each of those blocks?

Right. I can say strictly tight an edge. So if you saw, lemme see if I can quickly go back, um, to this.

If you have a look here, all this on the, if you look at these edges, um, each of these little blue entities is an input or an output to the block. Mm-hmm. Yeah.

Um, it's an MCP entry, a typed entry within the MCP protocol basically. Um, so depending on the building blocks capabilities and how you've defined it mm-hmm. Um, and what it's talking to, then you'll have a, a better understanding of that.

Each of these building blocks, um, uses a set of tools that actually talk to Kubernetes operators to do the functions in the background. Uh, we have a tool, which I think is actually in the next, Uh, That's Yeah, it is. Yeah.

Uh, no it's not yet. But we have, uh, a view of this which has, um, a tool called KTL. It was actually in that original drawing.

This is the tool that we use for managing the communication between the flow engine and the operators that do the work. Um, and it, it leverages, it's essentially an agent fall into, it's an, it's a MCP based agent for talking to Kubernetes infrastructure. So, um, it's, we're we're leaning into the whole AgTech model practice that's happening within the industry today.

But you've gotta managing infrastructure, You've gotta give it the right data model, right? Correct. That's the thing.

Right? That's What's critical. Feed it 75,000 YAML files, it's not gonna know what to Do Precisely.

Um, and YAML is a definition language at the end of the day, the way we use YAML is different in different layers of the stack. So, we'll, we'll unpack that in a second, but this is that CSP layer at the infra layer, which is what I started with, with ent. That's using very standard Kubernetes CR language.

The more things are in a standard definition layer, the more people can start to interact with them easily, rather than something that's in multiple different programming languages. Um, which allows us to abstract, but also really importantly, we can capture a state and that state can be repeated very, very easily. And that's a critical function when you start building infrastructure at scale.

Um, this is what's happening underneath the hood at the large service providers today. And we're trying to give those types of tools to the smaller service providers so that they can actually be competitive. So they don't need a hundred developers to manage just their front end.

'cause that's an expensive resource that could be better utilized elsewhere. Hmm.