Simplifying AI Development with Alex Yeh | KubeCon SLC 2024

This is Textron tv. com. Here we're live at CubeCon in Salt Lake City, where it's been snowing.

I've got a chance to break out all that winter, uh, clothing that I've had packed away in my closet. But we're here and we're interviewing people and meeting people. It's a great cube con.

I think they're expecting probably the largest cube con in North America to date. I want to introduce you to my friend Alex Yee. Alex is here.

He is the CEO of a company called gm. I Cloud. GMI Cloud.

Cloud. Yeah. And if you, if you missed it, I, I actually did an interview with Alex about two weeks ago, um, on regular tech strong tv, but we took advantage of us both being in the same place this week to, to catch up again.

So, first of all, Alex, welcome. Is this the first time you've been to Cube Con or, Uh, to Cube Con also Utah. And is this the first snow, uh, of, of the year for me?

Is it Good? Yeah. Well, I, I don't wanna embarrass anyone but one of our camera people, this is the first time he's seen snow coming down, too.

Oh, okay. He was very excited for it. Um, but I, I've actually been to Salt Lake City before, but it's a great cube con.

You could tell how big the absolutely showcase floor is, but there's also some great content. Hmm. Alex, for people who maybe didn't see our first text on TV interview and are not familiar with GMI cloud, give us kind of the short version.

Yeah. So first of all, GMI stands for a general machine intelligence. We're here to build the a GI future.

So does that mean Right? We wanna build, uh, a cloud or a AI platform that allows anyone, or any engineers, or any founders to build an AI application instantly, right? So think, uh, I would say Wix or Squarespace for ai, we wanna make it seamless, make it easy, abstract all the difficulties on deploying ai, uh, including infrastructure to model selection, to data ingestion, migration, all of that under one view, one panel.

You know what, one of the things I got out of our last interview was, look, there's gonna be plenty of infrastructure providers, hardware providers, right? That's right. But we're gonna offer you sort of, let's call it GPU as a service.

But that's not the GMI model. No, yeah, it's not pure GPU as a service. It's actually, there's prepackaged AI applications in here that, or maybe not applications, let's call 'em AI building blocks that allow you to build your application really rapidly.

It's almost, you know, rapid application development for AI apps. Correct. Uh, on state-of-the-art GPUs as well.

And that's, you know, it's kind of a unique kind of offering right now in the market, I would say. So, so I think here's what I, what I see, like, you, you, you are absolutely right. So the current state of the market is people are just selling boxes, essentially.

Yeah. Here's, here's some GPUs. If they break, we'll try to solve it.

And, uh, please don't come, come, come, come back to us if Uhhuh if it breaks that, that's kind of the mentality. And then if you don't, if you're not, uh, ordering, for example, this is pro for hyperscalers, right? If you're not ordering in a large quantity of say, say 10, 20 million bucks per year of cloud spend, then we're not even giving you a, I would say, a, uh, a account manager to support you and not to mention adding the features that, that you want.

Um, where we do differently is we want to offer a service. I think, I think GPUs are cool and, but people, well, people come to us, uh, buying GPUs, not because they think GPUs are cool, it's just buy for fun. Right?

But they're, but maybe for fun, but to build a solution, well, To, to experiment at this between a lot of them, To, it's to to experiment, to build a solution. And we're here to help them to build that solution. So there's founders who are building biotech, founders who are consolidating all the weather patterns together and predict, I would say agriculture yields or, uh, predict, I would say the, the, the next, uh, potential disasters with disaster recovery, all all that stuff, right?

And we want to enable them to build that. And it's, we see that it's so difficult to build an AI application. And so we, we just want to enable that and make, make their life much easier.

So instead of hiring 20, 30 people, including infrastructure, including, uh, ML engineers, including full stack engineers, you just hire data scientists, and then we handle all the difficulty, all the infrastructure. Uh, So I mean, obviously you think this, but this, this is the model for a new age going forward. I think so, right.

Don't, don't spend money trying to find difficult to find folks who, who, who play in that space, right? Yeah. Or as you mentioned, the full stack people, the the infrastructure folks, all of that.

Mm. Let's, um, and I just wanna make sure before we go further, mm. ai.

That's right. Okay. Wanna make sure people get that all?

Let's turn to CubeCon a little bit. Yeah. Look, cloud native has become sort of the, the compute stack of choice.

That's Right. Right. Especially Greenfield.

If you're building a new application on Brownfield, I think something like 80% will Everyone's building on on right. On Kubernetes containers, micro services. How is what you are doing at GMI work with that, play with that.

Uh, So we actually offer a lot more flexibility. So, uh, I gotta say two things. Go ahead.

One, we can connect there on-prem. If they wanna choose on-prem or, uh, our public cloud or hybrid, we can offer those solutions for them. And we have our Kubernetes, uh, man kuber managed Kubernetes environment sitting on top.

So we built a orchestration software, um, that's built on OpenStack and Kubernetes that we can decouple from our GP services. And so for companies that already bought the hardware, uh, and bare metal, frankly is a bit difficult to use. Mm-Hmm.

So we actually can license that entire stack to them so that they have a, call it AWS on the go and, and for their on-prem services. Yeah. So those are, I guess, one differences, but aside from, uh, using, uh, K eights, we actually can offer people, uh, manage, I would say bare metal to vm, uh, HPC slum container and ate.

So we have that option for The Whole spectrum. Exactly. Exactly.

So it's a, I would say it's a grander and larger architecture that allows people to have really, uh, high flexibilities on the type of workloads they want, because some people want route access, right. And then to, to, to, for, for their workloads, and they have their orchestration then, you know, we want, we don't wanna interfere with that. So, I mean, you as a service provider here, you want to give people ultimate flexibility, cover any option they may want.

Looking in your crystal ball though, do you see cloud native being the dominant compute stack here for, for AI apps going forward? Or do you think it's gonna be, I, I think so. I mean, if you think about GCP or Azure or, or AWS, how, how they, uh, grow to the size.

I mean, if you think about what these clouds are, they're essentially a wrapper on serv servers. Mm-Hmm. Right.

And then if you call AWS, like, oh, so they're just selling storage essentially. I think that's a, that's, that's not fair to trade to them, but that is the, the current state of the market is the GPUs. Can you imagine, like right now the state is actually, you can go to brokers and say, Hey, I need, uh, say one year of GPUs, and they'll just find you that, that I would say basically EC2, right?

Somewhere, somewhere from a random vendor. Right. It, it's unimaginable to, to, uh, I would say the tra traditional world.

And so I think that will consolidate, move to, uh, I would say a, a cloud environment, but frankly for last year or so, it's just because the supply chain is so constrained, right? All you have to do is just like, Hey, I got, I got a thousand boxes, so rent from me. Right?

But as that supply matches, ma matches up to demand, people will require more, I would say cloud native solutions or uh, or solutions that enable them to build better applications. They're, they're Not gonna Exactly. No.

Wait, that's, that's, that's not how it works. You Know, you said something in our last interview that I remember you said, I wanna sell the picks and shovels. Right.

And, and it's true. I think the people today who are saying, I've got GPU boxes are saying, look at this gold dust. Yeah.

Right? And eventually I Okay, Gold dust. Go ahead.

Actually, I, I wanna revise that. Okay. So go ahead.

Nvidia is the one that sells picks and shovels. I'm the one that buys a lot of, a lot of picks, a lot of shovels, and I, So, and Nvidia manufactures picks and shovels. You wanna be Walmart?

Yeah. Uh, not Walmart. Or I buy those, I bought buy those access and I wash it clean and lease it to people.

And when done with that, I take it by and wash it again and give someone else. And aside from just having patient shovels that to rent to people, I also have the gold panning, uh, thing. Right.

And also washer All, Whatever they need, like the goal for them is like, I want to help you find gold as soon as possible. So whatever machinery, we got it. Right?

I got it. But it's not just buying a GPU. No.

So all the stock from, from finding a gold, right? Yeah. And, and I think that's an important, again, I'm gonna emphasize that it's not even just the cloud native stack.

Yeah. It's the app building blocks on top of these things. Super important.

Yeah. So I'm gonna put you on the spot. We're here.

It's, it's early, right? We, we've doing this early in Q con, it's, there's still not even open here. Um, but we're almost at the end of the year.

Hmm. What are you doing for a second act? What's, what's new coming up at GMII?

Yeah, So you could talk about it. Yeah, yeah, of course, of course. So right now we're focusing on, I would say the infrastructure.

So we wanna build a vertical solution, but you need to have a very strong, strong foundation, right? Just building a tree in a very strong roots that goes into, that's why we, we chose a much bigger architecture using, uh, OpenStack so that we have a very strong foundation. But, so that is, we call it the cluster engine, basically a very powerful orchestration software that managed and overseas and predict all the potential failure of the, of the GPUs, right?

The next step is, I would say we call it the inference engine, right? So that, that will be rolling out end of this year, uh, sometime Q1 next year. 99%, which is, does not exist in this environment.

Right? Because the GPUs fail so much. Do they, I don't know if people realize that For, for the people who are, for the people who are training, uh, with their, uh, They realize it, they live, they Know they will know.

Like for, for example, like, uh, when meta, well, this is public. So when meta is training LAMA three, I think it failed a hundred plus times in a, in first 30 or 45 minutes. Really?

45 days. So on average, they break down every three hours. Wow.

I think their SA was like 50%. Can you imagine that? 50%.

50%. So I've been in, I was in the data center business Yeah. The turn of the century that send hope.

But back then, you know, we were promising five nines, right? That was the standard five nine. CPU is e much easier.

CPU Is much easier Obviously than, so GPUs is, is super hard to manage. Um, and so one thing is having a very, very strong, I would say SOP, to manage the GPUs, make sure when, when things fail, we have a very robust team that can re, that can react in, in 15, 30 minutes to, to help customers. But more important is, I, I think on the software, because we cannot prevent the, the GPUs from failing.

That's just, it's just a mathematical non-zero chance. It's Part of the, it's right now anyway, it's part of The Yeah, it's 200,000 points per, per server, right? So rules, yeah.

It's, it's math, right? Math, the big numbers. And so what we do differently is essentially we'll kick in all the, all the, I would say, uh, all the available GPUs in when, whenever one goes off.

So to, to the customer, they don't care, right? They just say, Hey, I got, I got 100 tarros. Right?

And then that's, that's what we will promise them and they can give us a constraint, what You do on the back end Exactly. Of your business. Exactly.

Exactly. So we, we we're rolling that out and no one's offering that this moment. I'm really Excited.

And you're calling that an Inference engine? Inference engine, yeah. I love it.

Yeah. Not to put the, you know, I don't wanna put you on the spot, but when do you think we might see this inference engine? Uh, So we're rolling out beta, uh, two to, uh, I would say pilot customers, uh, by before end of this year.

And then, uh, ga uh, between January, February-ish. And how do you become a beta customer? GMI Cloud ai.

Ai, ai, yeah. Or reach out, reach out to us and I'm happy to do a design partnership with you guys. I love it.

Yeah. Alex, I think you're gonna love your first cube con When you get back home, reach out to me. We'll do a text on TV and we'll compare notes and see how it went for you.

Sounds good. All right, man. Hey, thank you.

Congratulations. Great to see you in person. Good to see you as well.

Alex Yee, founder, CEO of GMI, cloud at GMI Cloud, do AI here at CubeCon. We're gonna take a break. We'll be back.

We've got a lot of Cube con coverage coming your way.