The Role of K8s in ML – The AI Times EP 4

Hello everybody, and welcome to another episode of AI Times. AI Times is sponsored by the AI Infrastructure Alliance. We have fantastic panel Today.

We're gonna get right into the meat of the topic. We're talking about, uh, MLOps and Kubernetes, and how do we combine these environments? How do we work?

How do they work together? How do we make this thing happen as we're doing more and more AI and ML in applications, whether it's at the edge or the core of the network and beyond. So, I'm joined today by, and I'm really standing in for Mike Ard here this episode joined of course, by, uh, Mike and my esteem's co-host, Lee Baker.

Lee, welcome. Why don't you kick off the topic and, uh, you can, you can toss out the first question to our panel. And panelists, please introduce yourselves when you, when you, uh, first speak.

Very good. Very good. Thanks, Mitch.

Uh, and welcome everybody to thanks for joining us. Um, so, uh, as Mitch has kind of very ablely introduced our topic for the day, we're gonna dive straight into questions. Uh, and, uh, uh, I'm gonna pick on one of our established panel, uh, who are, are very qualified to answer these questions.

So, uh, let's tackle this right from the very beginning. Uh, we're talking about Kubernetes. Uh, we're talking about that in the context of ML lops and data science.

But Basil, I've got this one coming for you. Do data scientists even need to deal with Kubernetes? You know, should that be kind of abstracted away from, from what their focus is?

And actually, should they be worried about the accuracy and training of their models? Should Kubernetes even be a consideration for the, for, for the modern day data scientist? Um, so yes.

Uh, thank you for passing the first question over to me. Keith Basil, general manager for the Edge Business Unit inside suse. Um, I think the general answer is they should not, right?

We should abstract away the, the lower level infrastructure that's running the applications, but in some cases, particularly at the edge, um, the data that they're building their models on, uh, we need to make sure that there's good connectivity, uh, especially in the industrial IOT space where we're connecting to various devices to pull data, to feed into those models. So if we focus on the plumbing and the glue to get our arms around the siloed nature of the industrial iot space, I think that's a good thing. But even that should be abstracted away.

So we're involved in a project called ocri, A K R I upstream to do that, to bring that siloed nature of the industrial iot space into a cloud native world to pass that data to the applications of, you know, higher up into the stack. So at the end of the day, just to summarize, it should be abstracted, but they should not know the ins and outs of Kubernetes. If I remember, OCRI is an open source project, right?

With your It, it is, it is. Yeah. Very cool.

So Leah, hopefully that was a, a good answer for you. Great response, basil. Uh, so, so let's take this back a little bit.

So Basil's saying kind of the, the, the data scientist should be abstracted away from the complexity of kind of, of, of orchestrating their Kubernetes cluster. But, so Alex, who does this fall to, you know, who is the role in the organization that has to bear the burden of responsibility? And is this a team sport?

And how do you, yeah. How do you establish kind of an m l LOP kind of best practice? Hi, thank you for, uh, for asking me.

That's a great question. Uh, I work as a director of Kubernetes here at Canonical, and so I get a pretty lucky broad view into, uh, some of the possible answers to that. But I'll start with humans firstly, right?

Because we've gotta think about the cognitive overhead of trying to compact this amount of data into one set of skills, right? One person. And that's, it's just not practical going forwards.

Things are scaling outta control in terms of what you need to know. So naturally, it falls upon organizations to just start to think about breaking these into functions. And one of the things we've seen recently is platform engineering is this, this new term for effectively cis admin, DevOps, and all those good things.

I do think that it naturally falls to two sets of individuals, primarily because those data scientists and ai ML scientists are trying to, um, build models. They're trying to add value, whereas the platform engineering team are trying to, uh, avoid entropy, right? So there's very different functions with their very different cycles of work.

So I think that ultimately if you are utilizing Kubernetes, you are looking at the tools and frameworks. If you're building it, you are kind of thinking about, well, okay, the substrate of, where do I put that, Kates, how do I make it highly available, and how do I also give them the right tools so they can just log in and start using their Jupyter Notebook, for example. So I do think there are two different functions, but there's such a lot of overlap these days that you'll find that your student grad scientist is building their own Kates cluster, just to get a feel for how this all this stuff works.

So in the future, we may well see that those lines begin to blur even more. Yep. Yep.

And, and Andrea, here's a prime opportunity to, to disagree with one of your colleagues. Uh, uh, is, is, so Alex is saying that you know, that you need to have a flavor, a, a, a, um, uh, a rudimentary understanding of all of this. Um, but what does that mean?

Is, is, is is native Kubernetes enough, uh, or actually, are we looking about tailoring a specific distribution or specialized tools to, to enable your kind of specific business use cases? Thank you, Lee. Uh, here's Andrea.

I'm the MLOps product manager within Canonical, looking after our portfolio and product strategy. And I'll have to agree and disagree at the same time. I think knowing Kubernetes is something that a lot of data scientists actually end up doing, but is not out of pleasure, but rather out of need.

And I, I do have a data science background. It was not necessarily something I was enjoying, but I was challenged with. And then if you want to evolve your product since it, it's quite a new function, you end up learning about it.

However, once you start as an enterprise, especially once you start thinking of production grade projects, you look at platforms that where you can run and automate part of your work, such as cube flow, when you think of open source, uh, projects or ML flow for, or feast for very specific, uh, capabilities, the challenge there comes into how do you connect or integrate all these tools in an end-to-end solutions. And that's where usually I think some companies, and we do it in canonical as well, integrate those tools into an end-to-end stock, such that data scientists, um, end up, um, being the profile that Basil described where they focus on modeling, they focus actually on data science, and they can run away from the Kubernetes needs at the same time. Alex is right.

I think long term is going to be a, a split in a way or another where you see data scientists, machine learning engineers, researchers, uh, very likely who focus on how to use those platforms that run on top of Kubernetes. But then you're also going to have platform engineers. ML lops engineers is also quite a, a cool function these days that describes the management of the tooling.

They, they manage the tools, they understand how to integrate them, and they ensure that data scientists at the end of the day can, can do their job and, and focus on, um, getting return on investments from their projects. So they don't get stuck in experimentation Very quickly into that, Andrea, like, if you think about something like Q Flow, you need to understand how service mesh works, object storage works, uh, Jupiter notebooks, Argo pipelines, that, you know, there's a huge amount of different knowledge there. And that's kind of precisely my point, is that my expectation, and this just leverages what, uh, basil was saying, is that this is a lot for one individual, and, and it detract.

It's not part of your job in the sense that that's not your primary function or output. So this is like a technical overhead that is becoming larger and larger. So I really see there being a bit of a, uh, a polarization there between the, the engineers who are producing the output function and those who are maintaining that platform.

Sorry, back to you, Mitch. Yeah, Alex, you were going right where I was thinking, which is, we talked about platform engineering. There's also the, just the software architecture, right?

In a microservices service mesh world, you know, the, uh, AI models, you know, things can be behind an A P I, right? And, and depending on Basil's point about different challenges at the edge or different parts of where, where the distributed application are the, whether it's MO code that's distributed in different, uh, microservices across that might be using common, uh, model services as part of your application. So my, what my point is, there's sort of an architecture role on top of this about how this is all constructed.

So that way data scientist isn't worrying about it. Is, is this a microservice? I don't really understand that stuff.

I know, you know, I know what I know really well. And then the issue is how do I get to the data? Is it, is it located where I can, you know, get a responsive re you know, answer to an e p I call about what my app is doing, am I on base?

It, it seems, you, me, that's also a part of this mix. And I would also like to add something here. I think there is also going to data science.

Now it's, it's very generic as a data scientist. You, you ingest the data, you claim the data, you train the model, you deploy the model, and sometimes you even monitor it or you look after that part. Probably the function itself is going to evolve and also suffer a split where data scientists, um, become specialists in one of the areas such that they, they understand better how to deal with data at the edge, for example, or how to, how to fine tune tune models.

And it's something that we see in the industry as as appearing. So I can, I can already hear our, our viewers running for the hills, uh, uh, uh, terrified by the kind of the complexity and constraints that they're hearing coming out of this. How, how can we, um, how can we encourage practitioners that, that Kubernetes is the way, as, uh, as, as kind of the Mandalorian might say, uh, and and, and to encourage them to engage and, and, and focus on that building block for their pipelines.

Yeah. Alex, what's, what are you saying? Uh, how, how can you kind of, uh, kind of console our viewers that, uh, that, uh, Kubernetes is the promised land and, uh, and we, and we won't terrify the horses?

I think, um, you know, I always try to orientate towards creating a pit of success, right? So an example is what we do is charmed cube flow. Take a, an application that's quite complicated to set up and make it really simple, make a one line installer, you know, that's why I wouldn't too successful.

That's why Windows is successful. So if you take that philosophy and you take it through to I am a data scientist, I just want a platform, I don't really care how it's all working, then you can build quite an opinionated setup and let users then just do their, uh, da data science and ML lops. Now, the reason why Kubernetes is, is such a, an interesting substrate is because it's exactly that.

It is pretty much the lowest part of the stack that works on the cloud. On my raspy PIs behind me, on my ibm, uh, uh, I think machines thinks then is that, and in the garage, it is something that is gonna be fairly future-proof, but let, but it has a common, uh, it's like a common Rosetta stone, isn't it? Many different types of tooling and application are interoperable with Kubernetes.

So therefore, you know, if you're investing your time and, and energy into that, that it's likely to be around a lot of people can utilize and leverage that. The one final thing I'll say is that with projects like Lang Chain coming out, which are now enabling the development, uh, through effectively FD SDKs to interact with inference APIs, you can start running local models in Ks. This is a relatively new field where you can use vector databases that are distributed across Kubernetes with an inference API in the front.

So suddenly you're no longer having to do remote inference, and people have this whole little factory where they can build train models and use them in Kubernetes and know the platform, to my knowledge, has that kind of expansive capability. So that's encouraging. So, so it's taking people on an incremental journey.

Um, uh, okay. So let's, let's, let's, let's take that and, and, and run with it. Andrea, you referenced some very kind of powerful, uh, open source projects.

You talked about cube flow, uh, you talked about ML flow, you talked about feast. Uh, where would you say the, kind of the state of open sources in the context of Kubernetes and, uh, what are you seeing people building, building on, building out on top of? Well, I think when it comes to ai, we really think of Kubernetes as almost a default option, because it does, even if we just complain that it might be difficult sometimes, it also gives freedom and flexibility.

So if I think of Charm Q Flow that, uh, Alex mentioned, it is Kubernetes that enables, um, and let us also do GPUs scheduling, for example, or enables us to integrate Charm Q Flow with charmed ML flow. And it's, it's fairly quickly done within a one command line such that, going back to what data scientists have to do these days, they just, we, we take away that burden. So they do it quickly, and then they can focus on, on what they, they are supposed to do, which is modeling, building, um, playing with data in order to, to get machine learning models.

So that's where I think the, the great benefit is. And now if you look at the open source world, I think in general, AI brought a huge shift. There are a lot of open source projects that are covering different capabilities.

Now, the challenge is that if I think of, for example, feast, it's, it's a feature store and on its own, it's not incredibly, or it is very valuable, but it's not, uh, something the data scientists get attracted to. They get attracted to it once they integrate it, and they can use it in an end to end, uh, scenario. And again, if I go back to Kubernetes and your initial question, I think that's where the value is.

You can use them all on Kubernetes without worrying too much that they will fail, they don't work. You need, you need to do a lot of operations. I think as worth pointing out too, Lee, that, um, so we're now, now I'm pitching products, but we mentioned a few, few companies.

There are lighter weight versions of Kubernetes as well, like K three s, which, um, seems a help, help pioneer. Uh, so there, there, and there's a K zero s and there, there are very much lighter weight versions of Kubernetes. You don't have to download or sign up for the service providers, big gorilla Kubernetes, you can get sort of get the nimble or version of, of ku, uh, to help you.

So it, it isn't a one size fits all when we talk about Kubernetes. And I think the other point about maybe to allay some of the fears is organizations are developing, um, uh, uh, uh, knowledge muscle about Kubernetes and how to deploy it and how to architect it. It's not the same as, you know, through two or three years ago where it's sort of like, wow, this thing's a pretty big behemoth, how do I do this?

I'm not saying everybody's an expert, but we, we've moved a long ways about how to architect and run, operate Kubernetes for a variety of applications and now including AI and ml. Yeah, Mitch. Uh, to add to that, just quickly, um, we also, uh, have Rancher desktop, uh, which allows you to just download single application and run Kubernetes locally, do all kinds of testing.

You can pick the version of Kubernetes. Uh, so it's a really powerful tool for, um, for developers and probably would apply to, um, you know, machine learning and data scientists. So, so, so Basil, let's, let's, uh, let, let's go with that because I, I'm intrigued to to hear from you around what is, um, uh, what is the implication for running Kubernetes in, in very kind of specialized use cases.

So, uh, kind of for, for our viewers, frame your definition of edge and kind of why Kubernetes is an important in that context. Okay. So we see the edge in three large segments.

Uh, the near edge, which is kind of the realm of the telecommunications companies. That's the closest to the centralized services, the data centers, the clouds, and then, uh, you'll see massive applications there. Uh, again, communication service providers globally, uh, take the, uh, uh, cloud native functions in the telco space.

For example, in the global race for 5g, that's all happening in that near edge space. When you get on the other side of the access device or cable modem, or a lot of demarcation, you get to what we call the far edge that's actually at a physical location, local network, run by the customer, private IP space, local Kubernetes, running at that facility on premises. Um, and then you have a lot of diversity there, and it takes all forms and shapes.

It could be a building, it could be an airplane, it could be a satellite, it could be a truck with a, you know, rolling data center, for example. But those are the far edge use cases. And then we have what's called the tiny edge.

This is the industrial IOT space where these devices are too small for classic, you know, full-blown lenux, uh, full-blown Kubernetes, and then containers on top of that. So they do have software though. And this is, I was speaking about earlier with the industrial IOT protocol space, right?

So we can reach out to these devices via M Q T, T O P C U A, whatever, and see them, discover them, pull data from them. Um, that's the market that we're gonna be focusing on, uh, in the coming year, uh, is the industrial iot space to bring that in. And particularly, and I I just mentioned that because we have some companies that are very, uh, uh, proactive and on the leading edge or bleeding edge, if you will, where they're looking at running their operations based on, uh, AI and M M ML models, right?

And so to feed those models, they have to have access to the data, the sensors, the actuators, right? Um, that are at that location to feed into that model. So this is why that glue, that infrastructure glue is the thing that I'm focused on.

So, but those are the three definitions near communications providers, uh, far pretty much all the edge use cases that you know and love. And then tiny is the industrial IT space. That's super helpful.

That's super helpful. And, and, and, and so Alex, as, as, as Basil's kind of framed kind of, uh, edge in as the context, where do you think, from your perspective, speaking to your kind of your customer base and users, where do you think we are in terms of on that timeline to, you know, re being reasonably confident to deploy models into edge environments, you know, pull data back, um, you know, manage kind of clusters remotely. Where, where, where do you think we are on that timeline?

I think that it's already happening. Um, so I have two products. One of those is microcases, and it runs in the data center and on edge on X three 90, power nine, you know, arm architecture, we have people running it, and we have it being, uh, currently approved for automotive and other industries, energy and gas.

So very similar to, to, to basil. And the anecdote I like to use about Kubernetes Edge is that if you've got three mobile telco base stations, one of those base stations goes down, the other two have to signal amplify, right? The gain has to go up so that more they can take the load of the additional cell tower cell signals, then you need to schedule more pods.

And that's what Kubernetes really lets you do. It lets you scale up more processes dynamically. So Kate's on Edge is a great use case, but as you said, managing on edge is super difficult, right?

Like, are you there? I can't get to your network, you're behind seven naps. So therefore, the dial home functionality is intrinsic to, to being able to have that working.

And for us it's snap. So Snap store, you know, you go Ubuntu, snap in store, snap actually has a, a dial home mechanism that we can see what snaps are installed and update them remotely. Now, you know, this is isn't any sort of pitch, but what I'm trying to say is that that's a difficult thing to figure out and to get it right is even harder.

Um, as for workloads and models, the challenge of models is size, right? Cause a model is effectively a database and, and vector database, and they typically can be really difficult to export onto a device. So I think what we'll start to see is that we're using silicon to do it.

I think we'll start to see RAM disks become more popular and there'll be, uh, preloaded models on silicon that let you run it at edge without having to have that, uh, that load off a disc. But there are still a lot of intrinsic channels, uh, challenges around inference, A P I C P U overhead. Now, I'll just finish momentarily, but what this also brings to light is that you need to start to have models that are trained not to be GPU centric models, but actually CPU bound models.

And we're seeing with the advent of dpu. So those dpu are data processing units, which are basically pieces of silicon that have really good GPUs and CPUs on them, that we have this more generic iot device that could potentially be used at Edge to do that kind of AI exfiltration and offload. So I think it's already here we see that the key players like Nvidia have already thought five years ahead, but the market is gonna be about five years behind before they pick up to it.

Yep. Uh, great snapshot. Uh, thanks Alex.

So, uh, so no one said it yet, so I'm gonna have to be the first person to say, um, uh, generative models, generative ai, you know, um, that's kind of the, the, the zeitgeist that has captured everybody's attention. Uh, and why is it so compelling? Well, because it abstracts a waste a lot of this complexity.

Um, we are seeing big organizations are trying to develop their own large language models, that's suddenly the kind of the, the direction of travel. I won't get into the merits of that, but, but, uh, where do you see kind of Kubernetes in the context of, of generative ai? Andrea?

I start, uh, by saying that what I think generative AI does is to, um, show people that there's another way to do things and to give them more time to do other activities that are more meaningful for them when it comes to both Kubernetes as well as open source tooling that we already mentioned. I think that's, that's where you can start developing your own large language models or models that are used for, uh, different projects. And in general, of course, there are constraints when it comes to generative AI in general that are worth, um, considering, but at the end of the day, it's still machine learning.

And that's what people have to, to bear in mind, it became famous because it, it, it just be showed people how easy it can be used. But at the end of the day, it's still machine learning and tools such as Kubernetes Q Flow, uh, are the ones that are going to be used to build the model, retrain the models, monitor the model, uh, so that data drift is detected, for example. And that's what's exciting about it, that you can use to build things that are really acu engage these days and are very innovative and very exciting.

Yeah. So, so Andrew, you, you referenced kind of data drift for the, for the benefit of our view is ex. Explain what kind of, what you mean by that and, and why monitoring models is so is, is so intrinsic.

Well, when it comes to, I'll start with model monitoring because data drift is part of it, but the truth is that once you have a model and you, you deploy it to production, it's it's performance. It'll start decreasing right away. And you, you don't do it once.

You always retrain your model data drift is, uh, the process of informing in a way or another, simplifying it, very much informing the developers that the data that is used for training or the data that that feeds back into, into the model changes. And it can be because of different reasons. Here, we can, uh, uh, raise security concerns, for example, that someone tries to, to retrain the model in a, in a negative manner.

But also it can be because people change their trend trends and habits change. And that's why it's always worth considering that you don't build a model once and then you, you let it get old there. You always have to, to keep it updated.

It's exactly the way that we do with, with the, we need to stay up to date with the latest technologies. It's kind of the same with models. We need to keep them up to date with the latest, uh, data sets, I'll say.

Yep. Yep. So, so, uh, that leads me to kind of a another area, which is, which is economics.

Uh, you know, why Kubernetes is crucial, not, not just for the reasons Andrew just described arou around kind of model monitoring and kind of performance, but also being able to, as I think Alex alluded to, kind of scaling up and and scaling down. Uh, uh, Mitch, you talked to a lot of people on these, uh, on these panels. What's your hot take on kind of, uh, enterprise's appreciation of the economic benefit of kind of containerization?

Yeah, I think it's a two-sided coin. There is the, um, capability to scale up and meet demand. There's also the kind of cost factor of just managing the cost of scaling up and you know, where you need it, when you need it.

And, and that can be a real benefit of, you know, leveraging Kubernetes and containerization and things like that. You just have to, you can't just kinda set it and forget it, right? And let it operate.

I think, I think the other thing maybe we should touch on about this is even just containerization of parts of the, of the AI model, of the services that are responding to applications. You mentioned Lang Chain, it's kind of my new favorite project thing to work on. Um, but, but there are ways of designing this so it can be scaled.

And I think that's an important factor of, you know, just a, a large, massive model that's gonna run somewhere in the cloud that may not be responsive to applications, isn't gonna be necessarily the right solution for everywhere. I, I love having balon cuz he truly is, if you ever wanted to know what the edge case is, he is the edge case by definitions. Cause he's working so much of the edge.

But I, I think the economic benefits are that ability to match what the demand is in the businesses, whether it's volume or it's location and distribution. Uh, Mitch, what I'd like to say is that, um, the, the business value, the application goes where it's best treated from the old, old money, uh, quote, oh, money goes where it's best money goes where it's best treated. So I, I think in our case, using Kubernetes at the edge in a kind of an edge native way, um, let's the application go to where it does the best value for the organization.

Paso, what did, what do you think that means for ki the role, uh, and kind of profile of, of ML lops teams, uh, uh, in, in, in organizations, you know, uh, the ML lops at the edge, kind of, it suggests that when you just, you do your day job, but you do it, you know, you do it in a, in a low compute environment, there's surely more to it. No, I, I think, I think, no. Well, I think you've answered it like, um, as what Alex said, you know, some of the, uh, models are fairly large and if you've only got, you know, Intel nooks at the location, then you are gonna be severely resource constrained.

Um, you know, especially if you don't have any GPUs or anything. So, um, the day-to-day work in a classic sense will remain the same, but you have to add the attribute of understanding what, what the constraints are at that location, because not only do you have a compute and storage constraint, you have a data delivery pipeline, uh, constraint because some of these, uh, remote locations are limited to satellite communications, for example, and you're not gonna send a giga data over that pipe. So, um, there may be some form of, um, silicon delivery as Alex, uh, as mentioned maybe Sneakernet even, um, you know, FedEx UPS to deliver something, right?

Or in some of our military use cases, there's a black helicopter that goes to a location, drops off gear and then flies out, right? So, uh, really interesting use cases, but you have to, I don't think the, the overall, the process by which they develop the value on top of AI changes is that they have to just take into account the, the, the environment. Yep.

Yep. Bazo topic we talk a lot about on the AI times recordings is, is how there is a, um, a fracture in the organizations as, as teams are becoming more technical, more complex, whilst at the same time kind of, you know, the CEO stands up and says, we're doing ai, um, and the expectations, you know, are parting like, uh, uh, like the c So do you think that kind of what you're doing with Edge is kind of in, is, is increasing that division of expectation or, you know, can we bring that back around? I I think we have to bring it back around.

So it, you know, I was listening to your question and immediately thought of the old days when everybody would cloud wash their applications. Like we're in the cloud now, and it was just, you know, some legacy app in a virtual machine deployed on Amazon, right? Um, so I think a lot of that you'll see here with Edge and also AI trying to AI enable these things.

Um, we, on our side, we're doing a very kind of measured pragmatic approach. Um, rancher, for example, will be releasing a Kubernetes rancher specific chatbot, um, in, at Cuon, for example. This is basically to augment the ops teams to get understanding of configurations and, and make suggestions and things.

Um, but you know, it's interesting because when you look at that as like a baseline, just generally the question becomes for a customer, especially some of our disconnected world customers, how do you ground that baseline to the customer's specific deployment and how do you do updates of the models and things like that? So these are some things that we are kind of testing in-house, um, and, and having solutions for, but it, it's a brave new world, but we don't see ourselves, um, as the, uh, kind of data scientists, we see ourselves always as infrastructure and plumbers to enable those data scientists. So our perspective is just a little bit different.

Yep. Yep. Um, great.

Great. Take Basil. Um, listen, we're, I'm gonna start to wrap up guys.

So I break out your crystal ball, uh, because, uh, we're gonna leap kind of, uh, five years, hence, uh, and, and, uh, I I want you to give me your hot take on where the, uh, Kubernetes journey has taken us. What does infrastructure look like, uh, and, uh, and, and kind of what are the pitfalls or, or positives that we've encountered? Okay.

So do gimme your best mystic meg, your, your best predictions. Alex, I'm gonna go to you first. Sure.

So in five years time, my project Kate's, G B t will be graduated from C N C F. The operator will allow you to run local models in Kubernetes that will have complete autopiloting for your cluster. There'll be a user base of 25,000 or so many companies adopting it, but this will be, um, comparable to their own journeys where they're starting to bring AI based operators, uh, into Kubernetes to start synthesizing and replacing SREs and lower their, not only their cost profile, but also their time to response.

I think they'll also see generally that there'll be AI specific hardware that's on C P U. So for example, laptops will be designed with an AI component that comes out of the box with a generic model, uh, for generative processing. So that's my, my prediction for the next three to five years.

Great. Love it. Love it.

Andrea, what, what, Andrea, what, what, what's your, uh, what's your take on this? What is the, uh, what does the ML lops team of five years look like? But when it comes to ML lops, I think on one hand we are going to reduce the skill gap on the market because at the moment there's clearly a challenge on finding talent.

Who knows both how to do data science on one hand, but also how to, to manage all these tools that are popping up, um, on the market. At the same time. I think that in two, five years, the industry and the landscape is going to be, uh, much more connected.

So all the tools integration that I was talking about are going to to be done through a much easier experience. Last, but not least, I will, I'll have to say that in general, I think, uh, the open sourcing machine learning models and open sourcing large language models is going to likely become a habit. So we'll see a shift not only having open source tools, which is quite normal these days, but we are also going to see a lot more models being open sourced in order to accelerate their development and to see where they go, because different people have different perspectives.

I'll say it's also worth mentioning, uh, I think it's the elephant in the room that no one talks about. They're going to be more regulations on the market in three to five years around ai, around ML lops. And whereas they're likely to be a bit challenging at the beginning, they're also going to, to set the standard, they're also going to, to help professionals go on the, on the safe side and understand how to protect their work, how to protect data that is being used, uh, behind, but also how to, to protect those who benefit from the, or who are those who are impacted by the, by the use of machine learning in general.

Yeah. Yeah. Guardrails for ML lops is a topic we visited before, and I think we will, we will visit again.

Um, great answer. Thank you, basil. I think I know what you're gonna say, but, uh, um, but, uh, let, let's, let's hear.

What's, what's your, what can I Say? I mean, Alex and Andrea basically stole, not stole, but they've set the, I mean, you can't do any better than that, right? And this, this actually speaks to the point of, of being an open source community, which I love.

So, um, you know, we all work for different companies where I think, uh, you two are the same company, but the fact that we are sitting here talking about some really cool things, um, is, is phenomenal because like, um, kt p t is is phenomenal, right? So I was gonna say what Andrea said is that the open source large, um, language models, I think that's gonna become more prolific, right? Because I think at the end of the day, open source is gonna win as it traditionally does, right?

So we're gonna build on that. Um, the other thing I wanted to say I wrote down here is that in the future, five years from now, I think we'll have private, um, you know, models and, and, uh, LLMs with dynamic or realtime kind of training that specific to that business function and that operation, right? Um, I think that's gonna be like the holy grail where you can ingest this thing, you can deploy this thing, and it is really productive and efficient to, uh, make your business stronger at the end of the day.

And it's gonna be a little while for we, before we get there, but things are moving so fast. It's, it's incredible. It's like this is a great time to be in tech, um, particularly with AI.

And, um, I'm just happy to, um, in my career path, be be in this thing because it's, it's a new day every day. And, and combining these things with the edge is just, just phenomenal. Yep.

Yep. Uh, amen to that. Um, uh, great answer.

Ba I appreciate that, guys. Um, I'm gonna have to wrap it. Uh, we, we could have gone on and on and on.

Uh, and hopefully we will do again, again another time. So thank you very much for joining us. Appreciate your contributions.

Uh, we'll make sure we get to some feedback from our viewers as well. Um, thank you from, uh, uh, AI Infrastructure Alliance for Par participating. Uh, Mitch, thank you for co-hosting, uh, uh, in Mike's absence.

Really appreciate that. Thank you everybody, and, uh, see you again soon. See you.

Thank you very.