CNCF’s Kubeflow Incubation Journey – Josh Bottum, Kubeflow

This is Textron tv. Hey everyone. Welcome back here to techron tv.

Uh, you know, we, we spend a lot of time covering the C N C F and the, I forget if it's 40 something or 60 something projects that are now under their umbrella, uh, including going to CubeCon, whether it's in Europe or, or North America, or we're looking at maybe possibly going to Shanghai in September. I think it is, though. We may not be going, depending on geopolitical things going on in the world.

Um, but I'm, I'm happy to have this next gentleman on with us. He, his name is Josh Bottom, and he is a interim steering committee member for a project called Cube Flow that we're gonna learn all about today. Hey, Josh, welcome to Text Drunk tv.

Hey, thanks for the, uh, hosting us here. We're very excited about the news and look forward to this conversation. Absolutely.

We're gonna jump into the news in a minute, but before we do, Josh, you know, I always like to give people kind of a table setting of who, who, who it is that's talking to them. They know me. I'm, I'm here every day, but they don't, may not know you.

Why don't you give 'em a little bit of the Josh bottom story? Sure. I'm, uh, about 37 years in tech.

Uh, last five or so focused, uh, a lot on ML ops and open source ML ops with Q Flow. Uh, I've worked both in large and small companies, but primarily I'm a startup type of guy. Uh, I've done many startups, uh, being like the first employee in a company for sales or business development.

Um, so I really look at the, the functionality of the software and how do you articulate the value and cube flow. Uh, what attracted me about Cube Flow and why I've spent the last five years working on building the community is that I saw this, uh, this Google based project was putting machine learning operations on top of Kubernetes. And I thought that marriage, uh, the operational aspect of, of Kubernetes really made a lot of sense for new applications, but also, especially for machine learning and AI type of application development.

Absolutely does. Um, and that, you know, 37, that's about where, you know, I think you and I have had similar paths there. 'cause I've also sales Biz dev, you know, on that side of the house and did many venture backed startups through, uh, until I started here.

But I started this company 10, nine years ago, 10 years ago. Anyway, um, you know, we mentioned ML Ops, we mentioned Cube Flow. How, how long has Cube Flow been?

Like, when, when was the project founded? So, Google founded it in 2017. We've had 16 releases.

If you could think about why people would choose to work with qpl, we have hundreds of companies that are contributors. And you could think that over the fif 16 releases over five years, there's at least 50 man years, I would say 50 years of work that have been contributed into the project from many of the market leaders. If you look through, whether it's a w s or Google or Azure or I b m, red Hat Canonical, you go through all the people that are distributions, those are incredible market leaders in terms of infrastructure understanding.

So you have that under low, under layer layer of the foundational software, but more importantly, you have all our application users, whether it's Apple or Roche or Zillow or LinkedIn, who have provided us feedback about how the application should actually run. So that collaboration, right, that investment into Cube Flow really adds a tremendous amount of value that is hard to get in a proprietary solution or to try to build it yourself. And so that's why I think Cube Flow has been able to grow and sustain itself over the, you know, five years since Google introduced it.

I love it. Excellent. And, and, and you're right.

I mean, you, you count man hours and man years and then months and weeks, you know, into these projects. That adds up pretty quickly. One of the things you didn't mention though, was the Linux Foundation and the C N C F, right?

And that, that gets to kind of our news today. Why don't you share with the audience a little bit, Josh, about our news today. So, you know, it's very impressive to me the amount of work that Google put in to this project Cube flow.

And you've watched what they've done with other similar projects, whether it's Kubernetes or Istio or others, that they have basically got up and running, set a culture. Uh, and by culture, it, it really meant how do you set up meetings? How do you run meetings?

How do you do, uh, summits, how do you do user surveys? How do you triage issues, how do you do roadmaps? All these things that they have built this culture, and now they've decided that just like some of the other projects that they've run in the past that have been tremendously successful Kubernetes, or for one, they're donating this into, you know, the C N C F and for us, uh, in the C N C F, I mean, it really does look like Q Flow is maturing to a new stage.

Uh, we've gone through many different stages with the project. We've been able to transition and keep people engaged. You can think of these frenemies, whether you've got Capital One and Freddie Mac and Intuit and other people competing both as users, but also you have all the other code contributors, which are like Google and a w s and Azure and I B M.

How did we keep that going for 15 or 16 releases in such a competitive area? And now the ability for Google to release that and allow for an open even more open governance, I, this is what I would say. I am so impressed on what Google had asked and, and set the vision to do from the community.

People contributed, and now that foundation is laid so we can go out and do this neutral, uh, governance that the C N C F offers to us. And what I am hoping as we go forward is somebody who's been involved in a lot of leadership and, and leadership in, in, in, in, in community often means work, right? Which I mean is, hey, to run a good community meeting means that you have to think about what the agenda is and how do you measure those agenda items, amount of time they get, how do you ask questions properly, who gets, you know, there is some management of that, but taking notes and just the basic blocking and tackling, uh, that you have to do.

The the point that I'm trying to make here is now that we're in the C N C F, we're gonna have even better, more process, more people broader reach, uh, and more integration with the C N C F Pro, uh, other projects, which I think is just gonna be fa fabulous for us to fix or work on the things that, that the user surveys continue to tell us we need to improve upon. Absolutely. And, you know, look, Google's a fantastic company as it relates to open source.

They, their, their contributions. You mentioned too, s t o and Coop Android and how many other, you know, major open source projects were, were founded by Google. But the beauty of the whole Linux Foundation and C N C F model is what I call the foundational era of open source, right?

We used to have, before that it was sort of a Big Brother era where you had a Google or an I B M, you know, or some, any company that would kind of own and manage that open source project. And the ones who did it best were good at bringing in, I don't call 'em frenemies, I call it co-opetition, right? Where they, they'd work together to, to set sort of a level playing field of which, and then from there they could build their castles and, and go in their different directions.

And I think that works to a point. But to really take it to that next level, you need a C N C F Linux Foundation kind of thing that says, okay, this truly is a fair deal for everyone. There are no favorites per se.

We're gonna do this as the community and everyone has a seat at the table, or everyone can have a seat at the table, and that's all you get asked for. Now, what I don't think we mentioned, and if we didn't, I want to make sure we get it out to our audience, is Cube Flow is coming into C N C F as an incubating project, right? Um, correct.

And for those who are familiar, you know what that means for those who aren't familiar though, right? There's, there's different levels of, or different life stages of a project through C N C F. Josh, I know, I'm gonna assume you know them, and if you wouldn't mind, but if you need me, I, I could jump in too, but it's your interview.

So look, uh, if you look at the Q Flow community, it, it's a very complex application. There's five working groups at least, plus a lot of contributing type of applications. So it, having this, uh, built, uh, 15, 16 releases, we have a release handbook.

We're a fairly 10,000, you know, uh, ac uh, folks on Slack. It's a decent sized community, mid-sized community, not large, but mid, mid-sized with process developed. And we've been able to show, you know, through our release team management that, you know, maybe Google ran eight of the releases, but of the other seven or eight that, that we're doing right now or have done other companies have picked in, in a ma you know, a, a whole smattering of different companies have jumped in.

So my point is, being able to manage the transition is very important. This is a transition now where we are not in the sandbox or the early phase, we're more in incubation. We need to go through a few things to be able to become a graduated project.

And those things where I think we'll get a lot of benefit from the c n and CF will be around security, running rootless containers, making sure there's even more, uh, and better types of isolation and user, uh, authentication and authorization. Not that we don't use the Kubernetes, uh, capabilities today, but we can always get better, right? Absolutely.

And that processes for documentation, processes for testing processes for best practices, we have some of those. I mean, I, I admit that just like any open source project documentation tends to be one of the hardest things to do. But now we have Che G P T, so who needs to any documentation you just asked that it's only two, only two years old, but you know, it's not bad.

I I hear you. I hear you there. There's a lot of people saying that too, by the way.

Right? Hey, we got Chad, g p t, we, you know, we'll get it done. Well, We actually started a project called Tribal Knowledge to, uh, fine tune or, or add in, um, more kuber cube flow based knowledge into a specific model that we were serving.

We have into An L L M. Yeah, I, you know what, I think a lot of us are working on that. Um, we're doing it here.

Josh, you know what? For people watching this out there and they say, you know what? I've heard of Q Flow.

I like it. I think it's a cool project. I think it's useful, man.

Now that's what C N C F I'd like to get involved. Where do you guys need help right now? Well, according to the user survey, which you'll see the post that just came up, we do this every year.

I started those maybe four years ago. This one, uh, again, shows that our top thing that we need to work on is documentation, really. So, uh, that, yeah, not surprisingly, but you know, my request of all Q Flow users is number one, if you're getting value out of Q Flow and you've used it, please work on one pr one poll request to improve the documentation, right?

8 is coming out on October 6th. Please jump in, get your hands dirty. It's not that hard to do a simple PR to improve the docks, but other places that people want help is, uh, an installation.

Now look, when you're talking about doing a Kubernetes installation, right? Um, yeah. Of a, a very complex set of software that has lots of dependencies, whether it's Argo, Istio, cert Manager, K Native, what have you, you're going to have some dependencies that are already installed in an environment, especially if you're running on-prem.

So installation is one of those things that people ask for help and documentation on. I continue to struggle with this as a, a community manager, is about how much we dive down and where do we draw the line as a community and say, this is the default canonical way to do it in a very basic environment. You need more help than that.

Either do d i y, figure it out yourself, ask the community, or go work with one of our 10 distribution providers that provide, you know, people that have invested in supporting the community, but also they, you have to pay 'em. And, you know, let's be honest about a lot of this. In order for our community to be thriving, the distributions have to be able to generate some form of revenue.

So the users that are getting benefit, they have to recognize either you need to volunteer time back or you need to work with the people that are providing support, that are providing those contributions to make the project better. Agreed. I don't want, I don't mean to embarrass you, but I noticed in our notes for this interview, it says, interim steering committee member, but you've been involved with Cube Flow, right?

A lot of people, Hey, Josh has brought a cube flow, right? Um, what, what, what's the deal with that? If, if you uncomfortable about it?

We don't have to talk about it, but I Good ask. This is, it's wonderful, actually. Look, I, I'm very, um, happy.

Let me say, I'm proud to be part of the Q Flow steering committee. I'm the only, uh, member outside of Google, uh, on the steering committee. Um, okay.

So, you know, with, uh, you know, work comes some responsibility. Um, the steering committee and the meritocracy, uh, culture that was developed by Google around Q Flow is not that the power sits at the top, you know, the power, which is really deciding what gets into the next release is, is actually at the working group level and at the contributor level. Mm-hmm.

So for me, I see those folks as the most important part of the community in terms of who's driving the direction. We're at the, at the, at the, in, at the interim City Steering committee, which now that we're through this part of being in the C N C F, we're going to have a more permanent steering committee and there'll be anu new set of elections. We knew this when we, when they asked me to start the process in October to get through the C N C F approval, that, you know, there would be elections that come up and we'll probably have staggered, uh, uh, you know, parts of the board that are, are added into.

But look, don't, don't just think that you're sitting at the top anybody who wants to be involved in any type of, of open source, and you just pushing buttons and, you know, somehow stuff magically happens. You know, all the good ideas that come outta your brain at two o'clock in the, uh, in the morning that you, you gotta have with the committee, you end up doing yourself a lot of times. So I guess the, the point of being on the steering committee is, is is something that is good on setting direction.

And here's the thing that I'm really focused on in the steering committee and I've brought up in, as I bring in, how do we partner with other communities, for example, with Ray Re or with ML Flow and all the other partners that we have to where it's a symbiotic growth relationship with those folks, right? And how does that draw in more contributors and more users to make the, the, the community more healthy? So from a strategic standpoint, there are a lot of other tactical things that we do in, in the in terms steering committee, like getting the IP transferred over to the C NCF F and other things like that.

But the fact is, the strategic part of it really comes down to how do you grow the community. Excellent. You know, what we, speaking of growing the community, what we haven't mentioned is a website, A U R L where people can go get in info, maybe a GitHub address something for people who want, maybe they're not familiar with Q Flow and they want to, you know, want to poke around a little bit.

Where do they go, Tom? Uh, excuse me, where do they go, Josh? org.

Um, the best thing that I would suggest to do is there's a calendar that you can find the Q Flow calendar. Join one of the meetings, they're every other week on Tuesday. Um, those are, we have a series of meetings.

We have meetings for the community and uh, then for each of the working groups, so inside of Q Flow, you may wanna be in the notebooks, uh, working group, which is how do you, you know, the I D E basically of how do you develop applications, right? Uh, so there's that part, or you might want to go to the training working group, which is how do you do distributed training of your ML models? And they do a lot of sophisticated ways, not just scaling up, but scaling down.

I mean, with all of this growth in Nvidia and G P U, you know what you really do need to learn how to scale down your application so you don't get those huge budget bills, or you can utilize your, you know, your, your, uh, infrastructure efficiently. But beyond training, then we have tuning. So now you have to tune your model because there's all these hyper parameters, parameters outside the model that need to be set.

We have an application to do that. It's called CIB or, and also Auto ml. It has two names, but there's a working group for that.

And then finally, the last two working groups are one, the Q four pipelines working group, which pipelines is the workflow. Once you have your model developed, how do you actually run through that in a repeatable way with error correction? So that pipelines is the most popular part of Cube Flow.

And uh, the point that I would make there is they have very, all the groups have very well run meetings. Cube Flow pipelines does, that's a good place to get started. If you're looking to get involved in Q four pipelines and use it, you probably will.

90% of the people use Q four pipelines. So that working group meeting, if you wanna get down and dirty, is where a lot of the application development works. And then finally they're serving with K ser, which is a sister project that's been integrated.

It was inside of Q Flow, then it spun out, it got big enough and became on its own. But we're still very much integrated both from an installation and a workflow. So you have an end-to-end from development all the way down to deployment inside of Q Flow with these five projects.

And that's how you get your hands dirty and get involved is at the working group meeting level. I'll tell you, Josh, once we got you wound up, man, you just let it all hang out there first. Thank you for all that information.

It's a lot to digest. The good news is, hey man, this is recorded. People can play it over and over again so they can, you know, there was a waterfall.

No, no, no pun intended. There a waterfall of information that that came out. Josh, I want to congratulate you though, because obviously you have a lot of pride and for all of the, all the hard work, sweat and tears you put into Cube Flow as well as the community.

And you know, part of this going into C N C F is an incubating project, is the recognition of, of the work that's been done here and, and the, and the usefulness of the project. So congratulations to you and the rest of the Cube flow team. Looking forward to having you on here and talking about your graduation one day soon, right?

As, as you move through the kinda lifecycle in, in uh, C N C F Linux Foundation. Thanks for being our guest today and we'll talk to you soon. Well, thanks for getting the word out and we appreciate all the efforts from your team and your audience.

I appreciate you. Alright, Josh bottom, interim steering committee member here at Cube Flow, now a C N C F incubating project. You'll probably be able to see them over in Chicago at Cube Con.

Cloud Native con, i, I wanna say in November, maybe it's October, but it's coming up this fall. We're gonna take a break here on Text Drug tv. We'll be back in a moment.

One last advertisement if I could. There is a go ahead. Uh, there is a Q Flow summit coming up.

Uh, we're planning it to be, uh, co-located with the K C D C N C F uh, uh, event in Dallas, Texas in early October. Uh, so we will be posting the CFPs and uh, the registrations here shortly, but we'll be there as well as in, uh, there will be representatives at the larger c**n, but the Q Flow summit tends to be a more one day very focused on the Q flow, uh, requirements and user a feedback. So, uh, if we put it in Dallas 'cause just like, uh, Chicago, easy to get to from both coasts.

Absolutely. And the weather might be a little better there. Alright, thanks a lot Josh.

We'll see you soon.