The Kubernetes Ecosystem with CNCF’s Taylor Dolezal – Cloud Native Now Podcast EP7

Hey folks. Welcome back to the Cloud Native Now podcast. I am Sharon Florentine.

I am here with my co-host, Mike Vizard. Nice. Hello everybody.

How you doing? Good. And uh, we have a very special guest on the podcast today.

I am thrilled to introduce you to Taylor Dolezal, who is with the CNCF. And I believe Taylor's title is head of Ecosystem. Is that correct?

That's correct. Happy Everybody. Awesome.

So I'm gonna start with a little softball and ask you to tell me what head of ecosystem means, and just briefly what that involves, and then we'll, we'll, you know, start throwing some strikes. I love it. I love it.

Uh, as, as head, uh, head of ecosystem at the Cloud Native Computing Foundation, that's a mouthful. So we typically shorten that to CNCF. Uh, I get to work with our end user community mostly.

Uh, I'll still talk with Google and Microsoft and Amazon and, and all the other clouds and other vendors. But, uh, my sole focus is working with folks that are end users. Those are people that are taking CNCF projects and principles and adapting and adapting them within their organizations.

So they're creating these incredible platforms to power how they work with the cloud, whether, uh, in the cloud, whether on-prem or some kind of hybrid solution in between really making sure that they get all the information and, uh, assets as part of the foundation. So it's really like boots on the ground folks who are taking this from theory and really putting it into practice, which is Equal. Exactly.

Okay. It's, And, and folks in end user organizations, you know, they, they have a day job. They're selling products.

They're creating, they're, you know, in manufacturing, they're making cars. They are running giant e-commerce platforms. And, uh, it's difficult to stay updated with everything going on, uh, especially with over 180 projects, uh, in the CNCF.

So, uh, we're trying to simplify that and make that a lot more accessible for folks to be able to keep up to date with what's going on. 'cause again, there are just so many conversations. Yeah.

What's your sense of the temperature of the community then? Because it seems like, you know, Kubernetes maybe is at the core of this, but there's a lot of different projects and it feels like it is taken a while for everybody to kinda warm up to Kubernetes. And I think the it ops team there, at least they're starting to kind of get it, but where, you know, when you talk to them, what are you feeling?

So, like any good technical professional, I'm gonna say, well, it depends. And there's, there's several different areas where people are taking a look at. And, uh, while, uh, a lot of folks will make fun of the, the cloud native landscape and all of the projects on that, uh, it really helps to zoom into that.

And so taking a look at whether it's runtime, whether it's databases, whether it's either, um, I believe there's about five different sections as part of the landscape that you can kind of focus in on. And then subsections as well. So whether you're looking to provision infrastructure, no matter where you're looking at as far as the phases of a, uh, adopting cloud native principles or projects within your organization, um, we, there's, there's definitely a project out there for you and, uh, within each of those communities.

It's interesting to see how people start to take something like Kubernetes, pull that in, and then they say, okay, cool. We have our orchestration, we have our deployments down. What about observability?

And then they kind of venture and just bounce around all of these other different types of projects and start to pull them into their platforms and then share out those stories of how they've done really well at that, or some really fun and exciting outage stories that they've hopefully fixed at that point. So depending on where folks are at in that, uh, adoption, uh, those adoption phases, that's, that's really where it varies. But in most cases, I'm, I'm, I get the chance to talk to people about their open source strategy too.

So do they completely pull in this project and just, you know, pull it directly from those releases? Do they fork that and then house that in internally so that they can work with security and other platform configuration type teams, or, those are the kinds of conversations I have, and those are always really, really good. It's always really introspective, and even even on a bad day, the respectful disagreement, you know, those are really captivating conversations, trying to figure out the why behind what people are trying to do with these workflows.

There was, there's no substitute for skinning your knees though, right? No. No.

Yeah, sometimes. Yeah, sometimes you gotta go through the fiery hoops. Well, and one of the things that, that Mike and I see a lot is, you know, we tend to, on the sites, publish things about, here's the next new thing, and here's how you can, um, take these cloud native open source projects and do these incredibly complex things and, you know, huge enterprise deployments and stuff.

But what we find is that a lot of folks that are reading our stories on cloud native now are a little further behind than we are in the press as far as like talking to these companies and talking to developers who are just starting out. Is that, are you seeing a little of that too? Like, a little bit, I don't wanna call it a lag, but, uh, you know, it seems like a lot of the user base is just now starting to get their arms around this.

Yeah, definitely. I think that whe before, uh, before publishing things out, um, there's a lot of reconciliation that typically happens, right? Uh, you don't wanna be first to publish something and then go, oh, shoot, we got that wrong.

Right? Um, e even taking a look at tools like, uh, I think it was subversion as far as like a version control system, not Git, but subversion, um, where people asked one of the first feature requests was for private branches. Uh, 'cause people were like, I don't want people to see my code just yet.

So I think it's a little bit of that too. And then it's really interesting to see some of the, uh, sort of forum members for end users. We have, uh, private meetings that they can join and discuss some of these things.

So before they go and adopt a new CPU architecture or a new type of compute in their favorite cloud, then, um, they can really level set and, and check with others and say, okay, you know, uh, Intuit, Mercedes-Benz, insert company here. Have you tried this? Has that worked for these kinds of workloads for you?

And then even level setting between these different types of business verticals as well. So finance might operate differently than manufacturing or, or education or video games. And so there's a lot of different things that people can investigate and check into.

So I think that's why there's a little bit of a lag on that front as far as sharing these things out. And at the CNCF, we're trying to cut through that and release more reports. Um, this year we, we, we have something called, so we have case studies, but then we also have end user journey reports.

We have one, and it's from 2019, uh, for Spotify. So it's interesting. Mm-Hmm.

But that's a little outdated. And so we have a couple that are, uh, going to be coming out this year, um, from some larger organizations. And we're, we wanna publish more of those because people are asking those questions like, how do you use this?

What can I copy? What can I use? And then our end user technical advisory board or tab just spun up, um, near the end of last year as well.

They're going to be publishing and overseeing some recommended architectures that end users are utilizing as well. So those are two different areas where we're really trying to meet the end users where they're at, answer those questions or, and spark new conversations around, is this really the best way to adapt, adopt, or use these kinds of tools together? Yeah.

So are you orchestrating essentially group therapy for the community? Is that how that comes together? In, in, in, in my house, I call it cloud native therapy.

Yes. That sounds like a column. Mm-Hmm.

You could talk about the cloud native therapy column. This is like, I, I, and you just, you just made me realize I'm missing my stress ball, uh, from my desk otherwise, i's gonna show you. But yeah, it's, I've got a little py stress ball, but, uh, comes in handy.

Comes in handy. I bet. I bet.

And you know, that's one of the things that we've talked about here on the podcast and on the site too, is managing the complexity. Because with all of the benefits that cloud native architecture and approaches can have, the big thing that people struggle with is that complexity. And, you know, so how do you, how do you help folks through that with security or just development or networking?

Networking especially is a huge complexity issue, Whether it's networking in person, uh, you know, with different people and organizations as well as, you know, inside of your Kubernetes stack or your cloud native stack. Uh, both, both, both are huge topics of conversation. And even end users are asking one, and they, they've asked me, they've asked one another, like, who is the best person to talk to you about this?

Because, uh, they, it's, it's, it's a lot to take on. Uh, when you start using Kubernetes, if you're using some of the primitives that are, uh, framed, well there, you know, deployments, services, ingress controllers, even deciding what, you know, which ingress controller you want to use can be a really deep discussion and take several hours or bike shedding or, um, deliberation, uh, between teams. But the more that you use of Kubernetes, of course, the, the more complex that it's going to be.

And, and we see that the longer that people use Kubernetes, the more things they want to take advantage of. They wanna rightsize their workloads. They, they wanna be using it as effectively as possible.

Um, that, that same adage of, you know, you, you can't have a hammer and then see every problem as a nail, you know, uh, or try to unscrew something with a hammer. It's gonna be really difficult to do that. We don't, I don't advise, please don't do that.

No. There's a better tool for their, for the job. So, uh, in tackling that, I, my, my biggest recommendation on that front is to iterate.

Just iterate, frame some metrics, get some SLAs, try to think about things in an iterative approach. Don't boil the ocean. Don't tackle this all at once, especially when dealing with those legacy systems too.

Mm-Hmm. By thinking about these ways that you can build trust with your teams too, right? Because you don't wanna do that and change everything all at once, then your management and other people aren't gonna have a lot of faith in you if you can't pull it off.

But by starting small and then growing bigger and bigger and bigger, as you start to get a handle on some of these things, that's the best way forward. That's where we've seen a lot of, uh, teams become high-performing teams, is by taking that approach, uh, rather than everything at once. But I get it, you know, when I'm cleaning my house too, I want everything all at once to be clean, but, uh, gotta take it one step at a time.

Yeah. I feel like we're on a, a journey that's full of these kind of little ironies in the sense that early on, a bunch of full stack developers, you know, not many of them spent five years convincing IT operations teams that Kubernetes was the way to go and the thing to have. And now all the IT operations teams have finally figured that out, and they're like, yeah, this is great.

We can centralize the management of all this stuff and we can be platform engineers and everything will be great. But the rest of the development community is now going, whose idea was this? And why?

'cause the, the abstraction level is a little too low for them. I think It's, uh, it's, it's interesting that you say that, uh, anecdotally, I, I used to work for Walt Disney Studios and was one of the teams that helped move the cont the workloads were already containerized, but we were moving it to Kubernetes and, um, kind of getting out of some other orchestration platforms and the entire time, months long transition to get all of these microservices, interconnect them, weave them together, um, that, that, that it was very much a DevOp dev versus ops story before it became a DevOps story. And the team kept saying, this is so hard.

I don't understand this, what's going on? Uh, you know, constantly trying to elaborate and give a little bit more insight into what systems engineering things were happening, why they needed to happen, why containers can restart or go away, you know, when it comes to these stateful workloads. Um, a lot of back and forth.

And then, uh, it made me learn a lot too. 'cause I had to, I had to be able to defend and explain this stuff or explore some solutions, uh, that weren't the golden path with these teams. And then by the end of it, they said, we get it.

This is, oh, this is, this is so nice. But it takes a lot of, there's a lot of friction and, uh, to get to that point, right? So, uh, I definitely see that with, with teams.

And then sometimes you don't make the right choice. And being, being nimble and flexible in making sure that you, atta you tackle this as part, as far as components, or this is the thing to do X, Y, Z, this is the thing to do. A, B, C, making sure that you kind of attach more to the interfaces and the specifications is also a great way.

'cause if something doesn't work, that ability to swap it out or that option to, you know, break glass in case of emergency, that that exists. So when teams see that complexity, that's typically what, what I try to point them towards is rethinking how they've composed all of these things together to create that giant platform that they have. Okay.

So we've made it about halfway through, I think, without AI coming up, but I'm gonna bring it up now and ask what the role of AI and generative AI is gonna be in helping solve this complexity issue. And, uh, what you're doing on that front now, where you expect it to go in the future. So ai, so I've told people you can't spell cloud native without ai.

You know, you can try, but you'll need to buy some vs. So, uh, true there. Uh, I'll, I'll, I'll try to be concise with this.

'cause there is just so much, and that's a very, uh, it's, it's, I need AI to understand all the AI news and help me summarize a lot of these things as I go through my day to day. Yep. But so Do we.

It's just like there's no, We have whole about it. So, Um, it's, and it's, it's fun to be in the hype cycle and kind of like see all of these things that are going on and, and try to, uh, get some sense as far as where, where we starting to develop primitives or, or some sane defaults on these things. And, um, zooming out, we're seeing, uh, we are seeing Kubernetes and, and platforms like slum used for training these models.

So that's one part of, of the problem. The equation within AI is getting these models together in the first place, being ethical about how you source that data, how you put that together, and you train these models and then hopefully they're, you know, smaller or host hostable, uh, by, uh, some of the Apache two licensed ones or these little bit more, uh, more in compliance with the LF and CNCF, uh, types of rulings on open licensing. The other side of that is inference, and that's the one that really has captured people is they're not so much in, you know, at, at least in the infrastructure side, it's not so much about the training, it's more about the, oh my gosh, how do we use this now?

Um, how do we think about hosting LLMs and, and, you know, small language models, large language models, vision language models. And so that's currently dis the discussion is how do we make that palatable on Kubernetes? What projects exist for that?

Mm-Hmm. Um, what I'm seeing, so l LMS are incredible at reasoning. So we used to have a whole bunch of if, if else statements or case statements, all these ways of, um, understanding, you know, conditionals with within programming.

But now we can send something to an LLM and get a reasonable or really intense answer back, uh, that, that helps us finish out our workload or add something to the user experience. And, uh, that needs a lot of GPUs. That's, that's a big part of the conversation right now.

But the CNCF is specifically interested in that infrastructure side and the application side. So we're starting to have another DevOps moment where it's AI engineers and INF and infrastructure engineers, and we're trying to help each other out and bridge the gap there. The one that's not so much a part of this, um, is the hardware folks.

So it's hardware infrastructure, application teams. Those are the three layers from what I'm seeing in the conversations that I'm having now. And we're trying to, so far it's been AI and ML engineers, um, doing their thing.

You know, we, we understand that infrastructure fairly well, but it's this inference side of things and trying to make, trying to optimize that and rightsize those workloads that's current, that's an ongoing conversation. Thinking about how we add features to Kubernetes or the community adds features to Kubernetes and how we can collaborate on that front. How is the inference engine any different a software artifact than any other type?

So are there unique attributes there that people gotta think through? There's a lot more consideration. So the best way to think about it is, is like another dependency within your stack, whether it be a database, a caching layer, that's how we can host it really well.

But the, the necessity for GPUs to be part of that stack is what makes it different, right? We can't just schedule it, give it this many CPU, this much memory, okay, we're good. You have to really dial in the GPU.

Do we use some of the cutting edge ones or can you know, does a GPU from three years ago work? Um, these both for training and for inference is, is something that we're seeing that exist as a problem today? So thinking about how Kubernetes kind of tiers these assets, do we tag, is there something lower level that we need, um, at the node level where we're actually physically connecting with this, uh, to our machine in a data center?

Those are more of the concerns. And then around the auditing and things that are getting passed to these, LLMs is also something that is being considered. So, um, if I'm a healthcare organization and I'm passing my medical record number or my social security number, do I even want that to go to an LLM?

There's a lot of additional layers around this job breaker of an LLM, uh, that, that folks are thinking about. Not applicable to every industry, but, um, uh, we're, we're also seeing a lot of costs on that front too. If, if we're using a vendor organization like OpenAI or Microsoft or Philanthropic or DeepMind, um, uh, we might not want, we, we might not be able to afford that.

You know, organizations might not be able to afford that in certain cases. So they might wanna leverage some in-house, uh, LLMs or S SLMs that they've trained before going outside and using that. But it's, it's really interesting.

So rather than a direct call to a database, it's a lot more routing and logic behind where do we actually want this request to go? Or do we want it to happen or fire back a message to the user saying, Hey, you asked for this, but this is kind of outside our policy. Can you reframe, um, this prompt that you've asked for?

Sorry, we can't, we can't do anything with, uh, social security numbers. Try again. Right?

But in the same time, it's not like I can patch these things, right? I mean, basically if something goes wrong, I'm ripping and replacing something that is measured in terabytes. Exactly.

Exactly. And, and that problem too of in batch workloads or these really complex kinds of, take this LLM transform it this way, and then you have a chain of that, then it's like, you know, okay, my workload blew up on step three of 12. Shoot, do we go back to the beginning?

That costs a lot of, that might cost a lot of money to run. Uh, we have thousands of records to do this. We have to, uh, work with, uh, a whole bunch of data or, you know, rethink our workflow, whether it's a retrieval assisted generation or something else.

So Q Con is kicking off this week, and, uh, Mike's gonna, Mike's gonna be there in person on the floor. And Taylor, I assume see your smiling face there as well. Absolutely.

Awesome. Yeah, I'll, I'll be sure to be caffeinated and hydrated. I'm, I'm looking forward to it.

Fantastic. What, uh, what are you, what do you expect to see? What do you think the, uh, besides ai, let's take that off the table.

What's the, uh, what's the big buzz gonna be this year? I think that, so Kubernetes is really starting to fade into the background in the best way possible, right? Similar to Linux.

We're, we're seeing a lot of folks standardize on Kubernetes like we had talked about earlier. And being that that's the case, more problems are coming to the forefront. So, uh, OpenTelemetry, for example, is, is that just has an incredible velocity right now.

Yeah. Um, so understanding your workflows better, getting this kind of telemetry incredibly important. Um, projects like Argo and, and Backstage are also picking up in intensely.

So I definitely see a huge lean towards platform engineering. And even though you said AI's off the table, that's, that's part of it. So GI ops, observability, telemetry, um, how do we compose these systems?

Are we doing the best that we can? Is a lot of what I imagine some organizations will be asking. And then the ever popular hallway track, uh, where you can just get to sync with some people either meeting with maintainers contributors or, you know, the, the nerd famous people that you've, you've had your eye on or love seeing their, their Twitter, their, their tweets and blog posts and things like that too.

So, uh, definitely I'm looking forward to a lot of the connection there. And just getting to interact and talk with people directly. Open source strategy is always a big one.

Granted, it doesn't have, like, that's not a main highlighter focus or something that's usually, uh, reported on, but that's, that's always a conversation in every single CubeCon that I'm really looking forward to. And then, um, some others that I have my eye on are, uh, web assembly, uh, at, uh, rust Wasm. We've had some, we've seen some directives from the White House around, you know, you should really think about memory management.

Uh, just because they sent that doesn't mean that the work is done. We, we need to figure that out, uh, as a community. And then, uh, taking a look at Edge Networks as well.

Um, that's always something I kinda see go up and down in terms of popularity. But, uh, when you're looking at latency time, especially around all of these new things that you need to add to your platforms, that's incredibly important. Uh, doesn't make sense to be sitting, waiting for that, you know, spinning, loading, buffering, uh, type of experience for your user.

And then last but not least, is the developer experience. We're seeing a lot more tools come out, um, how it, it these directly correlate to developer saving time and companies saving money is, if a tool's easy to use, you're not gonna spend hours researching how to put it together, how to make the best use out of it. So, uh, that's also something that's really been coming up as a forefront.

And, uh, I, I'm very excited about that. I love it when a tool's easier to use. All right.

Well, let me ask you one thing about that. Do you think our friends in Europe are further ahead in terms of their adoption of Cloud native than some of the other large continents that make up this world? Definitely, definitely.

In certain ways. I'd say that anecdotally, a lot of my conversations in North America are, uh, it, it does seem to me that it's a little bit more vendor driven, right? We have San Francisco, I, I live in Los Angeles and California, so, uh, haven't gotten down to Silicon Beach as much as I, I, I've, I should.

But, um, I, I'm, I'm seeing a lot of vendors drive the space and a lot of the innovation, at least here in North America. And that's, you know, that, that makes sense. That's kind of how it is within the ecosystem.

But in Europe, it's incredibly interesting 'cause we have organizations like CERN that are driving things through at a ridiculously large scale. So hearing some stories like that, or Mercedes-Benz, or SCNF, the, the Trane organization, seeing what is going on there, um, I, I'm, I'm having a lot more success interacting with folks from Europe and those end user stories, um, more so than vendors, at least, at least within that region. And because of the large scale of so many European organizations, there's a lot more.

Um, that's, that's, that's, that's really where I'm seeing a lot of good case studies and, and stories come out to help other end users and, and people k kind of move forward more quickly. Um, typically internationally, it's just very difficult for enterprises to, uh, you know, to really adopt Cloud native in a useful way and then proliferate that across their entire organization. That's constantly the problem is how do we knowledge share, how do we make this a little bit more consistent, uh, universally within an organization.

Tools like Backstage and others can help with that, you know, documenting and sh and again, knowledge sharing. But, um, it definitely depends according to the industry. But I really like what I'm seeing coming out of the, the European region.

And then we also see, you know, in terms of CubeCon attendance, it, it's, it's, it's huge. There's, even with travel cuts, budget freezes and things like that, we still are going to have so many people at CubeCon in Paris. All right.

Well, I think that will, uh, bring us to time folks. Thank you so very much, Taylor, for coming on and chatting with us. It has been great to have you, and, uh, we hope you will come on and check back in with us soon.

Thanks for having me. All right. And if I see you, if I see you this week, we'll grab a pilsner, right?

Oh, I love it. I'm, I'm down. I'm down.

Cheers. All right, perfect. All right folks.

Thanks. Well, we hope to see you again next week for another episode of the Cloud Native Now podcast. I'm Sharon Florentine.

And I'm Mike And we are saying goodnight and good luck.