Lin Sun on Building AI Agents in Kubernetes with Kagent

This is Textron tv. Hi everyone, it's Alan Shimo back here on Techron tv. My next guest is Lynn Sun.

Lynn is the director of open source for solo solo io and is also serving as A-C-N-C-F-T-O-C member and ambassador. And with cla uh, CubeCon just less than two weeks away. What a great time to have her on here.

Hi, Lynn. Welcome to Techstrong tv. It's great to have you.

Hey Alan, thanks so much for having me. Not a problem. So, Lynn, as I told the, uh, audience, your director of open source at solo solo io, as well as a TOC member and ambassador for CNCF, but you know, there's a lot that covers up a lot of history.

Give us a little of the history, a little bit of your journey. Yeah, so I am a immigrant to the us. I'm the person that carries $100, come to the us gosh, 25 years ago.

And I started my job at IBM. Um, I've actually worked at IBM for 19 years, uh, before I joined solo, right in the middle of covid. Um, I actually was thinking about retiring at, um, my job at IBM 'cause uh, I was working on it still open source, you know, something I'm really passionate about.

Um, but when the founder of Solo I Lavin reached out to me and said, I want you to lead up the open source team at Solo, I started looking at what is solo? So SOLO is in writing the business of cloud connectivity down, right? Um, connect microservices, how people expose their services outside of their Kubernetes cluster.

So SOLO is right in the domain of service mesh, uh, which is where my expertise was. So I decided to, you know, jump onto the, the hype of the startup hype. I, I feel like I would regret for not joining solo, you know, take the chance, uh, when one day I retire.

So that's how I end up, you know, being the number one employee doing open source at solo and, uh, I'm having a blast at Solo. I love it. So Edit is a friend of mine and I do tell her, send my regards, maybe I'll see her in London.

Um, you know, it's a great story, right? 'cause you mentioned Istio and IBM and Service Smash, and of course, you know, Istio though now is part of CNCF was not always right. There was a little history there with Google and how to that was gonna be dealt with and everything else.

Yes. But Solo is, has, has always had, I mean, for as long as we're doing, covering the Kubernetes space, which is probably going on seven or eight years now, right? The cloud native space solo has always been a, uh, a player when it comes to Service Smash and all of the things that service me Mes brings to it.

And, you know, our audience is a very technical audience and they're cloud native savvy. But for people maybe who aren't familiar with like where service Mesh fits into the cloud Native Stack, Lynn, how, how would you describe it? That's a great, really great question.

So Service Mesh is a term, I guess we started about eight years ago when we started Istio. Uh, there's other projects out there in the market. Linker D is another player.

Service mesh essentially is trying to solve the connectivity problem for microservices, right? Um, instead of having developers solving connectivity, uh, problems of microservices as libraries, um, in their application service message, solving the connectivity problem, uh, for microservices as part of the framework. So as a developer, when you develop microservices, you don't have to worry about how your microservices connecting to each other, how to have the connection, uh, secure through mutual TRS.

You don't have to worry about, uh, rotating your keys and your secrets. You don't have to worry about how do you observe your microservices consistently. So that's what Service Mesh is trying to solve.

It still is a really interesting project in the domain of service mesh. Uh, not only we are the most, uh, popular and most, uh, uh, deployed in production service mesh out there in the market, but I always view Istio as an innovator in the service mesh market, right? So two, almost three years ago we launched, uh, Istio service mesh in ambient mode.

What ambient is really driving IST mesh is towards being the mesh being totally transparent to the user, right? That's the word, uh, ambient. So when we started Istio seven, eight years ago, we always view service mesh as part of the infrastructure that user can run and forget about it.

But we couldn't accomplish that vision of service mesh with sidecars because with sidecar, with every single newer version of the IS issue, it could be a minor version or a fixed bag. User always have to restart their site car along with their application. They always have to worry about the downtime, the operation complexity.

So, fast forward to Ambient. Uh, by the way, we just announced ambient outreach GA in Salt Lake City, uh, at North America last year. So with Ambient, we really have the opportunity to provide a service measure without sidecar that's, uh, totally transparent to the user on the layer for layer, which provides mutual TLS and, um, and simple policy authorization policy on the layer seven layer with, uh, service measure, we're talking about retry timeouts, we're talking about traffic shifting on the edge, TTP layer, uh, we're talking about reach authorization policy that you can configure authorization policy based on method, based on whether it's GA or post based on certain particular path, right?

If we're getting really rich on the HT DP layer, that's when, um, waypoint come in. And what really interesting to me of is still ambient is we're bringing the vision of, uh, Waypoint as a gateway, um, but serving, uh, inter interservices traffic inside of the mesh, right? Essentially, we're bringing the ingress and egress gateway into ambient also as the gateway, uh, for interservice communication inside of mesh as Waypoint.

I believe this is also following the Kubernete gateway, API lead, right? Uh, for the audience who are not familiar with the Kubernete Gateway, API, it's the new networking API for Kubernetes. Uh, people view it as Kubernetes ingress V two.

Well, it not only allow users to control traffic for ingress, but also expand to mesh, uh, with East West. So I'm really excited about is ambient and the, particularly the architecture and the innovation with Ambient. Love it.

I want to talk about K Agent though, so I'm gonna shift gears here a little bit. Um, give us, give us the lowdown on that, Lynn. Yeah, totally.

Um, so as I mentioned, SOLO is in the business of for gateway and service Match, right? So we support hundreds of our customers running, uh, service measure, uh, using a CO and also Gateway. Uh, our gateway is glue and we recently donate the glue open source gateway to, uh, CNCF as a CN, uh, as A-C-N-C-F Sandbox project called K Gateway.

Um, so we have the, uh, hundreds of customers running these, uh, projects and or enterprise offering, um, in, in their, uh, data center or public cloud. And what we find out is our customer facing engineers often, uh, when they working with customers, um, they sometimes couldn't, most of the time they were able to solve, uh, the issues with the customer, whether it's configuration management or figure out how to configure policy across, uh, different cloud, uh, in service mesh, or whether it's troubleshooting, uh, pinpoint, uh, when there's multiple connections in the hub, which of the connections may be causing the problems. Um, we find out sometimes we had to bring in, uh, the top expert as solos.

Certain people like, uh, John Harvard, uh, we have to bring them into help troubleshooting problems. So as our company was scaling to a larger scale, we started to think about how can we free up our top developers from jumping to customer cause troubleshooting with customers to let them focus on writing new features for, for open source or for our enterprise offering so we can innovate faster. So, uh, we started to develop leveraging agent ai.

We started to develop a couple of, uh, agents based on our domain knowledge, which is, is still, uh, observability 'cause we always need to figure out observability for Gateway and me, uh, which is Kubernete, uh, which is, well our control plane and data plane mostly runs on, uh, helm. We use Helm a lot and our customer team. So we start to develop needs sample agents and uh, use it internally to, uh, help our customer facing engineers when they diagnose or help our customers.

And when they, we started thinking about why not just open source, uh, the agents we develop, uh, 'cause not only we develop the agents, we also develop the framework, um, by leveraging the Microsoft Auto Gen, uh, agent AI framework. We bring in UI and CRI for Kubernete and have the framework more integrated nicely with Kubernete. So we have the whole framework stack, uh, kind of developed alongside with the sample agents, along with, uh, a, a couple of dozens of tools where these agents consume.

So this is where K agent come from, right? Uh, we decided to having something internally useful for us and uh, open source, the whole thing. And, uh, we want to be able to benefit the broader ecosystem.

Certainly we couldn't make the agent successful by all itself because we only sampled, um, seeded K agent with a couple of sample agents. We develop internally. What we are really hoping for is, uh, in the next, uh, few months, um, we can bring the vision of K Agent Live.

Uh, what we really wanted to have is having every single CNCF projects out there in the CNCF landscape. I'm sure you look at the landscape before, right? IT 203 hundreds project.

It's a picture. Yeah. It's quite a picture.

Yeah, It's quite a picture and very, very hard to figure out how to navigate. And once you decided a few projects you want to use, it's also very hard to get started and troubleshooting when anything goes wrong. So what we had a vision, if Okay, agent is we are hoping it can serve as an inspiration for the CNCF community, um, that we provide the simple framework, the simple sample agents and tools and having the ecosystem, uh, help us to build the rest of the, uh, agents that we don't necessarily have the expertise.

So we're hoping to have project maintenance willing to join us on the journey of key agents. We're hoping to have, um, developer engineers, uh, platform engineers who have a lot of cloud native operation expertise. Uh, many of these, uh, cloud native projects join us and help us make key agents better, help us making key agents, uh, produce agents in the catalog with every single CNC of projects.

So people can navigate the landscape a lot easier by having agents sitting next to them when they explore any of the projects in the CNCF landscape. Got it. Lynn, we don't have a lot of time left, but for people who are listening and saying, wow, K Agent is something I'm really excited about, it's what we need.

Excuse me. In the cloud native community, what would you recommend for people who want to get involved in the K Agent project? Yeah, so we would definitely recommend the checkout K agent, that staff.

So that site is already up live since yesterday. Uh, we also have, uh, GitHub. Uh, so you can go to the GitHub icon on the website.

It's k agent Dash, uh, dev slash k agent. Uh, we would love to you to check out the project. They all get started guide on our website.

So you can play on the agent, like what I've been doing for the past few weeks myself. And, uh, you can interact with our sample agents, um, using natural language so you don't have to remember some of the commands or look up the project manuals. Uh, so we would love you, give us a star on the GitHub.

Uh, 'cause once the project reach, uh, 500 stars, we are really serious taking it to the next level, which is, uh, donated to CNCF as another sandbox project. That would be great. Lynn, congratulations on K and congratulations to all the folks at Solo.

I know you know it and the rest of the team has worked so hard at, at Solo and it, and it shows. Um, we will see you in CubeCon at London. Yeah, thank you so much, Alan.

I really appreciate the chat. We will have a big booth at, uh, London, uh, at CubeCon. io booth if you're interested in you to chat.

Uh, more about Key Agent. Thank you so much. Thank you.

Lynn Sun, director of Open Source solo io, as well as C-N-C-F-T-O-C member and Ambassador here on techstrong tv. We're gonna take a break. We'll be back.

Remember, we will be live all week streaming from CubeCon London. Should be a great time. We'll be back in a moment.