Pulumi’s Joe Duffy on New Infrastructure Libraries

Hey everyone. I hope you enjoyed today's Textron gang. It was a little hot and heated there with a couple of contentious subjects.

Um, if you couldn't tell, I am a big fan of working in the office, but it was a great show. We hope you're enjoying Textron Gang. Let's get onto the rest of our Textron TV today.

First up, it's me talking with my good friend Joe Duffy. Joe, of course, is co-founder, CEO of Lummi, and they recently announced some new infrastructure libraries, uh, may, uh, allowing you to use a Gen AI or abstracting with the Gen AI stack. Joe tells us all about it.

Here it is. This is Textron tv. Hey, everyone, welcome back here to techron tv and welcome to our next segment.

Uh, our guest for this segment is fellow who's been, uh, he's, I, I can't tell you how many of these he's done. He's been on with us many times, almost, I think from the beginning of drunk tv. It's my friend Joe Duffy.

Joe is the co-founder and CEO of Lummi. Joe, welcome back to Tech Drunk tv. It's great to see you, man.

Thanks for having me, Alan. Great to see you again. Pleasure.

Uh, so Joe, as I said, you've been on a long time. We, we've probably asked you this before, but I'm gonna assume a lot of people out here and a, they don't know who you are, and b, they may not know Lummi either or may they, maybe they think they know Lummi, but who really knows Lummi. Joe, why don't we give us a little bit of, of the background here on both you and Ulu?

Yeah, for sure. So, my name's Joe Duffy, founder, CEO pmi, uh, started the company actually about seven years ago, believe it or not. We're coming up on our seventh anniversary next month, um, which is kind of crazy.

Uh, before that, I spent, you know, over a decade at Microsoft, uh, working on developer platforms and developer tools like Dotnet, uh, was leading developer tools strategy before, before leaving to start Lummi. And the thing that struck us when we started Lummi was the cloud really changes everything about how we develop software, uh, from the design and architecture to the, you know, the implementation, uh, and of course how you run and operate it. And yet, you know, developers out there weren't thinking of it the same way.

They still thought of it as, you know, the world of two virtual machines and a database. Uh, and meanwhile, infrastructure teams were, were, you know, dealing with a lot of tools that paled in comparison to what we had built for developers over many decades with great languages and IDs and test frameworks. So we sort of thought to take a step back and, you know, we're sort of in the era of distributed computing and let's reimagine what that would look like in a modern way.

So that's what Lummi is best known for our infrastructure as code technology, which is open source. Uh, and we've since expanded into a broader cloud management platform with secrets management, configuration management, uh, search analytics, insights, and what we'll talk about more today, uh, AI for infrastructure. Absolutely.

And, and Joe, you know, chicken in the egg question. I think part of what has made Lummi successful is the overall success and acceptance of open source software, right? You couldn't have Lummi without open source in some, in some ways.

I mean, you could theoretically, right? But, but really it's, it's a big piece of that puzzle. A big piece of the equation.

Absolutely. And, um, so it, it, it's an interesting thing. com or do ai, I always forget Uh, dot com.

Uh, and yeah, just click one of the blue buttons, get started. It is, as you say, open source is in our DNA super important to us. com/pmi as well, if you wanna check out the repo.

com. Very cool. All right, so let's jump into this.

You guys have some new infrastructure libraries, uh, and they involved you and I, why, if you can explain it a little better than I can to our audience. Yeah, I think of, you know, we've been involved in AI in two ways. Uh, one, using AI for DevOps automation and workflows.

Uh, so we have, you know, uh, Lummi co-pilot, which is a chat bot that can cr generate your infrastructure's code for you. And increasingly over time it's just getting better. And we're, we're able to apply that to your unique, uh, organization's challenges.

But the second pillar is actually using PMI for AI workloads. Uh, when I think about AI workloads, I think of infinite compute, infinite data. Um, and, and PMI is really good at provisioning and managing infrastructure at that scale.

But the thing that we find is, you know, a lot of AI teams are not cloud infrastructure experts. So they, they have a, a thing running on their local laptop. Maybe it's a lang chain or some AI application, but when it comes to running in production, a lot of folks aren't sure exactly how to go about that.

And it turns out, when you talk to folks, a lot of the architectures are very similar. You know, there's kind of two classic patterns we see. There's, there's training the models and serving the models.

Those are heavily GPU based architectures, uh, very gnarly stuff. Often a lot of folks using Kubernetes for those. Um, but the second, which is what these new libraries are more about is the application.

You know, there's usually a front end, you know, whether it's a chat component or an integration into an existing app. Uh, and then there's, you know, um, resource augmented generation rag, which is, you know, Lang Lang chain is one way of stitching together a workflow of ai. But the thing is, running this in AWS or Azure or Kubernetes, it's tough, right?

And everybody's recreating the wheel. And so these libraries are meant to make those, you know, super seamless and easy. I love it.

Um, wanted to go back to this chat bot that you have built in here. Is that running off of, uh, your own, like custom, uh, LLM or SLM? Yeah, we've, we've built it to be agnostic to the LLM, so we can run it on, you know, uh, open AI or Azure open AI or anthropic, but we've extended it with knowledge of cloud infrastructure, uh, which is what we're good at.

And, um, and so it's very good at generating cloud infrastructure or infrastructures code to, to create that, that cloud infrastructure. And over time it just gets better. I mean, we've, we've worked on correctness and continuing to refine the models, uh, but it is, yeah, that, that's sort of our secret sauce.

Very cool. Um, let's talk about end users out here. People, I'm sure there are people on here right now who are our Lummi users, right?

Customers, how do they engage this? How, you know, how do they make it happen? Yeah, I think, you know, I'm increasingly telling people to start with the ai, um, because the thing that's really tough is, you know, you think of we have over 150 clouds supported AWS Azure, Google Cloud, Kubernetes, snowflake, CloudFlare Datadog, you.

And then each one of those has hundreds, if not thousands of services with different ways of configuring them. I mean, it's very daunting to get started. And so I often suggest people to get started with that and just describe in natural language what you're trying to do.

You know, Hey, I'm trying to build a microservice on AWS or, you know, I want a static website on Azure, and that's gonna guide you down, you know, the path that you need to go down. Um, if you know what you're gonna build, like, for example, one of the libraries we worked with, pine Cone, who's a vector database company doing amazing things. A lot of people using vector databases.

In fact, we use it for our chat bot for, uh, effectively, you know, compressing a lot of knowledge that we can then use to make the AI better. Uh, and so if you're trying to run Pine Cone in production, we now have a reference architecture for that. Uh, similarly, another library is Lang Chain.

Uh, so if you want to run Lang Chain, you've, maybe you've got it running locally on your desktop, but now you want to run it in AWS uh, our reference architecture that we worked with Lang Chain on is super easy to get up and running. So if you know that's what you want to do, just go straight to those reference architectures. Uh, they're up on our website.

Excellent. You know, I don't know if you knew this, but we, we did a hackathon here back in like late August. org, some other folks too.

Really, really good time. Um, and it, it, I really learned about how to train your AI and how to inject, whether you want to call it a small language module or, or what have you. Mm-Hmm.

Um, but my biggest takeaway from it, Joe, was it's real. Right now. Anyone who tells you it's not real, doesn't know what they're talking about, but the promise of what it could do as we continue refining this and getting better at it, is crazy.

I mean, I will tell you since August I've learned really how to do prompts, which is a nothing thing, right? Just how to write a good prompt it, and it blows me away. I, I can't, you know, as you sit here, so this is your first release with the, with these libraries, how fast, how far do you think we're gonna be going here?

Very fast, very far Short answer. Honestly. Things that typically take years are taking, you know, one year takes one month, uh, to make that amount of progress.

So it's very hard to predict. And, you know, that's one of the things that we face is, yeah, we can go spend a lot of time training and refining our own models, and yet GPT five is gonna come out, you know, sometime in the next few months, and it's just gonna eclipse everything we could have even imagined doing ourselves. And so figuring out where to spend the energy and the effort is, is definitely challenging.

But I think a lot of people using AI to apply it to problem domains. You know, we talk to customers all the time. Everybody has an AI slush fund, even, you know, fortune five hundreds.

It's, you know, finance, you know, healthcare, like literally every industry management is saying run fast. Just build value using ai. And that's very exciting.

But if you're an engineer in such an organization, like it's, it's pretty hard to know how to get started. Agreed, agreed. Um, the other thing though, and it's funny that I told you, I was on a webinar, uh, a round table before I came on here, the proliferation of, well, there's the proliferation of ais, but look, you could just make sort of a generic that plugs in, you know, different ais on, on the, on the go, but the proliferation, proliferation of tools that are then AI enabled.

Mm-hmm. Right? It, because it's also speeding that up.

It's speeding up a new generation of tools by leveraging this AI that I, I, at some level, I guess it's confusing for developers because, you know, they have a wealth of choices that maybe they didn't have before. But on the other hand, it's gotta benefit them at, because, you know, we've been through the technology game before, Joe, the, the kind of rises to the top and the crap settles to the bottom right. The market is ruthless like that.

Yep. And, um, you know, the, the tools that are gonna rise to the top here are just phenomenal pheno. I mean, the things that they could do is gonna be crazy.

How do you, how do you keep Lummi have its edge? Is open source a secret source for it? Is there something else?

What do you as the co-founder, CEO, I'm sure there's something you think about all the time. Yeah, I think the, one of the best bets we ever made was to bet on programming languages. You know, that that was our unique angle on infrastructures code, is we're gonna stand on the shoulders of giants.

And every benefit in the domain of programming languages now accrues to PMI as well. And that was true of, you know, test frameworks and ides and refactoring and libraries and package managers and, you know, secure supply chain. And, and now it's true of ai.

You know, instantly GitHub copilot was good at writing PMI code, uh, for example. And, you know, as the industry continues to evolve and make those things better, we, we benefit from that. I would say we also have our secret sauce, you know, our understanding of cloud infrastructure, all the metadata, the semantic understanding of the cloud, and we can use that to develop better models.

And so that's where we've been focusing energy. But my feedback to the team, there's a lot of people, I call it AI cookie licking. Everybody wants to lick the cookie so nobody can eat it.

Um, everybody's, everybody's doing that. But my feedback to the team was, sorry, that's kind of gross. But, uh, No, no, I like it.

It's a great analogy. Yeah, We used to use that phrase at Microsoft quite a bit, uh, but, uh, but my feedback to the team was, let's use ai, but, but let's not do it just for AI's sake. For every problem you're facing, take a step back and say, how would I solve this in an AI first way?

And if that genuinely leads to a better user experience or a better solution, let's do it. Uh, if it doesn't, that's okay too. Um, but that's been, that's to your point on the cream rising to the top, that that's been my philosophy and how to tell the team to make sure to focus on the cream versus the crap that's gonna ultimately fall to the bottom.

Absolutely. Absolutely. Joe, we're about outta time here, but you know what, I'm glad to see Lummi is riding the wave as, as you have.

I mean, you know, I'm probably working with you now, I you only four or five years, maybe more. Yep. Um, and you've, you know, you've always kept Lummi on the edge, so I'm not surprised to see you out in front on this one as well.

You need to go have, you're gonna have to come back though and tell us how things are going. Absolutely. I'd be more than happy to.

And thanks for having me All. Joe Duffy, co-founder, CEO of Lummi, go check out their new infrastructure libraries that allow you to, uh, work in gen AI here with all of the great languages and, and infrastructure that Lummi helps you with. We're gonna take a break on Tech Drunk tv.

We're gonna be back in a moment.