DevOps World London – From ChatGPT to Production – Patrick Debois, Showpad

This is Textron tv. Happily, a great pleasure of being joined by Patrick Deon. We're gonna be talking about, uh, an upcoming talk that he has at DevOps World London.

Uh, Patrick is VP of Engineering with Choppa. Welcome. Good to be chatting with you, Patrick.

Well, thanks for having me, Mitch. Uh, look forward to the event. Always.

Yes, it's gonna be a great event, and of course, I always enjoy talking with you. Um, love to hear a little bit about your, your talk that you have upcoming at, uh, DevOps World of London. I think it's about chat, chat, GPT, kind of the transition from in from that into production, kind of getting your stuff operational, using it.

Yes. Yeah, indeed. Um, I think there's, um, predominantly if you think about DevOps and the new Gen ai, there's two narratives.

One is about making people more productive with Gen ai. That's not what I'm gonna talk about. I'm gonna talk about all the people building gen AI applications and what you need to do to bring them into productions.

So typically people would think, oh, chat GBT is easy until you kind of have to build it yourself. And then there's a lot of problems that need to be solved. Um, so I try to do this both from trying to explain a little bit on, you know, what the different things are, models and rags and chunking.

So, but setting the stage, but then I go into the delivery of the app. What is the pipeline look like? What does observability look like, um, in this new world?

Uh, and I'm finally maybe closing off with some business considerations costs, uh, where we're gonna get the money, um, uh, and things like that. So try to cover the whole spectrum of somebody like really enthusiastic on in the technology. But yeah, trying to put engineering practices into it, uh, as we should.

I think. Mm-hmm. Very nice.

Yeah. Your last point about cost, just even know what, knowing what to cost out is, you know, the first step. Um, I really appreciate what you're saying about generative ai.

It's really easy to think about is the UI going into chat g PT or an API? But you get into the training aspects of it you talked about, and there's of course the large language models and whether you're creating your own that you're training, there's vector databases, way of accelerating performance, um, and there's kind of that cost of operating it too, you know, not just kind of getting that and paying, paying some, you know, server fees or whatevers. It's depends a lot on how that's used in your app and how resource intensive that it all is.

So you've gotta kind of build it and figure out what it's gonna take really to do all that once you get into production. Well, it's still hard to predict because, uh, what we've seen is that, um, different use cases will kind of, uh, result in different, uh, kind of pricing or kind of cost structures. So, mm-hmm.

It's, it's not a one size fit all fits all to kind of cancel, uh, it, there's a little bit of a debate saying, well, there's the marketing who will market the hell out of ai, but it is still about the features. So yeah, eventually we're gonna have to have the discussion whether the features get actually gonna be worth the cost that we're putting in there. Mm-Hmm.

Um, but yeah, it's a, it's an interesting challenge, a challenge that a lot of companies will have. Um, but one of those things, usually they, they're not immediately thinking about, uh, uh, well, they, they do see some limits. Like if you start with chat GPT, this is your token limit.

So people feel it. But then when you expand that to an enterprise, what about it not being predictable? What if it's not available?

How do we deal with this? And, um, and I think that's, um, the facade, or how do you say this, um, the image that Open AI has given us that it's so easy, you know, they make it, you know, it's insane the scale they're operating, but they, they, they seem make it so seamless, but it's still, the chat GPT that they're exposing is AB two C solution. So once you kind of make it like an enterprise, there's different constraints.

There's SLAs, there's like budgeting, like I said, uh, performance. Um, so there's a lot of more factors, uh, and I think the vendors are struggling to mirror the B two C kind of image. Uh, and there's still a lot of work.

So we learned a ton over year, uh, kind of being on that ride and kind of getting things wrong and swapping things out and in, uh, as we needed to. Um, so yeah, that's partly of, you know, the, the talk is lessons learned on all levels, uh, that we at Choppa have gone through the journey and things that I picked up from people, uh, doing similar things in the industry, so. Excellent.

Excellent. Um, where, uh, obviously gonna be talking about some learnings and tips and things that you've come across. You know, you oftentimes you give these talks 'cause you really enjoy this topic and you wanna help people and you want 'em to walk away with something really valuable.

What might be some of the things you hope people get from coming to this session at DevOps world? Mm-Hmm. I think it's, it's a strange thing to hope, but, um, I actually hope more people along this journey.

Um, a lot of engineers still dismiss this as a data scientist than a data thing. And I personally believe it is going straight to software engineering. Uh, it is way more about integrating than it is about that data, uh, giving the extra value.

So if, if, if people get over that mental blocker, it's not something that the other team is doing, you know, in DevOps would say the other silo is kind of working on it and we'll see what happens. Um, mm-Hmm. So for me, that would be a win that they kind of have the courage they see that they can do it.

Um, I hope to demystify some of it. That, that it becomes like clear, like there's nothing special. It is all about integrating stuff.

Uh, and that's, yeah, that would be, you know, my win if more people get in there and try to help us in this journey. And one of the reasons why that's important is that it is nothing that we've seen yet. We've been so focused on having tests who are predictable, uh, kind of, um, very structured.

These things don't lend them to be, you know, pr so predictable. You use another model, everything changes. You knew it was another problem, everything changes.

So it kind of, it does need so much more rigor of engineering to kind of, um, set this up correctly. And ironically enough, I I I say that we've been very good at dealing with uncertainty in ops, so there's a natural fit for you to dive into this world because it is a lot uncertainty that we're dealing with right now. Um, so getting more people enthusiastic about the journey, that would be, uh, kind of my big win there.

I think that'd be a fantastic win. And I, I'm in kind of anticipating in your talk, one of the things I bet people walk away with is the learning curve. Not, not the what it takes to learn, but the fact that this is an ongoing learning journey.

It is a day, okay, you gave me the five steps, I'm good to go and let's just yeah. Off and we're good. You know, plug that thing in and get it into production.

You know, it, it is, we're all still learning this, right? With generative AI and what it takes and what happens when you train the model and it suddenly goes a whole different direction to your unpredictability point. Um, or delivering data with a model with, uh, you know, an application into a DevOps pipeline or something.

You know, this kind of a little bit of different processes. So there's a lot, bunch of things that we're all kind of encountering and experiencing, and I think folks will realize that they're not the only ones learning this. Yeah.

And it's a, it's crazy, uh, the speed that like, you know, I have gray hair, you have gray hair, so I can talk to you. Um, we've seen some innovations in the industry over the years, but for me at least, the speed of this one, the impact after a couple of days, it has been insane. So that means there's no real good material and people are trying to make good material.

The material is outdated so fast that they can't keep up. Everybody's relying, like on LinkedIn, medium articles, kind of blog posts, some videos of an event, and then, you know, you watch them again a year later, it's gonna be all outdated. So we're all kind of getting, um, you know, in, in this mess.

And the, it's actually one of the reasons why I wanna bring this talk out is I wanna bring these stories into public a little bit. Like, you know, in the early days of DevOps, I wanna bring the stories out so we can learn from each other. Now, quite often people are keeping this close to their chest, like, well, it's hard.

We're apparently we're not doing so well. But once they got talking, it's like, well, oh, you have the same problem. So for me, that is, you know, kind of the, by talking about this myself, I hope to get other stories out there as well and see that we can learn the patterns, uh, and get better at this because, you know, it's, it's here to stay.

That, that's kind of my conclusion as well. You don't have to struggle on your own. We're all working through different challenges at different times, but maybe the same things we can help each other.

And that's how, that's one of the ways that advances so fast too. I know exactly what you mean. You go read an article or a blog post about how to do this, and then two weeks later there's another, oh, let's hook it up this way or with this vector database or with, you know, whatever.

Yeah. Um, but it, it is moving so fast. It's, uh, it's fun.

But that, that makes it fun too. But also it's okay if you don't know. Yeah, yeah.

It's a little bit like, like a kale surfing, right? You, you dunno. Exactly.

But, um, you could see it as a continuum of, you know, continuous integration, continuous delivery, continuous learning, and then you could say it's continu, continuously rearchitecturing, whatever you have, swap in, swap out. It's gonna be more and more. I've seen this with mobile, we've seen this with cloud.

You just, you know, any sauce thing, you know, today you're using a, uh, and a year later you swap in another vendor. So, so it is just, um, part of live of these kind of refactorings that you would call it somehow, or like, uh, choosing what's best for you. Um, but it, that is also a challenge because all the big vendors who are supposed to be delivering us this mm-Hmm.

They're all struggling. They have been taken by surprise, let's say it like, uh, in a, in a nice way. Um, so that is also not helping.

But again, we, we've already seen over a year, um, people are picking it up and yes, there's a whole discussion about models, but it, it's soon getting on par, right? So it will shift to something else. Like it's building the whole thing that needs to get better.

Uh, it needs to deliver more value. Uh, and that's gonna be the differentiator. So while you might think now is, you know, the one with the biggest model and the best model will win, uh, uh, I don't think that will be, uh, the case maybe in a year.

So Mm-Hmm. A series of races, not one marathon. Right?

Yeah, absolutely. Well, very good. Excited.

I mean, I hope, I'm sure the, the talk will go really, really well and sharing so much of your knowledge and experience you've gained over working on generative AI in the past year. It, it is interesting. December 5th is when DevOps world London is.

It's, it's a little more, but just a little bit more than a year. When November of last year when generative ai, I really kind of burst on the scenes for most people. So indeed, yeah.

Think back, it's only been a year. That's pretty shocking. Yeah, yeah.

Yeah. And if you look back at, um, you know, the things that I've shown then on slides that were kind of IDs became products already. So I almost like often when I have to like do a new iteration of the talk, it's like, okay, what products can I just, like, what research can I swap to product?

And it's, and so more and more is just, uh, getting there and then you, that makes you wonder what's the next then mm-Hmm. Because you could predict that this once you kind of know as the possibilities, but what is next? So Almost expect that there will be an next and a next.

Right. It's not we solve indeed. Well, great.

I have a great talk and please folks join Patrick, uh, at DevOps World London on December 5th. com. Super easy.

Uh, get that information and uh, see, uh, see your friends and see people like Patrick and their talks. Yeah. So thank you Patrick.

Great talking with you. Good luck. Yeah.

And I'd love to hear your story, so please come up to me as well and have a chat. Uh, I'm there the whole day, so would be happy to hear what you're, uh, you're struggling with or what you learned. Fantastic.

What a great opportunity. All right. Take care.

We'll see folks in, uh, DevOps world, London.