Integrating AI Co-Pilots and DevSecOps – DevOps Chats EP9

Hey everyone. I'm Alan Shimel. And I'm Mitch Ashley.

And you're listening to DevOps Chats. Again. Again, you know, b***h, you know what I realized?

So many of our friends who do dev, who do podcasts, they have like episode numbers. Episode 47. Episode 23.

I feel like I'm watching The Blacklist. We never really did the episode number thing. No, we didn't.

I I think it's just 'cause we did it as, as we could, you know, and I figure the, the numbers will take care of themselves, right? It'll be in the order they're in, I guess maybe should just go on the date. We should have someone, maybe one of the interns or something who count up how many chats we have just for the heck of it.

I think we, we'll get a number. Sure. You start eight.

Let's just Captains log. Yeah. You start there.

Yep. Engage. Um, anyway, so Mitch, you know, we, I I, I was doing some, uh, text drug TV interviews today about text Drunk Gang with you.

Uh, and, and it occurs to me, you know, 10, 15 years ago, if I would've talked to you about my copilot, you would've thought I was flying a plane. Exactly. Yeah.

And, and today it, it, it's amazing how quickly it's, it's kind of moved into the vernacular that a copilot is the AI assistant that helps you use a particular piece of software. And I, I, truth be told, and I'll, I'll say this, I first heard the term around Microsoft Mm-Hmm. Products.

Yeah. GitHub, uh, office 365, what have you. And, and to me it was almost like, oh, this is a new clippy.

Yeah. But it, it's not the Nu Clippy, it's Thank it's so much more. Yeah.

You know, it's so much more than that. But still it's that, you know, person in me. Um, but what we've seen is, is copilot has become like Xerox, right?

It's a, it's a, it's a term xenex. It's passed into the culture. It doesn't belong to any one company.

And, um, it's, it just means an AI assistant. And every, it seems like every DevOps tool vendor out here, of course, the whole spectrum of DevOps and even cloud native tools are coming out with copilots, with pilots. And, you know, we, we had this discussion on the gang, how many copilots are enough?

You know, cockpits aren't that big. How many copilots do I need and do I need a pilot to rule all my copilots? You?

That's why we have a copilot. I will note to your analogy we used on Textron Gang, which is airplanes only have one copilot. You don't dev when copilots on, there's copilot and one copilot.

Yeah. I think that can get by you. It's, I mean, it copilot is sort of the low hanging fruit of a gen AI right now.

That's the easy way to Mm-Hmm. Get the next step up from after chatbot. Um, but the challenge is, you're right.

How many do we need? How many every, every product has a copilot it, not not just DevOps tools, but every product, every application we're using now either has or will have a copilot. And first of all, I don't wanna pay 20 or 30 bucks a month for every one of those, right?

So which ones do you want and need? Now, if you're doing development, you know, the GitHub, Microsoft copilot probably makes sense, but there's others. There's testing, there's security tools.

There a lot of tools that have copilot. And each one is, they're at the same copilot. They're their own LLMs and interface and, uh, prompt engineering to use each one of 'em.

So it's like training. Train yourself on how to use 10 different search engines. When I just wanna know how to use one.

Same thing for copilots. I don't wanna know how to use all 10 of them, uh, because they're all different. Just make it easy.

I'd really just not one. Absolutely. Absolute.

Yeah. You know, I, I just did an interview with Scott Erlanger. Scott's the director of product marketing DevOps at at t Tricentis.

And they recently made an announcement. They came out with copilot. They did well at least for one of their product lines.

And darn if I remember one right now, but, um, which one it was right now, but it's on my interview on TechOne tv. But in any event, your point, it's an upcharge, right? It doesn't come by default with, with the product.

And you need to, you know, it's an option. Uh, I don't know for how long it'll be an option. I think it becomes table stakes and people are not gonna want to, people are just gonna expect it as part of it.

They're not gonna want to pay extra for it. Mm-Hmm. Sooner than later, right?

So, I, you remember when you used buy a spell checker? This is a long time ago, right? We would buy a Absolutely.

Remember that. I do remember that. We don't pay extra for that.

So I think that's, I I agree. You and I are on the same trajectory, which is absolutely, eventually this is baked in, right? It has to be baked in.

But I'm also wondering is I wanna get familiar with just one interface, one voice. Do I have, is my copilot the same AI all the time and I just plug it into different back ends depending on what app I'm in? Mm-Hmm.

But I only have to kind of learn one, I only have to learn to deal with one co-pilot. Yeah. Know, it's, it's like, um, I dunno if there's a good analogy, but you want, eventually, you want one co-pilot, one AI interface that we'll talk to the other co-pilots of the tools that you're using and it knows how to take your prompt that you wrote and customize it for how that, that specific LLM likes is engineered for prompts, right?

Instead of you learning how to do it for every individual LLM it seems like that that's kind of the real assistant you want is I don't really care where it comes from. I just want one way of querying, asking, requesting things to get done. Whatever looks like it, let it interpret it for me into the right format for a downstream LLM.

Agreed. I, um, Mitch, I'm, I, you know, I, I get that each one of these copilots, 'cause each of these applications that I'm using is like, is its own program, its own, I don't know if you wanna use the word platform its own thing, and, and they don't necessarily bleed over, but I, I just like, are we, are we thinking this through or we just mad rushing in that AIing everything right. In, in terms of how, how's, what's the end user?

You know what I mean? I, it, it reminds me of the days when you'd see people with those utility belts with like three different beepers and a phone and, you know, well, this message comes from here. This one I can only use for office work.

And this one is, you know, with my wife, that's the bat phone. And, you know, and, but now we just have one phone and that communicates, those are the days when you heard the, you heard the pager go off and everybody in the restaurant reached for their belt. Right?

Right. We go like that. It was like, are we in our awkward adolescent stage of copilots, I guess is maybe my question.

Yeah. I, it seems to me we're, we're like racing into this, of course, not thinking about, well, what's the right experience to create? Right?

But then again, we're talking about different products and different companies and everybody's trying to innovate. Uh, you know, I think it'll change fast enough anyway. We're gonna have sort of this awkward, teen awkward teenager stage for a year or two, and it'll settle into what it's gonna be next.

And hopefully it's better, you know, less, less complicated. I, I would definitely will be better. But you know what, here's the other thing.

The way AI is, is progressing. I mean, this is faster. We, I think we've gotten used to internet time, right?

Internet time, condensed timeframes, crazy AI is exponentially crunching internet time. Mm-Hmm. So when you say a year, I bet you in six months someone's gonna come up with the bright idea of a copilot front end, right?

That can fly every Boeing and Airbus jet out there, so to speak. And you just plug in, right? It, it's an API that plugs, plugs into a specific LLM given an, you know, an app, and maybe it uses the same, you know, big LLM but behind the scenes, depending on what that copilot is, is it an open AI copilot?

Is it lama is it philanthropic or, or a claw or what or what have you, right? And, um, and then there's an SLM that's application specific or an LLM that's application specific that plugs in on top of that. That gives me my, you know, my, it's, I'm, I'm getting a, a vision of Iron Man here, you know, where I, I get a, a, you know, a boost of a particular app's, you know, that knowhow over my base Ironman suit, you know, um, I don't, we go, you one Jarvis, that's what you want.

Yeah. Well, but, so in other words, Jarvis is my copilot, right? But Jarvis can plug into, and, and Jarvis always has that Jarvis intelligence built in, but then Jaris augments, right.

Whatever with some extra juice from, depending on what app I'm made. So I, so I think the analog to that is, you know, there were lots every day, every week, there's more AI announcements. This, this most recent week was what, uh, IBM and, and, uh, Dell and a number of other folks, and I can't remember, I think it was IBM, maybe it was Dell E essentially came out with a platform for managing your LLMs.

So maybe it's, oh, I didn't see that. What that is. Yeah.

That it was kind of one of those things thrown into the announcements, and I don't remember what the name of it was, but it, it made me think of maybe that's the answer is in the corporate environment, here's the LLMs, we're gonna use develop development using these five copilot testing, et cetera things, and here's the platform for doing that. I don't know if it gives you an interface to get to all of that, but maybe that's the destination is kind of, think of LLMs as we'll, have an interface to all those things and let let the company plug in the LLMs that we want to use and manage that infrastructure. Or third party does that as opposed to buying 20 of 'em.

And we all get to kind find our way through 'em. I, I think that's, you know, a likely future, a likely future. I, I think we also gotta think of, AI is supposed to make it easier for people, the testers for developers or security people, cloud native engineers, platform engineers.

We, we got to, you know, someone should do that user experience and say, okay, how do we make it easy to harness all this stuff for a typical worker? And then also these workers more, more often than not, are not individual, you know, it'll, it'll vacuum their team members. Exactly.

And so do you have team enabled co-pilots? Yep. Right?

That says, yes, Mitchell Allen has done this over here. So to be consistent, you should stay with that. Right?

And something like that, because that'll be a, a nice step up too, as these get more and more intelligent and, you know, in terms of how, how a group works, not just an individual. So I think that's another leap coming as well. Yeah.

I think that's the next kind of plateau is beyond individual productivity, people collaborating together, whether it's in a Slack kind of open collaboration or it's a development environment. Um, you know, imagine kind of stepping back up to your computer if you're a software developer, and you could very easily, you know, so where are we on the project today? And I could tell you, well, here's all the things.

Alan completed this area and it's all been regression tested and your code's looking good. Um, but it still needs to do these three things. And, but actually, you know, Mike did two of those for you.

Oh, okay. Good. I'll pick up here.

Instead of you kind of, right. Just figuring out the pieces, right? I mean, it's kind of like AI assisted workflows, right?

Team workflow. How cool. And I, you could start, I mean, think back to when we did VM in 2005, how cool would it have been to have sort of an AI assisted team workflow for remediations and Absolutely.

And stuff like that. Where, okay, go do those patches, assign that one to Mitch and let me keep me posted, and you know, what a what a what a great future. I mean, it's, it's almost thought tracky computer.

Well, I think it was, it is kind of where we're headed, which is now where you're like, do you trust what it did? So you have to look at everything and maybe, maybe you put it into production or put it through the paces to test it. Maybe you don't, but after a while, build up trust in this, right?

So yeah, it knows how to patch Microsoft servers. So I'm pretty good on if it's gonna run a, a patch routine for me. Okay.

I've watched it for 10 times. How many times do I have to babysit it? Okay, good.

Go do that now. Good stuff. This other thing, I'm not quite so sure you know, about, um, you know, building a, uh, a graph and doing analysis on it this way.

Let's kinda work through that together. Okay, good. Yeah.

AI's got that. I, I like the approach or I helped decide what approach options I want it to take. Okay, good.

And it can go back and tell me, like, this one looks a little different, dude. You like to look at some other options? Yeah, I would.

Okay. Or no, just, just the same one. You know, that kind of, that to me is the real assistant, not help me find some, I mean, yes, today copilots are this, here's some suggested code, here's some suggested or helped me find and look up stuff.

Well, here's some suggested, you know, uh, edits to your code, if you will, or something like that. Exactly. Yeah.

No, I look it, it's, it's fairly gonna be interesting to watch and it's definitely gonna have its effect on, on the way we all work, and especially around teams and coding and deployment and DevOps and platform engineering and what have you. You know, it's a, there be whales captain there, you know, there be whales. I I want to switch gears a little bit.

Okay, yeah, go for it. So, you know, we're gearing up. We're gonna be up in New York for platform combo.

We're putting on a satellite there. We CloudBees and some other folks, um, this whole platform. So I've been spending a lot of time, I did a webinar round table this week on for platform com.

Been talking to a lot of people, obviously around DevOps and, and you know, in our industry, and I think we were out at RSAA couple weeks ago, Mm-Hmm. I think one of the perceived truths out there is a reason why more organizations are more successful in dev utilizing a DevOps mindset of DevOps framework is that we've put, it puts too much on the developer's plate and developers want to code. And when dev coders are only coding 11 to 20 or 30% of the time, it's, that doesn't work.

We want coders coding at least 50% or more of the time. Mm-Hmm. And, and that inherently that shift left has become shifted to the developer.

And I'm not here to refute that. I'm not here to say that that's not the truth in some organizations, but what I am here to say is that really wasn't the intent behind DevOps shift left, just meant to get to stuff earlier in the development cycle where it's easier to fix problems where it's cheaper to fix security bugs where you can improve quality earlier, which allows you to get out good code faster. You does it mean the developer does everything?

And I think for too many people with DevOps, it became a case of let's make, you know, let's put it on the developer's plate. What I, I agree. Well, if you go back to, so if you wanted to reference the Phoenix Project or Gold Rats, the, the goal, right?

It's what, what what is sort of the deadly sin of, of um, kind of process engineering and, and, uh, constraint based management? It's work in progress, right? And it's things that are in a wait state.

Um, yeah. And the whole idea, if you, if you go back to that, take DevOps out out of it for a moment, but that's what DevOps is based on, is it's all the, the reason why you shift left or you look left is upstream is where those problems are created and downstream or where they create delays to go back and fix it, or it creates other issues that are, you know, in response to that problem. Well, it could be product quality, could be whatever, manufacturing defect.

And so if you take that logic and apply it to DevOps, you say, well then great for security. Whether the, where are the places before we get to test or before we get to CICD, um, where we're gonna improve, think of security as a quality process. How would you improve security?

Well, you might go all the way back to the spec, you might go to the design of the architecture. You might go to, um, some of the technology that you're using or platforms that you're using, you might go to tools that you're developing with, and then one at some point in that chain is the developer, right? There may be things there too that we can help the developer with, but I think it was just too easy to, to hand the developer scan results and say, here's vulnerabilities in your code.

Fix them and say, now let's just have the developer do more. And to your point, they a that's not fun. That's toil to them, right?

Um, uh, we remember the barking dog problem of intrusion prevention systems. We don't want that happening in development. So my point being, is it it's not shift left who, it's shift left.

How are we doing security and what changes can we make upstream that improve the quality of security? And it's incremental, right? It's not like one fix is gonna make it all solve all of our security problems.

It's an ongoing process. I I, I agree with you. It is, it is an ongoing process.

But you know, I, I think one of the premises of DevOps and TICD and software supply chain software factory as a, a place where we have, how and where we develop our software is that we want to ensure quality before deployment. We want to ensure security before deployment. We want to do more before deployment.

And that doesn't mean, you know, okay, so we're gonna pass to the left side of the factory, and the only people working on the left side of the factory are these developers. No, it means moving the security guys to the left side of the factory, getting the testers doing their scripts and coverage on that left side for the left in the factory. The platform engineers, the ops people, the SREs, they're designing these platforms with that in mind, that's where the guardrails are.

That's where the, you know, how the system is engineered. Mm-Hmm. That shift left to me and that, I don't know, I, I feel like somewhere along the line, maybe someone lost their way there.

So some of us have lost our way there. I don't think this is a hundred percent of it, but I think some of it is a long tail of the DevOps developers will do everything. We don't need operations or security or, you know, that whole mantra where it kind of started, that was one sect, if you will, of kind of people thinking about DevOps, which I know you or I, or many people didn't believe us, think that's not gonna happen.

Mm-Hmm. Can't believe it's gonna be that automated developers won't mind getting up in the middle of the night. And for every little issue, let, let me, let me paint a picture for you this way.

And you know, this is something, you know, I, you and I've explored is following shift left became, well no, it's also shift, right? Oh, no, it's also shift everywhere. So left shift left isn't just what DevOps DevSecOps is, right?

The kind of the model we've come to is in terms of creating software, you know, that is an output. Uh, it, it is the entire SDLC, right? All the way from design and planning through delivering code and your ability to deliver that.

And what are all the steps along the way where security is built or improved or incrementally added, made better. That's kinda the software layer. The, the other part of DevSecOps, which is more on our scene now, is the underlying tool chain and the supply chain of what's going into the process for external code and repositories and, and package managers.

The tool sets that you've, you're using, you know, for example, we're securing the CICD platform now, or using containers and even Kubernetes and things like that to isolate steps along the way. So someone interjects something into a step in the tool chain, it's not the whole thing is corrupted. We can fix that isolated, et cetera.

So, so to, to me, DevSecOps is now the entire process of both software and the underlying technology that's used to create it. So kind of think of it as horizontal as well as vertical and in the tools and software that we use and create that I think is what DevSecOps is now, or that's, that's DevSecOps next. That's what we're moving to.

Absolutely. Speaking of DevOps and DevSecOps next, how are we doing? When, when, when, uh, can our listeners, viewers get insights into your next big research report here?

That's a good question. Let's see. Um, I could literally, we're, we're in the, we're in the, in the wild is what I call it when route collecting data on our DevOps next survey.

Um, and we've already laid down a lot of kind of the ideas that are gonna be in this, uh, within this report. So we're cranking through getting a bunch of great responses. So I think we're probably another week or so away from collecting enough data.

And then we'll, we'll finish writing the report. So I think we're probably 30 days away or so. Okay.

I, I would imagine let's, let's think about the June, kind of end of June, early July when that's gonna be ready and in parallel in the back, I'm already working on the DevSecOps Next survey. Very cool. And these ideas about what's gonna be in that.

And then we'll work, we'll, we'll kind of have them overlap, staggered, and with the next one with that we'll work on whether it's observability or testing or whatever that may be. Mm-Hmm. We've got a bunch of ideas of what that can be.

Interestingly enough, in parallel, we're also working on AI and DevOps, uh, report an update to a report that actually Tricentis sponsored in 2022 that's kind of taking that angle of it. And we'll incorporate some of that data into our report as well as it'll be its own standalone reports. We got a lot of stuff happening.

Great stuff. Yeah. You heard it here first on DevOps chat.

Hey Mitchell, that about wraps up what I had today. Anything else? I think it's great.

Um, I would say as we're getting inundated by AI announcements, look, you know, look a little further down into the articles and look for themes. Even if it's about hardware, AI stuff. Look for the software part of it.

Look for the DevOps tool part of it. 'cause I'm starting to see more and more of that in these announcements, not just another copilot to our first, uh, discussion. So hopefully that continues the trend and that's it.

Very cool. All righty. That's gonna wrap up this edition of DevOps Chat.

We're gonna have a volume number sooner or later, but for now it's just another DevOps chat. I'm Alan Shimel and this is Mitch Ashley. And you just listened to another episode of DevOps Chat.