No No-Shift, DevOps Jobs, AI Tests Your Code, DevOps End Goal – DevOps Chats Podcast EP 5

Hi everyone. I'm Alan Shimmel. And I'm Mitchell Ashley.

And you are watching and listening to DevOps Chats. All right. Hey, Mitchell.

I think, you know, it only took us five or six shows, but we finally got our rhythm back. Gotta get that flow, get that rhythm. We got it.

We got it. We got it. Okay.

Uh, so Mitch, interesting weekend DevOps. We're gonna highlight, I think we're gonna highlight today two articles that, actually three articles that appeared on DevOps uh, dot com. But one of them was also the subject of a discussion we had on Textron Gang.

Mm-Hmm. And, and for those of you watching or listening who are not familiar with Textron Gang, it's a fairly new feature, only in the last three weeks or so, but it's our lead off for every episode, every show of Textron TV where we get the gang together. It's Mitchell and I and Mike Ard and Sharon Florentine, Amanda Rini.

And then we have various gang member guest speakers, John Willis, mark Hinkel, Tracy Bannon, Tracy Reagan, Bob Russell, um, Bob Russell, Bob Ruman, who, who's featured on the one we're gonna talk about today and others, you know, who come in as a special guest. It's 30 to 45 minutes of just covering what's news and, and what's happening and speaking our mind. So if we've now broken those out on Text Drunk tv, so you could watch just gang episodes.

I dunno, it still sounds a little funny to me. I feel like I should be in a crypto blood thing, but it's the old gang. So yeah.

The Texture on Gang episodes, and we're also spinning it up into a separate podcast, so you could check it out there. All right, Mitchell, enough of the commercials you wanna kick it off for today? Yeah.

Our, our, our good friend, we've known Mark Horn Beak for a long time. One of the people well known in DevOps, wrote a really good book around DevOps engineering approach to it. com called Forget Shift Left, why No Shift is the Future of Software Innovation.

And, and I take his article to be kind of a position paper on why this new approach of, well, maybe we don't need to shift anymore, meaning put testing, put security, put all the, you know, try to do those earlier in the cycle and we can more accurately, or or just as easily start to rely on testing software in production through the use of kinda really intelligent use of feature flagging so we can disable and enable things. And I imagine update them when they have issues and using AI to help us do testing. And that was the premise of it.

I, I'm curious your thoughts about, did it strike you like a eureka? Ooh, that's, that's, that's very cool. Did you kind of say, Hmm, wow, that's opposite of what we've been doing.

Is that gonna work? You know, I think it's kind of back to the future. Um, so let me, let me preface this by saying I have a tremendous amount of respect for Mark Sure.

Engineer and, and a real student of DevOps. And he was really helped us a lot with the DevOps Institute, stuff like that. Um, however, I think this is strike one here.

There's a big swing and a miss. Hmm. Um, and I'll tell you why.

To me, it sounds very much like you don't mind being my Guinea pig. Do you, I'm going to do some ab feature flag testing on you. Tell me what you think.

Oh, it didn't work for you, it crashed, it did this, it, you had a mission critical thing going on. But I was experimenting like, you're my canary. Um, I thought we moved past, you know, the idea that DevOps somehow means that we released software in embed, and then when it doesn't work, we say, oops, sorry, we're, we're looking for feedback loops.

Mm-Hmm. And that's why I say back to the future, it's, I think it, it's a DevOps chacha one step forward, two steps back. I remember I worked with a development team, test team leader of the test team says, yeah, I think our philosophy is if it works in production, it would've worked in test.

Right? Of course. And if it doesn't, it wouldn't have worked in test.

Now we know we don't have to go test, but I'm sorry I had that downtime. Yeah. Yeah.

She losing those customers. Well, uh, you know, so, so I had kind of that reaction also myself about it. Then I thought about, okay, so what in this strategy might be helpful to us?

Right. And I, uh, if I kind of pick in, pick in there, deep reach in there, the idea of more intelligent use of feature flags and could we use AI to help us find out the things that we can't find out anywhere else, but in production, not testing all of our software in production. Okay.

I don't disagree with that. And that might merit some. That might merit.

Yeah. And maybe that's what I mean. I agree.

There are things that you can only find out in production and that is a great reason why you have those feedback loops and why you iterate and reiterate and iterate again. However, doing it wholesale at the expense of testing prior to deployment, it don't, it does, just doesn't make sense to me. I think we're on the same page, so, you know, I'll, I'll give Mark partial credit.

How about that for, I think there's some interesting nuggets in there of maybe we can explore. It didn't on, on first blush, and I wish we had, you know, we could do a whole podcast with Mark talking with him about this kind of stuff. And I'm sure he's got some more thoughts you can't capture in a brief article.

But, um, I think you and I are on the same page Absolutely. In violent agreement here. What a way to start the show, Mitch.

What a way. Yeah, we put, did I pick the softball to start with? Is that what Yeah, that was an easy one.

Swatted. Back, back, back. It's outta here.

It's okay. Bounced off the wall. What else you got for us?

Hey, you know what, something that we do on a regular basis, just kind of given the times and, uh, folks looking for work that might be, you know, out of work or maybe switching jobs. One of the things that, uh, Mike Vard and the editorial team have been doing is they have this five great DevOps jobs, kind of rolling article dev, DevOps jobs opportunities. And I wanted to spotlight that and highlight that for folks.

Um, and, and you know, these aren't just kinda random. Oh, you, you might, well, maybe you should be a platform engineer or you should do something else. I mean, he literally is a, Northrop Grumman in, in Virginia is hiring a DevOps engineer as a consulting lead, McKesson, blah, blah.

You know, so they're real jobs and he's pointing those out. Now, will every reader go out there and apply for those and, and flood the zone? Maybe.

But I think the point is that there are good jobs out there and, and the editorial teams kinda try doing their part to, to help further, uh, the the job search process. I'm curious your perspective, were you part of them creating this or did they do that all on their own? And what do you think of it?

No, this was the editorial team actually, truth be told, I think it was the editorial and marketing team. Ah, okay. Because, you know, still what is the most, what is some of the most popular searches under DevOps and it's DevOps jobs.

And you know, for a while there it was impossible to hire a so-called DevOps engineer, um, there in such demand. But in today's tech climate, of course there are more layoffs. It's a little harder to find jobs.

And there are plenty of people, you know, back then a tech strong engineer, excuse me, a DevOps engineer. You had to know Puppet or chef. Mm-Hmm.

You don't see that as a requirement as much anymore. Um, but nevertheless, there are a lot of people with DevOps skills who were looking and we thought it was, it'd be a good sort of public service on a weekly or biweekly basis to highlight some really good positions out there. Good.

Well check it out. Hope our, our listeners watchers will go, uh, go check those things out. Share Yeah, thanks.

com. You could search 'em there and they're there. That's helpful.

com, right? Yeah, yeah, I do. It's one, uh, about tab nine, uh, releasing their, uh, testing platform, creating tests using generative ai.

But what's unique about that, they, they've had this capability kind of using the generalized lms, uh, but they've been working with Google and they've actually taken the next step, which is, we would call it, I think we've used the term do demo domain specific LLMs, where actually you load your code base into the LLM along with what it has as reference material. And from that it can generate test cases for you, a software, things like that. That's great.

So it seems like a first kind, at least that I've seen in this area. Maybe others are doing, it's, it the first time I've seen it specifically for testing. Exactly.

So I tend to call these things now s SLMs small language modules. Mm-Hmm mm-Hmm. Right.

So the, the, the AI still has that large lang LLM at their disposal, but first it looks for answers in the SLM and then, and then goes through there. Um, you know, as, as I, as we've talked about before, Mitch, right? I think we're just scratching the surface of use cases for these kinds of things.

This is the first time I've seen it specific to making a, a, a testing coverage. Mm-Hmm. Kind of a plan, but why not?

It sounds like a no brainer to me. Yeah, I think that's sort of what we all implicitly had in our heads of, well, to generate test cases we need to under understand our own software. So how does that get loaded into, you know, some backend that generative AI is going to use to provide those tests versus just being in an IDE and kind of interpreting as you're going along.

You need to know your code base too, to really do effective testing. It's not just unit testing, functional testing in A IDE. Agreed.

Agreed. com. Mm-Hmm.

Check it out. You know, so total, total boondoggle sidetrack rabbit trail, rabbit hole. This may every once in a while think of these questions.

Like there isn't really an answer to this question, but I thought, what happens when we're generating all this code and what, and, and it's happening. We're generating all this code using generative ai and it's s basing it on the code base that we fed it, and then the generated code gets added back to the code base that the LLMs using and it generates more code. And do you get into this kind of like in breeding?

Yeah, well, in breeding or an infinite loop. Some, some people call it southern ai, Southern ai. That's your words, not mine.

That's your heart. But, you know, you tend to get, when, when things, when things feed feedback, when you get these loops, you get feedback meaning like in the sound Yeah. In in the sound.

You get a, a feedback sound. Are you gonna start to get weird errors or weird outcomes? I think it would, you know, yeah.

I think it would definitely lead eventually to some poisoning or bias in there. Kinda reminds me, you ever see the picture of a guy looking in the mirror? He is looking in the mirror.

He is looking in the mirror that guy's looking at, you know, and, and it goes all the way through and it kind of uhhuh it gets a little freaky on you. Yeah. Uh, but it also distorts and I think that's what would happen here.

Exactly. Yeah. Distortion way.

Don't, don't kiss don't kiss your first cousin and don't create kind of that feedback on, on generative AI generated code. I'm sure I'm not the first sources to think of that, but it was like, Hmm, that's an interesting problem. I wonder, wonder what will happen with that.

Yeah, no, I, I mean it would be an interesting experiment to see how many generations it would take until poisoning sets in and you got, you know, kids with three eyes or something. Exactly. Um, or I could get rid of me really metaphysical.

And maybe there's another dimension of which that ai LLM is working just fine. You just need to grab that one. Perfect.

Yeah. Quantum computing, we'll fix it. Who knows?

Yep. Anyway, sorry. Sorry for the, the, uh, sidetrack.

No, no. Look, it was an interesting thought. That's why we do DevOps chats Mitchell.

So we can explore where no one has gone before. Exactly. Okay.

Five year mission. We're on here. Make it so make it, you know, engaged exploring.

We had one of our livelier, I would say most lively conversations to date, uh, on tech str Techron gang, um, the Bob, I think around Monday of, uh, uh, February 26. 26. Yeah.

Yep. It was, and it was, uh, what what kind of sparked it was this article that Don McVety wrote, essentially, what's the end, end goal of DevOps? Have we gotten there yet?

Dad? Are we there yet? Yeah.

And can we stop? I need to go to the bathroom. That part wasn't it, but the, are we there yet?

Yeah. What's in the article? And Well, to be fair, Mitchell, that was just the preliminary match for the main event of that text drug gang, which was around working from home.

But we'll save that for another show. Oh, that was, yeah, that was the cage match part of, yes. That was the case.

Death match. But this is just a basic MMA fight, you know, not the case, right? This was, this was your run in the mill, you know, but, you know, stop the bleeding.

Don Don's point was like, well, what's the business goal? What are, what are we tr instate? Are we trying to reach, I'm, I'm grossly paraphrasing what his article's about.

Mm-Hmm. And everybody should go read it. 'cause it asks a good question about business value.

And I think, I think you may have responded first and I followed up, but it, it was kind of that, you know, DevOps isn't a destination, it's a journey was really the answer. Yeah. It's, it's, we don't say what's the end state of agile or what's the insight of waterfall?

You know, and, and, and quite frankly, at least those have real definitions and there's a man agile manifesto and everything else. When you get something like DevOps that quite frankly, purposely was never had a manifesto or purposely never had a defined definition Mm-Hmm. Right.

And now you all of a sudden you want what the end goal is. The end goal is to make software better, faster. Mm-Hmm.

Make better software, faster, higher quality and metrics that are important to the business. Right. It's, you know, there's no end, there's the Dora metrics, there's other metrics.

I, and I get all that, but, you know, trying to pin the tail on the DevOps donkey with an end goal is just that a kid's game. Mm-Hmm. In my opinion, right.

It is journey. But Don's another friend, he's a smart guy, you know, so he's, oh no, Don, Don and Don's a thinker put himself out there with some ideas, right? Yeah.

No, Don's a thinker and he throws it out there. But I, you know, it also kind of reminded me of the people who were like, you know, culture sch small shirts all kumbaya, I wanna see the tools. Yeah.

Right. All that soft stuff. Yep.

And, and so, but they're missing the point on DevOps, which is the culture, which is the breaking down of the silos, which is the cross-functional teams working closely together so they could go faster and better. Mm-Hmm. It's the whole cams thing and everything.

So, um, I I, you know, I think Don was highlighting what many might consider a, a soft wide underbelly of DevOps, where I consider it its strength. Yeah. It's lack of specificity maybe.

Or it's open-mindedness maybe is another way to say it. Right? Fair enough.

It's adaptability to, okay, well this, and why I say adaptability is actually one of the things I strongly believe is by not dogmatically describing it or defining it, you know, you oftentimes hear mean, right? I talk about methodology wars, I remember the, this methodology is the right one, and they're all bad. Right.

Kind of religious arguments almost. And DevOps is kind of the, no, you know what, maybe CICD is the place to start, but we need help here. This is our biggest issue.

Let's work on this bottleneck or this problem, or where we're inefficient or whatever it isn't. Uh, if you're not doing it this way, then you're doing it wrong. No, you do it the way that makes sense to you, the principles and the ideas there where you're adopting.

Right. And there's tools and culture and process all in support of that. Agreed.

Agreed. But it's a good point. And I think Mitchell, it also sort of segues into DevOps next, are we got DevOps campaign that's coming up.

I mean, you're heading a big research project on this about where is DevOps? Do we need an end goal Where, what's the current state of DevOps, if you will? And where we going next?

You know, just, I don't want to put too much out of the bag, but in addition to survey and interview data, we're actually gonna rely on our own proprietary Mm-Hmm. com data that we've amassed, you know, over 10 years, we have a mountain of, and it's extremely valuable. Absolutely.

Yeah. To show, to show where, where people are going with DevOps next Mm-Hmm. And, uh, we'll be doing slices of that as well.

So we're, you know, looking forward to what you come up. No pressure Mitch, but looking forward to what you come up with there. You know, my, my goal with this report is that, you know, how they can predict that, uh, we're gonna have a, uh, hurricane by people buying all, all the milk and bread out of a, out of a grocery store.

A supermarket. Yeah. People Google, you know, I need a a, a backup power supply.

So we must be gonna have a storm. Maybe we'll be able to do that with a DevOps. Uh, well, as someone from Florida, I could tell you that s**t don't work.

It doesn't work. As soon as they say there's a possibility of a storm brewing in North Africa that may come across, we're out of water, milk and eggs around here. So the you're saying butterflies wings, butterflies wings flap in Africa, and Right.

Milk's gone. It, it's crazy. Um, Florida, but, you know, sometimes it become, become a self-fulfilling prophecy too, right?

Mm-Hmm. You get people GI was saying that tongue in cheek. Of course.

No, I, I know, but you know how I get when you mention hurricanes down here Oh, I know. It's your favorite topic Yeah. In Florida and, and working it out.

Well, we don't need measles vaccines here anymore. It seems It's, it's an individual choice. I don't know what you're doing down there.

You should bottle it, you know, or you know, I don't know what you do with it. I think we should shoot it is what we should do, but, and you know, we're allowed to have guns. No big deal.

I felt threatened. Oh yeah. Weren't they gonna pass like some open carry you don't need.

They did. Oh, they did? Okay.

Oh, yeah. Everything goes here. Oh, every, well, I called Florida the land of No rules except reading any book you want having control over.

Well, let's not make it a political thing, Mitch. Now you got, that's another podcast. We'll have to Yeah.

That's a different podcast. com and a very lively discussion on our Textron gang from February 26th. If you go on Text Drunk tv, you could probably get it there and shortly, depending when this plays, you'll be able to see it on a podcast on Apple Music or Apple Podcast, excuse me, or Spotify, or wherever you listen to your podcast.

Just look up Text Drunk Gang. Your favorite platform. Absolutely, Mitch, excellent show today on DevOps chats.

Man. It's, it's been fun. This, you know, the more we do this, uh, I know, I think it's for both of us, but it's such a flashback, reminiscent, we're still getting to do the podcasting.

Like, well, yes, but you know, what they say about, and I'm not blowing smoke here or anything, but you sometimes you hear people, the old people, not us, those old people. Yeah. Yeah.

You know, we were best friends for so many years, and even though I didn't talk to you in five years, it's like we get on the phone. 'cause we talked on phones then, and we, and we didn't miss a beat. Mm-Hmm.

It took us, I think it took us a couple of shows to get our, as I said in the beginning, to get our rhythm. But we, we certainly have, uh, rekindled the rhythm. And, and these are fun discussions.

I hope people watching and listening to this find it as fun. I remember we used to laugh a lot, and I think we're still laughing quite a bit. So that's a good way to go through life.

But if it's not entertaining to anyone else, at least we're entertaining ourselves. Hope, what the heck. I hope it's so funny too.

And useful for other folks too. Absolutely. All right, that's gonna wrap up, um, for DevOps Chat this episode.

I'm Alan Shiel, and I'm, and you just watched and listened to DevOps Chats.