Fireside Chat: Making DevSecOps a Reality | DevOps Onramp 2023
For many organizations, DevSecOps is aspirational. The idea of integrating security directly into the software factory is really compelling, but where do you start? In this fireside chat, John Willis and Techstrong Research GM Mike Rothman will discuss the best places to start your DevSecOps journey, quick wins to gain momentum, milestones on the way and pitfalls to avoid.
Transcript
you Hi, everybody. Welcome to Dev Ops on ramp here is another session for the devops omaramp conference and I couldn't be more pleased to welcome a friend of the firm right A friend of mine a friend of devops. Everybody knows him.
John Willis from welcome. I got to do a quick Old Man story. This one right do the Old Man story.
Let's go you're not as old am I but you're not too far away. So I'm going to get my coffee this morning this little Bookshop coffee place down the road here and and it's older General. Well dress guys doing he's walking from the front door all the way to the back and he's just excuse me, sir.
He goes, I just need to survey the place and I know he's sort of joking, right? So he walks back and he comes all the way back and he says I said to him I said, so what are your findings? You know, and he says well, I'm just seeing if there are any enemies in here.
I said my friend I've been on the search 64 years enemies are never in plain sight. Exactly. That's right.
That's right. And it's actually a great way to kind of kick off that you think about that. Yeah.
Well we're doing yeah, they devops I'm gonna remember that the idea of devops Armory is really to you know, get folks who are just getting going and devops it really trying to restart some of their programs and and some of their initiatives within devops and give them some real practical ideas for how they can do stuff. Now when you saw earlier today was Don mcveady talking about, you know, kind of the developer side of devops. So what I want and literally the guy that wrote the book, right, you know kind of it been very early in terms of a lot of the devops research and what's really dig into where none of the outside.
I really just sick outside of dead. SEC Ops really is and and I think you know the idea of our enemies hiding in plain sight or is really an appropo, you know type of issue because obviously we've got Supply Tax that we're trying to deal with but we also have to you know, kind of make sure that we can operate in a secure fashion anything that the developers pumped through the pipeline, right? We've got some mechanisms to you know, kind of check to make sure that we're not you know, really deploying potentially, you know codes with security defects and the like but it's hard.
It's complicated right we've got sres that are in there trying to make sure that things are more robust and resilient. Now, we have platform Engineers that are trying to you know, kind of build that you know platform in code that everybody else can look at so a lot of moving pieces John. So for those folks that are first getting going really trying to look at it from an op-centric point of view.
What are the first couple of things, you know, you tell them to think about and and maybe some tools that yeah, you know the thing I'm sort of, you know, sometimes I'm not the best intro guy, right because I watch I go to a conference and I see a presentation and I'm like, wow I was talking about that five years ago. I realized yeah, but it needs to be told right like if I'm not following who else is but I think the basics are very positive right now, but I'm I'm concerned about the basics the sort of I didn't realize that story was actually gonna be useful in this but the in playing site so on the basics really like I'll just sort of you know dump and then I want to get into the the not in plain sight discussion, right? Like I think what's going on with the s bomb I think the focus on securing the supply software supply chain all those discussions a lot of really good value stream mapping and a lot of this does come from an operations perspective, you know devops Capital.
Oh Ops and you know, so you're at you know, keeping aware of what's going on the s bomb work what's going on just you know, the general software supply chain discussions. I think the we the IQ of this discussion has gone up over literally just last couple of years. Maybe just a little Before pre-covid you start to seeing presentations of software supply chain and a lot of the vendors started refocusing their discussion.
This is solar wind sat a little bit of that effect. Yeah. Yeah, but that we added a little bit of sort of a rocket fuel to discussion good point right the but but again, it was a little bit of that happening though early or you know, but and then So that's all great.
And then I am, you know last year I was I another friend of the the tech strong group, right? I consider text wrong my family, you know, but you know, I think another is is the Jay frog, you know folk, they're just, you know, just wonderful, you know starting from the CEO and all down but I I gave a presentation there last year there. There's one of their swamp-ups and I and I used a Capital One breach.
I said, no, you know a bit of screen. It said no binary is harmed in this movie. Right.
So so again worrying about binary is dependency map like focusing on how do we get, you know sort of the s-bomb by the way, the s-bomb is going to be a mess beat. It's going to be a mess before we get it, right, you know, like we can save that for another things you bet. Yeah, like most things right, but the you know, you look at the Capital One breach and it really there was really no no sort of classic vulnerabilities, like nothing like, you know, that that sneak or you know crowd strike or you know, we can go down the list, you know check would of course because you know, the short version of it was you know, I I can't it to the Air France full for seven, right, you know everything that could have went wrong from a configuration perspective.
You know, the story that I had her old is they were in a hurry for a particular application. They got an exception they put up an open source proxy. They left the defaults.
One of them was bypassed. There was a crypto-minded weights in Waiting of just running curls against a billion URLs. Hoping that somebody Bypass on question mark and she hits the metadata server the metadata they happen to copy and paste from it to VPC definitions.
There's data sitting in S3 buckets that have been cleaned up. I mean, it's just and they're rolling IDs right like like everything that could go wrong. It hopes.
It pokes in on a happen, you know, rolling out a new release of an idea. They hit the metadata serve. They dumped, you know, permissive definitions.
They start, you know, rolling through as three buckets. What a day fine 145 million credit card application. There's no sort of CV there right?
There's no sort of nist definition that so I I think that I think that so I have a presentation I call software supply chain to missing links, right and you know, like yes clearly. I'm not saying don't don't focus on the important stuff that dependency map and all that stuff is you know is a major major problem for all infrastructure, but let's not use all oxygen Where we could be looking at leaky apis on malformed, you know configuration definitions things. Like what happened at Capital One where we're in a hurry and we sort of we don't treat the configure because in today's world, so I was reading the crowdstrike stuff, right?
There's a couple of reports that I'm not a security guy. You know, I sound like one I play one on TV, you know, Mike you're the real security guy right? But but I do since I've been into this Dev SEC Ops thing and people look at me as a leader.
I've been you know trying to do to do diligence. So there's a couple reports. I read every year and I like the crowd strike one because you know, they they sort of Simply explain stuff in a simple format for me so I can sort of well, that's a good one point, you know, and I think the thing that we're seeing is that I love this now, we're free like a 71% increase in malware free, you know attacks and the increase of like Cloud exploitations, right that that's goes obvious, but I think what's not Is what are the cloud exposure because you know, the increase could be the rising tide of more Cloud use.
Okay, and then it's really the same number or it actually is there's more configuration items in the cloud and there's more clever ways to attack where you don't have to you know, get get in that zero day vulnerability window. These are vulnerabilities are just sitting there. They're mistakes that were made that are just sitting there you need like like the crypto minor who attacks.
It's it's both actually, right? So the reality is you have more stuff to keep track of which makes it harder to keep track of right? The other aspect of it is that we are embracing infrastructure as code to a much greater degree and it's scanning tools, you know, you mentioned dependency maps and check marks and and a lot of the tools that are used to evaluate code early on in the process.
We don't have similar tools to do infrastructure. Yeah, there were a couple out there that was check off and that got acquired by Paulo. So, you know, there are a handful of tools that are really focused on on scanning templates.
But none that I would say or really that, you know kind of focused mature, you know anywhere near what we have in the in the software testing piece of it before it kind of rolls into the pipeline. So you have two of those things, right? You have those two things that are really combining in order to make it more difficult to ensure that you've got the proper posture in place, you know, as you're continuing for role things through this devops machine and that's what we're trying to get to right.
We're trying to get to where we're developers can you know kind of Pop something into a pool request it flows to the process. It gets deployed to production. Hopefully multiple times a day.
Well that requires a constant diligence across you or entire infrastructure that I don't think we're prepared to do right now and that's really a lot of stuff that we need to dig into and I think the you know, Mean when you think about classic, you know secops or devsecops or when you think about that, right, like most of the things we do well are from known patterns. Like we can identify a binary signature. We can identify some water we can identify, you know, even even I mean the truth and matters the only things were reasonably good at in sort of signatures of configuration are things like Secrets looking for patterns of Secrets or or you know, one of the things that like I got sort of baptized was a Docker, you know, that the early default Dockers had like literally default all these this config things just turned on you know, or you were in the name space of the host.
Right like you had to do a special effect. It wasn't even a flag until like a year into the product that you could actually change the name the username space of the container running on the house. So like if people didn't know like we like spend hours on that, but that's how I got baptized and things like TW Parking it'll basically became part of Prisma and and Evan IO is another good example, right like another one sold the Palo Alto.
Thank you Palo Alto for the makeup. I just there again. That was an interesting set of patterns.
Right? Really, you're sort of Norton Antivirus like things that people were doing wrong in the cloud the place that you point out gets hard is and I know there's some vendors now, we're looking for basic stuff in like terraform templates, right? And again, it's sort of the Evan and iOS Secrets management like but at the end of the day, these are sort of knowledge base can definition it's like, you know some, you know looking at somebody's pure Java code and trying to figure out like what did they really do wrong that creates sort of a buffer over whatever the things that like, it's just hard.
I mean it is hard to do that. I you know, I did somebody told me an interesting story really large large company. That we all have a divisive or two of I would know who that is.
Yeah uses and I don't know that I'm like want to endorse palumi. But this idea that like so what I was told is developers can only get configured. Templates of palumi-based sort of terraform Replacements and that's a pretty interesting idea.
So now if you could catalog You know create the sort of supply chain of a library like structure that you can only use endorsed and I know other people are doing this with terraform, but the idea that you know plumi has this interesting model that sort of it's sort of, you know creates these models almost like a programming language itself sort of an abstraction of an abstraction. Anyway, I think you know, that's the place we can get better just like what we do with code we had standard standard documentation. This is the way you code it XYZ Corporation.
This is the way you build terraform. This is the way you build configuration here at XYZ Corporation. So for those organizations and and I actually love that idea and that's something that you know, kind of I've been talking about within the context of some type of Center of Excellence right off center of excellence.
Maybe it's a security. Honestly. I don't care what you call it.
Right but that you have some group that takes responsibility to build a set of design patterns that other teens can use to really both accelerate the process so that they get It that is useful that has been vetted that doesn't have you know, some of these clear security defects, you know from day one and use that as the foundational aspect of how they build out, you know, their application and really providing value to the development teams that way, you know, again really to kind of go along what you just said John which is to to give folks something that is secure by Design, right? They can go and screw it up after that and we'll see that about a million times. But if they're just making things up front odds, or they're not gonna do that, right?
I think you're a foundation. Well it going back to the Capital One. Like there's a way to describe just like it like once you saw one problem, you need to not stop you need to look for the other problems.
That's all. So the Capital One was Russian is not your friend. That's right.
The you know, the Capital One is an interesting thing because now this is where I'm putting my own sort of theory on it, right but like they you know, Capital One is known as a Vanguard or Around Cloud security banking Financial, right, you know, they've sort of set the standard and here look what happened. And in my opinion what happened was they didn't have a process for out-of-process process right? In other words, right?
Like if they were gonna put up a standard proxy and they weren't in a hurry, right? Like there's no positive very minimal possibility that they wouldn't had a configured definition, but they didn't have is how to deal with an out of process process. So this having and I think this is the you know, going back to that large, you know manufacturer that like uses plume.
It could have been anything, you know, there's a early on I got to meet some some people at Fannie Mae early on and the original devsecops discussions. They they started running internal Dev check out stays and I got invited and and one of the interesting things that they were showing me that they did in their sort of pipelines and stuff is like they had like the olash like, you know, like like catch these things like, you know SQL injection or whatever just the things right and instead of just telling the developer. Hey, you know, we broke your build because of this go linked Olas and read a bunch of jargon based stuff by a bunch of security Knuckleheads, right?
Sorry. Oh the security people Are okay. Yeah.
I've just wasted another hour just to find out why you you know, and and then but they did is instead of linking you to some sort of security with security security verbiage they list they gave you an internal repo that had code examples of how not to do it. You know, so they literally blow then say, okay. This is the the Fannie Mae example for if you need to do this, this is how you need to set up the headers and code and over time.
They just create you know, just continuous Improvement model right like every time there wasn't an example. He got killed on a jira ticket got somebody took the responsibility said okay, you know said just tell me what I did wrong. Tell me how I did right.
So I mean, I think all that sort of plays very well into this. I think the other thing I wanted to talk about like which goes along with this is I'm starting to see these interesting new vendors. You know, how do you get Ops to be security focused?
I don't think you send all your Ops teams or developers too. How do you get developers to be Security First? I know Don Don's an amazing guy.
I love to make these, you know Lord and a great people. So I'm sure he did a wonderful job this morning, but the How do you get developers? You know, you don't I don't think you send them to security boot camps for like six weeks.
Right? Like, you know, they have day jobs, right? So how do you get them engaged in in this kind of conversation?
And I think this is happening almost serendipitously in that we're starting to see some of the Ops right Security Programs. And so, you know back in the day like you, you know, who could compete with the the complexity of you know of, you know, how either Jay frog or check marks or sonatype like, you know, there was a lot of Ip there. So like somebody like me like doing a startup and saying okay, I'm gonna decide to compete with sonatype.
Yeah, like good luck, but right but what we're seeing now with the Advent of some of these newer Technologies, you know, like I saw one the other day called codem and what they do is they sit on ebps. And they can now look at loaded code segments. And then they they sort of build a profile.
They say they create s bombs which I think is an interesting too from from your running the code, so it's real so they call it Dynamic SCA, right? And and so now If they see a code segment, that's not defined. This is the classic, you know, you know anti-palymorphism like well like that you're like like what if the code actually looks like this, you know the story about the the the Bitcoin operator attack right where he literally was sitting on you know, he got committer right here to commit rights and then he put his the ferris coating because he knew there was enough stream, you know, Bitcoin operated used this node model, right?
Like like there's no signature for that right like and or you know, the signature is the signature, right? But like what if like code gets loaded and now it's sort of like not defined or or it's it's immediately known, you know, the argument these guys make is like you can do a lot of stuff. In the pipeline to identify this stuff, but you don't yeah, we know what it's like until it's running and by the way, they they identified.
A lock for J vulnerability in a very popular monitoring observability tool. Right that Lily had been there now wasn't they could tell whether it was operations based, you know, whether it's actually being used or just loaded in this case. It was just loaded but the actual customer.
Demanded that they were going to turn off this product. There's a very well known product and then there's a another one. I saw the other day.
They're using open telemetry. Right like so I think this is interesting idea of of Flipping The Narrative of more real-time because the tools, you know, if you look at what's happening cncf and the open Telemetry stuff. It's incredible.
What's what okay, just the adoption for you know folks that you would think wouldn't have a vested interest in you know, kind of standardizing how they gather Telemetry because that's how what they do you Telemetry and and really kind of focusing their value on whether it's you know, pattern matching whether it's you know, again more insightful interface to deal with traces and metrics, you know, whatever it is. They're not differentiating on the fact that I can gather more data faster, right? You know, they really leaving that Telemetry now and I think if you have is a good example because like that, you know, these are actually, you know, Israeli security like really smart guys, right like but at the end of the day, you know, I'm not even to think like, you know what I could actually use ebpf and I could actually live in this signatures and I could build a product like that.
Whereas if you Go back pre-ebpf or pre-owned Telemetry like yeah, I'm not there's a lot more there that I you know, so I think like in a sort of SEC Ops conversation. I think there's an interesting of people who have operated infrastructure or certainly CDs interesting intersections of opportunities to create these clever new Solutions. Yeah, you know, what's interesting and I guess if you've been around long enough, you see the same stuff come around over and over and and both of us have been around probably too long and that front but it sounds a lot like, you know ID yes to me, right, you know kind of we're looking at stuff Flow by we're looking for problems of stuff that could be malicious.
Yeah. It's a different place to inserts all being largely, you know kind of an operational problem rather an infrastructure problem rather than just a basic attack on network problem, but we're seeing a lot of those same types of techniques used in new different places, and I think that's really the point right as Getting started in devsecops is yours really trying to do it right as opposed to reinvent the issues that everybody's already had there are new techniques. There are new technologies right?
There is the way whether it's through summer of Excellence, whether it's through platform engineering design patterns. However, you want to frame it. There's a way to get everybody on board early on in terms of what the right processes are right that the whole thing about we don't have a process for out a process processes, right which feels a little like, it's actually forgot to slow down in the light.
But you know, the reality is we don't right. So there's things we have to do to make sure that we're monitoring and and really critically evaluating the posture of all of these different components. But even the more we get into code the more we run code through these Pipelines the more opportunities we're going to have to insert some type of again both performance security availability.
A lot of these different approaches to ultimately deliver a more resilient environment. And that's what devops right? That's what devsecops is really all been about do it faster.
Do it more reliable do it with more resilience and and make sure that again you've got to handle on all these things happen because you know stuff is happening just the interesting thing about sort of secops very similar to the devops right like you like you can say, okay, there are people that hated the term devops now everybody's universally like oh, yeah. I I remember getting almost with like, you know rakes and pitchforks getting chased out of Alisa conference in Boston, you know, like because I I dared to say that you could figure service with Jeff right but But in the early days of devops one of the patterns that we we saw worked, which was go ahead and put an Ops person in a Dev team. And what it did which was really like when developers was making definitions of you know, sort of trying to Define like the the non-functional things right?
Like okay, let's put a directory structure here for the log. Right and the Opps person would be like, yeah, you know, what if you put it in slash whatever it'd be better for the operations, right? Like like that that that's one of the best sort of signal of why devops succeeded right?
And then, you know, when when devsecop Shannon leads coins eat like we you know, we we start building there's a bunch of people get went preserved about like that name is gonna destroy human kind, you know, and okay calm down, you know, just trying to get a point across but the point that got across is what we just talked about is we created an opportunity. For apps minded people to have a collaborative discussion which which were classic like for example, I don't I think that's a good observation about you know, I IDs systems like like, you know, but it's protecting the perimeter like so obvious people like yeah, that's great. But um, like what are we gonna do to protect the inner perimeter?
Well, why don't we use those same techniques? Well, actually the first go out of John was let's put those boxes in front of all of our you know data set. Yeah.
Yeah. Yeah, but I mean that to me the biggest takeaway about like this discussion about what's the outside of security which is you know in a sense the word devsecops. Worked just like the word devops work.
We got security people who had a classic mindset about security to be loud the Ops people to have a conversation to question. Some of the decisions that were long-standing patterns to say. Well, why can't we do it this way?
Like, yeah, that's a good question, you know, and I think that's what we wind up with these new sort of products. This is why you know crowdstrike is focusing on you know, 71% increase and you know non malware attacks, right, you know, so Yeah. As we've said right I'm kind of start a company.
Yeah a couple years ago to you know, kind of do you know more specific Cloud configuration checking guys that was gonna be an issue and and shockingly enough it has but the reality is Again by kind of focusing on that platform by focusing on these design patterns by focusing on vetting and testing a lot of the infrastructure that you have again. It's kind of new some of these innovations that are actually looking at the code that's actually run and and looking for things that are anomalous from that standpoint. It gives us a chance to not stop these attacks right not stop these but identify them faster and be able to get into a response and validation process earlier on to identify.
What is problematic and that's the point. It's a race right? Like there's no deterministic like solution product theorist that says, okay, if you do, you know, this minus that plus three minus two, There will be no more attacks right like like that anytime.
I see these vendors. We'll talk about zero, you know. Okay.
Yeah or anything man like yeah, we don't even get we started out there anything but there is no such thing in the physics World. There is no such thing as zero. So but anyway, but the point is the attackers are always gonna be clever as Shannon points out many times to me is it the attackers have now all the tools we have, you know, like the 15 years ago.
They didn't have the ability to to do compute power that we had to stop them. Now. They have the ability to do incredible stuff to attack.
So it's a race and the best thing we can do is make sure we're always thinking outside the box. We're always sort of applying, you know collaborative methods. You know, I'll say one more thing.
I know we're getting your time. I I like, you know, when we wrote the it's called the devops already made a governance which you know Shameless plug for Investments unlimited book, but you know, one of the things that happened was there a couple of banks in the first paper we wrote right and I remember John was a task at the time was it he's at PNC and at the end of the session he came up to me and we're friends, but he said I want to thank you. I'm like why just because Banks don't talk to other Banks about security.
And I've asked that question in front of people. You know, like I'll have an audience. I'll say how many people in the audience work for banks, you know 30 room raise your hand.
They'll say how many all actually collaborate on against your adversaries. And all the hands go down right like so there there's another you know, like, you know like this bread and butter in the financial fintech and protecting. Your fort shouldn't be one of them.
That should be basically well. I said that wrong but the table Stakes should be collaborative efforts of financial companies sharing information about Technology collaboration and we do it's you know, unfortunately, it's the third parties, right? That's right.
Yeah right until you know pieces right? But but as we're embracing a lot of these new tactics and techniques and I don't think we collaborate enough as an industry and that that really is a great point from that perspective. But but also mean I think that what we have now again for those folks first getting on to the journey, right, you know kind of starting to figure out how they're gonna Embrace that it is, you know more structure is better initially, you know, kind of when we started in all this space.
It was an early right because nobody know what the hell they were doing. So just did a whole bunch of stuff and you write a book and and I assess a bunch of things that God and we learn the hard way if you're getting on now at least a little bit okay change and all that other stuff, but you know, the reality is you can build with more structure in mind you can start to educate. your developers not to be Security people, right but to understand decent coding practice and really to be able to parse the stuff that gets kicked out if you break a build, right, you know, and that's obviously something we're not even deal with an effort but the same should go for infrastructure, right your Ops people, you know kind of the designers The Architects have to understand how that infrastructure and be compromised right how it can be attacked make sure that you're building some measure or protection and resilience into that environment and have tools that are constantly looking at, you know, the infrastructure to identify those issues whether they're in code great.
We'll fix them. But you know, the fact is, you know, you need to identify those and act on them as quickly, you know, and I think that is to continuous Improvement aspect of it. Like I was say that like, you know, like a lot of people think if they secure you double quote or at quote secure the secure so for supply chain, so they have a reasonably well, which or a reasonably mature death Tech right?
That's great. Okay. Well now like okay you Sort of like we're good.
What you do now is like, okay. What is the next thing like who builds the the pipeline? Is that secure because that's what the Auditors do right like, okay, that's great.
Now where's the code that builds that oh, we don't really have code. It's a bunch of scripts and like, where are they stored? You know, how are they sure right so that's right.
It's they, you know, the expression of turtles all the way down. Right? So like everything, you know again Capital One did a great job.
It isn't a criticism because like I mean like in complex environments, which all what we that's our domain like, you know, there is no sort of single stop everything. So but the thing was that like, you know at some point it's using me say, you know, so You know Monday Morning Quarterback to say, you know, like another discussion would have been like we really nailed how we configure things, you know tombow pallies to call it the clean room concept, right? Like they really had this thing down what they hadn't thought about is.
Okay. What do we do when we have exceptions? Right?
Like how do we sort of lock that down? How do we you know, what do we do? What's the process there?
Let's evaluate it's and it is in a day the devops portion of whatever you want to add to those characters is about continuous Improvement. And I think that's a great place to you know, kind of wrap up John right? Well, he there's so many and for a lot of folks that are here today.
It can be overwhelming right? There's so much to do. There's so many different options.
There's so many different knobs or so many different folks telling you you know, what's right and what's wrong. I hopefully, you know from at least this session, you've got a sense of you know, where your holes are on the back end of the process right the infrastructure and the operations piece of it. You'll hear from a whole bunch of other folks.
But I mean, I think the reality is like there's a lot of material that you can leverage a lot of it written by our John Willis here, right, you know has just done really just I didn't even work in terms of documenting constructs and work books and and different structures to be able to start to roll out devops in a way that gives you What you're supposed to be achieving with it right faster more reliable more resilient code. That's what this is all about. And everything is code now, so what that's on any any other parting shops before we before we wrap up a little bit here.
I mean, I think it is there is to take takes a village concept right? Like I learned incredible amount from you guys. When you doing all your Cloud security work, right?
I mean, we had a couple of calls and you know, I was really trying to figure out like what Shannon would tell me what you know, you know what, you know, Josh Corman would tell me and I like to celebrating stuff and and I think Allan used to be you guys and I got to sort of see some of the work you were doing. So it just takes a village. You just got to be constantly asking questions.
You got to find people that that sort of like make sense to you you made sense to me Josh Corman made sense to me Shannon Leets made sense to me, you know to trust your instinct from an on-ramp perspective this gazillion amount of noise out there and some of that noise. Take in the wrong direction. So trust your instinct on the people that you hear you listen to you know, it's it's what I've used.
I know it's what you've used in your career. You you trust your sort of instinct to say, you know, I think Mike knows what he's talking about. I'm gonna listen a little more to what he has to say, that's the sniff test and that and that's really great great advice.
And again, I think that the interesting thing to really focus on is the fact that the good news is you're not the first one do you the first one right into these issues? There are folks that have pioneered the way you're not taking the arrows in the chest, right? So so there is a benefit to that and that you can learn from all the mistakes that everybody else has made and again, there's a tonal resources out there.
Not just a tech strong, you know, a lot of different other places that you can get again a lot of guidance and a lot of help to move things along so General. Thanks so much. Hope you do all sorts of.
You know, you got a day job tell us it's a little bit of fun. Right the you know, I think that's the point right so I'm right now, you know for my day job, you know that if you want to see what what I've been up to for the last like five years really focused on a sort of automated governance or devastating. It's John it also calm the place that if you just want to have a conversation about stuff, you know, I think my goodness to like there were times I'd call Mike and we just meet for lunch and Atlanta and we just have these conversations about like we're we're very approachable both of us, you know, so and we sort of like to talk about this stuff right?
And so, you know, you know my box blue, but I think hopefully you guys have put it out because nobody's gonna know how to spell it at gmail account is sort of my permanent, you know where I'm always at. So that's the if you want to have a general conversation with me about any of this stuff for lean or agile or God forbid if you want to talk about Deming by everywhere. So anyway, don't ask that question if you don't have a couple hours.
Yeah. There you Courage. Yeah, this is fun.
Like it was great. That's great. John Willis.
Thanks so much. Let's wrap up now and we've got another session right behind this and another great, you know kind of example of how you can do devops in the right way as you're getting started. So with that we will At this one off and and head back to the studio.





