Techstrong Gang – April 2, 2024

Hey everyone. Happy day to you. We've got a great show lined up today.

Is DevSecOps there yet? I don't know. Maybe it's soup.

Is GI ops a real thing? We'll find, we'll talk about it. And then AI and testing is a failing the test.

All that are more for you on Textron Gang. Welcome back everyone. Alan Shimel here at Techstrong, uh, headquarters.

Welcome to this episode of Techstrong Gang. Let me introduce you to our gang members for today. First of all, our, uh, well, she's an editor here at Techstrong.

She's Core Gang from, uh, deep, dark, deep down in the heart of Texas. Amanda Razani Ani. Hello, Amanda.

How are you? Hello. Good.

Nice to have you on with us today, Amanda joining us. Amanda and I is our friend in Mitre, and she's a genuine gang member here. Pretty in pink.

If you, if you don't mind me saying that, trace Madden. Hey Trace, how are you? Hello everybody.

I am doing really well. Really well. Thank, good, glad to have you on.

And then joining me, har Heralding from the Bronx, New York here in Boca Raton. For now though, our Chief Content Officer, Mike Vizard. Hey, Mike.

Hey everybody. Alrighty. So, look, I've been out, I was outta CubeCon and then took a little time off and I, but I was watching a lot of the Techron gangs while I was away.

Great job Mike and team on making it happen. It, it was a really some quality events there. But let, let's dive into what we got today, right?

And we're gonna start off with a segment on DevSecOps. Is it soup yet? Is it there yet?

Um, I probably, I'm a I'm a much more glass half full kind of guy on this one. That may be some of you. But Mike, what do, what do you I'm, I'm on the glass half empty side.

You don that's why, that's what makes it happen. I'm interested to see what Tracy is on it. I know she's gone back and forth on it.

But anyway, what's behind this one? All right, So Tracy, I'd love to get your opinion on this. 'cause the folks at Jfr put out a survey and they were saying, um, well, it looks like people are having developers spend four days a week on remediating vulnerabilities.

But then their research team noted that they might be not spending their time on the right vulnerabilities because some of them are not as quite as severe as the security folks would make out, and others are not, um, actually present in their libraries or not internet facing. So what is your assessment of the state of DevSecOps? Part of my soul wonders, aren't we making a bigger fuss about this than it needs to be?

We are not making a bigger fuss about it than it needs to be, but we're not addressing it as well as we could. Right, exactly. To your point.

We've gotta help to prioritize so people understand which of the vulnerabilities that are worthwhile, because every vulnerability doesn't need to be fixed. Yes, I said that not every vulnerability needs to be fixed, to your point, if it's not in externally facing, if it's not, uh, exposing you directly, if it's in a little used area, if it's something that you are about to deprecate, there are things that you can do, uh, to avoid having any kind of vulnerability exposures. But it comes down to prioritizing what those things are.

And we need to not just drop this on the developers. Part of the issue that I had with that recent report is that it said the developers are fixing all of the security flaws. Well, shouldn't it be a mix of the developers and the security professionals who also are hands, hands-on with the code base.

Those together make it a better proposition as opposed to clubbing things onto the, the, uh, the folks who are not as adept from a security perspective. So look, I I've seen a raft of surveys that developers only spent 30% of their time coding. Right?

32%, something like that. Mm-Hmm. Shame on them.

I'd like them. Shame on us. They're the most highest paid people.

And what they want to do is code. Let them code. However, securities everyone's responsibility.

And I'm gonna tell you DevSecOps is there, DevSecOps is, is DevSecOps, is DevOps, DevSecOps is software supply chain security. It's another name now for DevSecOps. So DevSecOps, is it the, you know, I, I think what you get is when you get developer centric vendors, and I love my friends, the frogs, but you get developer centric vendors asking developers, what are you doing besides coding?

And anything they're doing besides coding, then coding then becomes the boogeyman. They gotta do a little security. No one's asking them to be security pros.

I yes, maybe four days a week. They spend some amount of time fixing vulnerabilities. But we need to use AI and automation.

Mm-Hmm. To make sure these vulnerabilities don't get into the code to begin with. It's generally not the code they wrote that's has the vulnerabilities per se.

It's the, it's the components they stitch together. Maybe they didn't use the UpToDate one or something. Tracy.

Tracy, to Your point, I would argue with you on that one though. Um, I would argue with you on that. So it depends.

So when we talk about ai, there are different types, right? Generative AI is, is the new kit on the block. And that's completely different than the AI and ML that we've been using to make code better.

That's how we get all of those emulations and simulations. That's how SpaceX was able to create the rocket in seven years and send it up, right? So there's amazing things that are happening there.

There are multiple reports, including one that just came outta Stanford University. That code that is resulting from a generative AI actually has a higher amount of vulnerabilities in it. So today we're adding today.

Yeah, today. And, and that's where I'm going with that is ground breaking potential generative AI with limitations with some things that we have to be concerned about with challenges. That's where we're at right now.

And that's why all of these reports can be very confusing because we're at an inflection point. Things are changing. Yeah.

No, but what I was, numbers Are gonna code less. I get But trace what I was getting at is the, no, I don't want developers to spend an inordinate amount of time on security. I don't think anyone does.

We want them to code. But when most of your vulnerabilities are coming from third party components, yeah, that's fine. We need to do better.

And whether it's generative AI or AI ml AI or, or some other types of automation, we need to do a better job of making sure that the code that's getting sucked into our code base is as vulnerability free as we can make it. So No disagreement. Yeah, no disagreement with that.

And it's a balance. All of my code is not external. All of my code is not coming from open source.

Right. So I'm just balancing you off Alan by saying Yeah, No, but, but about 70, 75% is, Well, well depend Depending on your domain space. You bet.

Let me ask you about who is responsible for patching, because I feel like we have this setup now where the security people are going, well, I need the developer to fix that. It seems like there's a lot of these things that are either at the OS level or some of the more common components that a security team can patch. And maybe we should let them patch more than we currently Do.

But what at, at before deployment or after deployment? A little Bit of both. So I, I think that's what the jfr and the GitLab and the CloudBees of the world, they're all doing, they're fixing vulnerabilities sometimes, right?

In the IDE, right? A a a, uh, a developer puts code into his IDE and immediately it comes up, Hey, this is, this has a problem. It has an overflow, it has a cross site, whatever.

And, and here's the fix. And you can fix it in real time. And the reason that's always the nirvana preferred method is it's a hell of a lot cheaper doing it then than it is to have that developer do the code, put it into, into a Git, and then sometime pre-deployment, or heavens forbid, post-deployment, a security guy has to go do it.

It, it, it, it becomes more expensive. So again, that's why I say by automation, and I don't know which type of ai, maybe it's all theis, right? Live in that.

It's A mix. That's a whole nother, that's a whole nother conversation. It'll take Us hours.

I I, I've seen demos of this in practice. I know Jfr, for instance, had, has it built in already? Do we need to reeducate the developers or are we just gonna give them tooling in the future and hopefully fix these issues as they go along?

'cause it's tools Alone are not, the answer tools alone are never zero, never. The answer tools are always there for humans to help humans. Education is key.

You've heard me say this for a couple of years. No, hobbyists. Our developers need to not be security hoists.

We need to bring the security people. We need to, whether we say shift them left or shift them all together, there needs to be some cross pollination. We need to educate the developers.

But developers are not the only ones responsible to Alan's point for security. It is shared. And we need to be talking about who's who in the zoo.

When you sit down and you talk about your secure software development framework, when you talk about your processes, what are the agreements on who's doing what? Because that is where the magic happens. When people truly understand where their responsibilities are and what they have tool wise to help them, not your security.

You're over there and because you said the S word, everything with the S word goes over to that group over there in the corner. And everything that has to do with coding goes over to the other guys on the other side of the room. We gotta stop that.

Yeah. I think proper communication and collaboration is very critical. And in this day and age, most people, I think across any sector should be expected to wear a few different hats and be capable of doing a few different things.

It it all coincides. You know, I, I've watched the DevSecOps space evolve for nine years now, right? We're doing RSA, I think it's the eighth or ninth DevSecOps, RSA, we're doing, I think an early mistake in DevOps or in DevSecOps, excuse me, was trying to make the developer security people and educate them.

You know, in all my years I've never met a developer developer who said, man, I love writing s****y code. I love writing code. Chockfull of vulnerabilities.

No, they're, they're craftspeople. These are, but I like to go fast. I they don't say, I like to go fast.

They say I have to go fast because I got deadlines and everything. Right? And I think quite frankly, when you look at some of the early DevSecOps companies, and I'm not gonna name names, they try to make security tools for developers.

Wrong, wrong, wrong. To Chase's point, it's not the tools. What you want is you just want to impress on the developers, security is quality.

You can't go fast and be reckless, right? You, you want to go fast, you gotta have quality too. It's not two outta three ain't bad.

You gotta have it all. And, and so the sooner we let the developers code and build as much automation and stuff as we can, knowing they know what their mission is and that security is quality, and then have the security pros running the security tools, the better off we'll all be. Developers will tell you that the tools that provided are slow and they Security tools.

Yes. And they will tell you that they generate far too many false positives. Absolutely.

It's far too much noise in the system. And they don't, and they can't make rhyme release Of it. And it's not only that they don't under, you know, the security tool tells 'em, well, this, this is bad because it's in violation of, I don't know, PCI four, oh, GDPR, whatever you want to, they don't know that stuff.

They just want to know, all right, where, where's the bad code here? What do I gotta fix if I need to fix it? Or if it can be fixed, you know, automatically.

But then they als Go ahead. Go ahead, Amanda, please. Well, they also need to know, should it be fixed?

Should they be spending their time fixing it? Because a lot of it, they should, you know, there should be understanding of, um, what's acceptable risk. And that doesn't need to be fixed and isn't an extreme problem at the time.

Agreed. There's guidance that has to go with that. Because the, at first they don't, until they're exposed to it, until they have help, I won't expect them to make that decision.

So there needs to be the architects, the engineers, the leadership, the security folks, they all need to be having the open, transparent conversation about what is acceptable risk. Right now, people are very, very panicked because there are so many security breaches that are going on. There's so many cyber risks that are happening, but people are panic stricken.

There's something out there. Yeah. It's a 3% chance.

We need to fix that. Well, okay, let's scale it back and let's talk about the likelihoods and let's talk about what the potentiality is for That point. Don't that on the developer's back chase, right?

No, I mean, No, no, no, no. That's what I'm saying is it is, it is a group conversation. It should never be on any one person's back.

I do want your opinions on the following though, right? We have seen vendors start showing up with DevSecOps platforms. It's the next generation of CI/CD.

Do we need to get to a whole new platform here and modernize it that way? Or can we achieve DevSecOps with our existing CI/CD platforms? Stop with the point solution.

This is the whole point of DevOps next, our research project coming out, right? Uh, and I learned this from Jody Bonai, the CEO at harness. The, the cobbling together of point solutions is never going to get us where we need to be.

We need, we need purpose built platforms that have security equals quality built into the platform. I, I feel strongly on it. I'm in complete agreement.

All the research that I've done, the organizations that I've talked with, it isn't often that I say yes. Yeah, no, I think it is the first chase. I think think we should end it right here then.

You know what? That's great. Write It down.

Put it, end it all. Now Let me just say this. We're gonna be talking a lot about this at our RSA DevSecOps and ai, uh, mm-Hmm.

Seminar, as they call it. Officially, it's an all day Mm-Hmm. Monday May 6th at RSA.

Check out our site. We get your free pass to get in there. But for now, we're gonna take a break.

We'll be right back. All right. And we're back.

And since we've been talking about some contentious issues in the land of application development, let's just jump into GI ops 'cause I'm scratching my head about this one. So, most recently, um, a cloud service provider called SIBO bought Cube First, it's a GI ops platform that was created by one of these accelerator companies. And God bless 'em for doing that.

Somebody needs to support the project. But at the same time, we saw WeWorks go under and we've seen some other moves. And I scratched my head over GitHubs because it seems to me, basically at its core, it says we need to manage the infrastructure as code alongside all our other software artifacts along inside the same Git repository.

Upon further review, it strikes me as kind of, well, oh, water is wet, sky is blue, and kind of obvious. So why are we calling this thing GI Ops as a separate motion? So I think this is a classic case of a product becoming a feature.

Okay? Number one. Number two, to me, GI Ops was always code for cloud native CI/CD, right?

With GI Ops, where, where some of the legacy CI/CD uh, tools such as Jenkins really didn't work so good in a Kubernetes type of environment. GI Ops was kind of designed from the ground up to work in that cobe environment, that cloud native environment. And so it got very, very popular.

Argo probably the most popular GI op, uh, project out there is, I think the third largest project in CNCF. And that's out of 180 projects. The only ones bigger than Argo is Kubernetes itself and, um, OpenTelemetry, right?

And then Argo. So if there's that many people using it, developing it, downloading it, someone's sneaking it in somewhere, right? The, the issue is though, does that make it a standalone tool?

Does it make it a standalone product or is it a feature of a cloud or, you know, another big, I mean, the people who were behind Argo for a long time were Codefresh our friend Dan Garfield, and they just got acquired, I think right before C**n. I, I'm at a loss at this second who acquired them. But we, we could get to that.

So I think, you know, this is a classic technology play. Product becomes feature. It's not that it's not real, it's real.

It's just not a product in and of itself Chasing thoughts. Oh, I'll, of course, of course. I have a thought on this.

I, again, this is gonna be a twofer for Alan. No. Yeah.

The, I always go back to what the intentions are of GI Ups not selling you a product, but talking about infrastructure as code. One of the strongest things we can do, whether cloud or not, is make sure that we're codifying everything, everything documentation, everything is treated as code with a sole source of truth. Those are kind of those immutable principles that we have to follow.

So GI ops, from that perspective, I don't like the fact that we have a different term for it. I don't like that we say GI ops. Why are we not just saying DevSecOps applied to infrastructure, applied to code, applied to testing and applied to all of those things.

Um, so I tend to agree with Alan that we're overblowing it a bit by calling it GI ops, by selling people GI ops platforms. It's all about being declarative, right? It's all about being declarative.

It got a leg up when it came to Cloud native, but we can definitely apply it anytime we have anything that work that, that is driven by configurations that can be codified, anything. So you can do that on premises. You don't have to have cloud.

It just got a leg up with cloud. We talked about this on a previous show while you're away, so I'd love to get your opinion on it. But GitHubs, I was away.

How am I supposed to know? Well, Well, GitHubs is, is GitHubs not akin to like platform engineering another term that we're throwing around, but doesn't really mean anything? Well, No.

Well, I, I have thoughts on platform engineering and I'm gonna come to that in a second. Um, look, it, it's definitely was a term that was thrown around and I, I give them my thoughts, you know, as code for Cloud Native. Here's what I found interesting though.

com over 10 years ago now, um, infrastructure is code was a real thing, and it's still a real thing, but it, it went out like the cool kids stopped talking about it for a while there, right? It, it went outta favor recently, like in CubeCon in Paris two weeks ago in, or a week ago, or however long ago it was. Uh, and even right before CubeCon, I, I was pitched by a couple of new companies who are in the infrastructure as code place.

And they didn't use the GI ops word, they used infrastructure as code. They said, and Chase, you're probably not gonna like, you won't agree with me on this one 'cause it was too much agreement. Um, but generative a, uh, generative infrastructure as code is a new company that, that's not the company's name.

I think the company's name is app cd, but it's generative infrastructure as code using generative AI to help with infrastructure as code. Um, we interviewed a few folks in Paris. The, the, the core idea was once the application is written, you'll apply generative AI to it to automatically configure the infrastructure as code required to deploy the application versus kind of the other way around.

Um, sounds kind of cool. I don't know which is the cart and the horse here, but Tracy thoughts, Um, well, so it, it, the cart and the horse, we could argue back and forth on it. I will say that generative AI right now, if it depends on who is creating the model, who is training the model where that's being hosted, if you're leveraging publicly available models, they are much less fine tuned to what you need.

They have much more, uh, problems with security if this group that is generating infrastructure has taken and either augmented with their own re uh, you know, retrieval, augmented, uh, or if they have taken and done a fine tuning on a model that they themselves are hosting prob you know, chances are that they are able to generate, uh, whatever the infrastructure could look like. But there's something people need to remember. And that is that generative AI is called non-deterministic.

What that means is that when you generate something, you can get 3, 4, 5 different options because it is not deterministic. It can't do mathematics. Generative AI does not do mathematics.

So while I agree that you could leverage some generative AI to help you jumpstart, I would not be dependent upon that wholly to do my infrastructure as code. I would look at it as a way to help me brainstorm and to understand what my options are. So yeah, it could work there, but I wouldn't depend on it, right?

To Tracy's point, uh, generative AI is what they call probabilistic, which is a fancy word for, it's probably right. So can you bank on that in scenarios where you need it to be Absolutely right. Or can you use it more for things where you're kinda, eh, we're close Enough.

So we're making a left turn here saying, is J AI gonna replace people or other things doing it? I don't think that was the point. I think the point was it was gonna help you.

Mm-Hmm. Do your infrastructure as code mm-Hmm. Not to replace you doing infrastructure as code, but the bigger point I was trying to make is that infrastructure is code is back, it never really left, but it's back and it's on people's minds.

And maybe that will replace GI Ops as a term. And, and we don't have GI ops as a standalone product category, if you will. Now platform engineering, different case platform engineering is just a new name for what ops has been doing for a long time, I think.

Mm-Hmm. And, and so, and if that's what makes the ops people happy, because they always felt slighted. They wanted ops dev, not DevOps.

So be I, we did the same thing with DevSecOps, right? The DevOps you get is the DevOps you deserve. Same thing with platform engineering.

So a couple of things I think is, I think platform engineering is about managing DevOps at scale. I'm okay with whoever wants to be in charge of that and play buzzword bingo. That's fine.

I would also say though, I'm really hopeful around Gen AI and infrastructure as code for the following reason. A huge percentage of the security issues that show up in the cloud are misconfigurations created using these infrastructure as code tools by developers who don't know enough about how to configure these platforms. And that's why we get things like open ports and all kinds of silly things going on that, you know, people say the cloud is not secure.

The cloud itself is probably more secure than on-premise. It's just the processes that we're using to provision it are not secure. So it seems to me Gen AI is gonna help a lot in that regard.

And maybe we won't have as many cloud security issues, but trace, I could be dreaming here. It, It will eventually, I'm gonna keep pounding on this right now. Groundbreaking potential, absolutely amazing potential.

And we're not there yet. Are we gonna get there soon? It's gonna take a little bit.

Every one of these models needs to have AI assurance against it. We have to understand if they have been fine tuned, if I'm training against the vast amount of code bases that are out there, I'm not training against the best of the best. I'm really looking at something else.

There's a new model or a newer model, which is called Super Coder two, which is not being trained off the vast amount of open source things that have tremendous amount of bugs against it. They're doing a mu, it's this Nvidia and HuggingFace have gone together. Those types of efforts are going to make a difference.

They're, they're just over the hill. So I would tell people, pump the brakes and be ready. But to Alan's point earlier, um, hopefully we can get rid of the term GI ops.

Um, I would probably disagree. Yes, Alan has time to disagree. I would probably disagree that platform engineering is only operations platform engineering.

We are going to see the intersection of low-code, no-code, and what we're calling as platform engineering because it's all about reducing the likelihood that a human makes a mistake. So we're gonna be spending more and more and more time, our software engineers are gonna be focusing on those platforms and creating opportunities for there to be more and more people, citizen developers, not all professional developers, more citizen developers that are able to create software that delights the end users. We're gonna see a big change there.

Okay, one last thought, one last thought. All of these acqui acquisitions and m and a, uh, activity in the GI up space, you're not hearing big multiples. And to me that's the, that's the, the truth, right?

That's, that's you, you're North Star right there. People aren't paying big multiples for these. so-called GI ops companies and GI ops, uh, platforms and tools.

So follow the money, as my grandma used to say. Anyway, we're gonna take a break here. Wrapping up our GI ops.

We're gonna be back in a moment about failing the test. You're watching Techron Gang. I'm Bonnie Schneider, sustainability contributor to the Techstrong Group.

I'm excited to introduce you to a groundbreaking new initiative from Techstrong Research, the sustainability pulse meter. The pulse meter offers valuable insights into how environmental responsibility factors into tech purchasing decisions for key players in the industry. Position your company as a leader in the industry and differentiate from your competitors with a sustainability pulse meter offered exclusively from tech Strong research.

All right, and we're back talking about our third topic, which is kind of related to the first two, but we're talking about the use of AI in testing. There's a survey out from our folks over at tricentis that shows a surprisingly large number of folks who are already using AI to drive testing. And we've already discussed whether it's deterministic or probabilistic, but it seems to me nobody wants to write the test in the first place.

So maybe this is a great use for AI and to begin with. So Tracy, we'll start with you again, but what's your sense of what's going on here with AI and testing? There's a study that came out in January, uh, that went across, I think 500 different businesses, and it showed up that only 47% of organizations have automated their testing.

Yeah. So what I said, 47% of these organizations have automated their testing. That means A, we've got a lot of automation to do, but b means that we have a lot of humans that are involved in manual testing into creating test cases, cases.

I absolutely love the use of today's generative AI to help with brainstorming. So if you, I'm a quality engineer. If I'm a test professional, if you sit me in front of A-A-G-P-T-A generative transformer with my requirements, with my test plan, I can actually go back and forth with chain of thought conversation and come up with fantastic expanded test cases.

So I'm a really, really big fan from that perspective. I'm also a big fan of using the existing ai, not generative ai, but AI tools for synthetic test data creation. Fantastic.

I wouldn't depend on generative ai and I make that delineation. I wouldn't use generative AI right now for my test case or for my test data generation. 'cause it's not good with math, right?

It's, it's non-deterministic. So one plus one could equal a whole bunch of different things depending on when you ask it. But we're also finding that if it's public data that you're using and you say, I need you to anonymize that data, that we're still seeing patterns that can track back to real humans that can track back to actual individuals.

So you gotta be really super careful if you're leveraging public models to generate your test data. But I'm a big fan of leveraging it for automating and helping to augment those test, uh, test professionals what they're doing every day. Huge.

Yeah. What I, what I found interesting in that survey was that, um, like you said, under half are, are using it right now, but 74% of the professionals that were surveyed would love to use ai, um, to help as an augmentation tool for the testing and believe that it would reduce a lot of the vulnerabilities. So, um, there's certainly, um, a wide acceptance of using it.

So let me throw one more stat at everybody since we're stat heavy today, only 3% of all of the folks who have been interviewed who want to use it, trust it. And there's a reason for that because it's early days and we know to not trust yet. We're gonna see more humans in the loop to start.

It's gonna be a higher volume of human hours that are necessary, but it's gonna help us to get to that eventuality of better, faster, higher quality, right? We're gonna see a lot on the horizon. We just have to make those investments now.

com. Mm-Hmm. It was before AI was a cool thing.

We used to just call it automated. Mm-Hmm. Continuous automated testing.

So let's take AI out of it for a second. Continuous automated testing. What has been the biggest change in the life of a QA professional over the last 12 years?

They have, they are spending less time having to actually write tests versus more because that, you know, through all the libraries and the seleniums and everything else, the tests are kind of out there. They spend more of their time deciding what tests should we run? What, what is my coverage?

What do I need to adequately test this application? What, what's my test coverage here? What kinds of tests, which tests do I run?

And the QA professional spends more of his time scoping, if you will, the, the coverage of their test coverage than they do actually quote unquote writing test. And I think that is where you can, whether it's generative AI or ML type of ai, I don't, um, and maybe a little of both, right? I think it could really help us better scope our test coverage and then whether those tests are running through, call it old school automation, newfangled automation or AI or GIB or whatever you want.

That's a, that's step two. But in terms of figuring out our scope for coverage, I, I think it, it's gonna continue to play a larger role. I think not all tests are the same, and it's getting hard to distinguish what can we shift left towards a developer that, you know, they can help them with their job without putting the, you know, them in charge of the hen house per se.

I, because they, you can't, you can't test your own code Per, don't make your developer your tester. I think that's bad juju, right? You just can't do that.

Your, your developer has enough on his plate. He's not your tester. And, and, and we've said this, you know, Mitch and I, Mitchell Ashley and I host, uh, DevOps Unbound.

Tricentis is the sponsor for three years. We've had this on multiple shows. QA professionals are not failed developers or junior developers or wannabe developers.

They're QA professionals. Leave QA to the professionals, not the developers. One of the issues that has come up is that a lot of organizations rely on third party services to run their tests.

And that's expensive. Can we reduce the cost of that? Maybe because we can run more of these tests reliably ourselves.

You wanna talk expensive? I, you know, again, Mitchell and I doing still secure, uh, the early two thousands, that was expensive, right? When you had to do unit testing and stuff and you had to, you couldn't stim simulate, you know, 5,000 iPhone threes or something, whatever they were back then, that was expensive.

Now, with everything virtualized in SaaS, I, I don't know if you could take the air out of that balloon by trying to do it yourself. I, I don't know if it results in that much savings, But the, let's say I do get an automated service. I know we've seen the emergence of these devices as a service platforms for testing.

They're considerably less expensive than what we were doing when we hired a small army of third party testers to go, you know, create the, run these tests for us, that was like cost prohibitive. And maybe why we didn't test enough That, that it absolutely was. I mean, it was, it was crazy.

We just didn't have the equipment. Now that everything's virtualized in a SaaS, I, you know, does it cost money? Yes.

Is it money well spent? Yes. Tracy, or application quality?

Is it gonna improve in the years ahead? Where are we? Ah, it is going to improve as long as, uh, I think we're going to see it get worse before we get better.

Quite frankly, I think in the next six to nine months, we may see some quality fumbles. We may see some breaches depending on the organization as they're rushing to change and try and improve with all of these new technologies and capabilities. However, we're gonna see some really tight code.

We are going to see automations, as Alan has said, we are going to see improvements in quality, dramatic improvements in quality over time. So in about two years, based on my research so far, I think we're gonna start to see some pretty awesome things. Amanda, did the, did the AI engine eat my testing homework?

Is that what we're gonna say? Well, I think a AI is going, I think AI is definitely gonna be a helpful tool that we see in the testing process. But I also think, um, if we can see, um, more continuous testing through, um, like, uh, separate sprints all going at the same time would be helpful and, um, more efficient.

And I, I would just, again, reemphasize not all testing is security related, right? There's UI testing, usability testing, scalability testing, right? There's, there's so many different flavors of testing today that an AI helps in some of 'em.

ML AI helps in others. Old school automation helps in others. There's a ton of great open source tools out there.

I mean, the, the testing frontier is, is so dynamic right now. So much more than it's been, you know, it used to be a lot of people did think testers were junior developers or developer wannabes, but testing professional, the testing profession today, the QA professional today is worthy of the name professional and the, the sophistication of the, of the whole ecosystem is so much better than it ever was that I do think, you know, overall our quality, our quality's better today. But we're also, the amount of code that we turn out and the coverage of testing that that code receives before it deploys is probably higher than it's ever been.

I'm a little dubious on one point though, and some of it comes back to this whole agile development initiative. But seems to me a lot of developers tell themselves the following, right? They go, well then we have an issue.

I'm kind of behind schedule. I'm skip the test because you know what, I'll just fix that in the next release of the thing and I'll just do the next update of that thing and I'll catch it then. And then the next project comes along, or the other project comes along, or there's, somebody's got a patch of vulnerability and then they never get to it, and then some days it bites them hard and other days they kind of get away with it.

And so we, I think we've become our own worst enemies in some regards. I mean, I don't know if I've seen that. I was gonna say I don't, I don't know if I've seen that quite the way that you're, you're talking about it.

Uh, most, most, um, development professionals, right? If we're gonna say quality professionals, we're gonna talk about development professionals. They want to do a good job.

And we've spent a decade plus helping people to understand that all that matters is quality working software that delights the end users. If that's not happening, just delivering crap fast is delivering crap fast. So I don't know of too many personally who run that gamble that I'm just not gonna fix it so I can get this other thing done.

We've, we're trying to enable that openness where you can turn around and say, pause, pause. Instead of this additional feature, we need to get this piece, the automated testing accomplished for this as well. So when they're checking in their code base, they're checking in the new feature and they're checking in the automations for the testing against it as well.

I would just say that half the phones on my, or half the AppSec on my smartphone would disagree with your statement. Well, but, but but to the point, how often do half the AppSec on your smartphone get updated? Probably once a week.

Maybe at least. Yeah. Because that's the difference.

When you were releasing code twice a year, once a year, once a quarter, it was very common to hear, you know, the following, we, we couldn't complete our testing coverage 'cause there just wasn't enough time or there wasn't enough budget wasn't enough time and then to, to go fix the, the bugs that testing turned up. There wasn't enough time. We had to make our deadline.

So we release code that wasn't fully tested, that wasn't fully fixed, et cetera. When you live in a world where you, you're releasing code 10 times a day, 10 x right? Like John Hall all, uh, or I mean, alright, maybe that's still extreme, but you're certainly releasing code daily, right?

That I don't think that's as good an excuse anymore. I've, every survey I've seen says less than 15% organizations are releasing code daily. I mean, we're just not that far along yet, But the, the act of releasing is different than having the ability to release daily.

So I may choose to not release daily because I have, I need to know that those who are downstream are willing to accept it. They're ready for what that change is. And it's something I put on the smartphone that is inherently easy to understand because I give you the little bootstrap overlay that says, look, this button has changed.

I can't always do that in business. I can't always do that in some of the different types of softwares that we're talking about. So sometimes I choose to not release.

But that's the difference between having the ability, which is where Alan is going. I have the ability to release as frequently and as often as I need to. And that is where we get to that higher quality because DevSecOps is about reducing our risk and improving our quality because the changes are smaller and more atomic and more testable.

Exactly. To, to Alan's lyric points That it's not my, that's like at the heart of Agile right there. Mm-Hmm.

You just, you just nailed it. I think that's a good place to end today's text on gang. Tracy.

Amanda, thank you so much for joining us, Mike. Excellent. We will be back.

Uh, this we'll be well, we, we won't be on Wednesday. We'll be back Thursday with a fresh Textron gang, maybe some special guests. We'll go from there.

Until then, though, this is Alan Shimel. You've just watched. Textron Gang Cloud native now is the web's leading resource for the growing cloud native ecosystem.

com is your destination for news, thought leadership, features and webinars on cloud native architecture, Kubernetes serverless, cloud native application development, microservices, service mesh, cloud native security, and more. Stay on the cutting edge of modern application development at Cloud Native. Now.