Quality Engineering: The Future of Software Testing – DevOps Unbound Roundtable

Good day everyone. Welcome. Welcome to another live Round Table of devops Unbound.

For those of you who are not familiar devops Unbound is a Bi-weekly, meaning every other week video show that we we produce here at Tech strong TV in partnership with our friends and sponsors that I sent this. worldwide leader in automated testing and then like once a month or so we do up this version of devops Unbound a live Roundtable version. Where in addition to having a fantastic panel as we always do?

We really open things up by opening it to you our audience to help drive the conversation. Before I get into today's session and our panel, let me talk a little bit about the odd to you our audience and I realize some of you maybe are just joining. We love having you involved.

We want you involved if you look at your web browser on the right hand side, you'll see. communication your big Market communication window and for most of you it defaults right into chat and public. If you type in there everyone in public everyone can see what you typed.

It's all good. If you put a question in there, we will try to answer it. We may answer it at the panel wide level.

We may answer it directly to you in chat. If you put in a chat anybody can answer it may not be someone on the panel even but so much the better. If you have a question directly for the panel that we'd like to raise to the whole panel level.

Please go to the Q&A section of that communication window. So up top where it says chat right to the right of it. Great out for most of you don't say Q&A.

If you click on that you can see that you can put in your questions there for the whole audience. What we have found and we're doing this now. I almost I think two years what we have found in two years of doing this is that the more audience participation we have the better the show is so if you've got questions, you've got comments.

You have thoughts. Don't be shy put them in. Why don't we can't get off?

I see some people we've got people from New York. Some people from La. So we're already by Coastal but we usually get people from all over the world.

I know there's got to be some folks out from India here from Europe perhaps in the UK. We've got Virginia. Come on, tell us where you're from it.

It's always one of the fun things about being on the Internet is seeing. Who's here? Next up though.

I wanted to introduce you to our panel members, but I think before I do that, let me quick word or what we discussing today on devops Unbound quality engineering the future of software testing quality and of the future of software testing, excuse me quality engineering focuses on the prevention detection and Remediation of digital experience issues. And that's a mouthful we could be talking about user experience security any number of things that fall under this broad umbrella of what we're calling quality. We're going to explore it some more with our panel and with you let me now get to our panel.

First up. I want to introduce you to it's actually his first time on a live round tables on our live Round Table anyway, so let's go easy on them, but let me introduce you to Lee Atkinson Lee welcome. Thank you.

It's great to be here and don't go easy on me. That's fine. All right, Lee.

What about maybe giving folks a little bit of your background? Sure. Sure.

So I'm a software Architect by training. I spent seven years with Amazon as uh, helping to build their service-oriented architecture back in the Amazon retail part of the organization back in the early 2000s and then moved AWS and helped build the elastic Beanstalk service. Then I moved on to New Relic and spent seven eight years with New Relic.

I'm a now mostly an author and a writer. I've got two books with arriving media architecture for scales. My main book and my second book overcoming IP complexities coming out this year still yet this year and I also write a lot of articles.

I write a week weekly article with container journal and I write for infoworld indigenamica and other places and I'm doing a lot of training courses with LinkedIn learning and then coming up with my own Training Academy sometime next year. Fantastic and Lee welcome and it's pleasure having you on. So as much as it's least first time our next guest he's a Mainstay for us here.

He's always here to help out and let his thoughts. I want to introduce you to Adam Kelsey. I got that right.

Absolutely. Hi, I lead or developer relations and internal platform products here at tricentes and I'm mainly focused on making developers awesome and making sure that that what we build and is high quality and and useful to the world. Absolutely, our third panel member really needs no introduction to our devops Unbound audience either.

She's the host of tech strong women as well as a frequent. Guest on many of our webinars and panels and she's also a board member of the open ssf former board member on the CDF. CEO founder of deploy hub for janitor of an open source project with CDF Aurelius first serious or till yes our chili.

Yes, who is the you mapped out? Well the world value right Tracy Reagan Tracy. I tried I tried.

Okay, you did fine. Yeah, it's a pleasure to be here. You know testing is actually where I sort of cut my teeth after I decided I didn't want to code anymore and it started what I had some code going into production that was old and I just said what happened in that process and I realized it was a bad build and that's how I started my first company.

Yeah. So I've been doing this for quite some time lifecycle management has always been a passion and I continue to do it and super happy to be here and having this discussion. Absolutely, so guys when I think about quality as it relates to Software.

As it relates to testing and security. I'm reminded of that commercial where they used to have the little kid who would come on and say yeah, I want to be obsolete by the time I'm 35, and you know, yeah, I want to be in a dead end job blah blah blah. and it's the same thing.

I've never met a developer. I've never met a test or I've never met a security or Ops or admin who raises their hand. And says, oh, yeah, I want to develop low quality software.

Right? I want to release something. That's just bug grid into terrible.

Right? We all have pride in in what we do and what we turn out and our work product. Unfortunately as we all can testify we've all been in the in the industry a long time.

You know, what I'm gonna stop us right there. I think I did introduce my co-host as I I usually do because I'm older than adults. Let me if you just Mitchell Ashley, who's my co-host and CTO here in Tech stroke Mitchell.

I apologize. No, no worries. It's helping a load a full question and talk to the audience.

So glad to be here CTO which extern group and partner with Alan. Wonderful panel, so let's jump in now. All right.

Thanks Mitchell. So is it I was so excited to jump into this. So no one wants to no one wants to deliver crappy code.

No one wants to deliver. You know something they're not that their names on and and they're not proud of it. But yet here we are.

We there's been plenty of crappy code and Bugger in software that has you know been released. Where's the disconnect? Right.

You can't let's not blame all the testers. Or should we right is is testing the gatekeeper of quality? QA quality assurance.

Is that the same as quality engineering? Right. Are they synonymous?

I don't know Adam since you work at tricentious and they're the testing company. I'm gonna ask you the hard question here. Are they Anonymous?

What do we doing now? So I I think quality engineering I think has two different meanings one is is to point out that that quality is an engineering discipline and that in that sense then then quality assurance in the testers and such is is part of quality engineering but the you know, we we need to treat that part of the the Engineering Process as an engineering discipline and not just a oh, we we let the testers do that and and hand that over to them the other Meaning of the phrase is that quality is something that has to be baked into the software that you can't you can't test your way into good software. You have to engineer it in from the start and so it's everything.

It's not just testing. It's how you design. It's it's secure by Design.

It's privacy by Design. It's it's how you Monitor and manage and maintain and and operate the software as well. Fair enough you can't include bad software to make it better.

Is that one way to put that right software is bad all the testing in the world isn't going to fix it. Right exactly, right. I mean if someone in testing once told me Testing doesn't make the software quality.

Just testing shows us if it's How crappy it is or how good it is, right? They don't don't shoot the messenger there, but it's not testing in and of itself. Is is not?

the the producer or the author of quality It's just testing it for Quality, you know? Tracy if I were a psych psychoanalyst, I would say there was something deep in what you said when you said well earlier my career when I will the words. I got tired of coding.

I was done code and And I became a tester. Okay, let's talk about this. Well, what happened was my car.

I would get code that was went to production. That was basically old code. And I know that it wasn't the tester's fault for missing it and I found it went all the way to the users.

And so I started looking at the the life cycle of of testing and I want to say that we have to we really need to disrupt the Persona around testing. We think about software testing and oftentimes we think about people just hammering on on a keyboard. I have so many good stories about testing it's hilarious, but I I there's one story I call it the fat man and the boardroom I was working for.

Ups and we were doing a massive massive implementation of their software all of their feeder system and they had all these truck drivers come in and we were supposed to do user testing and the first user came over turned the keyboard over and sat on it. And of course I got the and his point was you have to you have to build the software strong enough for for truck drivers because we do really dumb things. Now what it's what he said was there's chaos and software and I started thinking about that and I in my world I said the very first place that we really should be putting testing is in the software build process.

It's the very first step and when we think about quality engineering and we've done it for a very long time. We've maybe just not labeled it. I'm surprised it hasn't become testops.

That's what that name already, but good. Okay, but when we think about quality and digital products, we're thinking about every aspect of the of that digital product from its birth from the requirements. We could have mistakes and requirements.

We can have a mistakes in the build we can have mistakes in the code. We can have mistakes in the key value pairs. We pass to the cluster that's going to run it.

There is there are anomalies across the entire life cycle and testing is that exercise of creating quality in every single aspect of Life cycle not just the testers who get it and have test scripts that they're running or they automate it that is such a small aspect of quality engineering. So I feel like what we talk about partying has to be Broad. It is a creating quality or is it testing for quality?

And I think then that may sound you know, like semantics but I think it's an important distinction. I don't think testing creates quality. I think testing tests are quality verifies if I think it verifies to I would say that too but the point is an engineering we're addressing the entire life cycle.

Not just the last part when the software is delivered and you have people pounding on it. Yep, met you know Ford used to have the slogan quality as job one and whether that was true or what all went with that. But the basic the philosophy was at every person along the step along the process from the initial design of a car all the way through to the very end was responsible for quality.

And that's the way you make sure you have a quality product is you have everybody is responsible for Quality quality as they're most important thing. They need to worry about all the way through the process. You can't postponent To The Ends.

It's a process change throughout the entire food chain of a software product or an automobile doesn't matter. It's it's a process all the way through the system, you know leave that reminds me of while back there was the fruit of them Fruit of the Loom commercial inspector 12, right? That was the last person before it shipped that said this was a quality product and that sort of that checking it idea versus building and quality, right?

There's the idea right? He doesn't happen once it's through continuous repetitive improvements. A lot of small improvements Sometimes some big ones, but it's it's it's created as part of what you create not something you.

Find and well, you do find if it's there or not at the end, but you're not going to prove it that way. That's for sure. Yeah, and I should mention we have a poll question to do what you want to put that up about.

Sure. Let's let's run with it. So guys in the audience.

We'd love to hear you putting your thoughts on this. What is the difference between QA quality assurance and quality engineering no difference at all. It's a cultural shift quality engineering is much broader than quality assurance quality engineering requires more automation.

and if you want to put it in there. Have everybody join in we're getting a few should have our Jeopardy music. We have our Jeopardy, but you see what people think about the difference between B and C.

I'd love to get people to put into the chat or into QA about what they see the differences between right why they pick one or the other because right now you see you're running pretty close by to each other see edging it out a little bit. But I looked at those two and is like what really the difference here, and I know there is a difference, but I love to hear what people think. There you go.

Yeah the audience. Reaction. So yeah, if you wouldn't mind put in chat why you voted one way or the other and and we can discuss that I can tell you I I picked it's broader.

Because I I think it almost transcends culture, you know, one of the yeah, one of the things that I you know, one of the reasons I got into devops was security and what's become known as defects. I wanted the tenets of devsecops that I've always preached is security is everyone's responsibility. Well, you know what so quite so is quality.

It and maybe that is a cultural thing, but it's it's so much broader in my mind than just quality assurance. Which to me. I hear quality assurance, and I'm thinking I'm doing testing.

Right. I'm I'm assuring that the quality is what we expected to be. Cody decided here Dad goalie.

So I took a different different tact. I I absolutely see what you was thinking and I was thinking the same thing about broad terms thinking of doing broader, but I chose cultural shift and part of that is you know, I I talked about stoso which is the model that I promote and in my practice for a for how you managed services in the service-oriented architecture and organization that runs a service based application and the whole premise there is an individual team, you know, like the Amazon two pieces. Style team is responsible for everything having to do from that for that service from designing it to building it to testing it the quality to security to operational aspect to picking up the pager when it goes off in the middle of the night.

All of that is responsibility of that team and our teams like six to eight people so you know ownership is the key. I consider the ownership is a cultural Chef, you know, there's the throw over the wall to the next phase in the process is a Surefire way for things to be missed and skipped. But when you have owners of individual pieces and own a top to bottom Soup To Nuts everything having to do with it as a cultural change in most companies and that cultural change is what it really takes to make sure that your system stays operational scaleable high quality secure all of those things.

Pretty much and security. It's there's analogy that makes sense to me is you don't build a car in that ad airbags when you're done, right? All of that is part of safety is designed into the product quality is designed into the product.

As well. There's a there's a really great question in the Q&A. Someone imposed about we're always getting hit for feet more features Feature Feature Feature features.

And anytime we want to work on quality. They say well, but we want features. So it's it's not a feature you work on for quality.

It's got to be how you the whole process that you use and like I said earlier about that incremental Improvement not going to do it in one step. Yes, those two quality isn't the feature quality is just is. Well and do that part of your your philosophy the QA question, they're asking well, what do you do when they're asking for for features on your and you're just building quality and you you want to add quality in you have to be building the features with quality from the beginning.

It's not a process of adding something later. It's not a process of this is extra work. It's part of the work.

It's part of the engineering and if you're not looking at it as that, yeah, you you end up with okay, I built this and now let me go and and Harden it later and yeah, that's where we see companies that do anti-patterns like, okay. Well, we'll have a hardening Sprint before release or will if you're not if you're not baking that quality in you're going to run into that where we're customers or other stakeholders or saying, okay. Well, why are we spending the time on this?

You know, why are we spending the time on security on design on on anything else? You build it in from the very beginning? So good Twitter discussion just yesterday with someone who was talking about they don't have time or they don't you know, they're they don't have a large enough of an organization to have you know, service oriented architecture split up and all those things and it's a and it's like it's too much work.

I can't do that because I have to do all these features. I I have to do these features and nothing's more important. I can't think about scaling and think about quality.

All I need to think about is building these features and and that just brings you technical debt. That's what that does. It's yes you well, first of all, I want to thank participant 440 for asking that question and you know, we have to listen to that question and the question says hey, I'm a developer who's being pushed code code and end users want this software.

How do I build quality into that? And that is it's such a valid point and it's been a valid point for the last 30 years in software development and it will always be the problem. So I go back to the whole idea of where what can we automate to make sure that the quality is in there that has nothing to do with a line of code.

That might have been bad. I mean if I think if we really looked at the stats, and I don't know this for sure. But it's not just lines of code that are bad that causes quality issues.

There are so many other things that can be be offensive and create anomalies along the lifecycle. So the more that we can automate the more tooling that we can bring in the fewer scripts that we have that builds out our pipeline the fewer scripts that we use to build our code and the fewer scripts that we use to deploy code. The more elevated we have is the data that we need to make sure that we are baking quality into our process in terms of delivery.

So those developers aren't always blamed for a line of code that's wrong because that one is fairly easy to fix and it's usually caught somewhere along the line in user or acceptance testing. But if it's not it's not the end of the world. It's something that's fairly easy to fix if our pipeline is running properly and we have quality baked into that and I think that we're we're starting to have those conversations around both testing and security and we're going to start looking more policies.

What what and what's the data? There's so much data out there that we don't pull into our process. It's left underneath the devops pipeline.

It's left in chat sessions or Discord channels where people are talking about experiences. They have and problems they had and then that's none of that data is being brought into a central place so we can start doing proper machine learning and AI to really make quality a baked in process of the life cycle, and I don't blame a single developer. They make a code mistake because I know I made plenty, but if we can get code fixes out fast, and the rest of the stuff is working properly then we have a pretty clean system.

Absolutely, but you know, look, I'm probably the least technical person on this. panel today But I've been around developers and Engineers for a long long time and it's a consistent message I hear from them. Is that the pressure to deliver code?

Is the is the antithesis of delivering quality? And you know, hey, I got to get those features done. I got to deliver excellent lines of code and I I care I don't have time for security.

I don't have time for Quality. They'll fix that in testing. Right, and I I be honest with you.

I don't buy it. I don't buy it if you're pride in what you're doing. You don't want to deliver.

shoddy work product if you know you're delivering shoddy work product. And you still doing that job I ask why don't you become a test or like Tracy did or something, right? Be what?

What does that make? You know, we we've talked about this before thoughts on that then we have a great another QA. Question thing here and some great comments.

Yeah. What I would share is, you know, there was maybe not too long ago point in time where testing all happened at the end. And what always happened is time got squeezed in the amount of time left for testing got shorter and shorter same thing for security.

And so you just did whatever you could get done at the end and you shipped right and kind of like security as we shift test. We do more automation. We do develop testers do testing or built-in testing test driven development things like that.

I think there's hope of building testing into more quality into our software processes Tracy was talking about because it's systemic view of it. So it's never going to get done at the end. That'll now never solve it.

and to your point out and you got you have pride in what you're work is and so you have to work on the process as well as writing, you know, writing the application or whatever software that you're building. The quality engineering is all so more than just about testing. It's more than about verifying that what you've done is working.

It's it's even more than what we're talking about about everybody taking pride in and making sure you're building it in one of the things one of the trends that that I've seen is that more and more development teams and organizations are using methods other than testing to to ensure quality and to to everything from code reviews and and MOB programming and observability after the fact when you're deploying and and red green deployments and and all these different ways that we're that we're making sure that our software works is intended. It's not just about testing and I think quality engineering is is all of that. How do I track what errors are happening in production?

And you know, we we look at things like the the P90 and P95 rates of things. How do I know when those have Reach that threshold and what do they mean and and what caused it to go there? And all of that stuff is is about quality engineering that's all about building quality into your product and and making sure that it has that it you have a quality product even after the fact and it's not testing at all.

And remember a lot of companies if they're checking their code into get they'll do code reviews for one or two lines of code to try to catch that. So I you know, I always take the approach that everybody comes from a good intent and I believe that most individuals believe in that they're doing the best that they can even though they may say I didn't have time to test. Which happens it just does but the more that we can automate the more that we can streamline the process and think about quality engineering as that broader solving a problem from a broader View and capturing it, you know, implementing those policies you're going to have two code reviews when you get it I get check in before it goes into the build.

That's a first that's a first level defense as nothing to do with testing itself. It's just giving you know, if more eyes on the code to make sure that somebody doesn't see something that the original coder didn't see which in my day we didn't have that ability. I really really wish we could have because it would it's a feedback loop.

So how do we create that feedback loop in so many locations of the process? I think that will improve the quality overall. Guys, we've got some somebody else we haven't questioned.

I'm sorry deadly. Sorry, something else we haven't talked about yet is kind of is the environment the development environmental as well as the production environments. Chaos, testing.

You know, it's You know, one of the things that we talked a lot about at Amazon back in the day and I think it's being used more and more now with certainly that Flex promoted. This heavily is to create as much of a chaotic environment as possible within your production systems. Let alone your development environments so that as you as an engineer, when you build something you're building it into an environment that it has to be quality or it doesn't work at all.

So many software developers build in the vacuum, right and they build something that work once and worked on my desktop. So now I'll ship it and that's part of this mentality of I can do a lot of features and get them out, but they just won't be quality. But if you're excuse me your entire process of how you build it how you test it how you deploy it how you deal with the entire infrastructure all the way through and including production is a chaotic environment.

Used to having to build in quality features. You used to having to think about quality from day one and have that in your entire process as you go along that's part of the cultural change, but I think that environmental impact on building in It's a cruel world out there. So make sure your software works is an important way to do and Alan you talk about deaf secops.

That's extremely important for Deaf secops because that build security into the product as well from the very beginning. I'm absolutely you know, one of our participants put in here. The difference between QA and Q QE right the QE is to engineer the quality into the product and the QA is to ensure that the product has the right level of quality to ship.

I I like that I like that as I think that that's dead on in terms of I think the difference between the QE and the QA role part there. You know. Someone else wrote about that, SOA and and some proactive SOA and some new stuff coming out of IEEE.

I mean, obviously Quality engineering and quality control did the Nexus between them? There is a connection. They're just I think one of the most important things we want to take out here is they're not necessarily one in the same.

But they but they are in fact connected to that. something else that popped up was the this notion of Lee you mentioned with chaos, but it's it's not just the code Right. We live in such such a shift left in world.

Right where we think it's all about the code and all about deployment. But you know infrastructure has something to say there and when we talk about the quality of of our applications the quality of what we deliver, we we can't lose sight and you know and today SRE platform engineering is a very big thing. we can't lose sight of the fact that this code has to run somewhere whether it's serverless or Cloud native or what?

Have you? and so software Code doesn't exist in a vacuum when you have to run it on some on platforms and and so our quality has to extend there as well. To a certain extent with the Advent of the three large, you know public Cloud providers and we run stuff on public cloud.

Have we abdicated our responsibility for quality of those environments to Amazon or Microsoft or Google? I don't you know, that's part of them, but that responsibility, right? Yep, you know you the principal shared responsibility says that the cloud providers has responsibility for certain aspects of the entire Tech stock and you have responsibility for everything else.

And so it's your responsibility to make sure you understand exactly what the cloud provider is responsible for and you have our responsible for everything else. That's it's a you know, that it applies to security but also applies just to General quality and general infrastructure. Yeah and Davis and I'm gonna go back to harp on that data, you know each organization.

If you really want to start, you know, baking in quality. However, we want to describe it doing quality engineering every organization should be looking at where the where the where the bugs are coming from. Where are the anomalies is it is the testing is it in the deployment.

Is it in the infrastructure every time we do a deployment and there's problems a new a new release gets pushed out. Where did the problem occur? Because if you do that you're able to look to see what teams have done really great jobs of building in their quality and what teams are struggling but without again without the numbers without the data, it's hard to make decisions about how you can begin improving the quality within your organization because every single organization's culture is different and they have new and different problems.

There's those stats that say that the later you find a bug the the more expensive it is to fix and so testing earlier and and is makes it cheaper to deliver. But the cheapest bug to fix it. All is the one that never existed to begin with.

That that's a great that's a great point, right? But I guess it's it's a bug you never saw. What at the testing level.

right, but I so we've seen all these tools that it sort of testing the software literally as you're writing the code. It's testing the code. Right.

There is some. Folks doing that now. but bugs, I mean, you know, I don't want to call them bugs even mistakes or You know not just anomalies.

Yeah, I mean stuff happened. Yes exactly this features or features that you thought a customer wanted a customer wanted a different feature and that's a bug right? Yeah.

But in a lot of the time the the issue isn't that the software isn't working as it was designed. The issue is that you're design was bad to begin with right? My wife uses a piece of software that prints labels on things and they're they're continually saying, oh there's a problem with your your labels.

The UPC code is off the edge of the label. Well, it's because they're they're choosing to print right up against the edge of that label. So something slides slightly something slightly out of alignment.

It's working by Design. Testing says yes, it's printing it's printing exactly where we intended it to print. It's just the wrong place that it's not the right thing to do for the user.

And it could be part of this configurations, you know from an entire process begins with the original idea for why we want to write build software in the first place all the way through the product development the design the architecture everything all the way through to the very end and problems can occur every, you know everywhere in that process. How many products failed simply because it was the wrong product at the wrong time? That's a that's a bug that's quality issue.

Yeah. And that starts way at the requirements level right? Exactly.

Yes again require. It's it's it. How do you see it across the board but in Adam's example, there could have been just simply a configuration of the printer at the printer was printing it in the wrong place and I'm going to keep pushing that because when we talk about testing I hate to just focus on the software and the developers code and what was compiled and linked and sitting on somebody's machine and that they're hitting and and pounding on because from my experience and my years and testing it wasn't often that problem.

One of one of the biggest bugs that that we ever had to chase down was a It was it was in the build and a library was linked on the wrong boundaries. And every time you try to bring up the application you'd get what we used to call the os2 tombstone. Now that took us a long time to find it wasn't anybody's bad code.

It wasn't it wasn't any bad deployment. It was simply a tiny little parameter in somebody's build script that they passed over and then somebody else consumed it and it got copied a hundred times across the organization that had that that wrong boundary link and that caused the problem. So it there are tiny little details in this process the tiniest we most ridiculous little details and oftentimes it we lose quality in those tiny little details.

So it's not just the software it is how the software is configured and how it and then, you know, tools like infrastructures code was a movement that isn't that is a quality effort, right? That is a quality engineering effort because what we're doing is we're saying we're going to we're gonna take the the infrastructures configuration. We're going to put it in a file that we can now check.

Do something like get and we can start doing code reviews on our infrastructure configuration. That is a quality that is baking quality engineering into the process, even though the terraform people would never want to be heard as quality engineering tools, right? They're at infrastructure tool, but they are working to make infrastructure more secure not I hate to use the word more secure but certainly improve the quality because one person didn't log into the environment and make the changes on their own and nobody has any idea what happened and why now applications.

Don't run the way they should and you get have all this degradation and nobody can bring up there the basic, you know screens. So again, we really have to think about we talk about quality engineering it's not just software. It's not just programmers.

It is the entire Gambit from the day that we sit down and decide to build a build the software to when the user logs in. And there's a lot of pieces in between. I want to also highlight participant.

I don't know three one three at the bottom of their comment said something. I'm interested in the panels. reaction to it which is That adding quality adds more time to delivery.

And you know what that little nugget right there. That's the Crux of the whole thing. Does does quality actually add more time?

to delivery it saves time. As counterintuitive. I think I agree with you Lee, but You know, but nevertheless I think that's the prevailing notion that adding quality adds more time to this.

It has more time. It adds more. That's why it's a cultural change.

Yeah, yeah. So that it adding quality adds more time in the same sense that stopping at the gas station takes more time. Until you run out of gas and now you're walking and then you have all the time in the world.

Yeah. And you know, everybody's experiences their own and it may be in the case with that particular listener. Has the had that experience?

And there there quality process is just getting in the way and it's really not catching anything and I've seen that happen as well. so that could be the case, but in general if you really are committed to building a more a Pipeline of software delivery not to just deployment but from requirements to it you user sign on if you're building it in and automating it. It saves a lot of time.

It saves a ton of time because you're not having to spend those late nights looking for why a build or why something isn't running and it's because something was late was leaked on the wrong boundary, which is a horrible experience. Let me tell you. Yeah.

And Tracy could expand on that a little bit too. I think something else that I've certainly seen in my career is that it's the entire organization has to be part of this process and if you're an engineer and you get what we're talking about, but your management doesn't it doesn't matter right because you're never gonna have the opportunity to be on quality in you're gonna be asked where's that new feature? Yeah.

How come they're taking so long Joe's writing software faster than you why aren't you writing software faster? And that's the culture you're in and that's hurting you your company and your your opportunities there. This has to be a cultural change throughout the organization and you can't You can't do this on your own if your organization's not behind.

I agree with that. I also thinking can start at one team at a time. It is absolutely right.

So it might be hard for you one person to change the whole system. But a bigger part of the system, but still small can can take steps towards that and if those improvements show value people will glum on to it, right people adopt it. There are ways to kind of like absolutely agree.

I think what I'm seeing in some of the participants here are people who sounds like they they feel like it's not their choice and it's and and it's they can't make the decision on their own they need support to make that happen. We say maybe that support is management. Maybe it's other teams.

Maybe it's your manager. Maybe it's your teammates, but you can't do it on your own you need help you need support to make it happen. You can't build quality.

And if you don't have the infrastructure that's helping you make that happen qualities and you have to have the upper management is willing to put the money into looking at how to to build out the pipeline stronger add the tooling reduce to all of the manual efforts that can cause the quality issues. So Has to come from somebody above and I want to you know, kind of shift this conversation a little bit but, you know as an open source Community director for ortilius. we don't get a lot of testers contributing which is I think is really odd because What in an opens and there's a lot of developers who learn coding in the open source world, but they don't have a we don't have testing mentors to start teaching about what they're experiences with the software and creating the feedback loop.

So if you're a tester out there. Consider becoming part of an open source community and elevate your craft. It is an important important skill if we just talk about testing software which we have and I keep pushing us away.

But on this topic, we don't have a lot of people who sign up for open source communities who are experienced testers who understand how to build testing into the pipeline who understand how to test applications and can teach in Mentor. So, I think that the testing Community has some work to be to do to really get themselves elevated in the pipeline. What's your URL for your open?

Source? Project Tracy? io o r t e you have to spell that I owe their put it in chat.

I did. but there's so I mean, I know that Jenkins has some core contributors who are testers and they've done a really amazing job and some of them have those are the adopters oftentimes and they have they have been elevated up the kind of the the leadership chain within Jenkins, but I don't see it in in many of the open source communities that I work and I just don't see a lot of testers contributing and I would hope that they would start doing so Yep, we have a participant raising their hand. Well, he gave us a thumbs up.

I'm not sure what raising there. Maybe he was just testing out the chat window. But you know, one of the other participants said something and it triggered in my mind and not we, you know, we you know, we're coming up near well we suck some minutes left.

How do we Define what is quality? Is it something I know it when I see it. Right, they want to know, you know, they said user experience as part of quality.

Yeah, I don't know disagreement here. But how do you know you've delivered quality? I mean, where is it?

Is it you know, I hate subjective. I need a job objective. How do I know is it and really I think to me that's part of the role of QA, right?

That's why we are sure Quality quality assurance, but how do we know? Its quality in what way makes it quality is it I mean the ultimate user giving it the thumbs up or is it? You know what?

How do we know what quality is? It's the user getting the value out of it that you intended it to to have. Fair Fair I'll go I'll say okay Lee.

What do you think the musical court that went with that answer that? Yeah. That was the angels singing, you know, yeah.

Yeah, I was gonna say that I'm proving the opposite of right you can't unprove something, you know, it's you know that does this exist or not exist and it's it's proving a negative you you can't prove it, but you can look at data you can see the results whether that's users getting the value they expect whether they're giving your company money continuously and they keep doing that and your user base keeps growing or you know, whether they complain whether they they ask for features. You don't have whatever it is is data you can get that is indirectly related to whether you deliver quality, but I don't know if you can ever prove quality just like you can't prove security prove my software and secure. Well, you can prove it is if it doesn't people don't break into it.

Well the moment someone breaks into it. It's not secure anymore. Right?

But you know, it's it's you can't prove the negative, but you can use data. various levels to give you a qualitative analysis of whether you're meeting your quality objectives. In every step in the in the pipeline should have that data.

There should be you should be collecting data at every single every single stopgap. Now, we push we we deliver software all day long and a microservices environment that's even going to be faster. So we're gonna have to get better at doing this but Gathering the data and being able to react to the data where you can see where you have you're having these anomalies whether it be in the code whether it be in the build whether it be an the deployment of once you start Gathering that data, I think you can start understanding if you're improving the process but I think that proving quality it can be really challenging as Lee said it's hard to prove security because somebody's gonna figure out a way to break into it.

A great agreed. Um, if it is, you know qualities in the eye of the user right? It's a perception by the person who's getting the value or trying to get their expected value.

Whatever may be may not be what you built it for. But they're what they think the values that they're going to get out of the software and they'll either use it by it move on to the next thing or continue. You know, it's that's ultimately what matters.

Here I like what participants 660 right here quality is the extent to which it meets relevant requirements consistent with standards. Standards how much and how well it's been it's a good one too. Another one quality means object is perfect as per client requirements and their expectations.

So it's oh, you know, it's only as long as it meets what the client wants, it's good, right? I don't have to be the fastest runner here. I just got to be faster than you instruction and perform and always said good enough for who it's for me.

Yeah, that's what they expect. Yeah, and it could have been great quality to be way too late now. I need to change.

That that's well. That is a fourth dimension right now this time. Into the Twilight Zone.

Hey, you know what? It's about that time of the show that where I got to announce our for Amazon code Amazon gift card winners Cody if you've got them handy and you want to pop them out. But doesn't check.

If we could put him yes. They are in China. Yes, Alan.

Our four winners are Matia C Lynn M. Laura J and John B. There they are.

Okay. Congratulations to the four of you Cody will be reaching out with your gift cards. We thank you for participating.

So right back. I want to thank all of our this was a great audience today wasn't a panel. We got some great feedback some great comments questions thoughts.

They sounded really clued into this topic. So Thank you all for participating in that I'm not ready to wrap up just yet though. We still have a couple minutes here another good one here from participant.

I don't know three one three good enough is also called minimum viable product these days, right? And that so that I talked about MVPs, but yeah, no, we haven't and I don't know if that's exactly synonymous with quality either. You know, I still remember arguably it should be right.

Should it an MVP? Yeah. Well, if you're if your goal is to build the minimum thing that you need in order to meet the customer requirements.

That's quality, right? Everything else is waste. Yeah.

Yeah, I would argue in MVP has to be quality. We whatever quality is expected. which may not be perfect quality but Yeah, I I think it's an industry.

We've been bastardized that that mvp. I agree. We we do take a look at it and say oh that's a low quality version of what we ultimately have when the original idea was it is.

It's functionality and it's the it's the starting point by which we're going to start launching experience and experiments and start learning and and start building from there. But they're they're not low quality versions of those the highest quality that can exist on each of those things that you shift. Not a it the MVP of a car is not a low quality car.

It's a bicycle or a skateboard or yeah. Exactly. Absolutely.

Yes. And you guys remember rad? And development.

And they'll get the most get the lowest level product out the door and let the end user start experiencing it and pushing back feedback into what they want to see. So that's called the public beta now. So we didn't do public bandage.

But you know again the audience we have a killer audience today MVP is not necessarily quality. It's simply means what was possible to deliver. I I I'll take an amen to that right within a given time.

I think quality maybe is in the eye of the beholder. Another participant says what if the customer Demands a level of quality that is objectively garbage. Well, if you're definition of quality is it makes the hot customer happy.

I guess that's quality too. Is there an objective quality over and above what the customer wants? We entered to a higher authority as some gosh a hot dog commercial use today.

I don't disagree with that either by the way, right it one man's quality is another man's garbage. I don't know. Yeah, I mean it's true, you know.

If we think about some of the bigger tools that we're using right now, you know, when kubernetes first hit the the street and people started playing on it. It was far from perfect far from perfect and we kept as a community kept pushing back and they open source developers kept adding features locking it down securing it. So it you know, I guess you'd call it a fail fast method So getting that mvp out and getting new features added to it cleaning up problems that it's having.

May not be considered bugs. It may be considered a tool evolving. You'll never know if you have useful software if somebody doesn't use it.

No, that's the tree the first hey, we've got another QA here from participant for 40. And again, this is a common refrain I hear from developers and folks a lot. How do we push back on the assumption that the customers always right in my experience the customers wrong the vast majority of time, but maybe that's because they have a strong tendency to stay out of the requirements space and into implementation.

Well, I can tell you when the fat man and the boardroom sat on my keyboard. I went back and create a trap for every single key here. Because he was right and I had to he was he that was the customer and that's what he wanted.

So maybe so maybe that is the lesson right? How do you push back on the customer? You don't I mean I I'm not I'm you know.

Everyone is there Jerry Maguire? You know that. There's also customers in this customers, right?

There's customers that are more likely to be called clients were what you're doing is geared specifically to what they're asking for and there's customers like well, I'm building this new product I get this idea and I'm just no people will want it and you trying to build something that will draw people in and and it's it's those are different environments with different sets of expectations and different customer. Expectations and different custom different knowledge about what those customers want and one, you know exactly what the customer wants. They're telling you and they're paying you for and the other one you're trying to guess and hope that you guess right to bring that they're gonna buy your product.

And so there's nothing it's different and those environments as well it definitely and the trick there is to listen to what the customer is is asking for not listening to what they're saying, you know, if you if you have a seafood restaurant in in everybody that asks for a non Seafood dish you make exactly that dish over and over again. Suddenly you have you don't have a seafood restaurant anymore. You have a hodgepodge of a restaurant.

But if you hear that lots of people are asking for vegetarian dishes for some reason they they're coming to a seafood restaurant, but you're getting lots of requests for variations of vegetarian dishes. Hey, maybe put a veritarian dish on the menu and that's listening to your customer and making them happy. Absolutely.

Hey guys, we're right here at the top of the hour. We're actually probably over or which coming up on top of the hour. Tracy Adam Lee thank you.

Thank you so much for what it is a group has been a great panel. I want to really again. Guys, I I mean this sincerely it what a great audience what great comments and thoughts.

I learned a lot listening to see and seeing what you guys typed up Mitchell. I'm sorry, I forgot to introduce you see in the beginning again, but I'm gonna give you the last word take it. I think that's that's the anti-pattern.

Wearing that's okay. It's all right. Okay, you know I I the value out of this for me is the the different perspectives.

We all have it isn't one answer to one thing and it's most important that we learn from each other and learn from our experiences. So and I think that's what devops and bound does not plug our own thing here, but that's the Fantastic thing about bringing these experts together. So what a privilege to it's an interview part of it.

And I just want to mention one thing next Tuesday, if you're interested in this topic and you want to continue with it next Tuesday. The topic is test less quality over quantity. So some of you over there that have been saying testing testing testing slows me down.

Maybe that's that's a show for you to continue the conversation. And and by the way, that'll be next Tuesday on Tech strong TV, which starts around 9:30 AM eastern time, but you could play it on demand anytime after that during that you can just clap right to it and it'll be available on texture on dot TV as well a standalone. So do check out that episode next Tuesday.

Guys, thank you all thank you all for attending many thanks to Trey sentence for sponsoring this fantastic series of instructions. Catch us next Tuesday catch us on our next round table in the new year. For those of you who we don't get a chance to talk to have a merry merry happy happy seasons greetings and happy healthy New Years and keep keep writing quality product.

It's important this Allen Schimmel and we're out.