Creating Positive Digital Experiences Through Testing – DevOps Unbound Roundtable

Hey everyone, this is Alan shiml. Welcome to a special edition of devops Unbound live Round Table. com Security Boulevard container journal, and I have the pleasure of Hosting devops Unbound our regular video.

Program which is sponsored by the good folks of tricentes I try sentence. And today's special edition is is truly that it is a special edition. We've got a great.

A great panel for you. I'm going to introduce them in just a moment. But let me jump into our topic for today.

The title of today's topic is creating positive digital experiences through testing and it really what I hope is that we're going to delve into Just how important. Testing is to the success of our digital Transformations because at the end of the day, the tests are kind of represents or he stands in or they stand in the shoes. Of the customer and the customer is still the most important person in the equation.

We're going to guide guide further into that. Let me introduce you to our panel though to start our dive first because he's the top of my screen here. I want to introduce you my friend Brian Dawson Brian.

Welcome back to devops Unbound. Hello. Thank you.

Brian if you don't mind, why don't you tell our audience a little bit about yourself? Um, yeah, so I've spent about 30 years as hard it is to say that number and software development and delivery. I'm excited about this conversation because it gives me more of an opportunity to tap into what I'm proud of and that's that in those 30 years.

I've had the privilege of serving as an individual contributor as well as a leader in practically every phrase and software development and delivery from development to marketing go to market strategy pre-sales post sales and of course quality assurance, and I hold a special place in my heart for the role that quality and testing place. So I'm excited to talk about Absolutely, and we're excited to have here next up. I want to introduce Adam.

And I don't have the last name in front of me. So I'd like you to you might as well do it before I mess it up. Oh sure.

Now is Adam Centerfield? It's nice to see everyone again. I know have a lot of friends across the tricentes where else here so I cut my teeth and testing thanks to the MS Y2K bug.

So yeah going ever since and really excited for this conversation. You know, I've worked with and currently worked with a lot of amazing testers to have do a lot of awesome work have a lot of great stories to be able to talk about that work. Y2K Buck, yeah that brings me back.

Last but not least. She's a newly installed fellow. For my VM.

So let's give her a round of applause for that. It's my good friend. She's an author.

She's a Mainframe expert security. My friend Rosalyn Radcliffe. Hey Rosalyn.

Well, I I didn't say everything though. You got to introduce yourself. Thank you, and I'm glad to be here again, and thank you.

very Happy been. appointed I've been followed in my career. I have done just about everything as well except direct sales and management.

That's one really nice thing about being an IBM and a technical career path. And so I've had that opportunity to do lots of things. You shouldn't mention Y2K though because I spent What I Call nine months of Y2K interesting state.

So there are lots of stories to tell and that Y2K testing days, but I've actually spent time early in my career in human factors in user-centered design and testing from a user perspective and I think in this devops transformation and all the clients, I work with testing is what we have to automate and have to have in order to move faster with high quality. Testing is absolutely key and a critical part of this and that's why I wrote Enterprise bug busting because testing is a key piece of this transformation. Absolutely.

Yes. Let me introduce you to the last. Person on our panel today certainly not the least.

He's my co-host on devops Unbound as well as a business partner and friend for 20 years Mitchell Ashley. Hey, Mitchell want to introduce yourself down could be here, of course coasting with you CTO of Textron group and also principal at Textron research our analyst firm. Some of my favorite people in the world on our panel today and one of the things I'm most excited about is I have a suspicion that Brian is going to talk about gaming so I'm hanging out.

I'm hanging for that. You got me on the edge of my chair Brian right nice, you know. Well, okay, we got it out there.

So. What so let's jump into this though, you know. Look, I one of the first lessons I learned about testing when I really started talking to real life testers.

Was that testers are not? Failed developers Junior developers testing is not a weigh station on the way to becoming a developer. It is for some right some people do go through that progression, but it's not you know testers are not second class citizens in many case.

in many cases the the products that we end up with the end. You know the end product in our applications and stuff is directly in the hands the quality the success of it is really as much in the hands of the testers as it is the Developers. Right and and it's high time testers get treated and recognized for that.

But Adam, I'm gonna if you don't mind I'm gonna ask you to kick off here about this and talk about you know. Why how is it that testers are really the closest thing to to customer experience that we have? Yeah, sure.

So not only are testers really that the kind of you know final line or last line, you know, and in that chain before the software gets over to the customer, you know testers are really your you know your experts on how the products works right there. You're you know last line risk analyst, you know, I always like to think of testers as your your problem your problem solvers you're you know, folks that love puzzles critical thinking so they're really in there trying to figure out the way things work and ultimately, you know while I know a lot of testers who enjoy finding bugs and breaking things, you know, ultimately they like having a really nice working product and they work really hard a lot of them work, you know late nights. They work weekends on releases, you know really driving to make sure that the customer is very happy and oftentimes, you know, they're finding things that aren't thought about, you know, during user experience or requirements and there are such a becoming that customer to say Hey, you know this works, but it doesn't work the way that maybe the customers would like for Work and they're able to provide that feedback and you know, they they really do drive to you know wanting to have the best product possible.

Absolutely, absolutely and panel is as always feel free to jump in if you have any thoughts one way or the other with what someone else says another big thing in my mind though is the breath and width of tests, you know, we use the word testers. I don't even know if that's a good word to use anymore. Right because this testing I mean there's a world a universe.

of different testing right think about What testing was let's say when Rosalind and Adam you guys are gonna Y2K stuff right? I don't want this thing was that. All of the the infinite variety of testing that we see today.

Right. We we have people who you know, I forget what it's called, but it's you know visual testing, right? So what we see there we have accessibility testing.

We have security testing. Obviously, we have load testing unit. Testing right there is is it really fair to just Lump these all together and say oh yeah, that's testers.

That's testing Rosalind. Let me know you yeah. No, so I make a statement and I got to be a little careful.

Everybody becomes a developer whatever software Engineers. We're all Engineers. There's not a class of engineer and a class of that.

We're all in software Engineers. We're all building at least in the software case. We're all building software.

We're all doing this and whether or not we're our primary focus area is user experience or our primary focus area is performance or it is some area. We're all engineers and it is important to recognize. We're all Engineers.

We're all part of the product development team. We have our own Specialties. I don't like the term full stack developer partially for the reason the reality that it doesn't existence that sorry we need full stack.

Teams, we need people who have these skills and when I was early in my career, and I didn't know if I said it but it's like 35 years for me. So a little while I did user experience testing, you know, we actually got real developers sitting behind. I mean real People not developers real people sitting behind a glass wall watching them try to use our product to get the kind of feedback to understand what we could do better.

We you know, we have people who do the role making sure but they're all all part of this process everybody's part of this process and we have to recognize that they're all critical pieces of this process. Now one of the reasons testers get a bad name is because sometimes the testers really aren't testers sometimes their business analysts that got thrown into a job doing manual testing and maybe they aren't engineers and maybe that's not what they really want to do and maybe that's not their life. But if you look at the rest of us, we need to focus on this idea that it's an engineering profession and it's really important and performance and scalability and user experience are very important skills that we need people to have to make sure we deliver appropriately.

Yeah, I agree. And I know Brian go ahead Adam after you things. Yeah, so I really agree with that and I think you know Alan to you know, one of the things you mentioned is the the width of different types of testing.

I think that you know, it creates a lot of excitement and in the industry because you know, you can kind of come in as you know, coming out of college or switching over careers, maybe from support and to testing and say Hey, I want to get into the testing field then you look and see I mean there's you know dozens of different, you know Avenues and things you can learn and so it kind of helps you to not get stagnant in your career where you can always look and say hey there's this new awesome thing that I can learn and it does it creates a lot of excitement for folks and I really think that with helps with a lot of growth and the industry as well. Yeah, okay. You know and I I, you know, absolutely like to underscore first Rosalind last week.

We had a panel where I I declared I hate the term full stack developer. So that's why you heard me respond and I think in talking about that you land on um, an important concept that we need to do it better at addressing in the industry. And that's that we need full specs stack teams.

We have to respect domain expertise. Um, and we have to respect that everybody on the team has a role to play. However in speaking I think about it.

I met a number of people here actually doing a keynote at a QA Symphony quality Jam years ago and a number of discussions that came about me talking about devops in that setting was from QA leads that said, yeah, I'm really behind devops, but you know what my organization started to transformation and they didn't even bother to involve. And now we're working to deliver on Sprints, but nobody's accounting for qa's time. And that starts to lead into one of the the things I've seen sort of plagued the industry in the relationship between development and QA for a long time and it at times it's exasperated in this pursuit of speed and modern software development and delivery and that's to your point.

Allen QA is often treated as a second-class citizen. Um and annoyance and unnecessary requirement not as well, you know informed as us developers. My brother is is a QA leader at GoPro and one of the things he's talked about as well as other QA leaders I've talked to is it one of their primary missions is to ensure that their team gets a voice and they're comfortable and understand that they are as valuable as any deeply experienced developer and it's actually their duty to speak up as the domain expert when it comes to customer experience and quality.

Um, so I love this discussion Roswell and Adam. I love the points that you that y'all are bringing to the table. Integers important part of the voice to have and that's the voice of the customer right?

It's yes customer can be employed partner external customer paying customer Etc. And I think that's one of the it's easy to get into the middle of the software creation process and have the intent of what we want to have happen and there's always those. Well, we never expected anybody to do that with our software, right?

Of course because you're the person in credit you don't always think of those things. So it's interesting that the QA that the testing role as now all being seen as as a place where customer experience is brought back in or maybe it starts in from the beginning by the way, the only industry that agrees what full stack is is the restaurant industry at IHOP five so we can give the term back to them, right? Yeah.

I think it's really important that I The problems I think QA has I'm gonna cousin maybe a discussion argument in this one. Is there separate and they really shouldn't be separate. I I good I'm not gonna cause an argument having them as that QA team over there is exactly the wrong thing to be doing.

They need to be part of this product team from the very beginning because I need to be writing my tests my test automation while I'm writing my development so that they are together and one of the I'm gonna say one of the biggest problems with many companies agile Transformations is they throughout everything but the development yes and and they forgot everything else and so you ended up with scrumfall as I call it you had the development teams doing scrum and you fall into QA it doesn't work. We got to bring everybody together and recognize that they're part of the very beginning to make sure that you've got The right experience that you can automate what you're doing. I remember in one of our experiences having test and I got to be careful because their Engineers they're just Engineers, but they were responsible for making sure it worked those Engineers made a bunch of comments in the early development phases to make it easier to test because if we had better apis it's easier to test than it is to test the user experience.

And the other thing I've always said is I like I love automated testing automated testing. Everything should be automated except the user experience and there can we please have a human who actually might know what they're doing tested not a not a developer in this case. Yeah.

Well no developers a humans too. Okay, you may get full started I have with those developers. They still got heartbeats and blood pressure.

That stuff but yeah, but they may not be doing the end experience. They don't right. Well, they sometimes get lost in the trees for the forest.

But but here's an interesting thing right? com Seriously, no kidding around there was a a point of view that said devops signaled the end of the QA role. We were going to get rid of all the testers.

They were going to retire from being obsoleted 35, right and you better find something else to do. Puppy cock right? Obviously, we sit here now and laugh about it as much as we do about the Y2K bugs, but nevertheless it was a prevalent.

thought back then and The way I see it is this. When it comes to testing. Anyone who tells you we can automate all testing is full of beans in my mind.

Right, and I think another lesson we've learned with testing. is We we automate. What should be automated but just because we can automate it doesn't mean we should order made it.

Right. They were just some things Roslyn as you said customer experience. You can order me that and have AIML ET whoever you want doing it, but they really don't take the place of a person of a real live.

Person and right so there's so there's one school of thought that says hey testers in this new devops digital transformation World. They should just be spending all their time automating as much as they can. I'm not.

You know, okay. There's another school of thought that says. Look automate as much as you can but our best testers should them be doing higher value things that can't be automated.

Right and so spend their time on that and I I propose a third level over that that our best testers should be taking the results of automated testing as well as you want to call it manual testing. Instead of just reporting test results. Doing some analysis on those test results and making suggestions for how do we improve the software?

So it's not enough to say it failed this. But if we did, you know, if we did that it would be so much better usable right? And if you're gonna be a professional tester to meet like like Roswell and you're a fellow at IBM if we're going to have that highest level fellow tester.

Tester fellow, right? They they've got to provide that's their Beyond just reporting test results. It's that analysis and recommendation that makes them worth their weight in Platinum or whatever is expensive these days gasoline.

I don't know but you know, that's that to me is is what it's about. Yeah now short time yet. I I think I would add to that that I've I believe the automation enables creativity.

Right. So when you're able to automate the things that are monotonous or the things that need to be run every single time, it frees up your testers and your devops engineers and your developers to focus on you know, Innovation right to focus on potentially areas of the application that are very difficult to automate, you know, because I hear this right now with AI has been replaced testers. Hear that all the time.

It's like well, it should be enhancing testers. Right? It gives them another tool in their toolbox.

But you know, that's one of the things that I tell my teams is that you know, yeah, let's focus on automation. So you're free to you know, Focus your, you know attention elsewhere, right so you can really grow and one of the things that my team has done right now that I'm very proud of is they've built in accessibility to the automation testing framework because they had automated some things and they're saying, okay. Well what's next?

Well, we have this whole sector of users that really could, you know benefit from better accessibility on the website. So they built that into the framework. So that was Innovation.

It was born out of Automation and I think that's truly where it's at. Yeah, Alan. I I have to deliver on Mitch's prediction and I was going to talk about games and so you gave me the perfect, you know segue.

Um, so I I actually my you know, started my career and in, you know prior to programming and quality assurance and I spent a number of years. I'm doing quality assurance for games. And by the way, we use the term tester there, but I actually love roses Roslyn's case for using the term engineer a smart guy.

No recommended that we call them digital experience Engineers to talk about really kind of the spread and and full stack there. I agree but I heart back to games and there's you know, there were things that we need to fix that as I moved into the tools and Technology space. I I focused on fixing there were some things that I really appreciated about the relationship between quality assurance engineers and gaming and the devs and that's that that they were considered gaming experts and user experience experts.

So in addition to doing regression testing right going through test gifts to find bugs, they did two other really important things just kind of free roaming exploratory testing push the edges of the software try to use it in ways that a developer would never anticipate insurance was stable, but then also It would actually give feedback on the user experience. It's too hard to you know, to to beat this level. This is gonna discourage users and they'll fail to play the game and they would regularly report that feedback to the development team and they became the developers Partners.

They became the the the producers partners and as Mitch raised in engineering the digital experience for the game players, and I would like to see that better codified in our in our testing structures in relationships today. free s whatever the test Really? I mean, I remember when I was part of a team I had a bad habit.

Of having a system that I you know, I I snatched it so I could keep doing it but I grew it over the years. So I evolved it. I upgraded it.

I whatever I would find problems in that system. No one else would find because automated testing. I install new I install fresh.

I had a thing that had been upgraded through 17 versions or whatever. It was by that point in time. Well, mine was customer like and that was the point.

It was customer like testing and I would freeform do things and break things and now it's in the environment now you have to fix it, but you have to fix it so that you can fix it in the customer environment. Is that kind of thing that we need manual testing all It's not the I can test the API. I think a hundred percent API testing.

I should be able to get all the negative conditions the boundary conditions all that early kind of low level stuff automated and free up the time for this complex. scenarios that are just a lot harder to deal with and a lot take a lot more time take a lot more effort and you're never going to manage to automate that anyway, so let's let's allow that freeform that malicious that trying to figure out those security problems making sure we're doing security testing appropriately trying to break the system all of those things that we really need people to use their brain power to do just assure to accent that point Rosalind the back when Alan and I co-founded is still secure, you know, we spend a lot of time in front of customers and whenever someone would report like some weird edge condition products doing something strange. Yeah rarely happened, of course Alan but only a Mitchell's engineering team, but I don't know there's a whole another problem.

The first person I would always go to is hang and hang with the person who is the lead QA percentage you ever seen this. Have you ever tried this have we traveled is that something we were aware of because I go to the I'd have to go to Six developers to ask that question to get an answer. I could go to person who's tried it or not.

And there's so much creativity in the role of testing. I know we we tend to think of it as like a gallon said it's the people who can develop test or should try a lot of Code, by the way. Of forget that there's so much creativity in what I love is this conversation about it is about the customer experience at the end of the day.

When should the product work like this were like that? Yeah, the product person has their view of it. What's what makes sense from the customer what this makes sense from the person who's got to use this product and wants to achieve something with it.

And that's that was hanging. That was what he did on our team. It's a good Manning.

So let me let me go a step further here with that. Right and I'll hark back to our social security days with Mitchell. I think when we look at the role of the developer, the developers are given a requirement stock, right?

These are the features we want and go go write code for that to make those features alive, you know, and the thing. I do think Brian you're you know, it's not just games. It's it's every app that comes out.

It's that it's that tester who has to think about has to think like a user I don't think developers think like users. Their people they're humans, but they don't think like users. They think like look these are my requirements and that's what I got to develop they don't think about how is someone really going to use this well, and if I may just to interruptive defend developers, oh, they are humans and many of them are very good at thinking like users.

But when we talk about where full stack breaks, sometimes you need people wholly focused on a given domain. So it just happens to be the perspective. They focus on date today doesn't necessarily allow them to spend as much time focusing on the user perspective, right?

So so income people that own that experience, right or at least our similar to a security expert are ensuring that the whole team is thinking about that including developers. Sorry to interrupt balance one. I don't know right?

I think you did on Twitter because I'm gonna tell you something else I've learned. I mean look try scientists we work. Them a lot here on devops Unbound and other places we work with other testing companies in the space an interesting thing I've learned.

Do you know how many users of testing services testing? Providers like I try sentence wind up their customers and then they wind up going to work. For the testing company Adam.

I don't know if that's in your case, but the result and that happens an awful lot. You'll talk to people who said I was a customer of this company and I loved it and I I went to work there. right because at the end of the day Look, it's something I do here at Tech strung.

I try to tell my people if you don't have domain expertise around devops or cybersecurity or Cloud native, I don't care how well you know Twitter. I don't care. What kind of great writer you are.

Right, you're gonna have a really hard time keeping up with what we're doing right and getting traction. Because she got to have that domain expertise. So I find that in in testing more than other disciplines within this full stock genuine.

We live in right and we all get assimilated to testers. If you have people who've been there done that often Talk That Walk. And now come in as testers.

They bring an awful lot of of expertise that makes them better testers and I wonder how many people out here today watching this have followed that path to testing. right Whether it's in games, or it's in Securities and other perfect example, you can have the greatest developers in the world. You're gonna sit here and tell me how I should filter out vulnerabilities or what's a real alert versus or not.

You know, what is my daily kind of sock kind of you know, protocols and stuff if you haven't done that stuff forget about it. You could be the greatest developer in the world, but you don't know diddly about security. and it's Go Rose like that that is actually the real critical point of this.

We're telling developers that they have to be experts in everything and they can't be so in many in one my last role. I help teams build developer tools. So they're developers.

They know what the user experience should be their developers. Yeah, but they're not developers of COBOL applications, they're developers of development tooling so I still need somebody to get that other perspective security is every developers responsibility. But but that doesn't mean they can do it all so we we need to have those Specialists those experts who are gonna have that view into that world because no one person can can know it all it just we can't we can't do that.

We focus in our area if my job is to write that function. I'm going to write that function the best I can but I need that those other people around me to make sure that it's secure and it meets the user's goals and all of the other criteria. And I think in a lot of organ in a lot of situations and organizations that falls in the testers range.

Right in the testers kind of lap. They've got to be able they've got the testing the testing the automated testing regimen the manual whatever, you know, all of it has to have that goal in mind. And I you know, I wonder how many or testing.

Teams are really looking at it with that is their goal? Right versus finishing off a checklist of tests that need to be done versus the bigger goal of is this product rocking it is this product doing like the customer wanted to do and I think that's got to be the goal. And in mind, right?

Yeah, I think you have to incentivize right one of the things we can do is start to land on kpis metrics and measurements that kind of capture that so we incentivize QA with more than number of automated scripts written number of defects found and you know, what is what is the sticky time? We have a web application. What's the dwell time all web page?

What's the distribution of clickers? What's our bounce rate? Right?

Let's let's start to capture those metrics that give an indication of positive user experience and incentivize our teams with those. Yeah Brian if I can add to that as well. I think you know as Leaders, you know, really need to focus on metrics that don't antagonize the individual teams against one another right?

So you shouldn't be measuring, you know defects you against, you know, individual, you know lines of code or areas where we need to focus on team based metrics, right? How's our delivery, right? How are we you know, are we?

Do the customer are we making a net positive or net negative change right in our customer experience? And that's that. I think it would drive a lot of that.

You know tests are not as a second class, you know citizen but tester is a true core member of the team and I think that a lot of that falls on the leadership shoulders, you know, not only from you know, that metric standpoint but also, you know fostering that, you know deep knowledge and giving testers the time to you know, innovate and to really kind of drive and you know, look at new automation tools and things so I think there's a big component right for leaders really, you know stepping up and helping to make sure that you know, the testers can you know be the best that they can be If there's a case for extra changing the title to digital experience engineer, but you know, if you if you overlay agile and devops into our process that we do now if it's a crime if we're doing testing at the end, you know, not all the way through the process but that could be part of that experience that we are validating because it's it's when we've already got software built got it in different environments that we're doing QA or performing those functions on it. That's what comes out of it is just bugs but also a recreating the right user experience digital experience or multiple of them depending on the product. So when we get to the end, we've got the best.

It doesn't become the best at the end when we got all the bugs fixed if it's designed in front just like we designed in security up front just like we want to design quality. We design user experience do a great deal. Yeah that I just real quickly right that that I love what you capture digital experience is end to end not a point in time.

But what are our users experience from the first time they hear about our software to the first time they engage it to the last time that they engage it. I I love Dees digital experience Engineers great idea. I love d d e is alright that sounds like a button.

We should be giving our sticker and our net when we're not virtual. Well, we could do them in Virtual as well guys speaking of virtual. We're coming up on on our end game here, you know end time.

Well, that sounds bad. It's not end times. We're coming up on our time limit for today's panel.

But you know, first of all, I want to really thanks to everyone who's watching this on the tricentes virtual Summit, right? And and you know, but this one I think for Emir and one for North America and or Western East I think between them it's yeah thousands thousands of folks watching this and they're primarily an overwhelmingly interested in testing. First of all many thanks for tricentes for giving us.

It's opportunity to address the audience, but I'd like to ask each of you now. To talk directly to our audience and these are the testers. These are people who are involved in testing.

And all the in so many of the different facets of it. What advice can you give them right to to raise if we're gonna be digital experience engineers? What's your best advice to them to get you know to the to the promised land here Rosalind?

I'm gonna ask you to kick off then we'll do Brian then Adam and then Mitchell and we'll wrap up. So take your seat at the table. It's not devops.

It's does that QA inch Ops or whatever set of letters. I left out. You're part of the team you're there from the beginning and make sure you're really participating and that you take the role as making sure you're automating those things that should be automating and driving that digital experience.

So the user experience is what it should be. I love it. Yeah, I guess me next.

I actually have agreed with a hundred percent of everything Roslyn said so I'll just add a little bit look take your seat at the table and you know, we could be djws digital Justice Warriors, right? It's important that you're there to kind of defend the right and experience of users. Um, but I would also say as we all should should as team members collaborating take a more empathetic approach.

You don't have to be a developer, but do meet the developer halfway understand what they're experience is engages a team member and again, whether it's Dees or digital Justice Warriors, and we should take some role of that on we need you to have a seat at the table. Love it can't go wrong agreeing with Roselle. And she's a fellow.

I know it's easy. Guess what? She said?

Yeah. Yeah. Say always keep learning, you know, it's a big world out there and the realm of technology and I'm a big believer that you know, if you if you you need passion in order to truly, you know succeed at your work and to succeed in Innovation.

So, you know, don't give up. There's so many things so many awesome things to learn. So just just keep going forward.

Absolutely. I love it Mitch. I'm going to give you the last word.

Oh and I had believe it or not. It's not just about quality. You can have the highest quality product that it's still a bad product that isn't successful.

But if you shoot high for the digital experience for the customer experience and you can and shoot High whether it's think of the analog of that waiter who always remembers that you like sweet tea or when you go to the shop, they know you buy MAC software not Windows software by whatever version whatever it is, whatever the experience is that you go that is amazing. I wish that was everywhere. That's what you want people to think for your product for your software.

Shoot that if you go shoot high am I absolutely hey, we're gonna wrap it right here. I want to thank I sent this I want to thank everyone who's attending that I sent this virtual Summit for joining us today on this special edition of devops Unbound. There's a full.

Agenda of great sessions on here. So don't make this the only session you watch check them all out. Do watch us on devops Unbound every other week on Tech strung TV once a month live round tables.

Roseland Brian Adam Mitchell, thanks for joining us. Enjoy the rest of your summit everyone. Have a great day.