Techstrong TV – February 6, 2025

Hey, everyone, you know at the Super Bowl coming this weekend. Is it time to kill all the referees you're watching? Textron Gang.

Hey everyone, it's Alan Shimmel for Techstrong Text, strong gang. Happy Thursday. We've got a action packed show for you with some great people to discuss it.

Let me introduce you to our gang members for today. First of all, well, she's not at home in Colorado. I think she's out in California, but she's still joining us on the road.

It's good to see her. Kimberly Bates. Kimberly, how are you?

I'm doing great, and I think this first topic is gonna be a lot of Fun. Yes, it is. First, you know, there's been a lot of people waiting a long time for this, but, um, anyway, Kimberly, it's great to see you joining us from Las Vegas.

It's the Dynatrace twins, they're out there for the Dynatrace event. First, let me introduce, uh, Mitch Ashley, future chairman, vp, uh, for DevOps. Mitch, you're out in your hotel room.

I am work working on my tan, as you can see. Um, a little red from this of the morning sun here, but I don't know what our first topic is, but I'm sure AI can solve it. That's a pretty stock answer these days.

Also, joining us from Las Vegas, not quite as red and tan. I, I suspect he hasn't been out in the sun as much, but he's our chief content officer, Mike Ard. Hey, Mike, how are you?

So, I'm well, but I, so I'm assuming Mitch is my fraternal twin. 'cause you know, we, the identical thing's not working. No, well, twin sons of different mothers perhaps, but we'll go with it.

Joining us from Texas, is that, is that the, the Valentine's Day stuff in the back there, Ann? Absolutely. She's already, well, it's not, it won't be long now.

Uh, and a whole award. I think I finally gotten your name there. Nailed it.

It only took me three months. That's Funny. But I got there.

Thanks for, I appreciate the effort. Absolutely. Thanks for joining us.

And then, you know, saving the best for last as he sits on the throne in Silicon Valley, our one and only John Editor John Schwartz. Hey, John, how are you? I'm good.

It's happy to be here. Uh, I love the first topic, great timing. Absolutely.

With the big game this weekend. I hope, you know, we, uh, all have your Super Bowl plan set up. It should be a decent game.

I'm not gonna lie. I think I'd like to see Philadelphia win, but let, we'll talk about it later. Mike, what is this first Topic we're all excited about?

So, this issue is not just in the NFL, but it's coming to a head because of some, uh, calls made in the playoff games that people are, uh, upset about and continue to be upset about. In fact, I'll tell you how upset folks are friends of mine. I've known 'em all my life.

Our football fans have watched football forever, and they're not especially Bills fans or chiefs fans and, but they are hardcore fans with sports. They are not gonna watch the Super Bowl on Sunday. They are going to go to a Chinese restaurant and boycott this game.

These are people who've been watching football their entire lives, and they're basically fed up and they think that, you know, the referees and the umpires, including one in the major league baseball, fired one in the umpire just recently because of, uh, sharing a quote unquote, a account where a known gambler who, why would you need to do that? I have no idea. But John, I don't know what you're hearing, but people seem to actually be crying out for some technology help here.

Yes. So we have this fascinating, and I think extremely timely story on digital CXO on the idea of replacing or augmenting referees in, in the NFL and empires and MLB. So, as Mike and Alan, and I think we all know this major theme or conspiracy theory this season has been that the Kansas City Chiefs are benefiting from calls, especially calls that can't be reviewed like passenger for its calls.

In fact, yesterday, I believe this week during a media session, pat Mahomes was jokingly asked who his favorite referee was. He, he kind of laughed it off. But the point was made, I think what we have here, and, and again, it's a big backlash, um, this week, you haven't seen a lot of Super Bowl coverage.

You see mortal about the NBA trade deadline. And I think it's a fru utter frustration that's lingering over the, not only the success of the Chiefs, which is happens in the NFL. There's always a heavy or a dark or dark team that people root against that's dominant.

But the idea of whether the replay structure or even the officiating, even writ large, has a major problem. And I think that has been something that's been growing in, in terms of debate. For instance, if a referee misses a call, ostensibly it, the onus is on the teams to challenge the mistake.

And yet they're limited by what types of things they can challenge, which leads to these long delays and utter frustration. And I think people are looking at the idea of using ai. Same thing in baseball with the strike zone.

We have so many umpires with so many different variations of the strike zone. So we are seeing robots in the minor leagues starting to call balls and strikes. And I think there will be a push, I don't know if the NFL will ever accept it.

Perhaps they will eventually through their partners like AWS right? Um, this idea that you, you have a AI assistant of some sort to augment or maybe eventually replace the football officials. And again, in this story, which is interesting, this has been done in other sports and it's been done successfully.

You have, uh, the minor league games with, with robots. Tennis uses the Hawkeye technology to call faults and Lang calls. Soccer has something called var, which has been used in the World Cup.

And I think it's really interesting, again, as Mike mentioned, this element of the umpire being fired in baseball. You know, all of these leagues are so dependent and in fact are embracing gambling to the point where we start worrying about the i integrities of the game. And, you know, it's happened in the NBA and in the NFL.

There've been rumors for years dating back to the seventies about umpire, about referees, making dubious calls. So I think it's fascinating and I think it's something that we'll gain traction. But in the NFL, it's sometimes harder to gain traction on things like this.

But we'll see. You know, I I have a, I have a a lot of thoughts on this one. And, and I, let me confess, my name is Alan, and I'm a diehard sports fan.

I, I live for my sports. I I watch it, my kids, and it's a big part of my life. Let me first say that I think ref's, umpire's, officials blowing calls are part of the human factor of why we let the horses run.

As they say, anything could happen. You can get a call, a bad call, a good call, a not so great call, whatever. And that's part of the game.

It used to be very much, it's part of the game. Legalized gambling on sports, as widespread as it is now, has fundamentally changed my love affair with sports. I don't gamble.

Let me, let me say this. I, I'll buy some boxes for the Super Bowl, you know, and that's, that's like playing bingo. Um, I don't gamble.

However, the gambling and the amount of money that's at stake here is not just from my friend Tony's Uncle Vito, who used to hand out the little sheets to us. Mike, you remember this in New York, right? Mm-hmm.

You used to get a little sheet every day, every week, and you would pick your four winners. And if you, you bet a dollar you'd win 10 or something. You know, it's not like that anymore.

They've ruined sports with legalized gambling. Where now, right away the referees blew a call. Well, of course they, they, they got money on the game.

They, you know, they, they're sharing an account with a known gambler, right? This reminds me of Joe Namath getting banned from football almost. 'cause he had an interest in bachelor's three, right?

Where known gamblers hung out, uh, way before a lot of your time here on, on the, on, on the gang today. But so gambling has put such a magnifying glass on these things being officiated to the umpteenth degree, imperfect that it, you, you've taken the human out of it. You have to take the hu And if you take the human out of it, we all lost something.

Now, granted, I love the Hawkeye system in tennis. I go to the US Open every year. I love watching tennis like that.

And that system has taken it. I mean, I don't know, nasty Eli Netti would be upset 'cause he wouldn't be able to argue with the line judges anymore. Or the, or the umpire.

Remember it. Remember Eli or Jimmy Connors, remember? Hmm.

These guys, no. They put on a show when they argued. You can't argue anymore.

It, it's black and white. It shows you if it hit or it didn't. And that works great.

Balls and stripes maybe. I'm sorry, John. Yeah.

What were You gonna say? Oh, I was gonna say the one, one thing I, I, I should have mentioned too, and in, and in fairness to the referees in the NFL, the players are bigger. They're faster.

They make plays now that you, we couldn't even conceive of 10 years ago. So for the most part, they do a pretty good job. But it's always bothered me about the NFL officials is that many of them, for years, this was a part-time job.

It wasn't like MLB Where you No, that that was the knock that the NFL didn't pay their officials, like the professionals they need to be. And, and it's not like they don't print money over at the NFL and they can afford it, right. Compared to some of the other things they buy.

But that being said, I think there's a place for AI and automation and taking the official sort of out of it. But it, it varies by sport. And it vari, like for instance, the big thing in the Kansas City game was the fourth down spot.

It was the third down spot. It was a fourth down play, fourth down. When Josh Allen, to me clearly looked like he, he made the first down.

And they, those. But those are always judgment calls. Those are always 50 50th best.

That's what makes a football Well. And the question becomes, when you have the scramble that we have in football, whether or not AI can actually be accurate or not. I mean, it's, it's a very, very complex, you know, okay, so you've got this pile or what, you know, whatever.

I can imagine a bunch of different kind of scenarios where, okay, so yes, and I hear that if you put a chip in the, in the football, okay, see I got a chip in the football, but does that get the, the entire length of the football? No. You know, there are so many, you know, permutations on how the ball moves and how the players pile on top of each other, et cetera.

I think it would be a big challenge. Although, you know, there's amazing things that they do. So, you know, I'm all for video replay.

I think video replay has added a a lot of, you know, let's, let's get it right. I mean, in baseball it's fantastic, right? Because, you know, with the manager has within whatever it is, 30 seconds, 45 seconds to say, Hey, we wanna go to the replay.

They get so many replays a game and they do it. 9% of the time that replay is definitive, right? He's out, say for whatever.

So you, I'm not, you told me there's gambling going on in sports and cheating. Well, from, from says the man from Vegas there. Yes.

I hear that happens here, Ivan. I've not seen any of that. Uh, no, I I think I'm more in your camp, Alan, of, of, there are certain things that, that definitely could be augmented with AI or, or electronic measurement.

But even to your point ly about, you know, putting sensors in the ball, there's the ball going over the line. There's also the judgment, a call of were they down right? When they're piled up on 25, you know, 12 other people trying to push over that line.

Um, did their knee touch the ground or, you know, did their elbow head versus the hand? There's so many judgment calls in it. Um, and I, you know, is their favoritism?

Does Mahome get a, you know, get a, get a break? Well, probably, but so do a lot of people who, uh, do holding call, you know, are holding other players and they don't get called. 'cause it's fourth quarter and it's the last, you know, two minutes of the game or Yeah, that was a pass interference, but maybe not egregious enough.

So it it, there's too many judgment things in this to say, let's, let's have AI solve the problem for us. You know, you notice one thing, you notice the refereeing, I think in the playoffs is, is much better. They have all star teams of referees in these games.

And the flow of the game is much better. They don't call as many penalties, maybe because the teams that are playing are better and they don't commit as many mistakes. But there's a certain flow.

And my fear is that when you, uh, apply ai, how are you gonna apply it to a judgment call? And is that gonna mean that we're gonna have stoppages constantly? I think we have too many stoppages as it is between injuries.

And I, I think you will to turn down the, the ai, you know, uh, right. Enforcement. You could not watch an NBA game with AI because there would be a penalty called every play.

Well, There's every in football on every play, right? Same thing. Exactly.

And I don't, but but at the same time, I don't wanna impinge the integrity of the umpires and referees. But I will say this, it costs a lot of money to go to these games now. And if I'm gonna go spend that kind of money, bring my family, have the whole thing, and to have the game essentially ruined because somebody's having a bad day, I can't get behind that.

We need a different answer here because I need, you know, it's not enough to say, you know, oh, that's part of the game. That was part of the game when it cost me 20 bucks to go to the game. Now it's costing me $200.

I need a better answer. So You're telling me must Have been, you know, and I get back to where Alan was, what Alan was coming at is the joy of the game. You remember that?

What, what is that movie? Any given Sunday? Yeah.

Oh, that's a good movie. Anything happen? I mean, that's the beauty of football is that it's just so unpredictable.

I'm, I mean, yeah, I think the others are too. But they're just so many weird things that can happen on the field and everything else. And that's what makes it so exciting.

And I, Alan, it's kind of what you said. It's like you throw the money in there, everything changes because you've all of a sudden got folks doing whatever. And I was, I, you know, I didn't know we were gonna be doing this topic, but on the way to the airport yesterday, I was listening to, and they were talking in the news and they're talking about that the ticket sales, the resale ticket sales for the Super Bowl are down.

The prices are down 30%. Really? Yes.

Hmm. Now, and they said that ticket, the lowest cost ticket is still $4,000. Yeah.

But so I was thinking, okay, so is that because of the controversy? Is that because we're tired of seeing the Kansas City, you know, at, at the Super Bowl? Probably the latter, maybe, you know.

Um, but yeah, I'd read, I had read actually that it Was a b it's a much bigger stadium. I guess it has an additional three or 4,000 seats than Vegas did. Some it may have to do with, so that could be a factor in the game too.

You know, the president's going, other people are going, I don't wanna go. Yes. You know, the only, The only thing, the only thing that's gonna get the NF NFL's attention is if the ratings are down, then they'll address it.

I mean, they're just like bottom still, bottom line oriented. I, I'd rather Solve concussions Than, you know, we're worried about. Nevermind that.

Honestly, I think ratings are gonna be down based on just my little survey of my friends out there who are not having it. But Chinese restaurants sales are up. Is that what you tell?

What About Taylor Swift? Taylor Swift will be there. Could be.

So, um, Taylor, That's, you know, I'm reminded of, you know, being Jewish. Where do Jewish people go on Christmas? Chinese food, um, Chinese, these Restaurants.

Right. Chinese restaurant. So I, and, and I can't imagine, and That's why movies always get launched on Christmas Day, right?

Yeah. Yeah. I, um, look, it's not just to be fair though, let's not pin this all on the NFL and the NFL referees.

There are plenty of baseball guys who, who, you know, there's no joy in Mudville tonight. 'cause they called us Strike three. And you know, I, again, I think that's just, and Mike, I disagree with you.

I go to the game to see my team play. They lose because the ref made a bad call. You know, I scream, I'll kill the bugs or whatever, or, you know, but I, it's one thing to say, all right, he's human and he had a bad day, or he made a bad call.

It's another thing to say, oh, he's got money on the game. Yeah. And I don't care whether it's, whether you're Pete Rose as a manager.

Mm-hmm. Who did it, or there were a couple players last season who got suspended in baseball around betting. If you, if you are involved in pro sports with the betting that's going on there now, you've gotta be above reproach that they can never say one of the motives were you have money on the game.

That, that would p**s me off. It does. I don't know.

I'm going to, I'll disagree with you on this point though. Like, it used to happen when there was a bad call. What?

Once a month. Now it's every game. And that's what's different.

And that's what people are complaining about. It's every game now. It's not just a once in a while kind of thing.

It's a, every baseball, football, basketball. You Know what, you mentioned basketball. I watched game, I watched my Knicks pretty religiously.

I watched the New York Knicks got my NBA subscription. I think the NBA does a hell of a job with their replays and stuff. They, you know, I, I've been watching it a lot.

I'm not gonna say they make every call perfect, but they go to the replay a lot. And, and the replay usually is pretty definitive. I, I think replay here is more effective than AI Anyway, though, guys, we we're 20 minutes in on this and we're never going to come to an answer.

Stay tuned. Hey, if you are not watching the Super Bowl Sunday, instead of eating Chinese food, watch Old Textron Gang. We got a bunch of episodes.

Goes catch up on ai. Do something positive. Let's watch it during the Commercial gang binge.

We, we don't have a TechOne gang commercial Mitchell for the Super Bowl. Oh darn. Okay.

No, we don't got that kind of budget. Come on though. Maybe, maybe ads will go down next year.

We could do a TechOne gang or bought us. I thought you had your free cash. Well, I was hoping to get invited to perform at halftime, but they got someone else for that too.

Alright, we're gonna take a break here on Textron Gang. We're gonna be coming back. Let's talk about something else you're watching.

com is the leading resource for news analysis and education on challenges facing the cybersecurity industry. com covers all aspects of cybersecurity, including data security, DevSecOps, cloud security, application security, network security, security threats, and more. com has the largest selection of security content featuring breaking news, blog posts, podcasts, and more.

com to learn more. com. Home of Security Bloggers Network.

Hey guys, we're back in. ai that you should all check out, but it basically suggests that despite all the hypes surrounding ai, it's just too hard for us to implement. And it requires humans who have super skills and not everybody has those skills.

So we're not seeing AI become pervasive the way we might have initially thought. And it's just too damn hard. And you've been tracking the space for a while.

What's your sense of, you know, are we kind of like the techies are all over this AI thing is gonna automate the world. And I think the average person's kind of looking at it going, well, that's a lovely idea, but I can't do it. I don't think we help ourselves by changing jargon every 15 minutes.

So for example, AG agentic AI is the new term. Vanti about it sounds like a company name. It's intimidating.

People have to read up what it is. They may not understand when we really could just say agent. Um, I think it's just a type of AI that can make decisions and adapt and using agents, but yet we've coined a term.

So I think as technologists, we have to resist the temptation to brand and name everything because that intimidates people and users, right? But I think that, you know, the thrust of the article was really that democratization has stalled, uh, lack of democratization has stalled, uh, trust. And the complexity is too much.

So Kempel was basically arguing that everyday users have to have specialized skills to adopt ai. And you know, we need things like no code, uh, tools and accessible interfaces, open model. And I think something that he said that really resonated with me was that we need open source foundations and data transparency.

I think there's a lot of lack of trust. Look at deep seek, for example. Everybody was on that train and then they realized, oh wait, it's China.

Oh wait, where's my data going? And then it just sort of started a chain reaction of distrust along with it. But every platform faces it because of the lack of transparency.

I mean, need to make the same argument about centralized versus decentralized, right? Uh, are these platforms taking off that are decentralized? Not as much.

Uh, so we're sort of in a will they, won't they, you know, like a Ross and racial situation because I think a lot of organizations find it too complicated. And in some cases, I have to be honest, AI is really a solution looking for a problem, right? So we're sort of in the, in the in between middle earth phase where I think we're gonna get there.

But I think that he's right. Kembel was absolutely right. We need more user-friendly, accessible, and trustworthy AI so that people can understand, which is, and I'm, I know I say it a lot, but I love perplexity, um, because it's actually utilizing deep seek.

But it, it shows you when you use deep seek, whether you agree with the China part or not, it shows you it's chain of thought. It tells you why you get to your reasoning. And even just that subtle change has built trust with a lot of users despite who, who has released it.

So I think that that's sort of an indicating note that Kempel is right. People want more transparency when it comes to AI adoption. Yeah, it's a, it it's a really good point.

Um, I would say that the user experience with generative AI is broken. We're starting with, you know, everything is a chat for one. Not everything is, is well well suited to towards the chat.

The, the other is the fact that you have to select which model you're gonna use. 5 sonnet versus other alternatives. You know, there's a lot of belief that, um, LLMs are essentially gonna be commoditized 'cause there's so many of them.

And we'll, perfect how to train 'em and build them. And yes, they'll increment and get better. I kinda look at it as, I don't care what dictionary, you know, word uses to spell check or gram what Grammarly uses to, you know, grammar check my writing, just do a good job of it.

I really don't wanna spend any time checking it. And the fact that we require developers testing people everyday users of ai, you know, people just using their computer for work, whatever might be, you know, select your engine, your your provider for choice. Choice of choice.

I think, I think we've kind of got it backwards. Let's like, make the interface what the task is and not what the technology behind it of the 25 choices you've gotta choose from. Right?

I totally agree. And I think hugging faced does a great job. Um, they make it feel like an open marketplace.

Uh, it's approachable, but again, for developers, it Users complexity. Yeah. A user would about it, right?

Having the open marketplace is, is is a complexity. I I have a different take on this. I don't think it's the complexity per se.

I, I think first of all, you've gotta, not all AI users are the are alike. It's not a monolith, right? Tech folk who are used to, you know, dealing with user interfaces and, and maybe understand a little bit about what's going on with different models and stuff is one set.

I think when you look at the non-tech crowd, I think there's two main things that are holding back greater adoption. Number one, FUD for your uncertainty and doubt every day they're hearing how dangerous this could be, how he could take your job away. How the Chinese are spying on us by doing it.

How, you know, they're sucking all your information and making it available every day's. A new story here that I think, you know, non-tech folks, non-security folks are half scared to death to use the technology. And it's kinda like moths to a flame of those who do, right?

They use it basically for poly tricks and cute, nice to haves, but are not using it the way we would want them to use it. And the way this we're talking about here, the second piece of it is, most non-tech people don't understand what to use it for. They think it's a better Google go search using ai and you can search using ai.

Don't get me wrong, but it's not, the power here isn't searching, right? Uh, you know, how many, how many calls were, were disputed by NFL Refs or by NFL coaches in 2024? That's a great Google, right?

But it's not an A Right? But it's wrong. It's wrong a lot of the time, right?

That's, that's the Thing. And that's the other part. It's wrong, wrong, but you shouldn't be using it to search that.

Now, I do believe, and I've seen it now in our Google workspace, and I've seen it in my Microsoft office and I see it in my email program that I use and some of the others, you know, the co-pilot kind of thing where it, it is built into the app you are working on. It is a little easier than just using a standalone chat bot, right? Like I open chat GT or Claude or, or one of those on my desktop.

Um, but I, I think in, you know, I had this discussion with my youngest son last night. He's a TV broadcaster and we were talking and he said, dad, you know, local TV cable is, is getting killed. Everybody's cutting the cord.

I don't know if I want to go work for another local TV station or do I wanna be a digital storyteller and go work directly for a sports team or, you know, 'cause sports is where he is at. I said, well Brad, if you're gonna be a digital storyteller, you better figure out how to use AI to tell your story. 'cause you're 23 years old and you're not gonna have a long career as a digital storyteller, whatever that is.

Um, if you're not going to harness AI because you will be replaced if you don't leverage it. He said, nah, you'll always need a person. You'll always need humans.

There's only so much it could do. And I think that's the other thing is there's a big feeling out there that it does this much. It's never gonna really do this much.

And I think those people are wrong. And I, okay, so the premise of this is partial is how it's slowing adoption because it's so complex. And that's probably very true.

But guys, how far are we into this? I mean, we are talking about how big AI is and how it transforms everything we know about business, everything we know about customer service, everything you know about retail, all these kind of things is, is massively transformative. And what we're fine, what you're looking at is, okay, so we're bumping up against the wall on different things.

Oh, it can do this, but it can't do this. Okay, so then we get to do invention. Oh, it's, it's gonna eat up all of our energy.

We gotta have, build all these atomic power plants. Okay, well, maybe not, maybe, you know, some of the learnings that we're getting from Deep Sea as well as IBM and how they're gaining efficiency with some of the things we're doing, we don't have to do that. So this is, you know, we're only into a very, very small, you know, maybe, maybe not even 10% of what this this can do.

And that means invention. And so I think, you know, tech people want things to move really fast. You know, it doesn't move that fast.

You know, Microsoft wants everybody to adopt, you know, copilot. So they pound on us for like, I don't know how long to doing it, but they can't even get us off Windows 10 for crying out loud Windows 10. There are people weren't running Well.

Said. Yeah. So, You know, and, and you know, they're threatening us this year.

It's like, I think they'll capitulate because we're saying absolutely long Live Windows 95 long wi live Windows 95. I don't think, and I agree with you Kimberly, I don't think that we've had AI's killer moment yet, right? Like augmented reality had its augmented reality had its killer moment when everyone was using Snapchat filters, not realizing that they were using augmented reality, right?

They were using it mass adoption without acknowledgement of the technology. AI has not had its killer moment because although people are technically using it with voice assistance agents, whatever, I don't think it's reached that moment where we're all dying to use it, but we don't know that we're using it or we're not aware as Aware. And I don't think we're going to, I think we're gonna get into this world as, and I think about the enterprise.

So I keep going back to how they're, what they're adopting, what they're doing and the kind of applications they're starting on and the applications that they're doing. We're gonna see this customer service capability. It's gonna change how we do, how customer service happens.

So we won't even know that's going on. We'll just know that it's changed. And that same thing is gonna happen with, you know, some of the logistics.

We're gonna see logistics change as they apply that kinda things. I mean, just that mere thing about what, you know, when I was talking to Novartis or they're presenting at AWS and saying, you know, they went from 50 people filling out the forms that they have to fill out in order to go to drug over to the FDA to being able. And in eight weeks or 13 weeks or however long it was down to like three days, it's like, holy crud, that's transformative.

So there are these pockets of transformation, it's just that we don't see it as a day-to-day thing. It's just going to happen over time. Well, there's two types of change that, that occurs with technology.

One is doing what we do, but doing it with the technology that either does it faster, cheaper, maybe it eliminates a task, whatever it might be. The other is doing something you couldn't do before. It just wasn't feasible possible.

Or we didn't even think about solving it that way. You know, may maybe, uh, you know, one might be, uh, you know, discovering new proteins is something, an activity that we have that we do. And say in labs already, we just hadn't used ai.

Now we do, et cetera. But um, you know, maybe we create a new element. Nobody thought I could create a new element.

Now I can do that with ai. I'm just making up something. But something that was brand new, never, never thought of before.

Those, those are the kind of real, to me, that's the real transformative change. The, um, doing things better, faster, cheaper, or not at all is the evolutionary part of change. And we're still mostly in that phase, but 90.

But until those two moments happen, there's gonna be this disconnect between the type of experiences you expect daily from AI and what you're being fed with ai, which is just the conundrum. Plus, as Alan said, you mentioned the fudd factor. Something like Deep Sea comes along, immediately becomes number one free app on the app stores, then only to be told the users who've downloaded, only be told 24 hours later what's happening to your data and what's China doing with it.

These things work against, against ai. It's just this kind of conundrum of hype versus reality. And I think to some extent, deep Seek was exposed in embarrassment of riches.

We had all just accepted the way that things were being done was how they needed to be done. And here they came and they unseated us. Although I really wanna know what chips they used.

I don't think that that's been public yet. But that was a game changer in a way that I think almost needed to happen because we just had sorted, accepted this mass consumption of energy was a given. We accept, we accepted that this was the path and they showed us, actually, no, it's not so good.

Good for them. I don't think. I don't think, just just to go on the record here, um, I don't think Microsoft's gonna give up on Windows 10.

What they're gonna do is launch Windows 12. They're gonna call it the AI enabled version. And because 13 is unlucky, they're only gonna stay on even numbers from there on out.

So we'll go from 12 to 14 and that's how they'll get us all on board. I actually give them a lot of credit because as an SEOI am seeing Bing in the top 10 of traffic, as you know, uh, search engines bringing traffic to my clients for the first time probably ever. Bing is, Bing is not, not ads organic, uh, because of of chat GBT and because of the integration.

So good for them. I, I don't disagree with you, but I have noticed that I am being pulled into using Bing against my will. And then I have to look at it and then I go, oh crap, that was a bing result.

And then I go back to Google to get the result that I am looking for. 'cause the Bing result is still kind of crappy. That's just your bias against all Microsoft.

I, I'm a big Windows user, but is still good, Not as good. Whatcha you getting from Google? They're a little Gemini thing and then a bunch of sponsored ads and then all the way down on the bottom, you actually get a search result.

It's ridiculous. That is ridiculous. That is a Well, it's turning.

They're hurt, they're hurting traffic. Right? The more people get from Absolutely.

You know, the zero click content, the less they're gonna click on sites. It's leveled out many, many sites Yes. 'cause of those overviews.

Yeah. And, and I guess that's what they wanted, but to me it's not great search anymore. No, well they're doing it on purpose.

'cause they wanna be the provider of the content, right? They're essentially not being a facilitator. They wanna become the provider of the content using AI results that they pull from the people who create the content.

Yet, yet another company taking content without permission. So is there any, uh, uh, Anna, maybe you have this data since this is what you, what's your core business is. I mean, I personally, I I don't even read it.

I just go down because I don't trust it. I mean, I, AI generator, this goes back, this goes back to what you were talking about. I don't trust the data that's getting presented to me by Google on, on that summary.

Um, I just kind of start pinging down 'cause I wanna go something that's been vetted by somebody, a person, and that's not been vetted. So is are people doing that? Are they, are they just ignoring that and going down or what's happening with it?

Um, Well most the time You need a summary, right? Do you really want a summary of this? I'm not looking For that, right?

Well, the, the overviews were only as of a month ago, only in 17% of, of queries. I think that has ratcheted up Oh, a lot. 'cause as they're making a mad, they're making a mad push for Gemini.

Like, you'll notice Gemini, if you're in Gmail, is offering to read your email for you and summarize it. They're, they're pushing it down our throats. I personally always read it because I'm trying to dissect how it's putting itself together.

Where is it pulling sources from? I'm reading it for a curiosity and an analysis viewpoint, but often I do skim below it. And those statistics I don't think are readily available yet.

But I have a feeling that, um, the average consumer is maybe not so discerning. They're like, oh, answer easy. Good if, especially if it's something simple, right?

Like, what is a business open? Or how do I get there? You know, those sorts of queries know the more complex stuff.

I hope so. I hope we're not blindly trusting them, but I I'm afraid a lot of consumers will. I wanna right click and delete it.

That would be nice. Just a little X and close all thing. All right.

Hey, we gotta take a break. We're gonna come back and discuss, uh, our third segment today. But you're gonna have to wait for us to come back.

You're watching Textron Gang, Discover Textron Group, the epicenter of tech innovation. We are your go-to for reaching IT leaders and practitioners worldwide. Our secret impactful content that sparks awareness, engagement, and top quality leads with us.

You'll access editorial websites, streaming videos, virtual events, custom content analyst research and more. Join our satisfied clients. Let's revolutionize your tech journey.

Contact us today and tell your story to the world in the most powerful way with Textron Group. Hey everybody, that's my Alan Chimble invitation right there. Um, We, uh, Mitch and I are out at the, uh, Dynatrace conference this week.

And Dynatrace, if you don't know, is a provider of an observability platform. And they've been around for a long time. And we're, the show is really about how to bridge this divide between IT operations folks that have been using various platforms to observe it and what developers see.

Because we all have heard the story about developer says, well it worked on my laptop, but it doesn't work in the production environment. And this conversation's been going on longer than anybody cares to admit. In fact, you know, I would say three quarters of applications when they're first built probably don't run on the production environment they're intended to until they get tweaked and somebody kind of bangs them in shape.

But Mitch, you're here with me. Are we making progress here? What's your take on what's going on with Dynatrace specifically?

But in general, can we get better at this? 'cause it seems like, I feel like I've been having this conversation for three decades now. You mean if it works in production, it would've worked in test, is that what you're saying?

Yeah, Something like Something. No, it's interesting where, you know, we used the term shift left about security. We're seeing shift left happen with observability, right?

Moving earlier into the kind of workflows that happened before things get into production. I think, you know, a year or two ago it was all focused around how do we consolidate to one source of truth between securities, the SecOps, um, use of security and, and it ops use of observability, uh, for those applications. Now it's how do we, how do we get a window into observability data that's captured in testing but also in production all the way back up into the development part of the process.

And that was a big part of what dynatrace's announcements are. And there are a few companies that are focused on this area, which I think is really encouraging is literally as a developer being that because they ingest the code base and, and, uh, analyze it against the, uh, the telemetry data that they're getting as a developer, you can say, I'm looking at this piece of code. What are the log messages or alerts or alarms or telemetry data that was captured as part of this?

'cause I'm trying to understand if that's the error that I'm really trying to track down or in operations, right? You can say, here's where the issue, uh, the symptoms of the issue that we're seeing and immediately go to some of the parts of the code base that that might be. But it also applies for testing, you know, know, solving bugs and things like that.

So why wouldn't you want to benefit from what we're seeing in production, which is ultimately why we're writing in the code and also what we're seeing in test, uh, because that's part of kinda getting through that process into production. So there's a lot of good things. Dynatrace is one of the companies that's doing this new rec, new Relic is another, um, honeycomb is very developer focused observability platform.

Of course the, you know, the other players, Splunk and others have been around for a while and, and they can play in development, but some are, are catering more to earlier in this awkward development lifecycle. Y you know, it's funny, observability le let's face it, observability originally was replacing a PM, which, you know, when I look at the software development life cycle with deployment being kind of the center and stuff left and stuff, right? A PM was all on the right side, right post-deployment, how's my application performing?

But now observability and we saw at a cube con, I suspect we'll see the cube con in London in a few weeks again, or two couple months again. And, and here at Dynatrace. And what you're saying, Mitchell, the action in observability now is on the left side and, and, you know, predeployment and moving further into the developer.

And I, I didn't see that coming to tell you the truth. I I, uh, I didn't think that that was Where, Where, where the action would be. But plainly, that's, that's where these folks are, are moving, right?

It's like Predeployment and connecting development and ops together. So, sorry Mike, I didn't mean to step on you there. Go ahead.

No worries. But Kimberly, I was involved in this philosophical debate last night and it meant something like this. Wait a second.

Were there beers involved? It might have been a couple of beers involved, just chuck it, but, but the car conversation, Old fashion whiskey, I was, is it, is it, is it incumbent upon the developers to make their applications work in the production environment as dictated by the IT ops folks? Or is it incumbent upon the IT ops folks to figure out a way to make the application run and who's, you know, responsible or who should give more here in this conversation?

Because, you know, ultimately the application drives the business. So it's, you know, who, what's, what's the dog and what's the tail here? Well, I think it depends.

Classic engineer, That's definitive right there. It really depends upon what part of the application. You have a future as a referee here, perhaps Ball this part of the line.

It's the DevOps. If the ball went on this side, the lining side, the problem. So I mean, so on the, on the observability, I mean this is what you're talking about is where, where things get glued together, right?

So you, if the application has to run appropriately itself, okay, so it's whether that, so how it gets deployed, how it's connected to probably in gonna say these things, I may don't, probably don't have any idea what I'm talking about here, but how it's connecting to the database, the live database it's working on, how it's, you know, it's working with the other applications that have to run there, you know, when, when, when they come up, et cetera. So there's, I think there's probably scenarios where it's an IT operations problem and there's scenarios where it's a DevOps problem. So that's why when you're saw saying shift left, you're not just gonna shift it over to the guy.

This says it's all your fault, all your problem, all your issues I need. And that's where the observability probably comes into play is that I'm getting more of a, you know, full war look, look at how this thing is running and operating. Is that good?

Yeah, no. So I, look, I believe developers should not develop in a vacuum, right? Because that never ends well.

And so Kimberly, to your point, when you're a developer and you're writing an app that's gonna make a database call, it's incumbent upon you to make sure that that call actually does get made and that, you know, your SQL is right or whatever you're using to make, you know, make the call the query and that that information is passed through and it, and it moves on to the next, it moves on to the next task. Um, I, I was, and I've always felt even before DevOps, I've always felt this way that, you know, the developer sits there and says, well, it ran on my machine. That that's nonsense, man.

You know that you didn't do your job. Well, I, Go ahead Anthony, as we go. Go, Mike.

No, no, No, Kimberly, you go, go ahead. Well, so if you, that's why, I mean, if you think about it, Alan, was it two, three years ago when platform ops became a thing at Q Con, right up until that time, the only people that were at Cube CubeCon were developers, right? And they said, I don't need operations because I can stand this entire thing up myself and I can run it.

Well then things kind of got bigger and once we started getting bigger, we started dividing the responsibilities a bit more. And then we said, okay, so there's this other thing called platform ops that we have to have. But so dev operations and platform operations gotta get together.

And oh, by the way, there's this other guy called SEC too. So it isn't, one person doesn't have all the knowledge for this. They, it's just impossible to do.

It's too com too big and too complex, which is why they're having the AI engine there, et cetera. So, Mitch, I, and I'm Italian, I'm not Italian, but I'm still using my hands on. I I see that.

But that's good. Well look, I'm, I'm gonna, I'm Gonna surprisingly disagree with Alan. Um, all the time that we see this, it's like developers go out and they go, well the database, the production environment is using.

So, so I'm gonna go out and, you know, get a different database. It'll be MongoDB, a document database, I'll run it myself, I'll manage 'em myself for about nine months. And then I get tired of that.

And then I take this giant turd of a thing and I give it over to the it ops people and say, now it's yours to run. 'cause I'm busy, I gotta go write another application. This happens all the time.

And so, Mitch, you're laughing. Yeah, I just, your choice of words just struck me odd there. Well, yeah, I personally think, you know, let's just wind it all the way back, you know, development and production is the answer to all this.

'cause then you don't have a difference between your development environment and your production environment. I say in all just no, I'm not suggesting that you that we do that. I think Kimberly, you're definitely onto it, which is connecting that what you said with Alan's point is part of platform engineering is creating the platforms that are gonna help you get through the process and get to production.

So you're not go building your own environment to develop, do development in that is wildly different or even mildly different from what's in production, but it causes you to trip up in getting it there. And that's part, I think that part, part of the rule, maybe even the responsibility of platform engineering is to do everything we can to make that process flow, flow as smoothly as possible. So as we move from development through test into production, whether it's containerized or it's platforms or whatever it is, that helps us keep those environments as consistent as possible.

I agree. com, another text drug site. You can learn all about this there, guys.

Did I tee that up? Yeah, we, we've gotta end it. But before we end, let me just say I would discount anything Mike Ard said about disagreeing with me.

'cause the rumor is he does share an account with a known gambler. Um, I hope you've enjoyed this text, hang with and thanks for joining us. We'll see you soon, Kimberly, as always, it's a pleasure to have you on.

I know you're headed on a little bit of a cruise vacation. Enjoy it. I hope the weather's great for you Mitchell, Mike, enjoy Vegas.

Get home soon, John, keep an eye on Silicon Valley for us out there. Okay? All righty.

Until next time, happy Thursday, we'll be back tomorrow. Of course, we have a full text drunk TV lineup immediately following the gang today. So stay tuned, stay here on your favorite channel.

We're coming at you. This is Alan Shimel, we're outta here. This is Textron tv.

Hello everyone. Welcome back here to Techstrong tv. I got another first time guest for you here on Techstrong tv.

His name is Mark Cusack. Mark is the CTO of a company called Yellow Brick Data. Um, and we're gonna find out about yellow brick data and, and talk a little bit about Kubernetes.

But first let's talk a little bit about Mark and welcome him. Hey Mark, welcome to Tech Drunk tv. It's great to have yarn here.

Hello Alan. Very nice to be here. Thanks for the invite.

Pleasure. Mark. Um, I mentioned you're CTO at Yellow Brick Data and we're gonna jump into kind of about yellow brick data in a moment.

But before we did that I wanted to kind of just get an idea of your, of your, uh, background and of your path to becoming, you know, sitting here today as the CTO. Yeah, and I guess Alan, my background is somewhat unusual as actually I started out in academia working as a physicist going way back into the dim distant nineties. And so I ended up, um, doing an undergraduate, um, physics degree, um, postgrad PhD in, in the theoretical physics with a strong kind of bias towards, uh, distributed computing actually, which is kind of the link to the present day.

But, so my career took me from academia into government research, into distributed simulation systems and then into the startup world where we spun some of the technology that we developed in, in government agencies out into, into my first startup, which is back in, uh, 2004 called Rain Store, which is all about, um, archiving massive amounts of data from relational data warehouse systems. And then that company got acquired by Teradata in 2014. So I joined Teradata for a few years and ran the data warehousing product line there.

And then I flipped out about four, four years ago or so to Yellow Brick to become CTO there. And that's where I find myself, What a great story, theoretical physicist. You ever look back and think to yourself, man, if I had taken the fork of the road on the left versus the right, what would I be working on?

Now I do think that, but I always keep my kind of eye as to what's happening in, in the world of physics. And you know, there's a lot of crossovers when you look at this sort of advances in quantum computing and over, over the last, actually last six months or whatever. And I look back at my work, so 20, 30 years ago and there's a lot to kind of get excited about there.

So, but what is interesting is when you start to apply ideas in a totally different area into what you're working on today and that kind of cross-fertilization of, of ideas across different disciplines, I think is, uh, pretty interesting. Absolutely. Very cool stuff.

Mark. So were you at Yellow Brick data from like day one kind of thing? Or or you joined, they were already out.

Give us the background on Yellow Brick. Yeah, no, yellow Brick was actually well established when I joined. The company was founded in 2014, um, with the idea of how that they could apply the new emerging N-V-M-E-S-S-D technologies into high performance analytics and data warehousing.

And so the founders came from Flash storage companies, um, as well as, um, established database companies. And so I, I joined kind of quite late on in the day why, why they've already been in market with a product for three years when I joined. Give us a sense then, I mean we, we get an idea of why, where it comes from and, and what they were doing.

Mm-hmm. Let's fast forward, you know, 'cause there's another one of those 10 year overnight sensations, right? You guys are added there over 10 years.

Fast forward to today. Tell us about Yellow Brick today. Yeah, Well I should just let your viewers know.

I mean, yellow Brick is an SQL data platform. We're essentially a relational database, but one that's really tuned for high performance descriptive analytics. And so you look at our customer base, it's financial institutions, insurers, telcos, government agencies, and they typically want to modernize their existing old data warehouse infrastructure with yellow brick.

And typically they've also got a lot of private data. Some of the crown jewels of their enterprise data is stored in a data warehouse of course. And they want this data to, in some cases re be retained on premises and in others they want to deploy that data around analytics on it in the public cloud and in some cases some hybrid combination of, of that setup or even multi-cloud as well.

But ultimately they want to keep control of their data and that's what Yellow Brick enables them to do. So we have customers from Redshift, from Teradata, from Oracle, from IBM, SQL Server migrating to us. And the end of the day, what do they get?

They get, um, better outcomes from their data, happier users because the thing is a lot, lot faster significant cost savings and big returns on investment. I, I just gotta ask 'cause I'm curious, has, has this whole AI thing had an effect on the yellow brick business? I don't think we would be a credible technical company if we didn't have an AI roadmap to our, to our investments.

And, and so we do and we've done a lot of work, particularly in a couple of areas where one of the most, well about a year ago we added capabilities for yellow brick to operators, a vector store in similarity searches for retrieval, augmented generation applications. So, you know, augmenting knowledge and injecting knowledge into your chat conversations. So we, we have that capability, but more recently we've been looking at, um, converting natural language text questions into SQL and have that executed directly on yellow Brick.

Oh, that's nice. So that's something we're actually working on. There's gonna be quite an exciting release of the product in, in a couple of months.

There'll actually be an SQL co-pilot, if you like, that allows you to Yeah. Give the scammers and Databases that's, you have, you can talk natural and it, and it translates to SQL. Correct.

If you could do that without, you know, if you could do it with, I mean, it's always the same story with ai, right? If you could do it without hallucinations and without mistakes, man, that would be so cool. But, you know, well, so I have a lot of friends who grew up being SQL DB admins.

Right? And I don't know how, how they would view that. Is that kinda taking their job?

Or is that just gonna set the world on fire for us? Well, you know, I, there are, I think you have to split the use cases into two. Here.

You've got this kind of, uh, uh, kind of holy grail of having any business analyst who's, who knows nothing about SQL being to ask any business question they like of their data and getting an answer back that they can make serious, you know, life altering business decisions on the backlog. Yeah. That, that's one pile.

The other is, hey, maybe this is a useful productivity tool to allow me to roughly generate a starting SQL point that I will then double check and verify, uh, and then add that to my, my, uh, production. Not that, you know, that's not very different than you hear from testers. From coders, right.

Either. Right. On one hand I could say, oh my goodness, the sky's falling.

It's gonna replace me because when you know, we're gonna go from 27 million developers to 500 million developers, 'cause everyone could develop code, you just tell the AI to, or how am I gonna leverage this, the 10 x my my worth, the 10 x my productivity? Right. And I always want to be on the 10 x side.

Yeah. And I think that is the pragmatic approach. I think, uh, we're, we're deluding ourselves.

If we think you can take the natural ambiguity of the English language or any other language and convert that unambiguously into a SQL statement that will run and give you the right result, we're far away from doing that. But getting that 10 x performance improvement, that's what we're at yellow brick kind of thinking more about. Absolutely.

Hey, before we jump into anything else, yellow brick's, uh, website. What, what's the website? com.

Very simple. Yep. Great.

Alright, let's talk, if you don't mind, let's pivot a little bit to our topic of discussion today, which is how Kubernetes delivers scalable analytics in hybrid cloud situations. And as we were talking off camera, I think in order to have this discussion, I think we have to first define hybrid cloud, right? When, when, when I, yeah.

I, you know, cloud came on the scene 2005, 2006 for me is when it hit my radar. And, um, you know, we initially it was public versus private cloud, right? That was the big thing.

Where are you gonna keep your stuff? And then the answer became, well, both. And that was very easy to call the hybrid cloud.

I've got some in the public cloud and some back in my data center and, you know, calling that private cloud went outta style there for a while. Right? It was just back in the data center.

But, um, it recently, it's, it's come back. But the other thing that really took me by surprise, 'cause I didn't see it coming, was what we call multi-cloud. Where, you know, I got some stuff in a WSI got some stuff in Google, some stuff in Microsoft, maybe, maybe a little bit in the Oracle cloud.

Ah, maybe I got some stuff back in the data center too. And, you know, I've got, and I I I put stuff into different clouds based upon what's the best tool for the job. Right, right.

I don't know if we saw how big that was going to be in relation to the ne more narrow definition of hybrid cloud, which is just some form of public private. What's your take on that? And, and, and if we could define that, then let's talk about how we use Kubernetes to deliver the scalable analytics for that.

Yeah. As, as far as yellow brick is concerned, we think of hybrid, uh, cloud and multi-cloud in facting. Sometimes we com combine the whole concept into hybrid multi-cloud.

If the idea is I will place my data and my data warehousing workloads on the basis of data gravity, data sovereignty, um, cost security and other considerations. And so we are all about, it doesn't matter really where you deploy yellow brick and your data, we want to give you the same experience everywhere on any public cloud, uh, on a hybrid combination of those multi-cloud combination rather, but also running in your own data center as well. That's really what we're aimed at.

We want to give freedom of choice and flexibility about where you deploy those workloads. So for us, hybrid cloud is just picking the right tool for the job, as you say, and placing data at the right place at the right time. I agree with you.

I think that's a great way of looking at it. And what you call it, you called it the, the hybrid multi-cloud data sort of model. Yes.

It's a mouthful. Yeah. We need, we need, we need, uh, some initials there.

Well, I'll work on it anyway. Let, let's now turn it over to Kubernetes, mark and talk about how do we leverage that here for scalable analytics. Yeah.

Now, you know, we, when we were embarking on this hybrid multi-cloud journey, um, there were a number of considerations that we wanted to make sure that we tick the boxes on. One of which yellow brick software had to run anywhere in the data center and in the public cloud as well, and a hybrid combination of the two together. It needed to be elastic.

You know, we needed to be able to scale compute, uh, oh and storage in independently of one another in all of these environments as well. All modern data warehousing solutions today provide that scalability that you, you scale your compute to fit the tasks at hand, for example. And then last but not least, it needed to be resilient as well.

We need this thing to, to be capable of supporting business critical operations with 24 7 availability. And so when you look at Kubernetes as an orchestration framework, it really does tick all of those boxes. It becomes this kind of cloud operating system for us where we can deploy the same containerized micro, uh, services architecture that yellow brick has in any one of these deployment options and have the same experience here as well.

And you know, what has been very interesting is there was the, a big lift to kind of get yellow brick in our first cloud deployment running in the elastic Kubernetes service EKS in AWS, but then the barrier to migrating it to a KS and Azure and then GKE got lower and lower and lower. Yeah. And we've just done our most recent call OnPrem to, um, red Hat OpenShift as well.

Yeah. And so now we have Kubernetes coverage wherever most enterprises would, uh, would care for it. Absolutely.

You know, I I, I made, uh, a reference to it earlier in, we are seeing more and more what I used to call private cloud back in the data standards, right. Whether it's Red Hat OpenShift, which is a, a dominant one. And uh, what what's the other big private cloud open source?

Uh, it was like whole consortium of people. Rackspace was behind it, right? I Mean, this rancher and Zu and other kind of, well, Rancher Rancher is now part of, uh, of, uh, of, uh, not of Tu Seus.

That's correct. Rancher is Seus Tanger of course. Is is Broadcom.

Yeah. Right. Yeah.

No, no. But the, the private cloud stack not open. Was it open cloud?

I think it might have been Open, Open stack was The open stack. That's it. Yeah.

Yep. You know, that kind of has been rejuvenated lately too. 'cause that had gone an dormant for a while.

So we we're definitely seeing a lot of that. Um, but Mark talking about scalable analytics here, you know, it, it begs the, the real point, which is, look, our data warehousing is, you know, it, it's, it's consuming space, whether it's public, private, a little of this, a little of that, a lot of this, and a lot of that. It, it, it, it just seems like there's never enough and, and we're, you know, we battle is it cheaper to do it here versus there?

And what makes more sense? And I need to segregate data, you know, and, and maybe store it based upon its particular value. Um, how does Kubernetes help us maybe with some of those kinds of analytics and those kinds of data points that we need to make those important decisions?

Well, I, I think actually to some extent, Alan, the, the, the decisions are somewhat orthogonal. I mean, our customers make decisions on where they're going to place their data and workload on the basis of the business problem and use case at hand, right? So, um, Kubernetes, I think doesn't impede what we do.

Um, now, I, I have to say though, uh, when, when we deploy Yellow Brick and Kubernetes at the moment, we pretty much totally, um, take over that Kubernetes deployment, that Kubernetes cluster, the only workloads that are running in the Kubernetes cluster that we are running in a yellow brick workloads. And we do that for performance reasons as well. Um, we, we do a lot of work at the very lowest levels of the Linux operating system to bypass main memory, to have direct access to the NVME drives that we access the store and retrieve data from.

Um, we, we introduce our own threading models within Linux, and we do a lot of low level work, which means that we want to, within a particular Kubernetes compute node, take over all the resources on that box. And so we don't sort of allow, we put anti affinity rule rules in place in Kubernetes to stop other pods kind of coming into our sphere of influence. So I guess that's a long way of saying that we, we kind of isolate ourselves from other considerations and, and give it an environment that's completely dedicated to Yellow Brick to run on.

Love it. com, you mentioned the website. Any particular path they should follow on the website or for specifics on this?

Yeah, there's a, there's a ton of reference information blogs, um, follow, follow us on LinkedIn as well to, to get more information from you. org site, you'll find, uh, a paper that we published at their conference about a year ago that gives us the, the full kind of Kubernetes architecture breakdown as well. But tons of information on the website.

Excellent. Mark, thank you so much for coming up here on Text Trunk TV with us today. It's been a delight.

Please do. Come back, keep us posted. You know, this is, you know, it's all about the data.

Stupid, right? That, that's the lesson that we've learned over the years in, in back here. And it seems like yellow brick's right in the middle of it.

So do keep us posted. Will Do. Thanks Al.

It's been a pleasure. All righty. Mark Cusack, CTO Yellow Brick data here on Tech Drunk tv.

We're gonna take a break. We'll be back. We've got more for you.

Hello and welcome to the digital CXO podcast. I'm Amanda Asani, and with me today I am happy to have Dan Evans. He is the Senior Director of Smart Cities and Smart Lighting at Itron.

How are you doing today? I'm well, Amanda. Thanks for having me on.

Can you share a little bit about your background and then a little bit about Itron and what do y'all do? Sure. Yeah.

Um, so I'm based out here in California, in Silicon Valley. So as you can imagine, I've popped around to a few technology companies. Uh, that's kind of my roots is around, uh, tech and, and networking.

Uh, so I've, I've, uh, done a few stints at companies that were introducing broadband internet to the home over cable infrastructure. That was back in the nineties. Uh, I've been at companies that build wireless telecommunication systems, not not the cellular, uh, phone, uh, type, but for more, uh, internet services providers.

Uh, and then I joined a company called Silverspring Networks, uh, in the late mid two thousands, which was really focused on the utility industry and, um, enabling communications and software for utilities to, uh, allow them to start deploying what was called smart meters at the time. So a, a an electricity meter that had a communications capability. Um, and then in 2018, I think Itron acquired Silver springing Networks.

Um, and Itron, sort of segueing to sort of who we are, um, is a company that is, you know, largely focused on, uh, helping utilities and cities become more resourceful with the resources that they, uh, operate and, and and maintain. So we work with utilities across all of, uh, um, the commodities, electricity, water, gas, um, as well as with cities, uh, in, in the cases where they manage those commodities. Or in my world, um, as managing the smart cities line of business, we look at some of the challenges that cities are facing today, um, and, and how we can, uh, address those.

And, and we've kind of leveraged a lot of the core infrastructure that we've developed for that metering world into what became the smart lighting world, which we'll talk about a bit more today, and then taking it beyond lighting into some of these other use cases. So it's been, you know, a a almost about 17 year stint within this space, uh, with the last 10 years really focused around smart cities. Wonderful.

Thank you for sharing, and you're certainly the right person to speak with today. Then, since our topic is about the evolution of smart street lighting, and we're hearing so much about smart cities these days, so, you know, streetlights are no longer about just elimination, they're becoming the nervous system of smart cities and collecting data, enabling services that were not possible a few years ago. So can you elaborate on that and what benefit are c seeing?

Sure. Yeah. No, this, uh, this phenomenon, if you will, uh, started kind of in the 20 10, 20 12 timeframe.

Um, and cities and utilities, depending on where you are around the world, I, I did mention earlier, but Itron is a global company. So we see this phenomenon, uh, evolving in many places, uh, in, in utilities in North America and the US specifically, a lot of the streetlight infrastructure is owned by the utility. So there was a natural conversation that we were already working with those customers to take it to the next level.

And in the other parts of the world, that infrastructure tends to be owned by the cities. But either case, they were looking at older technology. So we're talking about the lamp technology.

You know, a streetlight has evolved from a gas lamp, uh, fixture to, you know, something that was electrified and that electrical, uh, technology has evolved. Um, and the, the phenomenon that we were, we caught up with was LED technology. So now it's everywhere, right?

It's, it's in our phones, it's in our cars, it's in our homes. Well, it's on our streets as well. So you may live in a town, uh, if you look outside of the streetlight, it has been changed in the last 10 years into an LED streetlight.

Um, what you may not know is whether or not has a smart controller on top. So for the same reasons we got into the smart metering space, um, it basically enables communication to that asset. And why do you need a communication to a smart streetlight?

Um, it's because you can, uh, be more proactive in maintaining it. So, um, you and I, we've all seen streetlights that are out, right? And, and you're wondering, okay, did somebody call that in?

Has somebody reported that, uh, I'm not gonna bother, you know, a week later it's still out. So, okay. Clearly nobody reported it.

Let me report it. So a a, a smart technology can be introduced there to, to sort of be the virtual citizen. And as soon as there's a problem on the streetlight, an alert will go back to the operator, whoever's responsible, and allows them to be more proactive in their, in their maintenance.

So it makes us as citizens happy. The, uh, economics of the, of the city from a commercial perspective go up because people feel safe, and the quality of life is high. So, you know, the LEDs were coming in, which meant that whoever owned that light was gonna roll a truck to replace it.

So, uh, our, our solution and, and we're not the only one who offers this, was when you go out and upgrade that light, add a smart control solution, because you probably will regret not having done that, um, in, in a few years. And now you, if you have to go back to that light, the economics of that, uh, just, just don't pencil out. So that's kind of the benefits are, you know, you get the visibility improved maintenance, you can also have an improvement on the energy consumption.

So just by, by going from the older high pressure, sodium metal, metal halli streetlight technology to LED roughly gets you a 50% savings in energy. So that's good. But we're encouraging our, our customers and our install base to not stop there, add the controller, because these streetlights actually have this dimming capability.

So just like you have a dimmer switch in your home that can adjust the light level, now that functionality exists on a streetlight, why would you want to do that? Well, you could have situations where you've got traffic, the, the, the drops during the middle of the night, no one's on the street, so you can bring the lights down to still a safe level, so you're illuminating the ground, but you don't have to be bright a hundred percent. So that can get you some more energy savings beyond just the LED.

Um, so it's a combination of energy and, and maintenance cost savings that has really, um, driven this adoption of the smart lighting control solution that we've seen around the world. And we have about 4 million of those, uh, str smart streetlights under management here at idr. Yeah.

So there are many aspects that are improved from it from a safety energy and cost. Um, so with that being said, are there other places that you could put this sort of smart technology besides street lamps? I just wonder how, how else can cities benefit from this type of technology?

Yeah, so I mean, the, the, the nice thing about a streetlight pole is in most parts of the world, it's energized all the time. There are some situations where it go, the energy goes away during the day and it comes back again at night. But, um, a lot of, uh, cities are looking at how do I leverage that asset for more than just this smart lighting solution, which, you know, is good in and of itself.

Um, so we have worked with, at Itron, we work, uh, with a, a, a host of, um, technology partners who bring, uh, their solutions and we look at how we can integrate their solution with ours. So in some cases, I'll give an example where you might want to monitor the air quality on the street level, you know, at a very granular level, um, more so than what you might have using sort of federal level or government run facilities that report on, on weather or air quality in and of itself. Um, those can be attached to the streetlight and powered by the streetlight 'cause you've got power there.

And then you can actually leverage the smart controller to pull data from the sensor and then push that data up into the cloud into the software that we manage, which provides now the city with a single pane of glass view of the solution that they've deployed. Um, and so more and more of these different use cases, again, tied to the challenges that the cities are facing, whether it's, you know, climate change, uh, traffic congestion, as cities are growing, um, you know, public safety of, of infrastructure, um, there's a number of different challenges. And we work with this partner ecosystem to bring together their solutions with ours so that the city sees this as a, an investment that they can leverage and build upon, right?

Starting again with the lighting and, and then adding more on there. Once you have that, now you've got, you know, you've got your lighting data, you've got data coming from a variety of different other sensors, whether it's traffic or, or air quality, then you can start bringing these data sets together and actually making some analytics at a minimum and saying, okay, interesting data. Let's do some reports or take action.

So, um, you know, for example, my, my, uh, mention earlier of the ability to dim lights, well, if you had a traffic monitor on that street or distributed on the main roads, then you would know definitively that your volumes are down. Or if there's an event all of a sudden in that part of town that normally doesn't happen, the sensors would detect your traffic levels are actually higher, so you, you shouldn't dim when you normally would. Right?

So it brings the data sets together. Um, similarly examples of, uh, I wanna manage my traffic congestion. Okay, well, as a result of managing that, maybe you deploy a smart parking solution to help people l you know, get to the parking spot as as fast as they can get some off the road, removes the, the congestion.

But at the same time, you've got an air quality benefit too. 'cause your pollution from the vehicles has just dropped. Well, how do you know that unless you've got the data?

So it's really bringing the data sets together in that kind of cohesive manner that are, you know, that we bring to our customers through our, our software platform called City Edge. Yeah, and I can imagine by collecting that data, as you said, if there's an area of town that's more congested, um, over a period of time, they might know, okay, we might wanna have a few more police officers patrolling the area, you know, um, to help you make it more safe in that area. Exactly.

Yeah. I mean, public safety is one of the key domains that we look at, and that ranges from, you know, uh, areas where lighting isn't present, right? That presents a public safety sit, uh, uh, situation.

You may have flooding, uh, so a roadway could get flooded for, you know, heavy rains, uh, and the, and the creek or the river has overflowed that causes, you know, a public safety issue if, if the city can't get their maintenance crews out there to close off the roadway. So again, the technology has the ability to, to, to detect those situations and proactively get, uh, get the crews who need to be out there addressing that problem, uh, for the benefit of the citizens. And again, these are all use cases that the city will benefit from and, and help their city grow to the next level, which is a, is a challenge all all the cities around the world are facing.

Yeah, and it sounds like from what you're telling me, with all this energy savings, uh, on the electricity side, I know that that the, the grid is a, a big issue for big cities. So if they're trying to build out, or even smaller cities that are trying to build out, there's only so much, um, energy available, so that would provide more, Correct, correct. And that's, that's, that's another area that I try is spending a lot of our, uh, kind of research and investment is how is the, our customers, the utilities, uh, being challenged in new and different ways as more things are, are entering the grid scenario.

So electric vehicles is a big area where we're, we're looking at how to help our utility customers manage the load as more and more of those appear right in, in our, in our homes and businesses. And, and again, taking it to the city level, cities are electrifying their fleets. So all of these trucks I talked about, rolling to fix the problem on the street, on the streetlight, those are all moving to electric vehicles as well.

The buses that are being run in the infrastructure are moving to electric. So what happens at the end of the day when all those fleets come back to the corporate yard, they're all gonna plug in, and that's gonna create a huge demand onto the grid, which causes instability if the city, if the utility hasn't planned for that. So we offer solutions that can come in and help manage that load, uh, and, and supply balance, which the utilities ultimately have to be doing every day to make sure they're not having any outages or, or concerns with the, the power quality they're delivering.

So it is a challenge, um, that we at Itron are, are helping our utility customers address along with the other, the other big movement to renewable energies, right? So changing the type of energy, uh, supply from what might have been a, an oil or coal base power plant to a wind farm or a solar farm, which are, are great for the climate and, and, um, the overall environmental impact, but they bring a challenge in their dependability, right? The clouds move in, solar, uh, output drops, the wind goes away while your wind farm is not producing anymore.

So again, this is a, a challenge that we're helping with our, our utility customers, uh, balance, uh, all of these different supply sources with the demand and helping manage the demand to match. Yeah, absolutely. So you would say that the environmental impact is far reaching?

It definitely is. And, and again, when, when we look at, uh, kind of the market here, uh, that is a point of overlap, the climate concerns and, uh, everybody has carbon footprint reduction goals, Itron has our, our own, uh, set of goals. Uh, the utilities have theirs, the cities have theirs, um, and everybody wants to move in that direction, uh, a as well as sort of the other impacts to the, the environment, uh, that are out there.

So, so utilities are being encouraged and moving, uh, uh, rapidly towards renewable. Um, but it does introduce some of those, those new challenges as, uh, as cities are growing and, and cities, uh, like the one I live here in, in the Bay area, you know, data centers are popping up everywhere, right? And thank you ai, right?

That brings, uh, all sorts of compute power that's needed. Um, which, you know, computers run on energy. So as a data center is built, you know, I look at it and go, okay, well my local utility is now gonna have a challenge of feeding that data center.

'cause that's a 24 by seven demand, right? It, it's not like us when we go to sleep and the energy drops off, that data center runs 24 by seven. Um, so they've gotta be able to, to have good visibility into their infrastructure to know, uh, how they deliver that without bringing instability into the grid.

So for a city just trying to enter into the smart city space, what advice do you have for them? What, from your experience, what challenges do you see occur when trying to integrate some sort of smart city plan and what advice can you give? Yeah, no, that's a great question, Amanda.

I would say, you know, it starts with understanding the problem you're trying to solve. And, you know, as we go and look at these surveys, there's a number of challenges that cities face. Uh, and I think we, we we're open and, and collaborate with the cities to, to, to identify what are the key challenges they have.

Um, definitely look at, uh, solutions that have maybe a broader, uh, fit as far as, um, you, you may start with one application that addresses one challenge, but think a little bit holistically in a higher level. Um, and, and what I mean by that is, a city might have a street lighting department and they might have a water utility and they might have a, a transportation or traffic department. What we typically see is those city departments operate in silos, you know, and there's not a lot of crosstalk that happens typically now you're up at the city manager or even mayor level, depending on the size of the town or city, where somebody's looking at this across the board and saying, okay, I have a fixed budget.

How do I make the most and bang for my buck? You know, when I make an investment choice, am I just solving the problem of the, of the lighting guy, or do I really wanna look at a solution that's gonna address something for this traffic, uh, person as well? Um, so looking at sort of a more of a holistic and homogenous, uh, deployment of technology.

And then when you get into the technology and, and the selection, you know, look at what's out there on the market, look for open standards, right? Um, it's all very important to be able to grow on top of technology, uh, rather than pick technology that might have a four or five year lifetime and then it's gone, or the company's out of business or, you know, those are sort of long-term decisions that they make. Um, the other thing is, you know, you know, it's fine to start small, it's time, it's fine to start with a pilot, decide kind of, does this give you the data you need?

Is it meeting your expectations? Um, because that is very useful and important when you go back to the city council or to your, you know, investors or whoever is funding to say, I want more money to do X, y, or Z and here's the data I got from the, the pilot I ran a year ago that shows that I'm, I'm on the right trajectory. It's gonna give me the benefits, me and my citizens and, um, that my communities, the, the benefits that that we say it will.

Um, so that that's sort of the trajectory and the journey that they should get on. When you think about the future of smart cities, do you have any ideas rolling around for new integrations or tools that you think would solve some kind of problem? Wow.

It, it's a very wide space is what I would say. Uh, Amanda, I've been doing it as I said, probably for the last 10 years. And, and what we saw is that that lighting space really grew and it, and it popped, you know, let sort of use those terms where we definitely saw an uptick.

Um, we are still kinda looking at the market and saying, okay, where's the next big pop gonna come from? And there are a number of different use cases, uh, and, and challenges that we are, uh, that we are trying to address in collaboration with the cities. Um, I think the, the key and, and maybe I'm jumping a little bit on the AI bandwagon here, but AI is all based upon the data underneath, right?

It's, it's applying logic, it's applying algorithms to data. And so, um, for us at Itron, I mean that's sort of, that's kind of where we started with, with the metering space and the smart metering space, and then the lighting space. It wasn't just to enable the connectivity, but it was to put software in place that was collecting data.

And now you have the data, what do you do with it? You know, we often hear from customers and cities, okay, there's so much data, I don't know what to do with it, right? So I think the next level is really, uh, uplifting data into an outcome.

And, and, and, and looking at the analytics behind that, what is the challenge? What is the outcome you need? And let's not get lost in the data.

And I think, again, AI is just another way of saying, I'll give you the answer you need because I'm crunching all that data underneath. And then once you have that outcome, you can be making incremental improvements and optimizing your system. Uh, and you know, there, there's a feedback loop.

So the data will tell you, oh, that didn't work, or, yes, that did work, right? You got the outcome that you, that you needed. Um, so it's, it's just that, again, the growing need for data, uh, the growing need for the analytics to then be layered on top of that and then delivering ultimately an outcome to the customer that they don't have to worry too much about all the, uh, the sausage making as we would say underneath the hood.

I just get what I need for the benefit of my customers. Okay. Well, if there was one key takeaway you could give our audience today, what would that be?

Don't be afraid of technology. I mean, the, the, uh, depending on where you live in the world, again, I live in Silicon Valley, so it's, it's almost a foreign concept to not know what tech is. But, um, you know, having worked with large, you know, fortune 500 companies and, and, and who are in a space, maybe that doesn't move as quickly.

Cities are the same, right? And there's an evolution of the staff, and I think the people part has a lot to do with it. Um, and so as more and more people adopt technology, I think they should look at how they can leverage that for their infrastructure.

So when we talk about smart cities, you know, it's, it's really about the city. It's about the, the, the, the, the delivery of the services that they deliver to you and me as citizens within that city and looking at how technology can leverage that, optimize that, you know, get you more bang for your budget. 'cause the budgets are, you know, not going up, you know, they're usually going down.

And this is where technology and automation can often come in and replace maybe a task that, um, uh, was costing you a lot more before. So that's the theme. Whichever one of these smart city use cases, you know, is the next one that that comes across as the big, the big winner.

Um, it's all gonna be based around that. So, uh, I think that's the key message and the key takeaway about um, uh, you know, not to be afraid of the technology adopted and you know, companies like Itron gonna help you navigate that journey from, from beginning to end. Wonderful.

Well thank you so much for coming on the show and sharing your insights with us today. My pleasure. And thanks again for having me, Amanda.

All right. And thanks to our audience, stay tuned. There's more.

com is the leading resource for news analysis and education on challenges facing the cybersecurity industry. com covers all aspects of cybersecurity, including data security, DevSecOps, cloud security, application security, network security, security threats and more. com has the largest selection of security content featuring breaking news, blog posts, podcasts and more.

com to learn more. com. Home of security bloggers network, This is Techron tv.

Hey guys, thanks with Throw, we're here with Nick Klowski, he's the senior research director for Ryans. And we're talking about a new study that they did with Arco that looks into, well just how the role of the CSO has expanded and what are the implications thereof. Hey Nick, welcome to the show.

Great to be here. Mike book, I think everybody we talked to will tell us that the CISOs playing a larger role in organizations. In fact, in some places the CISO now ones IT and security, but I don't think that's the standard operating procedure just yet.

But as it's clear that security is a bigger focus, what has been the impact on all this increased responsibility on these CISOs Elements of scope creep And what we see is kind of two forks happening. On one side you have the CISOs to kind of have the scope creep thrust upon them and are given extra responsibilities, extra functions and aren't necessarily given rewards, aren't necessarily given a bigger role and more influence within the business. And it's just kind of like, oh, all the problems are getting dumped on those CISOs and those folks are getting burnt out fast and looking for new jobs 'cause they just want to get out.

And then you have the CISOs who are being given real ownership of parts of the business, for example, the full ownership of it, given the opportunity to really have more influence at the executive level and real opportunities to grow the business and partner with other executives and those CISOs who are getting elevated with meco creep are finding it generally very rewarding to have all those extra opportunities and extra responsibilities because it's giving them a bigger role within the business. Is that emotional rewarding or financially rewarding? All of the above.

Uh, because a lot of times, you know, at least in my experience, so almost everybody gets added responsibilities to their job over the years. And it takes a while before anybody recognizes that maybe I have a different job and I need a different title. So we will CISOs as they continue to get elevated and responsibilities evolve into something else.

Yeah, so for example, we see this dual IT and security ownership situation where you might have a CISO and CIO kind of role, the folks who are CISO and CIO see a substantial increase in compensation, substantial increase in satisfaction, and generally speaking more of an executive role within the business. Whereas if we see CISOs who just take on discrete functions of it but don't get the full CIO function, they could own as much as half or more of it. Their compensation stays pretty much the same as other CISOs.

Their satisfaction drops and they start getting frustrated with how they're just given all the problems to solve but not given commensurate rewards. The folks who are taking on more responsibility outside of traditional cybersecurity are their common attributes or they maybe they've got an MBA or something and they go do something different that their colleagues are not doing. We do see some commonality in that the CISOs who are getting more opportunities have diversity of experience, whether that is working across multiple industries, whether that is making an effort to get involved in risk committees or AI committees or compliance committees within the organization kind of work cross-functionally and build relationships cross-functionally.

But ultimately what we see happening is CISOs are just extremely strong problem solvers. And as businesses have bigger digital risk problems, the CISOs the best person to solve 'em. They're, they're the people who know enough about the compliance angle, enough about the technical angle and enough about the security angle to actually problem solve some of those risk issues and they're getting thrust into those roles.

The key thing is figuring out how can I influence these decisions in partnership with other business leaders as opposed to how do I become the defacto owner of a whole bunch of risk that really needs to be owned by business units. And to that point, are they kind of, um, as they assume those responsibilities, um, are they engaging with business leaders more in terms that the business understands? 'cause I think part of the problem I've seen over the last year, and I think we've talked about it in the past, is that security people, they finally get access to the boardroom but the boardroom still doesn't know what they're talking about.

Yeah, we are seeing about 50% of CISOs report to the board either quarterly or monthly and year over year just fewer and fewer CISOs are are never reporting to the board. Many CISOs are now starting to get active on subcommittees, which is where a lot of the board work really gets done. And then in general we are hearing more and more about CISOs getting executive exposure and looking at the rest of the C-suite as their peers.

We are planning to update next year's survey, which we're probably gonna launch in late March, early April, with some really specific questions to get a sense of how frequently CISOs are meeting with executive peers and which ones they're connecting with more often When I do gain access to the board, I think one of the things that becomes quickly apparent is that the board cares a lot more about compliance than a lot of things 'cause compliance is where the fines are and that's part of the oversight. So is that driving more security people to take out responsibility for compliance because it's kind of like the path to the board To an extent I think compliance can work as a forcing mechanism to get the board to care about Security. But more often, or I should say more substantially, I think what we see is a lot of compliance solutions end up being security.

It ends up being figuring out how you're gonna protect and safeguard data once it's actually in your systems and where that data's living and how that data is rooting through your systems. And that just requires security input. And so security's gonna have such a big stake in solving a compliance challenge that it makes sense that security is getting a larger stake in understanding what those compliance challenges are in the first place.

One of the things that I noticed is that sometimes when security people get exposed more to the business and the port, their appetite for risk starts to increase. They're actually start to think about things a little bit more in terms of well what is the impact of the business? 'cause the business leaders are always assessing risk and to them cybersecurity is just one more.

Um, so does it change the way the security people think? Oh yeah, we see a strong correlation in our data between exposure to the board and satisfaction with the business' alignment to security priorities. And we think two things are going on at the same time and they're kind of coming together to make that satisfaction high.

And that is board members and business leaders are becoming more aware of cyber risk and more open to real conversations. And so CISOs are starting to feel heard and CISOs are becoming more aware of business risk and understanding the priorities and what level of financial risk within cyber is tolerable and more willing to take risks than they might be if they're kind of siloed in the back office thinking about all the threats and worrying about what happens if there's a breach. But come to understand, oh okay, the business is aware of this risk, the business is choosing to take this financial risk.

I have the backing of the rest of the organization. I can feel comfortable with this risk. So do you think that the CISOs that understand that and have that level of conversation are arguably less stressed out than their other colleagues who are always kinda walking in every morning going, I don't know what's gonna come next, but it could be a disaster?

In some ways they also tend to have a lot of executive presence and don't show that stress as much. Sometimes they actually are less stressed out or whether they're just good at showing it. But also those CISOs tend to have been put in a position by the business to be less stressed and have more resources on the team below them so they can delegate more and focus on more strategic issues.

A lot of CISOs are starting to elevate their executive presence and their wherewithal on business acumen, but they're still forced by the way the business regards the role into kind of the back office tech function and they're working to get the business to change too. One of the other things that I sometimes wonder about is when I see senior security or even IT people for that matter get closer to the business, they start to lose touch with the IT and the security teams underneath them. So how do you kinda maintain that relationship while you're establishing these other relationships in a way that doesn't, you know, where they underlying folks who report to you start to uh, I don't wanna say they become suspicious, but they become a little more, um, negative.

In some ways this can be easier for CISOs than what I've seen in practice because generally speaking, CISOs are such curious people who care a lot about doing good meaningful work. And when when you have that attitude it's easier kind of, you know, it's almost like great to talk CISOs into going out of being in the weeds and care about the strategic corporate stuff because they want to step in and helps their teams. They want to understand the technical problems, they want to go learn a new programming capability and they have to learn to monitor that part of their brain so they can do enough of that to stay plugged in, but not so much of it that they can't have that strategic influence over the business.

Do you think there'll be more of a, a split, like when I see in the IT world there's a CIO and these larger companies frequently now there's a CTO or somebody who's involved in the actual tech. So will security teams evolve where the CISO will be complimented by somebody who is really the security operations lead or somebody like that? I mean we see large security teams having functional department heads for SecOps, for architecture, for GRC, for ASEC product sec, all kinds of assumptions depending on the specific needs of the org, it's really just a matter of scale.

We're also seeing in those kind of dual CISO CIO roles, you might have a person who is a CISO and CIO and title who then has a head of IT reporting to them and a head of InfoSec reporting to them because it gives them the ability to be very strategic while those functional department heads are leading the technical execution, How do you perceive the relationship between CISOs and the CIOs then evolving? Because in some ways it feels like, you know, we want them to collaborate more, but you know, people start to get jealous of their resources and prerogatives, It's gonna vary a lot from organization to organization. But what I'm seeing in a lot of healthy situations is where even if the CISO is reporting to the CIO, that reporting is almost clinical in nature.

It's about, you know, getting through the reviews and the formal HR things you have to do and and function. They operate more as peers within the business, both with access to the executive teams and they serve as partners and they both have a shared vision for where they want it and security to go. Those two units work extremely collaboratively and they are able to have a strong, healthy relationship where they are working toward common shared vision for how security and it can make the business better and deliver value.

In your experience, are there courses for security people to take to kind of get this kind of business acumen? 'cause I mean there's no shortage of technical courses for security folks, but I wonder if we need something that helps them understand the business and kind of have those conversations with people. Yeah, there are some emerging programs.

We at Ions have an executive competencies program where we offer coaching from recently retired or active CISOs and VSOs and a variety of asynchronous and synchronous learning opportunities to develop those business skills within a cyber context. We see programs at universities like Carnegie Mellon where some of our faculty are professors teaching business skills to CISOs and even like always like the NACD will kind of help CISOs understand governance or help board members understand cyber. It's a growing need.

And of course there's always the option of pursuing an MBA or something like that if you want to get really deep in the business. I can't help but wonder if AI tools, whether it's chat GPT or whatever, is gonna make it easier for the security of people to understand a lot of these business rules. I mean, ultimately most of what we're doing in business is not rocket science.

It's been fairly well documented. There are tons of business courses and is that all just gonna become accessible content? I think a lot of it's accessible already.

The nuts and bolts side of it, CISOs are generally very smart and very curious and they can pick this stuff up fast. Generally what we see as the tricky part is all of the soft skills, the human interaction, the emotional intelligence, the relationship management and influencing where in most business functions, you gradually build those things up with other business leaders organically as you move up the ranks. Whereas in security, you're moving up the ranks due to technical achievement often then you get thrust into this role where you're now meant to connect with other business leaders and you don't already have those relationships necessarily.

You don't already have the shorthand that the business is using for specific problems and you have to go and kind of build, other people may be building up over five or 10 years of osmosis fast and that's where things get challenging. What relationships can you build to fast track that process? One of the other things I've noticed is that more businesses today, when they all align with somebody else, there is a bigger concern about the security of the two firms and how that's gonna be maintained is there're more of an effort to, for the CISOs to kind of collaborate with the, with other CISOs from different organizations to kinda address those business concerns and that becomes part of the job.

Yeah, we're seeing a growing trend, whether it's through ISACs or there's through organizations like us and all of the events that we host to bring CISOs together. There is a growing effort to create a stronger community among CISOs that folks can share and find that balance between how do we talk about our problems, whether it's the very real threats we're facing or whether it's the business challenges without losing the competitive advantage that we may gain through having an excellent security program. And it's just finding ways to open up lines of communication in a healthy, productive way for the CISO community.

So ultimately, what's your best advice to aspiring CISOs out there? I mean there's a lot of them, I think they wind up, you know, being in the technical track, but at some point if they want to become the next generation ciso, they need to have some sort of business experience. So how should they go about getting that Find side projects that get you excited, that are adjacent to your technical areas of expertise, but not directly in them, whether that's an AI steering committee, whether that's a compliance committee, whether it's a customer trust initiative and volunteer for those kinds of programs.

It'll get you exposure to other executives or business function leaders who will then see the kind of value you can offer and bring you in and sponsor you when business is trying to solve other problems. And then gradually you're gonna get more exposure to the business, the business is gonna get more exposure to you and you're gonna demonstrate the value you can offer to the organization beyond simply the security nuts and bolts. Alright folks, you heard it here.

You wanna become a ciso? You gotta expand your reach beyond just the technical side of the job. And that all requires talking to business folks and ultimately you need to be seen.

Hey Nick, thanks for being on the show. Thanks for having me, Michael. It's great to be here.

All right, and back to you guys in the studio. Hi everybody, and happy new year. Thanks for joining us for another episode of Techstrong Women where we feature amazing women doing amazing things in tech.

I'm Jody Ashley, executive producer here at Techstrong, and I'm here with my co-host Tracy Reagan, creator and CEO of Deploy hub and very busy lady when working with the Linux Foundation. I'm sure it'll come up today. Before I introduce today's guest, I wanna give you a quick update about what's happening here at Textron.

com, so be sure to go and check that out. We have a lot of virtual events happening, uh, predict 2025 is coming up. If this airs after that, you can go out and watch it on demand and I would recommend it.

It's gonna be an awesome virtual event, so you wanna be sure and check it out. com and be sure to tune in every day to Techstrong TV for great shows and interviews. Okay, Tracy, it's 2025.

What's on your mind today? So over the Christmas holiday, I I, you know, I, it didn't do a whole lot, but I would still watch kind of news coming across, particularly around cybersecurity and Space Force because it's something that I'm particularly interested in right now. And this, this article came across about the DOD and, um, you know, tackling Weapons cybersecurity and it talk, it talked about, you know, that there's work that's being done to address cyber threats all the way down to like code level.

But something in that really bothered me and it said, let me see if I can, I'm looking at the quote. Um, basically it said that they know that there are, uh, vulnerabilities out there, but they're willing to take the risk not to address them. Um, I'm not sure why that would be the case.

I don't understand it, uh, because it bothers me that we can do better. And even in weapons security, I, I I feel like there is a lack of real understanding what these vulnerabilities are across the whole spectrum of cyber security. So to just say, you know, the, the risk is there, we understand it, but we're gonna move forward anyway, um, is kind of a bother.

It kind of reminds me of the recent fires in, uh, California, that area, Pacific Palisades that they've been, they've known for quite some time that it's a high risk area for, uh, flooding and fires. Uh, but we did, how much did they do to, to, to make sure it something as catastrophic as a firestorm didn't happen, or how, how prepared were they? So I feel like we've gotten into a, a place, maybe this happened in 2024 or maybe it's always happened that we're complacent when it comes to, um, predictions, right?

Predictions about what could happen in weapons cybersecurity and saying, we can take the risk even though we don't know completely what we're talking about, we're okay with taking the risk or is there something more we could do with protecting something like the Pacific Palisades from Firestorm? So it, it bothers me and as, as we go into 2025 with Gartner predicting a tripling of vulnerabilities, I feel like we've just been bombarded so much with these kinds of threats that were just, we're numb to it. So that's my concern for 2025 and I, I feel like it's a discussion that should be had within all organizations right now about how proactive we need to be.

Yeah, I can imagine reading that drove you bonkers. It was kind of shocking, right? It was like, Yeah, something like Space Force Sa something like Space Force, you know, that a general would say, you know, I accept the risk without any clue of what they're, what I'm actually accepting.

I'm just gonna move forward. God knows what's gonna happen in 10 days. Well, the people who are attacking us are not that complacent.

They're on their toes, right? Yeah. So we have a formidable, uh, uh, component, uh, component opponent out there that we need to be serious about.

Yeah. But we have an incoming president who wants to change his mind again and move space force to the state of one of his cronies. Like he, before, before the last election, he was moving it to Alabama and then Biden said, no, it's staying in Colorado.

Which obviously I pay a lot of attention to living here. And now he's talking about moving it again. So let's not focus on the secure side, let's focus on moving it and wasting a ton of tax dollars in the process.

But that's a whole nother conversation. Yes, it is. All right.

Well we have a really cool guest today. I would like you to introduce you to Sal Kimmich. Is K Kimmi or Kim?

They're both good. They're both Good. I like to say it right though, so well welcome and tell us a little bit about yourself.

Yeah, actually, um, it's probably, I'd love to dive into a little bit the commentary on the DOD. Um, so I have a pretty unique background in and with open source in that throughout my career I have inhabited almost every profile of an end consumer that you can map. So I have been an consumer in a federally funded program between both the US and the uk.

I have been a machine learning engineer working within the DOD. So my first contracting role in DC was with the Missile Defense Agency. And then I moved to go work with the US Air Force, their Kessel run software incubator.

I then only left security clearance because I got married and moved to the uk and I did ask, can I work remotely for this skiff in a foreign country? And they said, no, obviously not. We read the contract, you know how this works.

Um, so I jumped into for the first time the corporate layer of open source, uh, which is generally what most people get exposure to if they're using, using advertising or marketing to understand it. That's the only layer that they'll ever experience. But it's really, really important and we probably should dive into why they would accept vulnerabilities, um, in the DOD specifically, uh, because it's changed a lot in the last five years.

And I think that's really positive the ways that it's changed. Um, so if you are an end consumer of open source as a federal developer, there's a couple of really interesting things. So number one, you're never going to upstream.

If you upstream once onto the thing that you were consuming in the last couple of years, you would not just immediately lose your job, you would lose your security clearance. You would never have a career again, right? And it's not one or 3% of open source consumption.

That is specifically in this case, federal. We're not just talking about the larger and global government consumption. That's a different number.

And that varies particularly by the European country that you're dealing with. And they've got government style OPOs in order to be able to engage with it. But I had someone come up to me at a conference earlier this year, it was someone really early in the career, and they did ask me this question, what percentage of open source consumption do you think is federal?

And I was sitting at a table with a color, couple of other open source leaders and I said, Ooh, it's literally impossible to know that answer given the design of the system. But I would estimate somewhere between 30 and 35%. And then the only reason why I'm willing to say that publicly as something slightly more than a conjecture, even though that's all it is, there's no stats.

I then turn to someone who works in a major corporation that I know has not just a general osbo, but a specific federal osbo. And they did not speak a word, but they did give me odd and a shrug as if that is about correct. Right?

So for every two of the developers that developers that you're thinking about consuming it and upstreaming to open source, generally there's one that is consuming that information and has to have alternative pathways of communication, mainly regulation in order to make sure that those things are secure. And I think this is really, really interesting when it comes to security vulnerabilities of the supply chain. So the first job that I ever took coming out of security clearance consumption and coming into the open and general corporate layer of production and open source was specifically sonotype.

I did that because I think that they have a really, really interesting and pretty direct approach and engagement. They are really focusing on making sure that they can secure that supply chain or that end consumer class. Now, I think in order to not be so afraid of vulnerabilities as they exist on the internet and on platforms like GitHub and GitLab, you have to understand that all of these things are built over kernels.

And there are many different kernels. I've mostly in studied and investigated the Linux kernel. There are other kernels as well.

And even the Linux kernel is not a single kernel. There's about seven of them that are really, really important. There's three of them.

There's like the main line, the main kernel, which most people generally use. And then there's a long term kernel of which they're very, very sincere in making sure that no vulnerabilities come into place. And then number three, when you're dealing with vulnerabilities and open source, you have to become extremely familiar with understanding the zero day marketplace.

So when there is a critical vulnerability that has been observed and been highlighted in the days and sometimes weeks before, a zero day, a zero day just literally means you have zero days to patch zero days. That's what it means. It is bad, it's immediate, and it's pervasive.

Um, and so when you received a zero day vulnerability, you have to understand that all of the work has already been done to secure the critical infrastructures that you depend on. Now, this is not just the DOD, these zero days and the work done before the zero day hits. Public and corporate are protecting things like major cities, water filtration systems, those largely run on things like Kubernetes these days.

So I think that's really interesting to dive into and to to consider. It's a very different world of open source. Um, but it's increasingly important.

And the nature and the style of leadership within the DOD has changed. This is not so much due to the leadership in the executive branch. They are separate, but it has changed because of one very, very specific condition.

Uh, this is the fact that generally, depending on the country that you're dealing with, it takes exactly four days of unlimited assault onto a foreign territory before you go into a condition of what is defined as protracted war. When you're in protracted war, you begin to engage in a very, very different series of process, specifically in the chains of command within DOD, so that you can be highly responsive to it. So that's had an impact on the way that open source interplays with it.

But also there's no difference in the actual nature of playing with the human gen like DTUs that is open source. You have to use it because it is where the progress is made. You have to use the intelligence of the commons in order to get the right answer.

And then you have to set up additional processes to make sure that is maintained as secure. Um, so I I really enjoy watching that space and I also really am now getting to watch it from afar. 'cause I'm absolutely just engaging in the corporate layer.

Um, which doesn't typically have this kind of insight once you're in security clearance, unless you lead the country, you're probably gonna be in security clearance the rest of your career. But in the, uh, Gartner, uh, report, I, uh, I only read snippets from it, it said that 58% of they said that code level vulnerabilities would probably triple with about 58% going after government and cyber infrastructure, right? Are, you know, our utilities, our, uh, healthcare, the, the, the infrastructure, the technical infrastructure that we depend upon, that's where those vulnerabilities will be targeting.

Um, and I, you know, I, it would be a curious thing to be able to get an SBO m from every single one of those cyber, uh, kind of infrastructure teams on the code they're delivering and look to see exactly what's what, what open source packages they're consuming, because those you would think would be the most then critical ones that we should be monitoring. You know, and at least minimum require for those teams to have an open SSF scorecard, right? At minimum to show that they have some commitment to adhering to security policies.

So it's a, it's a, it's an interesting topic and I think that, um, there's part of us, and I'm reading this really interesting book, book called Sapiens and it talks about how we um, as our brain kind of developed, what drives us, this is a weird one, is gossip and fiction and it has for thousands and thousands of years. Sounds About right. I know.

And we will believe anything we choose to believe, right? So it's easy to say, that will never happen to me. It's easy to say, I can excuse those risks 'cause I don't believe it will ever happen because we wanna believe in fiction.

And if somebody tells us we're okay, even though we may know the data shows differently, we're gonna believe what we want to believe. Really interesting right now. Now Sal, you have a PhD?

Uh, yes. Interesting story. I don't, but I can explain why.

So, um, so I, most of my undergraduate training was funded by, uh, the National Institutes of Health. And it included both a total consumption of my cost. So it included everything down to my rent and my healthcare.

And then I was immediately positioned to do an accelerated PhD between the US and the United Kingdom, specifically working on real time signal processing, um, for medical interventions for the human brain. So I have this great and interesting background and one quick note there, if you can get one of these unlimited, uh, government funded undergraduate degrees. I went and I checked the contract that I was signing and it said, we will pay for all of the classes that you need to complete your degree.

And I said, wait, is this limited or is this unlimited? And I found out it was unlimited. So I in fact left my undergrad with two majors and two minors because I didn't have to pay for them.

I could just pursue it as true education, much more European style. So I got a degree in cognitive science with a focus on neuroscience where I was doing all of my statistical work. And I got another degree in political science with a focus on public law.

I really enjoy. And I find it very interesting to look at history from the perspective of codified law because it gives you much more information about who was in power, how is that power titrated and how is it maintained or lost. You can do that by analyzing law much better than you can by sociology.

Um, but then I jumped into this accelerated PhD, so it was a three year minimum. I already had a first author paper route. And if you wanna look at anything from my security clearance or my academic background, just don't search sal, search Sarah, SARA, I go by Sal because I asked mechanical Turk what three letter moniker was easiest to remember and signaled authority.

And then I used that in order to enter open source quite literally. And when I, when I look at gender and pronoun dynamics, I really, myself, personally don't care. Any pronoun said to me with respect will be treated with respect.

However, generally if it's in writing, I'm going to prefer they them because I don't want to be indexed into a specific profile that could have a bias and an algorithm. And 100% of the time, if I'm pursuing a promotion, I will request that we use he him pronouns. Not so much because I believe there's gonna be any bias from individuals that have previously worked with me.

But because it's very likely that there's an internal system that is relying on an algorithm that probably does have bias. So let me just bias it in the right direction for myself. But here's why I don't have a PhD.

It's a great story and it comes from a very good mentor. So I had two different mentors. I had one at the signal processing lab at the National Institutes of Mental Health in dc technically Maryland.

Um, and then I had another mentor who was the head of the art and sciences, uh, section of the University College London, who was generally just there for life advice. And, uh, I had put together a online course that taught about a thousand people how to, uh, put together a machine learning pipeline specifically for brain imaging. And if they completed that and they did a peer review style, uh, or prepared for peer review style paper, then I got AWS open source to fund the credits for them to be able to complete it.

I got that done, I put that out online and I immediately started getting inbound requests for jobs. I turned most of them down 'cause I didn't find them interesting. But there was one job that sounded very interesting because for about four months, the CEO just kept on calling me up and we would have discussions about potentially what I would do if I went into security clearance.

'cause I was not interested and I had to be convinced. Um, the CEO was also previously a, uh, a, uh, fighter jet pilot. So very interesting because they were leading based on the profile that is impacted by the end consumption of open source, right?

Very serious. They know that if they get this wrong, right, if we mess up this vulnerability chain that will result in a death, right? So sincere and that kind of leadership style is much more available in systems outside of the corporate space.

Um, and I always look for it, but the reason why I don't have a PhD is because my advisor on the UCL side, I said, okay, unfortunately I really do think there is a job here that I am inspired by and would really like to do. And he said, Hmm, how much money are they offering you? And I said, this much money.

And he said, oh, okay. If money matters to you, I need you to know that that's more money than I'm making right now. I said, I, I think that matters to me.

And then he said, okay, you know what? Go do this. Go do this for a year, 365 days from when we stand down your PhD research.

I want you to send me an email and let me know if you wanna come back and finish. And, uh, I remember the day, 'cause there were moments in and out of my first year of getting involved and stood up in federal software production where I didn't know if it was the right fit for me. And, uh, but it happened to be that on day 365, I was working remotely in Barcelona that week.

So I wasn't producing code that week. I was just attending internal meetings. You cannot produce code outside of a skiff.

But I was doing a, like semi vacation working on a beach in a foreign country. And I thought to myself, I can do this while making more money than I would make in literally the highest leadership position that I could ever possibly get into in open, in, uh, in academia. Uh, so yeah, it's just because money mattered to me and because I had been able to raise the signal on all the things that are important to a corporate producer or to a security clearance producer.

Can you demonstrate that you can do the work? Yes. I already had a first author paper out, so I didn't really need to wait.

I had already gotten it done. And then number three, can you excellently communicate and propagate not just your understanding of the topic, but the ability to actually do the topic to other people. Now if you have those three things, it makes it very easy to get a very, very good and interesting job because there are so few people with that combination of skill.

And um, yeah, sometimes I fantasize about going back to academia, but I just cannot pull myself to do it. 'cause it used to be that I had to be in academia 'cause I needed access to supercomputers. And I really particularly love the supercomputer at NIH because if you work in this space, high performance computing of any type, you know, that our clusters are called, uh, bale wolf clusters, but not at NIH.

We named them bio wolf clusters. And I forget overthinking that. I I just love that.

Um, but, uh, you've had A very interesting journey then into employment as a woman in tech. You know, it is so many avenues and I don't think we've heard this avenue before. You know, that you Yes.

You basically did it. And the, um, the idea of getting, basically getting your education covered. Mm-hmm.

That's amazing. So how did you find out about that? Did, did you just stumble across it or did somebody point you in the right direction?

Yeah, well I had very sincere financial need. Um, so I was looking for the best opportunity out there. And I got involved in research the second that I got to school, quite literally the first quarter of my first year as an undergraduate, I went to uc, San Diego.

And, uh, there was a professor there that was doing research on the cognitive design of cockpits for Boeing. And I myself am a pilot. That's why I'm always interested and have a portfolio that keeps leaning into aviation.

Um, but uh, they had shown us some transcripts that I just knew could not be correct. 'cause you have to use alpha numerical when you're talking to a, uh, a control center. And I said, Hey, I think I can just correct these for you.

Um, and that was how I got involved my first year, my first week of undergrad in research. So it really, and and, and this is true. So when I, you have to be so sincere about research itself.

All of the classes that you ever take at any university that you ever take, you will never be better than everyone else in the room. And there's already gonna be 30 of you or 300 of you. But when you're pursuing research, you have the ability to see if there is knowledge that needs to be re produced, go and pursue that knowledge and then share that knowledge as widely as possible.

So the first study that I was ever published on was on, uh, cockpit design of Boeing seven 30 sevens. And to this day, in my own consulting work, I use that all the time. I typically go and I'll speak to like mid-sized banks or something that has a critical service to it.

And I simply explained to them this, you now exist in a world where you had site reliability engineering and you understood that that was real time. But as we think about cybersecurity and the conditions that we have been growing into, cybersecurity is now a real realtime event. It has to be acknowledged in real time.

It has to be patched and as near to real time as possible. So I go in and I will teach them to use their dashboards like a cockpit combining both SRE and cybersecurity whenever possible. But here's the second layer of that, that's really important, especially if you're paying attention to, say the Cyber Resilience Act right now.

Um, there's something different about aviation than software, and I think these are going to converge. We're going to create a thing like a com, like a compliance crab. It's all gonna look the same at the end of the day as this evolves.

So if you are in a commercial aircraft, you're gonna have a black box. If the thing fails, there is going to be a perfectly preserved audit log that should allow them to understand exactly what went wrong. That is essentially the ask of the CRA.

They need you to have a verifiable and reproducible audit log of your cybersecurity methods and operations. And you should make that as automated as possible and work into your operational design. Um, and I'm super excited to see that.

'cause I think that's really important work. And when I look at the way that compliance and regulation are evolving for software generally for open source to some degree specifically, but generally in the sector, I do think it's really appropriate to go look at the past 50 to 60 years of aviation compliance and understand how similar those things begin to look. I could not agree more.

You just de just described what we've been doing at Artelia and Deploy hub. We used to call ourself the black box of software because the problem is is that the, the, the pipeline itself for every co when we were doing monolithic, we, this argument of didn't hold as much water because everything you did in the pipeline related to that one software solution that you were delivering to end users in one big monolithic ball, right? So you could have a black box, you could see, you knew where at least what, where the logs were.

But when we're fragmented with hundreds of microservices that make up a single application, that black box is a hundred black boxes that it, nothing is, nothing is centralized. And you don't know if they're all, um, living by the same security compliance. You don't, you have, it's very hard to see that.

So centralizing this kind of data in, in the way you just described should be applied to every piece of software that we, we push out the door so we have a full view of it. And it has to be versioned. It can, it's not just for the application at the time that it's executing, it's over.

It's the history that gives us the insights. It's the change, right? It's the change that shows what went wrong.

Mm-hmm. Um, so yeah, there's so much to be done in, in software for this discussion. We recently, this continuous delivery foundation, of course I'm pushing it recently started a new SIG called the CICD Cybersecurity sig that we're really gonna look at models because pro, part of the problem of building that black box is that DevOps engineers don't necessarily have time to go figure out what they need to add to every single workflow.

And this is going to be a manual effort to build that black box. We gotta make it easy. We gotta make a, a model that people could say, here's a an example plugin that I can use.

Here's an example command line interface that I could use to generate s bomb for god's sakes. Something as simple as that. So I'm glad that you bring that up because it is incredibly important for software as we move forward.

Now I wanna talk about your background. You said you were in Barcelona, but now you are in Italy. Tell us what you're doing in Italy with, uh, awarding open source.

Okay, well first off, I think I do a lot of personal travel now because when I was on government funding as an undergraduate, the one thing they would not let you do is study abroad. They'd let you go study at MIT in the summer, but not abroad. And I wanted to see the world.

Um, so, uh, for the last three months I have been here in Kunio, Italy, which is not a well-known place, it's not a very large town. It's sits on a wedge in the Alps. Uh, and it's extremely protected traditionally from uh, like land attacks.

Um, so I came here 'cause I was really interested in this place, which is well-known to people that study sovereignty as a physical location where this city itself has remained sovereign to both political influence and religious institutional influence, which is very unique to Italy. Um, and to kind of just observe that and understand that. So I'm here 'cause I'm doing my own midlife study abroad, but, um, I'll point you right up to the ceiling real quick because you should be able to see it is beautiful masterpiece.

Absolutely. I know, I thought she was sitting in the Sistine Chapel for a minute when she, when she logged in. I'm like, oh no, that's actually a real room.

Yeah. But, uh, that was commissioned by the family, the body family and the 17 hundreds. That's their crest right behind me.

Um, and uh, I came here specifically because, you know, I've, I've got insight into the government layer, government consumption layer. I've been working in the corporate consumption layer. Um, but there's something that everybody forgets and it's that open source is also just incredibly fun.

Um, when you look at vulnerability, sustainability, maintainability, you have to recognize that these are all building blocks. And some of them are created specifically to be supporting critical infrastructures. Those are well protected.

Those are well maintained. They'll be sitting in something like an antitrust. But there's a lot of one-offs, really interesting things that are produced and open source that aren't meant to have a general audience.

And if they are, it's a very small audience. So we're doing a series of awards. I'm working with Art Farrow on this and I'm waiting for whatever his videos come out to be.

'cause I said the one thing I'm not is creative. You do that part. But, um, we're doing a series of awards based on every single Greek muse and we're gonna go find the open source, either project or committed commit, uh, community, um, that really aligns to those values.

Are you working in science? Are you working in art or music or in historical preservation? Um, if you're doing something like that in open source, I think it's really important to remember that that whole world still exists.

And then to also understand this, um, it's very, very true that there's an absolute alternative to burning out in anything. And you can call it something very simple, just call it burning in. Like stop paying attention to your retention statistics at a corporation.

Pay attention, right? If they're about loss, really pay attention to what is it that you're doing when you're doing it right. Um, and one of those things is allowing people to have and to develop their passions with technology.

So I'm using this opportunity as a time to help to highlight people that are genuinely showing something that is so passionate that I find it interesting and inspiring and worth sharing. I have one last question before, 'cause I know we're gonna run outta time, but I really have to get this question out because if there is somebody who's watching this who is an undergrad, which I hope they are, how did you find your research project and was that a government grant that the, uh, uc, San Diego was involved in? Yeah, so I, I mean, honestly I started applying for funding in my first year.

Whatever I could find, like, is there an associate, so you Yourself were looking when you were applying for funding, where were you applying to? So I started at the institution and then I started looking, uh, specifically into my, uh, degree program. And I started going and getting the professional level education that you need and pursuing external organizations.

So two things that really helped there. I was working with the cognitive science department and they had a bursary that was available exclusively to graduate students to support their research with training. Okay.

It's not exclusive if you go and ask. And so I went in and I got some funding to be able to pursue independent training. That's how I got connected originally with the Martino Center outside of, uh, or in Boston.

And the, uh, like brain, uh, and Cognition Institute from MIT. Um, so I went and I pursued education that was at one level higher than what was expected for me at my level because why would you wait to get it done? And then number two, I just break through whenever I see an arbitrary gate being kept closed and I will ask the questions, what are the conditions by which I can open this gate and I will ask it to the person who has the door locked.

Um, one of those conditions was very important to me. Uh, so I really wanted to join the association for the Scientific Study of Consciousness because I was studying real time interventions using FMRI brain imaging. Um, and I was told at the time that, that's great.

We'd love for you to participate in our student committee, but that's for graduate students. Now, one year later, I show up to the same person who helped me in my PhD as well. I show up to the same person who had that door locked and I said, hello, I am still an undergraduate.

I have full funding, not just for myself, but for my research. Does that satisfy the condition of being a serious researcher in this space? They said, yes.

They let me join. I was immediately working with the professionals in that field. Um, so go and look at gates, see if they're actually closed, see if you can get them open.

And if they are closed, make the conditions discreet, get them in writing and see if you can fulfill them. When you're fulfilling those conditions, great, you're done. You're set.

Now there is another thing that's really important. I pursued biomedical research. So in order to do that with human subjects particularly, you have to be working under something that is called an IRB form.

So the in Institutional Review board, um, I have a curd of many undergraduates old in one of those themselves under their name. But I was pursuing independent research. I was creating my own research designs and then using the funding to get the data done and then to produce those methods.

Um, and that worked. I just didn't tell myself that any of those were conditions just because there are arbitrary and they exist to satisfy a societal expectation of when you'll be ready to produce intellectual property. And if you're pursuing open source, you're ready already.

It's why you're here. Um, but one thing's really important because it's mentorship, and I know the best mentor that I ever had in life was Dr. Lisa Eer.

Uh, uh, Dr. Lisa Eiler, uh, from the VA hospital in San Diego. And I remember going to her early on and I was stopping around and asking every single lab that I went to, do you think I can get a first author paper done as an undergraduate?

And I had some people actually laugh in my face when I said, that doesn't matter to me. That's just a closed door. I'll knock on the next one.

But I went into her office and she said like five words to me that were so powerful 'cause I had never heard them before. I've been well supported, well coached my whole life, but no one had ever just said about something I wanted to do. You can and I'll help you.

So simple. But that's not something that women hear. Women versus men are much more likely statistically to hear a no when making requests around funding, when making requests around promotions, all of these things.

Um, and she just recognized something burning in me, the fact that I was really burning into consciousness studies and to modeling and interacting with consciousness as a computational design. Um, and she fully, fully supported me. And I will always be grateful for that.

And it's something that I make sure to say explicitly to anyone that I am mentoring, find out exactly what it is that they wanna do in life, see if I can support it. And then I do everything in my power to do that. Sincerely, You can and I can help you.

Those are very, very powerful words, right? Mm-hmm. That's amazing.

Absolutely amazing. It, you know, we hear, uh, the journey of women all the time and mentorship is always at the poor of very successful. Absolutely.

Mm-hmm. That's what we hear. And it's not just mentorship from other women, it's mentorship from men as well.

Mm-hmm. Yes. Mm-hmm.

Yes, absolutely. Men are part of the solution. They're so much part of the solution.

Yeah. They're Also part of the problem, but that's what Yes. Yeah.

Yes they are. And I don't know how much time we have. Yeah, We're pretty much there.

You ladies. Um, Can we just ask a question? Yes.

You ask your question. Recommendation. What is a book recommendation?

Tracy? Always add recommendation. Uh, so there's two books that are super important.

Um, actually I have one of them sitting right over there. It's, uh, cybersecurity for Generative Systems. It's very good.

Um, another book that you should read if you're really interested in understanding the state of cybersecurity is, uh, the Cyber Deception book. So there's a Cyber Deception 1 0 1 book. It comes out for, um, FinTech services, uh, about every two years.

And it basically explains how you create honeypots and artificial systems in order to observe Adversarials attempting to get into your system without letting them do it. Um, that still is incredibly important work, and it's one of the most evolving areas of cybersecurity because now it's just agent on agent artificial intelligence. Um, but I do wanna jump back to one thing that I think is really important to consider and think about, especially at the corporate layer for vulnerability, uh, uh, analysis and awareness generally.

There's two approaches to it that we can take. One of them is the one that most people are currently taking, and it's basically doing a scan semantic analysis and identifying either the vulnerable project or the vulnerable code snip, uh, that's incredibly computationally extensive. And it may also encourage people to be pursuing a vector of com of compound vulnerabilities.

Always remind people that log four J in itself was not a vulnerability. It was a compound vulnerability when in place with j and DI that made it harder to catch for a while. Um, but there's another approach to this that is entirely different and it is using category theory in order to find those conditions.

So applied category theory is a way to begin analyzing vulnerability. Uh, and it would allow you to find categorical conditions and to avoid not just a single code snippet, but to actually be able to see, and when I say categorically, it means we have set condition A feeds to set condition B feeds to set condition C. We now know exactly how many projects have that logical, substantial backend, and we can remove that vulnerability, whether or not it looks the same, we can remove that logical compound across languages, across semantic complexity.

That's a direction that we absolutely have to go into. And it's not something that is a far out there idea. There are some r and d spaces that are looking into this.

And you must understand very importantly that this is an idea that particularly the US pays attention to NIST organizational design, all of the things that it gives down for us to be compliant to are, and to be a CT compliant. So if you're interested in this space and you wanna understand and start thinking about it, then go to the top of the supply chain. Well, mental chain of understanding.

Can we categorically provide the best solution possible? We're gonna do that with applied category theory. It then comes down, it gets right now interpreted into a semantic language that we're communicating out.

And that's where there's a lot of lossiness in communication 'cause it's human to human communication. But if in the next 10 years or so, and I always say apply category theory is the answer to everything, we just haven't found it yet. And that is so true.

Um, but if we can close that gap, uh, we're gonna be able to avoid those conditions, not just in the current reality of production, but also moving into a quantum compute reality where they also will be able to have a much more efficient way of scanning if they are doing it as an adversarial. So we want to make sure to categorically remove those logical conditions moving forward. And that I think is the most interesting area of cybersecurity Right now.

Well, thank you so much. Um, that's a great place for us to wrap today, and we really appreciate you being here. Tracy, you got anything else before we wrap this?

I'm just glad she mentioned Quantum. Yeah, I know you're into that too. All right, well, thank you so much for being with us today, Sal.

And thanks to our audience for joining us for another, um, fun filled and very technical episode of, of Techstrong Women. Um, we're excited you were here and as I said, keep watching Techstrong tv. There's a lot of lot more shows to watch today, so stay with us.

Thanks again. Have a good day. Hi everyone.

Welcome to another episode of DevOps Unbound. You know, uh, we were reminded by our producers right before going live on the show that the very first DevOps Unbound was in August of 2020. So we have been doing this now, going on four years, two months we're, we're into our fifth year of DevOps Unbound.

And that to me is just mind boggling. You know, it was, I don't want to take too much time away from what we want to talk about today, but DevOps Unbound was originally the brainchild of, at the time the CEO of T Tricentis, my friend Sandeep, Ari, and myself. And we brought Mitchell in very early.

And at the time it was the Tricentis CMO, Brent, and I forgot Brent's last name. And we, we concepted out this idea of doing a bi-weekly video podcast series that would explore all aspects of DevOps. And even though Tricentis, and this was point of years ago, right?

Tricentis obviously very focused on continuous testing and testing, but Tricentis folks thought it was very important that we explored the full spectrum of DevOps. And over that time, we certainly have. And not only that, but we've stayed current as New forces.

And, you know, new technologies came into the DevOps space, right? We're gonna be talking about one of them today, ai, but, you know, I don't want to say attaboys or Pats on the back, but man, four plus years riding. Congratulations to our Tech strong team and our tricentis partners who co-produced this.

A special, special shout out, Jody, Ashley and Ly nor who never get on camera, and I'm not gonna force them to get on camera now, but these two gals, you know, week in, week out, month in, months out, they, they pick the, the topics they source the guests, they get the abstracts done. They, they make the trains run on time here. So shout out to Jody and Lanier for all, all of their work over this time.

Let us now though jump into today's show. Today we're gonna be talking about AI governance, very timely topic in DevOps testing and everything else. Uh, before we jump into it though, let me introduce you to our amazing panel.

First of all, she's a frequent guest on Techstrong events. She's a friend of Techstrong, as a matter of fact, she's in the Techstrong. Actually, two of our folks here are in the tech strong gang, but she is the CEO of Deploy hub.

And, um, also an open source board member extraordinaire. Tracy Reagan. Hey, Tracy, how are you?

I'm doing great, Alan. How are you today? It's great to be on this conversation.

You know, I'm always, you know, ragged about ai, so I'm glad that you invited me to this one on this call. I hope that we really kinda dig into the technical standards because that's the area of interest I have, because if we have technical standards and potentially we can start building those standards into our DevOps platform. And I say that because I know that we, you know, we're, we need to build security into DevOps platforms, and we have this on our tail building some sort of transparency and accountability into the DevOps pipeline for, uh, for ai.

So a huge new area, and it's great to be on this call Joining Chasey and I from Tricentis. He's, he's been on a few times before, over the last four and a half years. Martin Klaus.

Hey, Martin, how are you? Hey, Alan. I'm doing well.

Thanks for having me on the call again, appreciate it. Uh, and it was great to see you in person a few months ago. Um, yes.

So I am responsible for, uh, customer engineering at Tricentis. I've had multiple roles including product marketing, product management, and one of the things I get to do in my, in my role is to actually work with customers who are, um, defining their own AI strategy. And so I, I'm involved in a lot of conversations where customers are trying to figure out how, how do I introduce AI in my organization?

How do I make sure it's safe, it's secure that the data stays in our data center and doesn't get leaked, uh, out on the internet. Um, and also how do we test AI technologies, uh, so that they're safe to be used in our environment? And so that's a really fascinating, uh, conversation to have with customers of different vertical industries and different segments because they each have slightly different requirements, but they're also together all looking for the same thing.

So I'm looking forward to the conversation today and talk about regulations and governance. Fantastic. Thank you, Martin.

Um, next, she's also a Text Strong gang member and frequent text strong, uh, event guest, our own Hope Lynch. Hey, hope, how are you? Hi, Alan.

Uh, so happy to be on today. And funny enough, Tracy and I had a conversation recently where we started digging into, uh, AI and some of, uh, what we saw as the issue. So I'm so happy to be here today to talk about governance because it's, it's a critical topic.

It's great to be able to elaborate on it for a lot of people. Absolutely. Thanks, hope, and it's great to have you here.

Thank you. Next, well actually last, but certainly not least, he's my co-host all for every single DevOps Unbound episode we've ever done. He's also the CTO here Ad Techstrong, as well as VP analysts for DevOps at the Futurum Group.

Mitch Ashley. Hey, Mitchell, how are you? I'm doing real well.

It's a, it's an interesting time to address this. Dora just came out with their report and kind of put the pin on the AI donkey saying that's causing us to be not as efficient, which I don't believe that's really happening. But, you know, so getting into governance, we are trying to figure out ai, so it's a good time to be talking about governance, technical standards, regulatory, how do we do this, right?

So, so folks, look, it's been, I think about two years now, right? Since chat GPT, just about two years since Chad GPT burst on the scene, I think it was November or two years ago. And, um, and it's sucked the oxygen out of the air of every conversation we've had on tech sids just about it seems we're always discussing it, and we've seen the gamut of this is gonna be the greatest gift to mankind ever, right?

I'm reminded of that old movie where they have the, the book to serve how to, how to serve, or how to serve man. Mm-hmm. And it's a recipe, um, verse versus, you know, the, we must put the brakes on AI and stop it because it'll be the death of us all.

AI will fight this imperfect something out of a Star Trek, be jerk kind of episode, right? And, and seek to discharge humankind. Um, I'm a firm believer that progress stops for nomad.

And, and I think that's certainly been the case with ai. It has moved ahead, full speed ahead, damn the torpedoes. However, in typical fashion, we have seen some governments, many in the US unfortunately, but some governments have started to try to put some governance.

That's what governments do. And governance, um, you know, around the use of AI and some private industry public consortiums are trying to establish rules of, you know, what's ethical, what's right, what's not right? Where does it tread on humans and human rights?

Where does it help? You know, what, what's the right, what's, what's wrong? And right here to do.

Obviously, some things are clearly wrong, right? I mean, using it to, for mal, you know, ma living mal, I always mispronounce that word. Using it for bad purposes right?

Is never a good thing. These deep fakes, and we're seeing it around election season now, you know, prevalent use, no one thinks that's the right use for it, but there are right and wrong. Um, Tracy Hope, I know you guys, both guys both have strong feelings on this hope.

I'm gonna let you go first, if you don't mind. What do you think? I think, uh, as far as governance, one of one, one thing that comes to mind as you were mentioning, um, you know, the eu, they have the EU AI Act so that they can have comprehensive enforceable standards.

There is work, I think also happening in Canada. Australia, they're sort of looking to see what's happening, uh, in the eu, but in the US I think it's gonna take a little longer. But I do also think if there is a company that can say that they are taking these steps to have critical oversight governance, so that, um, you know, it's not a black box, maybe it's explainable, they can ensure that there's no bias.

Uh, there'd be a little bit of a first mover advantage there. I know that, uh, some financial organizations, I think Capital One is one that is taking steps in that direction, but there is a long, long way to go, uh, to get AI governance across industries and to be something that I think is pretty commonplace. Arnold Thoughts?

I, I wanna point out that there, there is work being done by the US government as well. We have the Yes, true, true. We had the Algorithmic and Accountability Act passed last year.

Mm, that's right. And they, and they kind of addressed many of the same things that the, you know, the EU is trying to address, which is this idea of accountability, really. Mm-hmm.

Um, you know, uh, particularly around what, and all of these, all of this governance is really just around high risk AI applications. Mm-hmm. You know, surveillance, um, hopefully medical and warfare.

Mm-hmm. Uh, but we, the, the US is MA is making some progress, but I do feel like a lot of the governance has been turned over to the, um, European Union, uh, ever since the GDPR we're still like, you know, they're doing data stuff, so we don't have to, but we, but the US government is still, they're looking at it that that bill was passed, and, you know, it says, Hey, you've gotta make sure that you're developing, uh, fairness and, and, you know, minimizing bias and promoting transparency and mitigating any kind of discriminatory, uh, discriminatory outcomes. Mm-hmm.

Um, you know, all of this, when I think about it, and it was kind of morbid, but I think about the opening scene of robocop, right? In Detroit, And recently there was this insane article that came across and, um, tech on text, uh, crunch about Silicon Valley having a discussion if AI weapons should be allowed to kill people. Oh, wow.

Okay. You know, it's like, duh. No.

Well, that is, I think that's the first lore of robotics, right? Yes. Mm-hmm.

Yeah. Asmas last asmas lot of robotics. And that's what that scene went from.

The robocop was Right. Be the robot goes, and, and, you know, slaughters the entire board. Mm-hmm.

So, you know, so I'm sorry. We have, why we have, well, we have governance, um, and it's only around these, these high risk systems. I feel like we have a long way to go to start trickling it down to all systems that are being written and being able to take these technical standards that like NIST is working on, and apply them through the DevOps pipeline to start actually working to make sure that they are transparent and that some of these tools now can be applied, uh, to the process mm-hmm.

Opening, I get it. Of, uh, of Terminator one too, by the way, crushing the skulls of humans. But Terminator robots, um, sorry, Martin, I was just gonna jump in and, and say to me, it's fascinating when something comes along that's akin to security, it's akin to data that gets this governance, and how do we do it properly so that it doesn't escape from the lab or escape from the application data doesn't leak out, um, or in this case, do harm.

I mean, you know, other software could do harm too. Algorithms could as well, you can argue whether social media does that, but AI is really jump to the top of the list of what are we gonna do? What do we have to do to, to secure this?

And also make sure it's not used for nefarious purposes. So, Yeah, sorry. I think it was interesting that, that you see folks like Sam Altman or even Elon Musk, uh, even asking governments to step in and help create some rules or guardrails because to see the potential of where this technology can go, I'm not sure we're quite at the turning level yet, or robocop, what have you, but because it's going to be an evolution, uh, but the evolution is happening really, really fast.

And I think we're already a little bit on the back footing, uh, with, uh, some regulations as it relates to use cases outside of business. Uh, for example, you know, uh, kids in school using AI systems to do the homework for them, uh, which, you know, now puts education systems at, you know, the back footing in terms of like, how do we deal with the situation when we require essays to be written as part of like entrance, examin entrance requirements for universities. And now anybody can just generate an essay in the voice of Ryan Gosling or whoever, uh, you wanted to in imitate and, and sprinkle in some grammar mistakes and punctuation to make it appear as if it was original authentic work.

And so that's just one example of where we're thinking completely new on chart territory because, uh, we don't know yet know how this technology can be used and applied. In some cases. We're thinking, especially in the business context and the kinds of customers that I'm talking to, um, the, what I'm hearing is that there's definitely a need for transparency.

And I think that's one thing that we could sort of check out very easily, because companies are asking for where is the data gonna be? What is being processed? What happens in transmission?

What about encryption? Um, who wants to the results? Who wants to, you know, the models, what models are you using?

And then how do I customize it and make it my own, because I do not want my data to get leaked out on, on at all. But it's a very broad field, and I think the way, um, for example, the European Union and or what the House White House has also been proposing is a good one to say. Let's think about this from a a point of view of, of risk.

What is unacceptable risk? As you mentioned, Teresa's surveillance, um, or, uh, protected groups, uh, in, in society and other things that, or medical, uh, applications. There may be areas of unacceptable risk where, you know, we, we need to actually have some laws in place to, uh, to control those things.

And then you work your way down in terms of high risk, low risk, and other use cases where, you know, uh, it's, it's a less of an issue, but this is a extremely complicated, important topic that affects all aspects of, of life. I, I, I don't necessarily disagree, but you were talking to someone who's from the generation of no, no calculators allowed in the school test, right? I, how many did, did you, were you allowed to bring calculators into your test?

No, no, no, no. But kids today, they bring scientific calculators in. They no longer have to worry about doing those equations and figuring pie and all of that.

It's all there for 'em. I think we're gonna come to the same thing. I, I think, and call me radical, but I, I think when we get to new technologies like this, our, our part of our makeup is to think, go slow.

Go slow. This could have repercussions we haven't thought through. And we know, quite frankly, that that never works, people, right?

If you outlaw guns only, guns only outlaws have guns, people are, they're still gonna be a segment of the market that's gonna go as fast as they can. Um, I think almost these governance breaks, if you will, these governors on the use of these technologies is to give our society a chance to catch up, a chance to get acclimated, a chance to get comfortable with what this, what these technologies can do for us. And I think if we recognize that, we could look at them in a whole new light.

But I also think it begs another question of how do you test for AI governance? Right? I mean, Lauren, I'll throw it at you.

You're, you're ous, you're the worldwide leaders. How do you test for AI governance? How, how would you, you know, who's, who's minding the, who's watching the watchers here?

Yeah, I don't think necessarily that it's possible to test for governance when you need to be able to, uh, it's more about transparency and really sort of, you know, like in the case of security, right? Like companies are used to, you know, expose and report and, and, uh, sort of showcase, uh, what sort of, uh, security policies and governance they are implementing the products. And, uh, and we have standards around that.

If you look at SOC two, and if you look at many different encryption standards, um, I, I think on the security side, we have gone of, uh, have come a long way already. I think that could be analogous to AI as well to some extent. Um, but the thing that I see with our customers that, that we speak with is that like, how do we effectively test an AI system that it gives you reliable results, um, uh, in addition to how it works at the, you know, uh, is it safe to use, right?

So is, is the outcome useful and applicable, uh, and safe to use as well as IT technology safe to use as well? That's where I think we're in also new territory, because if, um, AI models are passed the Turing test, then it's a knowledge system, an expert system that, you know, uh, can involve with time, then it's going to be much more difficult to, you know, come up with prompts or tests and parameters to say, yes, today the answer I got back was acceptable. But what about next week, next month when the NOWLEDGE system has evolved and now it has been enriched with new information or with new, uh, you know, adaptations or weights, learnings that will, will now yield a different result.

And I think that's a challenge with testing that it's no longer a point in time activity. You have to almost like have a, an ongoing monitoring system in place. Um, but you also don't know what you don't know and what you should be testing for that could be exposed through some sort of loophole.

And I think that's what some companies are finding out the hard way. Like if you look at some of the, you know, Canada examples, for example, where it gave wrong responses, now there's a lawsuit and so forth. Um, and that's a challenge I think with testing ai.

What we do internally is we're trying to sort of adopt this monitoring model, uh, where we ongoing, uh, on a regular basis benchmark test and validate, um, and have sort of a, a red team approach as well to say like, how can we expose, um, the models in a way that, uh, the results we're getting are no longer in line with what we expecting to do. Um, unless there's a new way to, uh, figure out, you know, testing of not the testing models, uh, I think, you know, that's going to be the approach that a lot of people will, will need to take as well. Yeah.

A couple of things I would like to uh, insert here. For anyone who's listening and it's trying to, I guess, wrap their heads around, how would I start to do these things? There are tools out there, and you still have to know what you're doing, but you could look at, um, uh, Q Flow, ML flow.

Um, they help, they can help streamline building, deploying, managing your machine learning models, actually at scale, um, open source tooling. So, uh, it definitely, you know, there's always a learning curve for these things, but if you're looking for some tools, uh, that can help you get started and figure out what you should do, um, those are two pretty good ones I think that folks can use. Then I wanna add too, on this topic, um, there's, there's, there's two levels here.

We have testing and we have compliance. So if we look at what we've done through, you know, over the last five years in security, we have done everything from adding signatures to, um, repost scans to things like open SSF scorecard that looks and determines how safe software potentially, um, could be. I really believe that these types of tools will also be created for looking in, uh, how compliant AI software is.

Now the problem with compliance is that the data is often fragmented, and maybe you can look at a GI repo for one particular component to see, uh, how, how compliant that component is. But it is a first step. Compliance is a, it will be, it is important today in security in software, and it will be important in securing and making sure AI software is safe.

So, while testing is important, understanding the compliance levels of the code that's coming across the supply chain is not gonna change. We're just gonna have different types of tooling to make sure that the, you know, that it's, that the model is fair. For example, how do we scan for that?

Not sure. I think IBM has something called like IBM 360 that, that does some kind of fairness check. So, as, uh, you know, hope pointed, pointed out, we do have tools out there, and that's why the technical standards become more interesting, because if we can define what those technical standards are, we can also define the compliance levels that we need to achieve.

And while we have seen a lot of these governance docs, um, from NIST to the eu, uh, talk about some of the basic pieces, we still haven't seen a really clear roadmap for what is compliant, and we haven't done that for, for code in general. So maybe we'll get there. We're trying, but we have to look at the compliance question and how do we track and report compliance.

So the companies who are writing software and consuming these, uh, large language models, have a way to judge how safe they are. You know, in some ways this is, you know, there's compliance against standards. I you have to have something to test against, right?

Which I think is lar largely what we're saying, and there's two forms of it. One is IEE and nist and, and, and regulatory types of definitions of what those standards are, what what you're testing against. Oftentimes though, compliance isn't testing against someone else's standard.

It's testing against your own standard. So a lot of compliance frameworks are all about what is your policy for, for this part of security? Like if you go through a web trust compliance process.

So it's all about, here's what our policies are to protect data, to secure this process to the handling of customer information, whatever it might be. It doesn't tell you what this process should be. You have to define your own.

Then the compliance is, we have validated that we, we have systems and processes in place to ensure that we follow those processes. And if we are in fact following them, uh, my, I have a sense that this is kind of where we're going. There'll be some things that will help guide us, um, and getting some insights to, to what we should be testing against.

I I almost think we're living in this world because it's so wide open and what you could do with AI that organizations are gonna have to publish what their, kinda like their privacy policy is or what their, their, uh, uh, online community behavior policies are. Same thing for around ai. And then do they in fact meet those?

Because otherwise the requirements could be so vast. I'm not sure you could really test against everything and really know whether is this a safe system? Is this a safe ai?

Yeah, I think Mitch, that's a great point, and sorry, um, Joe, you go bar. I was gonna add one more thing. What I'm seeing already is that, um, this is not a simple question or answer already, and it's going to get even harder to answer because, um, it's going to be a more of a blended approach as well.

Like right now, I think oftentimes it's pretty obvious when you're interacting with an AI chat bot on somebody's website, because that's an easy way to have some sort of customer facing interaction. But where, uh, it's much harder to see where AI was used in process is when you have like a composite blended service where AI tools are going to be part of the outcome. And so you can see this an art, you can see this in, in, in creative, um, uh, disciplines.

Uh, you can see this even like in, you know, from a business standpoint where, you know, or, you know, we were just joking about the after for today's topic was, you know, run through an L lab to say like, how, how can we clean this up and bring it down to 120 words or like that, right? So in those situations where AI is used as a tool, uh, uh, as part of a larger workflow, then, you know, how can you know that level of transparency, you know, be sort of transported to the, to the end user as well? Like, how do I know that the, a piece of artwork that I'm purchasing as an example, I'm not a a a deal at all.

I'm just bringing a theoretical example. How do I know this is original work? Um, or it was used or, yeah, obvious was used as a tool in the creation of, of the artwork or in an email or whatever it, what have you.

Uh, I think that's where right now the world is moving where AI services are more, uh, an assistant role, an agent role to basically assist with the creation of, of new work in a much more productive way. Um, and then the, the, the notion of compliance and governance and transparency is going to be even harder to, uh, to say like, do we need to put a label on anything that, you know, has, has been touched, involved in some sort of AI tool in any creation of it? So just to correct my, uh, the name of that tool, it's a open source tool.

It's called AI Fairness 360, and it is an open source. I just Googled it. Um, and it's for ML models, so something you can put in your DevOp pipeline, right?

There you go. That's, you know, there any examples where things have really slowed down, right? I mean, you think about adoption of the internet, adoption of the cloud, social media, um, I mean, the only things I can think recently around AI are Microsoft copilot recall where we got ahead of our, so they got ahead of themselves and someone thought it was a good idea to snapshot your screen every, every second, but not securing that information, the market reacted.

The other is Apple's reaction to the EU AI Act, which is, uh, we don't, basically, I took it as we don't understand your re regulations well enough to know whether we could follow that. So we're just not gonna bring our AI to the EU until we, we feel that we can meet that compliance. It wasn't that we don't like your standard, it was like, we just don't know what it is.

Other than that, there's very few places where some, a regulation has stopped ai, uh, in, in its tracks. And, and I think one of the things we're gonna need is internal. A lot of this is judgment call, right?

And so much of it is we're gonna need the internal board of here's how we're using ai. We're, we're thinking about doing this with it. Are we delivering on the promise to our customers of safety of, uh, protecting their data transparency?

We said we're gonna be transparent while are, we're being transparent enough. There's so much of a judgment call to this that you almost have to have some kind of an internal mechanism to say, no, maybe it's not a gate review that everything has to go through. We can use some tools, like you're talking about Tracy, to do some of those things, but it's, it's like, here's our stated policy of how we're gonna use AI and, and what we're gonna do and not do, and the line's not always clear.

So how do we help clarify when we need to make those calls? I have a more fundamental question, which is, what is the purpose of all this regulation and governance? It's to build trust, isn't it?

To build trust in AI in its use where I think the, the best way to build trust in AI is to use it and see for yourself what, what's real and not, maybe not true, so good. Um, but fundamentally, you having these regulations allow you to trust AI more, right? I mean, Martin do.

The, the fact that the EU has this AI act, it's not even, I mean, it's been passed, but I don't think it actually goes into effect effect until next year. Um, but by having that act, you say, oh, AI's a little more trustworthy, a little safer to use. Now I have more trust in it, because if it doesn't, why are we doing this?

Yeah, I think the, this is a good point because the, the, the what, what builds trust is obviously, uh, transparency, uh, is one way to say, look, you know, there are regulations out there that requires out there, here's what we are doing in order to make you feel comfortable that the technology that we're delivering, the products and services that we're offering are safe to use an environment. And so, for example, uh, just be super practically for a second, at tricentis, we've actually established an AI trust center that you can look up on our website where we publish, uh, our data privacy, our, the, the kinds of technology that we're using, our security, uh, uh, that we're using. Uh, and we're also publishing the, the, the guiding principles that we use now in internal development, uh, to basically give customers that want to know, uh, a place to go to, to find out how do we, how do, what do we do internally and, and, and what do we do?

And these are things that we are going to, uh, update and, uh, keep current on an ongoing basis as new regulations come out. Um, I think there will still be an opportunity for like standards like ISO and others to, uh, create more formal, uh, certifications as well that, uh, vendors like us can, can sort of, um, uh, can achieve and publish it as well, just like Tracy, as you mentioned on the security side, where we have a lot of requirements already that, uh, companies can, uh, adhere to. And then also be publishing that and say, here's what we're doing, and if you have more questions, let us know because we wanna work with you to understand what specific requirements do you have that we can, uh, support as well.

And then it's about sort of, you know, showing and demonstrating and, um, and putting, um, you know, the proof in the pudding, if you will. I think this, this question of trust is pretty, is really important. Um, I, I, you know, IE even in tech strong gang, I've said many times, I don't trust an autonomous driving caller yet.

I, so trust is an issue, but part of that is awareness, right? Um, if I have, if, you know, when I was driving a Tesla, I could have put it in auto, you know, kind of an autopilot, but I never did. But I was aware that, that I was making that decision.

Part of the EU Act is says that, you know, if you're kind of at a minimal risk, at minimum, you need to indicate that this is, this is AI generated. So we know in the United States right now, there's, you know, public service announcements going out about the potential of robocalls or AI generated, um, robocall that says go vote at some other location. The person that's listening to that doesn't know that that's an AI generated, uh, phone call.

So, you know, the awareness of the fact that you're consuming AI data is super important. Anything that comes across that's been generated by ai, it should have some kind of a stamp on it. Some kind of a, you know, a watermark that, A hologram or something.

Yeah, yeah. Something like the doctor in Star Chek Voyager or something. Right?

Exactly. Exactly like that. Um, because then we can, then we are aware that we are looking at something that's been, uh, that, that is actually AI and not human.

And that's important. It's very important, actually. One of, uh, one of the things that comes to mind when you say, um, do I get a sense of confidence knowing that these rules and regulations exist?

It's almost like here in Charlotte, North Carolina, let me tell you, speed limit can be 70, it can be 55, it can be whatever it wants, right? But here in Charlotte, 80 miles an hour, 90 miles an hour, pretty much everybody is doing it, and you very rarely see anyone, you know, hold over. So is the speed limit actually meaningful if there is no enforcement?

Right? I think one of the things that will give confidence is, uh, and, and not that you necessarily want to see governments going after, you know, every one, but key stories about, uh, enforcement and how it actually has benefited people and, and made a difference, right? Having the rules is great, but understanding, um, in, in common scenarios and common use cases, real world things that have an impact, um, I think that is a big confidence builder when people can see it and connect to it in that way.

So I will just like rule of thumb, so I could save someone a speeding ticket here. Yeah. In, in Florida, it's a 70 mile an hour speed limit.

And the rule of thumb is they won't pull you over up to 80. Anything over 80 you're subject to get pulled over. Don't take that to the bank, and please don't say Alex in North Carolina, they often say five miles.

Uh, but there, there, there are no, there, there are no, but, But you know what, bringing it back to ai, this is a perfect example where if you have autonomous driving one of Tracy Reagan's favorite things mm-hmm. And you tell, and you tell that AI program, Hey, not to exceed 75, you don't ever have to worry about it because it's not the human who's gonna go as fast as they can. Mm-hmm.

If the AI is told 75, the AI is going to go 75, assuming the AI behaves as intended, and we have trust in it. And that's the perfect example, right? Yeah.

And if there's a bug and it goes 80 and you get a ticket, whose fault is next? This is true too. And are the tickets Now automated?

Because The card, You know, Worried is hooked in the law enforcement worrying. We're all worrying about a scenario that's gonna go away. If you remember back to the future, national Miller Brown says, where we're going, we don't need roads.

Exactly. We don't have the, this gigawatts, gigawatts, whatever. Go jc.

This brings me to another topic around governance. There has to be industry standards. Uh, you know, financial is gonna be different than, you know, traffic control.

It's gonna be different from, um, welfare or warfare or, uh, or, uh, I don't know, surveillance. Every, I think every industry is gonna have to look at it. I think the Food and Drug Administration has done some work in healthcare, AI and healthcare defining standards.

But what will be, you know, so the question is what's this? What are gonna be the standards where the industry itself, right? The AI industry itself.

So I keep going back to compliance. If you define standards and industry specific ones, uh, and then we start figuring out a way to measure the compliance of those standards. I think we're making some progress, but bad actors are gonna do things without pursuing any standards as hope indicated.

How do you enforce this? You know, I have a 25 mile an hour si sign on my dirt road. Do I, you think anybody's gonna ever give me a ticket on that?

No. 'cause some, some neighbor put it up there and I fly by it at 40 miles hour, More than 10. But, but, you know, some neighbor put it up, Some neighbor put it up, right?

But it's, so it's kind of like, you know, it's a neighbor putting it up, the EU iss putting up this sign that says you go 25 miles an hour. Um, you know, do we honor that? And I think that most of us will try when it comes to this topic, but I do think industry standards are gonna be, uh, are gonna be as important if not more important than higher level, you know, federal level government standards on these topics, because they're very different When it, when it comes to governance.

Now we see this, right? They're in different governance issues for the fin, you know, what we call the highly regulated industries, financial and healthcare and, and government work and stuff like that. And, and we've adapted to that.

And as an industry, we, you know, people comply with those different vertical standards. And they're not all industry standards. Some of them are government.

Um, I, I would imagine we'll see the same thing with ai. I just, I just, you know, I have that hologram issue where ethical people will act ethically, but unethical people will almost certainly act unethically and they may not make their hologram have the, the, the, the watermark or the, or the, or the standard, and I guess maybe this is true with a lot of governance, is the good people do their best to do good, and sometimes negligently or inadvertently, they, they may miss a compliance or governed standard, but AI has the ability or the potential, AI has the potential for people who are not ethical to really abuse the system. And I don't know if having AI acts in place, you know, we have to go to criminal stuff.

I, I, I don't know what the right answer is, But, you know, I think even the criminals are gonna have their own governance standards Really With their organizations, right? It may not be a governmental or federal standard, but these criminal organizations, some of them are very large, right? Large, they're, they're gonna have their own standards.

And it, it makes Like la cosa nostros kind of, you know, you Yes. Only go out with your wife on Saturday night, not the, not the girlfriend, but whatever. Right?

Right. You know, sometimes lack of a, a federal, well, a federal standard, I mean, that's where we are now. And states are already enacting their own laws around ai.

Colorado has a law, I don't remember the name of it, but it's basically says, if you're, if you're working with ai, that is high risk, you have to, um, there's some transparency requirements, and you have to have some type of review of what you're doing to make sure that it isn't endangering people. Something like that. But that, that's Colorado.

Who knows what, you know, Wyoming's gonna do California or Idaho. Yeah. So, so we're, I mean, we live in a global world, but if every state has a different policy, different law, that's a complex web to try to build products and use ai, I mean, We've seen this Before.

Think, Alan, your point, your point is, is spot on, you know, this is, the whole process of governance is really around, um, uh, kind of a democratic system between the EU and between all the states and the us. Uh, everybody has to act, uh, in a gentlemanly way and, uh, agree to those standards and comply to it. And, and that's what drives it.

And if, if we don't, um, we choose not to, um, there's very little accountability, or it's very hard to even find out that you're not, uh, complying to those standards. So governance is hard. It really is, uh, especially when you're trying to define governance through a, a democratic process where everybody makes their decisions across these countries and are relying on everybody to, um, be honest, and not everybody is.

Guys, we're over time. I gotta be honest with you. I apologize, but it was an interesting conversation.

I wanted to let it go a bit. I think we could all agree at this. I think there's still a book to be written, or at least a few chapters in how AI governance is gonna take shape here and what its effects going to be in a global marketplace.

And it'll be up, uh, you know, the ethical people, as we said, will act ethically, the Tricentis of the world will try to give people a sense of, Hey, we could test for this, or we can, you know, given transparency, we can show you that, you know, to, you know, be the transparency that it's in compliance or what have you, test for it. But a lot of it's good, I think is the book is still yet to be written. So we're trying, is I, I guess the, the right thing, right, is we're trying, and it, and this is still evolving.

Anyway, Martin, Tracy, oh, thank you guys for coming on our DevOps Unbound episode today. Mitchell, thank you as always for co-hosting. Thank you Tricentis, for partnering with us for these four plus years.

Now. We look forward to many more reminder for those folks out there. This, what you just watched was a prerecorded version of DevOps Unbound.

Every two or three versions, we do a live audience, and you get to ask the questions, and you get to make the comments, and you get to participate in this discussion. And we love having you. So stay tuned for our next live round table of DevOps Unbound.

But until then, this is Alan Hummel for Techstrong Group. Have a great day, everyone. We'll see you soon.

Hey, everyone, you know, with the Super Bowl coming this weekend, is it time to kill all the referees you're watching? Textron Gang. Hey everyone, it's Alan Shimmel for Textron Textron gang.

Happy Thursday. We've got a action packed show for you with some great people to discuss it. Let me introduce you to our gang members for today.

First of all, well, she's not at home in Colorado. I think she's out in California, but she's still joining us on the road. It's good to see her.

Kimberly Bates. Kimberly, how are you? I'm doing great, and I think this first topic is gonna be a Lot of fun.

Yes, it is. First, you know, there's been a lot of people waiting a long time for this, but, um, anyway, Kimberly, it's great to see you joining us from Las Vegas. It's the Dynatrace twins, they're out there for the Dynatrace event.

First, let me introduce, uh, Mitch Ashley, future chairman of vp, uh, for DevOps. Mitch, you're out in your hotel room. I am work working on my tan, as you can see.

Um, a little red from this, the morning sun here, but I don't know what our first topic is, but I'm sure AI can solve it. That's a pretty stock answer these days. Also, joining us from Las Vegas, not quite as red in tan.

I, I suspect he hasn't been out in the sun as much, but he's our chief content officer, Mike Ard. Hey, Mike, how are you? So, I'm well, but I, so I'm assuming Mitch is my fraternal twin.

'cause you know, we, the identical thing's not working. No, well, twin sons of different mothers perhaps, but we'll go with it. Joining us from Texas, is that, is that the, the Valentine's Day stuff in the back there, Ann?

Absolutely. She's already, well, it's not, it won't be long now. Uh, Ann, a whole award.

I think I finally gotten your name there. Nailed it. It only took me three months, but I got there.

Thanks for, I appreciate the effort. Absolutely. Thanks for joining us.

And then, you know, saving the best for last as he sits on the throne in Silicon Valley, our one and only John Editor John Schwartz. Hey, John, how are you? I'm good.

It's happy to be here. Uh, I love the first topic, great timing. Absolutely.

With the big game this weekend. I hope, you know, we, uh, all have your Super Bowl plans set up. It should be a decent game.

I'm not gonna lie. I think I'd like to see Philadelphia win, but let, we'll talk about it later. Mike, what is this first topic we're all excited about?

So, this issue is not just in the NFL, but it's coming to a head because of some, uh, calls made in the playoff games that people are, uh, upset about and continue to be upset about. In fact, I'll tell you how upset folks are friends of mine, I've known 'em all my life, are football fans, have watched football forever. And they're not especially Bills fans or chiefs fans, and, but they are hardcore fans with sports.

They are not gonna watch the Super Bowl on Sunday. They are going to go to a Chinese restaurant and boycott this game. These are people who've been watching football their entire lives, and they're basically fed up.

And they think that, you know, the referees and the umpires, including one in the major league baseball, fired one of the umpire just recently because of, uh, sharing a, a quote unquote account with a, with a known gambler. Who, why would you need to do that? I have no idea.

But John, I don't know what you're hearing, but people seem to actually be crying out for some technology help here. Yes. So we have this fascinating, and I think extremely timely story on digital CXO on the idea of replacing or augmenting referees in, in the NFL and empires and MLB.

So, as Mike and Alan, and I think we all know this major theme or conspiracy theory this season has been that the Kansas City Chiefs are benefiting from calls, especially calls that can't be reviewed, like passenger for calls. In fact, yesterday, I believe this week during a media session, pat Mahomes was jokingly asked who his favorite referee was. He, he kind of laughed it off.

But the point was made, I think what we have here, and, and again, it's a big backlash, um, this week, you haven't seen a lot of Super Bowl coverage. You see mortal about the NBA trade deadline. And I think it's a utter frustration that's lingering over the, not only the success of the Chiefs, which is happens in the NFL.

There's always a heavy or a dark or dark team that people root against that's dominant. But the idea of whether the replay structure or even the officiating, even writ large, has a major problem. And I think that has been something that's been growing in, in terms of debate.

For instance, if a referee misses a call, ostensibly it, the onus is on the teams to challenge the mistake. And yet they're limited by what types of things they can challenge, which leads to these long delays and utter frustration. And I think people are looking at the idea of using ai.

Same thing in baseball with the strike zone. We have so many umpires with so many different variations of the strike zone. So we are seeing robots in the minor leagues starting to call balls and strikes.

And I think there will be a push, I don't know if the NFL will ever accept it. Perhaps they will eventually through their partners like AWS, right? Um, this idea that you, you have a AI assistant of some sort to augment or maybe eventually replace the football officials.

And again, in this story, which is interesting, this has been done in other sports and it's been done successfully. You have, uh, the minor league games with, with robots. Tennis uses the Hawkeye technology to call faults and Lang calls Soccer has something called var, which has been used in the World Cup.

And I think it's really interesting, again, as Mike mentioned, this element of the umpire being fired in baseball. You know, all of these leagues are so dependent, and in fact are embracing gambling to the point where we start worrying about the i integrities of the game. And, you know, it's happened in the NBA and in the NFL.

There've been rumors for years dating back to the seventies about umpire, about referees, making dubious calls. So I think it's fascinating, and I think it's something that'll gain traction. But in the NFL, it's sometimes harder to gain traction on things like this.

But we'll see. You know, I, I have a, I have a a lot of thoughts on this one. And, and I, let me confess, my name is Alan, and I'm a diehard sports fan.

I, I live for my sports. I I watch it, my kids, and it's a big part of my life. Let me first say that.

I think ref's, umpire's, officials blowing calls are part of the human factor of why we let the horses run. As they say, anything can happen. You can get a call, a bad call, a good call, a not so great call, whatever.

And that's part of the game. It used to be very much, it's part of the game. Legalized gambling on sports, as widespread as it is now, has fundamentally changed my love affair with sports.

I don't gamble. Let me, let me say this. I, I'll buy some boxes for the Super Bowl, you know, and then that's, that's like playing bingo.

Um, I don't gamble. However, the gambling and the amount of money that's at stake here is not just from my friend Tony's Uncle Vito, who used to hand out the little sheets to us. Mike, you remember this in New York, right?

Mm-hmm. You used to get a little sheet every day, every week, and you would pick your four winners. And if you bet a dollar you'd win 10 or something.

You know, it's not like that anymore. They've ruined sports with legalized gambling. Where now, right away the referees blew a call.

Well, of course they, they, they got money on the game. They, you know, they, they're sharing an account with a known gambler, right? This reminds me of Joe Namath getting banned from football almost.

'cause he had an interest in bachelor's three, right? Where no gamblers hung out, uh, way before a lot of your time here on, on the, on, on the gang today. But so gambling has put such a magnifying glass on these things being officiated to the umpteenth degree imper, that you, you've taken the human out of it.

You have to take the hu And if you take the human out of it, we all lost something. Now, granted, I love the Hawkeye system in tennis. I go to the US Open every year.

I love watching tennis like that. And that's system has taken, I mean, I don't know, nasty Eli Natasti would be upset 'cause he wouldn't be able to argue with the line judges anymore. Or the, or the umpire.

Remember it. Remember Eli or Jimmy Connors, remember Mac? Mm-hmm.

These guys, no. They put on a show when they argued. You can't argue anymore.

It's black and white. It shows you if it hit or it didn't. And that works great.

Balls and stripes maybe. I'm sorry, John. Yeah.

What were You gonna say? Oh, I was gonna say the one, one thing I, I, I should have mentioned too, and in, and in fairness to the referees in the NFL, the players are bigger. They're faster.

They make plays now that you, we, we couldn't even conceive of 10 years ago. So for the most part, they do a pretty good job. But what's always bothered me about the NFL officials is that many of them, for years, this was a part-time job.

It wasn't like MLB No. That, that was the knock that the NFL didn't pay their officials, like the professionals they need to be. And, and it's not like they don't print money over at the NFL and they can afford it, right.

Compared to some of the other things they buy. But that being said, I think there's a place for AI and automation and taking the official sort of out of it, but it, it varies by sport. And it vari, like for instance, the big thing in the Kansas City game was the fourth down spot.

It was the third down spot. It was a fourth down play. Fourth, fourth down, where Josh Allen, to me clearly looked like he, he made the first down.

And they, those. But those are always judgment calls. Those are always 50 50 at best.

That's what makes it football. Well, and the question becomes, when you have the scramble that we have in football, whether or not AI can actually be accurate or not. I mean, it's, it's a very, very complex, you know, okay, so you've got this pile or what, you know, whatever.

I can imagine a bunch of different kind of scenarios where, okay, so yes, and I hear that if you put a chip in the, in the football, okay, say I got a chip in the football, but does that get the, the entire length of the football? No. You know, there's so many, you know, permutations on how the ball moves and how the players pile on top of each other, et cetera.

I think it would be a big challenge. Although, you know, there's amazing things that they do. So, you know, I'm all For video replay.

I think video replay has added a, a lot of, you know, let's, let's get it right. I mean, in baseball it's fantastic, right? Because, you know, with the manager has within whatever it is, 30 seconds, 45 seconds to say, Hey, we wanna go to the replay.

They get so many replays a game and they do it. And, you know, I would say 99 point, 99% of the time, that replay is definitive, right? He's out say, for whatever, I'm, you tell me there's gambling going on in sports and cheating.

I, I'm just, well, from from it says the man from Vegas there. Yes, Exactly. I hear that happens here.

I haven't, I've not seen any of that. Uh, no, I, I think I'm more in your camp, Alan, of, of, there are certain things that, that definitely could be augmented with AI or, or electronic measurement. But even to your point, Kimberly, about, you know, putting sensors in the ball, there's the ball going over the line.

There's also the judgment, a call of were they down right? When they're piled up on 25, you know, 12 other people trying to push over that line. Um, did their knee touch the ground or, you know, did their elbow head versus the hand?

There's so many judgment calls in it. Um, and I, you know, is there favoritism? Does Mahome get a, you know, get a, get a break?

Well, probably, but so do a lot of people who, uh, do holding call, you know, are holding other players and they don't get called. 'cause it's fourth quarter and it's the last, you know, two minutes of the game or Yeah, that was a pass interference, but maybe not egregious enough. So it it, there's too many judgment things in this to say, let's, let's have AI solve the problem for us.

You know, you notice one thing, you notice the refereeing, I think in the playoffs is, is much better. They have all star teams of referees in these games. And the flow of the game is much better.

They don't call as many penalties, maybe because the teams that are playing are better and they don't commit as many mistakes. But there's a certain flow. And my fear is that when you, uh, apply ai, how are you gonna apply it to a judgment call?

And is that gonna mean that we're gonna have stoppages constantly? I think we have too many stoppages as it is between injuries and challenges. I, I think you would have to turn down the, the ai, you know, uh, right.

Enforcement. You could not watch an NBA game with AI because there would be a penalty call every play. Well, there's Every play football on every play, right?

Same thing. And I don't, but, but at the same time, I don't wanna impinge the integrity of the umpires and referees. But I will say this, it costs a lot of money to go to these games now.

And if I'm gonna go spend that kind of money, bring my family, have the whole thing, and to have the game essentially ruined because somebody's having a bad day, I can't get behind that. We need a different answer here because I need, you know, it's not enough to say, you know, oh, that's part of the game. That was part of the game when it cost me 20 bucks to go to the game.

Now it's costing me $200. I need a better answer. So you're telling Me it must have been, You know, and I get back to where Alan was, what Alan was coming at is the joy of the game.

You remember that? What, what is that movie? Any given Sunday?

Yeah. Oh, that's a good movie. Anything could happen.

I mean, that's the beauty of football is that it's just so unpredictable. I, I mean, yeah, I think the others are too. But there are just so many weird things that can happen on the field and everything else.

And that's what makes it so exciting. And I, Alan, it's kind of what you said. It's like you throw the money in there, everything changes because you've all of a sudden got folks doing whatever.

And I was, you know, I didn't know we were gonna be doing this topic, but on the way to the airport yesterday, I was listening to, and they were talking in the news and they were talking about that the ticket sales, the resale ticket sales for the Super Bowl are down. The prices are down 30%. Really?

Yes. Hmm. Now, and they said that ticket, the lowest cost ticket is still $4,000.

Yeah. But so I was thinking, okay, so is that because of the controversy? Is that because we're tired of seeing the Kansas City, you know, at, at the Super Bowl?

Probably the latter, maybe, you know. Um, but yeah, I'd read, I had read Actually that it was a bi it's a much bigger stadium. I guess it has an additional three or 4,000 seats than Vegas did.

Some it may have to do with, so that could be a factor in the game too. You know, the president's going, other people are going, I don't wanna go. Yes.

You know, the only, the only Thing, the only thing That's gonna get the NF NFL's attention is if the ratings are down, then they'll address it. I mean, they're just like bottom still, bottom line oriented. I, I think I'd rather Solve concussions Than, you know, we're worry about.

Nevermind that. Honestly, I think ratings are gonna be down based on just my little survey of my friends out there who are not having it. But Chinese restaurants sales are up.

Is that what you're telling? Yeah. What about Taylor Swift?

Taylor Swift will be there Could be. So, um, people Taylor Know, I'm reminded of, you know, being Jewish, that where do Jewish people go on Christmas? Chinese food.

Um, Chinese Restaurants. Right? Chinese restaurants.

So I, and, and I can't imagine, And that's why movies always get launched on Christmas Day, right? Yeah. Yeah.

I, um, look, it's not just to be fair though, let's not pin this all on the NFL and the NFL referees. There are plenty of baseball guys who, who, you know, there's no joy in Mudville tonight. 'cause they called us Strike three.

And you know, I, again, I think that's just, and Mike, I disagree with you. I go to the game to see my team play. They lose because the ref made a bad call.

You know, I scream out, kill the bugs or whatever, or, you know, but I, it's one thing to say, all right, he's human and he had a bad day, or he made a bad call. It's another thing to say, oh, he's got money on the game. Yeah.

It's, and I don't care whether, whether you're Pete Rose as a manager mm-hmm. Who did it, or there were a couple players last season who got suspended in baseball around betting. If you, if you are involved in pro sports with the betting that's going on there now, you've gotta be above reproach that they can never say one of the motives were you have money on the game, that that would p**s me off.

Or does I'll, I'm going, I'll disagree with you on this point though. Like, it used to happen when there was a bad call. What?

Once a month. Now it's every game. And that's what's different.

And that's what people are complaining about. It's every game now. It's not just a once in a while kind of thing.

It's a, every baseball, football, basketball. You know what, you mentioned basketball. I watched game, I watched my Knicks pretty religiously.

I watched the New York Knicks got my NBA subscription. I think the NBA does a hell of a job with their replays and stuff. They, you know, I, I've been watching it a lot.

I'm not gonna say they make every call perfect, but they go to the replay a lot and, and the replay usually is pretty definitive. I, I think replay here is more effective than AI Anyway, though, guys, we we're 20 minutes in on this and we're never going to come to an answer. Stay tuned.

Hey, if you are not watching the Super Bowl Sunday, instead of eating Chinese food, watch old Textron Gang. We got a bunch of episodes. Go catch up on ai.

Do something positive. Let's watch it during the Commercial Strong Gang binge. We, we don't have a tech gang commercial Mitchell for the Super Bowl.

Oh darn. Okay. No, we don't got that kind of budget.

Come on though. Maybe, maybe ads will go down next year. We could do a text on gang or thought for Term bought Us.

I thought you you're free cash. Well, I was hoping to get invited to perform at halftime, but they got someone else for that too. Alright, we're gonna take a break here on Text Drunk Gang.

We're gonna be coming back. Let's talk about something else you're watching. com is the leading resource for news analysis and education on challenges facing the cybersecurity industry.

com covers all aspects of cybersecurity, including data security, DevSecOps, cloud security, application security, network security, security threats and more. com has the largest selection of security content featuring breaking news, blog posts, podcasts and more. com to learn more.

com. Home of Security Bloggers Network. ai that you should all check out, but it basically suggests that despite all the hype surrounding ai, it's just too hard for us to implement.

And it requires humans who have super skills and not everybody has those skills. So we're not seeing AI become pervasive the way we might have initially thought. And it's just too damn hard.

And you've been tracking this space for a while. What's your sense of, you know, are we kind of like the techies are all over this AI thing is gonna automate the world. And I think the average person's kind of looking at it going, well that's a lovely idea, but I can't do it.

I don't think we help ourselves by changing jargon every 15 minutes. So for example, ag agentic AI is the new term. Vanti about it sounds like a company name.

It's intimidating. People have to read up what it is. They may not understand when we really could just say agent.

Um, I think it's just a type of AI that can make decisions and adapt in using agents, but yet we've coin a term. So I think as technologists, we have to resist the temptation to brand and name everything because that intimidates people and users, right? But I think that, you know, the thrust of the article was really that democratization has stalled, uh, lack of democratization has stalled, uh, trust and the complexity is too much.

So Kempel was basically arguing that everyday users have to have specialized skills to adopt ai. And you know, we need things like no code, uh, tools and accessible interfaces, open model. And I think something that he said that really resonated with me was that we need open source foundations and data transparency.

I think there's a lot of lack of trust. Look at deep seek, for example. Everybody was on that train and then they realized, oh wait, it's China.

Oh wait, where's my data going? And then it just sort of started a chain reaction of distrust along with it. But every platform faces it because of the lack of transparency.

I mean, you could make the same argument about centralized versus decentralized, right? Uh, are these platforms taking off that are decentralized? Not as much.

Um, so we're sort of in a will they, won't they, you know, like a Ross and Rachel situation because I think a lot of organizations find it too complicated. And in some cases, I have to be honest, AI is really a solution looking for a problem, right? So we're sort of in the, in the in-between middle earth phase where I think we're gonna get there.

But I think that he's right. Kembel was absolutely right. We need more user-friendly, accessible and trustworthy AI so that people can understand, which is, and I'm, I know I say it a lot, but I love her perplexity, um, because it's actually utilizing deep seeq.

But it, it shows you when you use deep seek, whether you agree with the China part or not, it shows you it's chain of thought. It tells you why you get to your reasoning. And even just that subtle change has built trust with a lot of users despite who, who has released it.

So I think that that's sort of an indicating note that Kelo is right. People want more transparency when it comes to AI adoption. Yeah, it's a, it it's a really good point.

Um, I would say that the user experience with generative AI is broken. We're starting with, you know, everything is a chat for one. Not everything is, is well well suited to towards the chat.

The, the other is the fact that you have to select which model you're gonna use. 5 sonnet versus other alternatives. You know, there's a lot of belief that, um, LLMs are essentially gonna be commoditized 'cause there's so many of them.

And we'll, perfect how to train 'em and build them. And yes, they'll increment get better. I kinda look at it as, I don't care what dictionary, you know, word uses to spell check or gram what gram Grammarly uses to, you know, grammar check my writing, just do a good job of it.

I really don't wanna spend any time checking it. And the fact that we require developers testing people everyday users of ai, you know, people just using their computer for work, whatever it might be, you know, select your engine, your your provider of choice of choice. I think think we've kind of got it backwards.

Let's like make the interface what the task is and not what the technology behind it of the 25 choices you've gotta choose from. Right? I totally agree and I think hugging faced does a great job.

Um, they make it feel like an open marketplace. Uh, it's approachable, but again, for developers And users, complexity, yeah. The user would, having the open marketplace is, is is a complexity.

I I have a different take on this. I don't think it's the complexity per se. I I think first of all, you've got to, not all AI users are the are alike.

It's not a monolith, right? Tech folk who are used to, you know, dealing with user interfaces and, and maybe understand a little bit about what's going on with different models and stuff is one set. I think when you look at the non-tech crowd, I think there's two main things that are holding back greater adoption.

Number one, fud, fear, uncertainty and doubt. Every day they're hearing how dangerous this could be, how it could take your job away. How the Chinese are spying on us by doing it.

How, you know, they're sucking all your information and making it available every day is a new story here that I think, you know, non-tech folks, non-security folks are half scared to death to use the technology. And it's kinda like moths to a flame of those who do, right? They use it basically for poly tricks and cute, nice to haves, but are not using it the way we would want them to use it.

In the way this we're talking about here. The second piece of it is most non-tech people don't understand what to use it for. They think it's a better Google go search using ai and you can search using ai.

Don't get me wrong, but it's not, the power here isn't searching right? Uh, you know, how many, how many calls were, were disputed by NFL Refs or by NFL coaches in 2024? That's a great Google, right?

But it's not an A thing, Right? But it's wrong. It's wrong a lot of the time, right?

That's, that's the Thing is that's the other part. It's far wrong, but you shouldn't be using it to search that. Now, I do believe, and I've seen it now in our Google workspace, and I've seen it in my Microsoft office and I see it in my email program that I use and some of the others, you know, the copilot kind of thing where it is built into the app you are working on it is a little easier than just using a standalone chat bot, right?

Like I open chat GPT or Claude or, or one of those on my desktop. Um, but I, I think in, you know, I had this discussion with my youngest son last night. He's a TV broadcaster and we were talking and he said, dad, you know, local TV cable is, is getting killed.

Everybody's cutting the cord. I don't know if I wanna go work for another local TV station or do I wanna be a digital storyteller and go work directly for a sports team or, you know, 'cause it sports is where he is at. I said, well Brad, if you're gonna be a digital storyteller, you better figure out how to use AI to tell your story.

'cause you're 23 years old and you're not gonna have a long career as a digital storyteller, whatever that is. Um, if you're not gonna harness AI because you will be replaced if you don't leverage it. He said, nah, you'll always need a person.

You'll always need humans. There's only so much you could do. And I think that's the other thing is there's a big feeling out there that it does this much.

It's never gonna really do this much. And I think those people are wrong. And I, okay, so the premise of this is partial is how it's slowing adoption because it's so complex.

And that's probably very true. But guys, how far are we into this? I mean, we are talking about how big AI is and how it transforms everything we know about business, everything we know about customer service, everything we know about retail, all these kind of things is, is massively transformative.

And what we're fine, what you're looking at is, okay, so we're bumping up against the wall on different things. Oh, it can do this, but it can't do this. Okay, so then we get to do invention.

Oh, it's, it's gonna eat up all of our energy. We gotta have, build all these atomic power plants. Okay, well maybe not, maybe, you know, some of the learnings that we're getting from deep seek as well as IBM and how they're gaining efficiency with some of the things we're doing, we don't have to do that.

So this is, you know, we're only into a very, very small, you know, maybe, maybe not even 10% of what this this can do. And that means invention. And so I think, you know, tech people want things to move really fast.

You know, it doesn't move that fast. You know, Microsoft wants everybody to adopt, you know, copilot. So they pound on us for like, I don't know how long to do it, but they can't even get us off Windows 10 for crying out loud Windows said there are people weren't Running well.

Said. Yeah. So, you know, and, and you know, they're threatening us this year.

It's like, I think they'll capitulate because we're saying absolutely Long live Windows 95 long wi live Windows 95. I don't think, and I agree with you Kimberly, I don't think that we've had AI's killer moment yet, right? Like augmented reality had its k augmented reality had its killer moment when everyone was using Snapchat filters, not realizing that they were using augmented reality, right?

They were using it mass adoption without acknowledgement of the technology. AI has not had its killer moment because although people are technically using it with voice assistance agents, whatever, I don't think it's reached that moment where we're all dying to use it, but we don't know that we're using it or we're not aware as aware. And I don't think we're going to, I think we're gonna get into this world as, and I think about the enterprise.

So I keep going back to how they're, what they're adopting, what they're doing, and the kind of applications they're starting on and the applications that they're doing. We're gonna see this customer service capability, it's gonna change how we do, how customer service happens. So we won't even know that's going on.

We'll just know that it's changed and that same thing is gonna happen with, you know, some of the logistics. We're gonna see logistics change as they apply that kinda things. I mean, just that mere thing about what, you know, when I was talking to Novartis or they're presenting at AWS and saying, you know, they went from 50 people filling out the forms that they have to fill out in order to go to drug over to the FDA, to being able in, in eight weeks or 13 weeks or however long it was down to like three days.

It's like, holy crud, that's transformative. So there are these pockets of transformation, it's just that we don't see it as a day-to-day thing. It's just going to happen over time.

Well, there's two types of change that, that occurs with technology. One is doing what we do, but doing it with the technology that either does it faster, cheaper, maybe it eliminates a task, whatever it might be. The other is doing something you couldn't do before.

It just wasn't feasible possible. Or we didn't even think about solving it that way. You know, may maybe, uh, you know, one might be, uh, you know, discovering new proteins is something, an activity that we have that we do and say in labs already, we just hadn't used ai.

Now we do, et cetera. But, um, you know, maybe we create a new element. Nobody thought I could create a new element.

Now I can do that with ai. I'm just making up something. But something that was brand new, never, never thought of before.

Those, those are the kind of real, to me, that's the real transformative change. The, um, doing things better, faster, cheaper, or not at all is the evolutionary part of change. And we're still mostly in that phase, but, But until those two moments happen, there's gonna be this disconnect between the type of experiences you expect daily from AI and what you're being fed with ai, which is just the conundrum.

Plus, as Alan said when you mentioned the FUD factor. Something like Deep Sea comes along immediately becomes the number one free app on the app stores, then only to be told the users who've downloaded, only be told 24 hours later what's happening to your data and what's China doing with it. These things work against, against ai.

It's just this kind of conundrum of hype versus reality. And, and I think to some extent, deep Seek was exposed in embarrassment of riches. We had all just accepted the way that things were being done was how they needed to be done.

And here they came and they unseated us. Although I really wanna know what ships they used. I don't think that that's been public yet.

But that was a game changer in a way that I think almost needed to happen because we just had sorted, accepted this mass consumption of energy was a given. We accept, we accepted that this was the path and they showed us, actually, no, it's not so good. Good for them.

I don't think. I don't think, just just to go on the record here, um, I don't think Microsoft's gonna give up on Windows 10. What they're gonna do is launch Windows 12.

They're gonna call it the AI enabled version. And because 13 is unlucky, they're only gonna stay on even numbers from there on out. So we'll go from 12 to 14 and that's how they'll get us all on board.

I actually give them a lot of credit because as an SEOI am seeing Bing in the top 10 of traffic, as you know, uh, search engines bring traffic to my clients for the first time probably ever. Bing is, Bing is not, not ads organic, uh, because of of chat GBT and because of the integration. So good for them.

I, I don't disagree with you, but I have noticed that I am being pulled into using Bing against my will. And then I have to look at it and then I go, oh, crap, that was a Bing result. And then I go back to Google to get the result that I am looking for.

'cause the Bing result is still kind of crappy. That's just your bias against all Microsoft's. I, I'm a bing Windows user, but Bing is still good.

Not what you getting from Google. They're a little Gemini thing. And then a bunch of sponsored ads, and then all the way down on the bottom, you actually get a search result.

It's ridiculous. That is ridiculous. That is a Well, it's turning.

They're hurt. They're hurting traffic. Right.

The more people get from Absolutely. You know, the zero click content, the less they're gonna click on sites. It's leveled out many, many sites Yes.

'cause of those overviews. Yeah. And, and I guess that's what they wanted, but to me it's not great search anymore.

No, well, they're doing it on purpose. 'cause they wanna be the provider of the content. Right?

They're essentially not being a facilitator. They wanna become the provider of the content using AI results that they pull from the people who create the content. Yeah.

Yet another company taking content without permission. So is there any, uh, uh, Anna, maybe you have this data since this is what you, what's your core business is. I mean, I personally, I I don't even read it.

I just go down because I don't trust it. I mean, I, AI generator, this goes back, this goes back to what you were talking about. I don't trust the data that's getting presented to me by Google on, on that summary.

Um, I just kind of start pinging down. 'cause I wanna go something that's been vetted by somebody, a person, and that's not been vetted. So is are people doing that?

Are they, are they just ignoring that and going down or what's happening with It's the I, Right. Do you really want a summary of this is, I'm Not looking for that. Right.

Well, the, the overviews were only as of a month ago, only in 17% of, of queries. I think that has ratcheted up Oh, a lot. 'cause as they're making a mad, they're making a mad push for Gemini.

Like, you'll notice Gemini, if you're in Gmail, is offering to read your email for you and summarize it. They're, they're pushing it down our throats. They're, I personally always read it because I'm trying to dissect how it's putting itself together.

Where is it pulling sources from? I'm reading it for a curiosity and an analysis viewpoint, but often I do skim below it. And those statistics I don't think are readily available yet.

But I have a feeling that, um, the average consumer is maybe not so discerning. They're like, oh, answer easy. Good if, especially if it's something simple, right?

Like, what is a business open? Or how do I get there? You know, those sorts of queries know the more complex stuff.

I hope so. I hope we're not blindly trusting them, but I I'm afraid a lot of consumers will. I wanna right click and delete it.

That would be nice. Just a little X and close the whole thing. All right.

Hey, we gotta take a break. We're gonna come back and discuss, uh, our third segment today. But you're gonna have to wait for us to come back.

You're watching Textron. Okay. Discover Textron Group, the epicenter of tech innovation.

We are your go-to for reaching IT leaders and practitioners worldwide. Our secret impactful content that sparks awareness, engagement, and top quality leads with us. You'll access editorial websites, streaming videos, virtual events, custom content analyst research, and more.

Join our satisfied clients. Let's revolutionize your tech journey. Contact us today and tell your story to the world in the most powerful way with Textron Group.

Hey everybody, that's my Alan Chimble invitation right there. Um, we, uh, Mitch and I are out at the, uh, Dynatrace conference this week. And Dynatrace, if you don't know, is a provider of an observability platform.

And they've been around for a long time. And we're, the show is really about how to bridge this divide between IT operations folks that have been using various platforms to observe it and what developers see, because we all have heard the story about the developer says, well, it worked on my laptop, but it doesn't work in the production environment. And this conversation's been going on longer than anybody cares to admit.

In fact, you know, I would say three quarters of applications when they're first built probably don't run on the production environment they're intended to until they get tweaked and somebody kind of bangs them into shape. But, Mitch, you're here with me. Are we making progress here?

What's your take on what's going on with Dynatrace specifically? But in general, can we get better at this? 'cause it seems like, I feel like I've been having this conversation for three decades now.

You mean if it works in production, it would've worked in test, is that what you're saying? Yeah, Something like Something. No, it's interesting where, you know, we used the term shift left about security.

We're seeing shift left happen with observability, right? Moving earlier into the kind of workflows that happen before things get into production. I think, you know, a year or two ago, it was all focused around how do we consolidate to one source of truth between securities, the SecOps, uh, use of security and, and it ops use of observability, uh, for those applications.

Now it's how do we, how do we get a window into observability data that's captured in testing, but also in production all the way back up into the development part of the process. And that was a big part of what dynatrace's announcements are. There are few companies that are focused on this area, which I think is really encouraging, is literally as a developer being that because they ingest the code base and, and, uh, analyze it against the, uh, the telemetry data that they're getting, as a developer, you can say, I'm looking at this piece of code.

What are the log messages or alerts or alarms or telemetry data that was captured as part of this? 'cause I'm trying to understand if that's the error that I'm really trying to track down or in operations, right? You can say, here's where the issue, uh, the symptoms of the issue that we're seeing, and immediately go to some of the parts of the code base that that might be.

But it also applies for testing, you know, solving bugs and things like that. So why wouldn't you want to benefit from what we're seeing in production, which is ultimately why we're writing in the code, and also what we're seeing in test, uh, because that's part of kinda getting through that process into production. So there's a lot of good things.

Dynatrace is one of the companies that's doing this new rec, new Relic is another, um, honeycomb is very developer focused observability platform. Of course, the, you know, the other players, Splunk and others have been around for a while and, and they can play in development, but some are, are catering more to earlier in this software development lifecycle. Y you know, it's funny, observability, let, let's face it.

Observability originally was replacing a PM, which, you know, when I look at the software development lifecycle with deployment being kind of the center stuff left and stuff, right? A PM was all on the right side, right post-deployment, how's my application performing? But now observability and we saw it a CubeCon I suspect we'll see the CubeCon in London in a few weeks again, or two couple months again, and, and here at Dynatrace.

And what you're saying, Mitchell, the action and observability now is on the left side and, and, you know, pre-deployment and moving further into the developer. And I, I didn't see that coming to tell you the truth. I, I, uh, I didn't think that that was Where, where, Where the action would be.

But plainly, that's, that's where these folks are, are moving, right? It's like Predeployment and connecting development and ops together. So, sorry Mike, I didn't mean to step on you there.

Go ahead. No worries. But Kimberly, I was involved in this philosophical debate last night and it meant something like this.

Wait a second. Were there beers involved? It might've been a couple of beers involved.

Okay. Just joking. But, but, but the core Old whiskey I was is it, is it, is it incumbent upon the developers to make their applications work in the production environment as dictated by the IT ops folks?

Or is it incumbent upon the IT ops folks to figure out a way to make the application run and who's, you know, responsible or who should give more here in this conversation? Because, you know, ultimately the application drives the business. So it's, you know, who, what's, what's the dog and what's the tail here?

Well, I think it depends. So it depends. Classic engineer, What definitive right there Depends upon what part of the application.

You have a future as a referee here perhaps. So the ball hit this part of the line, it's the DevOps if the ball went on this side lining side, the operations problem. So I mean, so, and the on the observability, I mean, this is what you're talking about is where, where things get glued together, right?

So you, if the application has to run appropriately itself, okay, so it's it whether that, so how it gets deployed, how it's connected to probably, and I'm gonna say these things I may don't, probably don't have any idea what I'm talking about here, but how it's connected to the database, the live database it's working on, how it's, you know, is working with the other applications that have to run there, you know, when, when, when they come up, et cetera. So there's, I think there's probably scenarios where it's an IT operations problem and there's scenarios where it's a DevOps problem. So that's why when you're saying saying shift left, you're not just gonna shift it over to the guy that says it's all your fault or your problem, all your issues I need.

And that's where the observability probably comes into play, is that I'm getting more of a, you know, a full wor look at how this thing is running and operating. Is that good? Yeah, no.

So I, look, I believe developers should not develop in a vacuum, right? Because that never ends well. And so, Kimberly, to your point, when you're a developer and you're writing an app that's gonna make a database call, it's incumbent upon you to make sure that that call actually does get made and that, you know, your SQL is right, or whatever you're using to make, you know, make the call to query.

And that that information is passed through and it, and it moves on to the next, it moves on to the next task. Um, I, I was, and I've always felt, even before DevOps, I've always felt this way that, you know, the developer sits there and says, well, it ran all my machine. That that's nonsense, man.

You know, that you didn't do your job Well also, I thought, Go ahead Anthony, As we go. Go, Mike. No, No, Kimberly, you go, go ahead.

Well, so if you, that's why, I mean, if you think about it, Alan, was it two, three years ago when platform ops became a thing at Q Con, right up until that time, the only people that were at Q Con were developers, right? And they said, I don't need operations because I can stand this entire thing up myself and I can run it. Well, then things kind of got bigger, and once we started getting bigger, we started dividing the responsibilities a bit more.

And then we said, okay, so there's this other thing called platform ops that we have to have. But so dev operations and platform operations gotta get together. And oh, by the way, there's this other guy called SEC too.

So it isn't, one person doesn't have all the knowledge for this. They, it's just impossible to do. It's too com too big and too complex, which is why they're having the AI engine there, et cetera.

So Mitch, I think, and I'm Italian, I'm not Italian, but I'm still using my hands on. I I see that, but that's good. Well, look, I, I'm coming East.

I'm, I'm gonna, I'm gonna surprisingly disagree with Alan. I'm all the time that we see this, it's like developers go out and they go, well, the database, the production environment is using. So, so I'm gonna go out and, you know, get a different database.

It'll be MongoDB, a document database, I'll run it myself, I'll manage it myself for about nine months. And then I get tired of that. And then I take this giant turd of a thing and I give it over to the it ops people and say, now it's yours to run.

'cause I'm busy, I gotta go write another application. This happens all the time. And so, Mitch, you're laughing.

Yeah, I just, your choice of words just struck me odd there. Well, yeah, I personally think, you know, let's just wind it all the way back, you know, development and production is the answer to all this. 'cause then you don't have a difference between your development environment and your production environment.

I say in all Jes, no, I'm not suggesting that you that we do that. I think, Kimberly, you're definitely onto it, which is connecting that what you said with Alan's point is part of platform engineering is creating the platforms that are gonna help you get through the process and get to production. So you're not go building your own environment to do development in that is wildly different or even mildly different from what's in production, but it causes you to trip up in getting it there.

And that's part, I think that part, part of the rule, maybe even the responsibility platform engineering, is to do everything we can to make that process flow as smoothly as possible. So as we move from development through testing into production, whether it's containerized or it's platforms or whatever it is, that helps us keep those environments as consistent as possible. I agree.

com, another text drug site. You can learn all about this there, guys. Did I tee that up end?

Yeah, we, we've gotta end it. But before we end, let me just say I would discount anything Mike Ard said about disagreeing with me. 'cause the rumor is he does share an account with a known gambler.

Um, I hope you've enjoyed this text going day with us, and thanks for joining us. We'll see you soon, Kimberly, as always, it's a pleasure to have you on. I know you're headed on a little bit of a cruise vacation.

Enjoy it. I hope the weather's great for you, Mitchell, Mike and Joy Vegas. Get home soon, John, keep an eye on Silicon Valley for us out there.

Okay, excellent. All righty. Until next time, happy Thursday, we'll be back tomorrow.

Of course, we have a full text drunk TV line up immediately following the gang today. So stay tuned. Stay here on your favorite channel.

We're coming at you. This is Alan Shimmel. We're outta here.

This is Textron tv. Hello everyone. Welcome back here to techron tv.

I got another first time guest for you here on Techron tv. His name is Mark Cusack. Mark is the CTO of a company called Yellow Brick Data.

Um, and we're gonna find out about yellow brick data and, and talk a little bit about Kubernetes. But first, let's talk a little bit about Mark and welcome him. Hey Mark, welcome to Tech Drunk tv.

It's great to have you on here. Hello, Alan. Very nice to be here.

Thanks for the invite. Pleasure. Mark.

Um, I mentioned you're CTO at Yellow Brick Datar, and we're gonna jump into kind of about yellow brick datar in a moment. But before we did that, I wanted to kind of just get an idea of your, of your, uh, background and of your path to becoming, you know, sitting here today is the CTO. Yeah.

And I guess Alan, my background is somewhat unusual as actually I started out in academia working as a physicist going way back into the dim distant nineties. And so I ended up, um, doing an undergraduate, um, physics degree, um, postgrad PhD in, in the theoretical physics with a strong kind of bias towards, uh, distributed computing actually, which is kind of the link to the present day. But, so my career took me from academia into government research, into distributed simulation systems and then into the startup world where we spun some of the technology that we developed in, in government agencies out into, into my first startup, which is back in, uh, 2004 called Rain Store, which is all about, um, archiving massive amounts of data from relational data warehouse systems.

And then that company got acquired by Teradata in 2014. So I joined Teradata for a few years and ran the data, a warehousing product line there. And then I flipped out about four, four years ago or so to Yellow Brick to become CTO there.

And that's where I find myself, What a great story, theoretical physicist. You ever look back and think to yourself, man, if I had taken the fork of the road on the left versus the right, what would I be working on? Now I do think that, but I always keep my kind of eye as to what's happening in, in the world of physics.

And, you know, there's a lot of crossovers when you look at this sort of advances in quantum computing, uh, and over, over the last, actually last six months or whatever. And I look back at my work was 20, 30 years ago, and there's a lot to kind of get excited about there. So, but what is interesting is when you start to apply ideas in a totally different area into what you're working on today, and that kind of cross fertilization of, of ideas across different disciplines, I think is, uh, pretty interesting.

Absolutely. Very cool stuff. Mark.

So were you at Yellow Brick data from like day one kind of thing? Or, or you joined, they were already out and give us the background on Yellow Brick. Yeah, no, yellow Brick was actually well established when I joined.

The company was founded in 2014, um, with the idea of how that they could apply the new emerging N-V-M-E-S-S-D technologies into high performance analytics and data warehousing. And so the founders came from Flash Storage companies, um, as well as, um, established database companies. And so I, I joined kind of quite late on in the day where, where they've already been in market with a product for three years when I joined.

Give us a sense then, I mean we, we get an idea of why, where it comes from and, and what they were doing. Mm-hmm. Let's fast forward, you know, 'cause This's another one of those 10 year overnight sensations, right?

You guys are added there over 10 years. Fast forward to today. Tell us about Yellow Brick today.

Yeah, well I should just let your viewers know. I mean, yellow Brick is an SQL data platform. We're essentially a relational database, but one that's really tuned for high performance descriptive analytics.

And so you look at our customer base, it's financial institutions, insurers, telcos, government agencies, and they typically want to modernize their existing old data warehouse infrastructure with yellow brick. And typically they've also got a lot of private data. Some of the crown jewels of their enterprise data is stored in a data warehouse, of course.

And they want this data to, in some cases be retained on premises and in others they want to deploy that data and run analytics on it in the public cloud. And in some cases some hybrid combination of, of that setup or even multi-cloud as well. But ultimately they want to keep control of their data and that's what Yellow Brick enables them to do.

So we have customers from Redshift, from Teradata, from Oracle, from IBM, SQL Server migrating to us. And the end of the day, what do they get? They get, um, better outcomes from their data, happier users because the thing is a lot, lot faster significant cost savings and big returns on investment.

I just got asked 'cause I'm curious, has this whole AI thing had an effect on the yellow brick business? I don't think we would be a credible technical company if we didn't have an AI roadmap to our, to our investments. And, and so we do, and we've done a lot of work, particularly in a couple of areas where one of the most, well, about a year ago, we added capabilities for Yellow Brick to operate as a vector store in similarity searches for retrieval, augmented generation applications.

So, you know, augmenting knowledge and injecting knowledge into your chat conversations. So we, we have that capability, but more recently we'd be looking at, um, converting natural language text questions into SQL and have that executed directly on yellow brick. Oh, that's nice.

So that's something we're actually working on. There's gonna be a quite an exciting release of the product, uh, in a, in a couple of months. So there will actually be an SQL copilot, if you like, that allows you to yeah.

Give the schemers Database. That's what envision you have, you talk natural and it, and it translates to SQL if you could do that without, you know, if you could do it. I mean, it's always the same story with ai, right?

If you could do it without hallucinations or without mistakes, man, that would be so cool. But, you know, well, so I have a lot of friends who grew up being S-Q-L-D-B admins. Right?

And I don't know how how they would view that. Is that kind of taking their job or is that just gonna set the world on fire for us? Well, you know, I, there are, I think you have to split the use cases into two.

Here. You've got this kind of, uh, uh, kind of holy grail of having any business analyst who's, who knows nothing about SQL being to ask any business question they like of their data and getting an answer back that they can make serious, you know, life altering business decisions on the back. Yeah.

That, that's one pile. The other is, hey, maybe this is a useful productivity tool to allow me to roughly generate a starting SQL point that I will then double check and verify, uh, and then add that to my, my, uh, production That, you know, that's not very different than you hear from testers. From coders, right?

Either. Right. On one hand I could say, oh my goodness, the sky's falling, it's gonna replace me because when know we're gonna go from 27 million developers to 500 million developers.

'cause everyone could develop code. You just tell the AI to, or how am I gonna leverage this, the 10 x my my worth, the 10 x my productivity? Right.

And I always want to be on the 10 x side. Yeah. And I think that is the pragmatic approach.

I think, uh, we're, we're deluding ourselves if we think you can take the natural ambiguity of the English language or any other language and convert that unambiguously into a sequel statement that we will run and give you the right result. We are far away from doing that. But getting that 10 x performance improvement, that's what we're at yellow brick kind of thinking more about.

Absolutely. Hey, before we jump into anything else, yellow Brick's, uh, website. What, what's the website?

com. Very simple. Yeah.

Great. All right. Let's talk, if you don't mind, let's pivot a little bit to our topic of discussion today mm-hmm.

Which is how Kubernetes delivers scalable analytics in hybrid cloud situations. And as we were talking off camera, I think in order to have this discussion, I think we have to first define hybrid cloud, right? When, when, when I, I, you know, cloud came on the scene, 2005, 2006 for me is when it hit my radar.

And, um, you know, initially it was public versus private cloud, right? That was the big thing. Where are you gonna keep your stuff?

And then the answer became, well, both. And that was very easy to call the hybrid cloud. I've got some in the public cloud and some back in my data center and, you know, calling that private cloud went outta style there for a while, right?

It was just back in the data center. But, um, it recently, it's, it's come back. But the other thing that really took me by surprise, 'cause I didn't see it coming, was what we call multi-cloud.

Where, you know, I got some stuff in a WSI got some stuff in Google, some stuff at Microsoft, maybe, maybe a little bit in the Oracle cloud. Yeah. Maybe I got some stuff back in the data center too.

And, you know, I've got, and I I I put stuff into different clouds based upon what's the best tool for the job. Right? Right.

I don't know if we saw how big that was going to be in relation to the ne more narrow definition of hybrid cloud, which is just some form of public private. What's your take on that? And, and, and if we could define that, then let's talk about how we use Kubernetes to deliver the scalable analytics for that.

Yeah. As, as far as yellow brick is concerned, we think of hybrid, uh, cloud and multi-cloud in facting. Sometimes we com combine the whole concept into hybrid multi-cloud of the idea is I will place my data and my data warehousing workloads on the basis of data gravity, data sovereignty, um, cost security and other considerations.

And so we are all about, it doesn't matter really where you deploy yellow brick and your data, we wanna give you the same experience everywhere on any public cloud, uh, on a hybrid combination of those multi-cloud combination rather, but also running in your own data center as well. That's really what we're aimed at. We want to give freedom of choice and flexibility about where you deploy those workloads.

So for us, hybrid cloud is just picking the right tool for the job, as you say, and placing data at the right place at the right time. I agree with you. I think that's a great way of looking at it.

And what'd you call it? You called it the, the hybrid multi-cloud data sort of model. Yes.

It's a mouth mouthful. Yeah. We need, we need, we need a, some initials there.

Well, I'll work on it anyway. Let, let's now turn over to Kubernetes, mark and talk about how do we leverage that here for scalable analytics. Yeah.

Now, you know, we, when we were embarking on this hybrid multi-cloud journey, um, there were a number of considerations that we wanted to make sure that we tick the boxes on. One of which yellow brick software had to run anywhere in the data center and in the public cloud as well, and a hybrid combination of the two together. It needed to be elastic.

You know, we needed to be able to scale compute, uh, oh and storage independently of one another in all of these environments as well. All modern data warehousing solutions today provide that scalability that you, you scale your compute to fit the tasks at hand, for example. And then last but not least, it needed to be resilient as well.

We need this thing to, to be capable of supporting business critical operations with 24 7 availability. And so when you look at Kubernetes as an orchestration framework, it really does tick all of those boxes. It becomes this kind of cloud operating system for us where we can deploy the same containerized micro, uh, services architecture that yellow brick has in any one of these deployment options and have the same experience here as well.

And you know, what has been very interesting is there was the, a big lift to kind of get yellow brick in our first cloud deployment running in the elastic Kubernetes service EKS in AWS, but then the barrier to migrating it to a KS in Azure and then GKE got lower and lower and lower. Yeah. And we've just done our most recent call OnPrem to, um, red Hat OpenShift as well.

Yeah. And so now we have Kubernetes coverage wherever most enterprises would, uh, would care for it. Absolutely.

You know, I I, I made a reference to it earlier in, we are seeing more and more what I used to call private cloud back in the data standards, right. Whether it's Red Hat OpenShift, which is a dominant one, and, uh, what what's the other big private cloud open source? Uh, it was like a whole consortium of people.

Rackspace was behind it, right? I mean, this rancher and Zu and other kind of, well, Rancher's now part of, uh, of, uh, of, uh, Knight of Tu Seuss, correct. Rancher is Seus Ger of course.

Is is Broadcom. Yeah. Right.

Yeah. No, no. But the, the private cloud stack not open.

Was it open cloud? I think it might have been Open cloud. Open Stack was the Open stack.

That's it. Yeah. Yep.

You know, that kind of has been rejuvenated lately too. 'cause that had gone an had dormant for a while. So we we're definitely seeing a lot of that.

Um, but Mark talking about scalable analytics here, you know, it, it begs the, the real point, which is, look, our data warehousing is, you know, it, it's, it's consuming space, whether it's public, private, a little of this, a little of that, a lot of this, and a lot of that. It, it, it, it just seems like there's never enough and, and we're, you know, we battle is it cheaper to do it here versus there? And what makes more sense?

And I need to segregate data, you know, and, and maybe store it based upon its particular value. Um, how does Kubernetes help us maybe with some of those kinds of analytics and those kinds of data points that we need to make those important decisions? Well, I, I think actually to some extent, Alan, the, the, the decisions are somewhat orthogonal.

I mean, our customers make decisions on where they're going to place their data and workload on the basis of the business problem and use case at hand, right? So, um, Kubernetes, I think doesn't impede what we do. Um, now, I, I have to say though, uh, when, when we deploy Yellow Brick and Kubernetes at the moment, we pretty much totally, um, take over that Kubernetes deployment that com Kubernetes cluster, the only workloads that are running in the Kubernetes cluster that we are running in a yellow brick workloads.

And we do that for performance reasons as well. Um, we, we do a lot of work at the very lowest levels of the Linux operating system to bypass main memory, to have direct access to the NVME drives that we access the store and retrieve data from. Um, we, we introduce our own threading models within Linux, and we do a lot of low level work, which means that we want to, within a particular Kubernetes compute node, take over all the resources on that box.

And so we don't sort of allow, we put anti affinity rule rules in place in Kubernetes to stop other pods kind of coming into our sphere of influence. So I guess that's a long way of saying that we, we kind of isolate ourselves from other considerations and, and give it an environment that's completely dedicated to Yellow Brick to run on. Love it.

com, you mentioned the website. Any particular path they should follow on the website or for specifics on this? Yeah, there's, uh, there's a ton of reference information blogs, um, follow, follow us on LinkedIn as well to, to get more information from you.

There's academic papers out there, uh, that you can get out. org site, you'll find, uh, a paper that we published at their conference about a year ago that gives us the, the full kind of Kubernetes architecture breakdown as well. But tons of information on the website.

Excellent. Mark, thank you so much for coming up here on Text Trunk TV with us today. It's been a delight.

Please do. Come back, keep us posted. You know, this is, you know, it's all about the data.

Stupid, right? That, that's the lesson that we've learned over the years in, in back here, and it seems like yellow brick's right in the middle of it. So do keep us posted.

Will do. Thanks, ally, it's been a pleasure. All righty.

Mark Cusack, CTO Yellow Brick Data here on Tech Drunk tv. We're gonna take a break. We'll be back.

We've got more for you. Hello and welcome to the digital CXO podcast. I'm Amanda Ani, and with me today I am happy to have Dan Evans.

He is the Senior Director of Smart Cities and Smart Lighting at Itron. How are you doing today? I'm well, Amanda.

Thanks for having me on. Can you share a little bit about your background and then a little bit about Itron and what do y'all do? Sure.

Yeah. Um, so I'm based out here in California, in Silicon Valley. So as you can imagine, I've popped around to a few technology companies.

Uh, that's kind of my roots is around, uh, tech and, and networking. Uh, so I, I've, uh, done a few stints at companies that were introducing broadband internet to the home over cable infrastructure. That was back in the nineties.

Uh, I've been at companies that build wireless telecommunication systems, not not the cellular, uh, phone, uh, type, but for more, uh, internet services providers. Uh, and then I joined a company called Silver Spring Networks, uh, in the late mid two thousands, which was really focused on the utility industry and, um, enabling communications and software for utilities to, uh, allow them to start deploying what was called smart meters at the time. So a, a an electricity meter that had a communications capability.

Um, and then in 2018, I think Itron acquired Silverspring Networks, um, and Itron, sort of segueing to sort of who we are, um, is a company that is, you know, largely focused on, uh, helping utilities and cities become more resourceful with the resources that they, uh, operate and, and and maintain. So we work with utilities across all of, uh, um, the commodities, electricity, water, gas, um, as well as with cities, uh, in, in the cases where they manage those commodities. Or in my world, um, as managing the smart cities line of business, we look at some of the challenges that cities are facing today, um, and, and how we can, uh, address those.

And, and we've kind of leveraged a lot of the core infrastructure that we've developed for that metering world into what we became the smart lighting world, which we'll talk about a bit more today, and then taking it beyond lighting into some of these other use cases. So it's been, you know, uh, almost about 17 year stint within this space, uh, with the last 10 years really focused around smart cities. Wonderful.

Thank you for sharing, and you're certainly the right person to speak with today. Then, since our topic is about the evolution of smart street lighting, and we're hearing so much about smart cities these days, so, you know, streetlights are no longer about just elimination, they're becoming the nervous system of smart cities and collecting data, enabling services that were not possible a few years ago. So can you elaborate on that and what benefit are c seeing?

Sure. Yeah. No, this, uh, this phenomenon, if you will, uh, started kind of in the 20 10, 20 12 timeframe.

Um, and cities and utilities, depending on where you are around the world, I, I did mention earlier, but Itron is a global company. So we see this phenomenon, uh, evolving in many places, uh, in, in utilities in North America, in the US specifically, a lot of the streetlight infrastructure is owned by the utility. So there was a natural conversation that we were already working with those customers to take it to the next level.

And in the other parts of the world, that infrastructure tends to be owned by the cities. But either case, they were looking at older technology. So we're talking about the lamp technology.

You know, a streetlight has evolved from a gas lamp, uh, fixture to, you know, something that was electrified and that electrical, uh, technology has evolved. Um, and the, the phenomenon that we were, we caught up with was LED technology. So now it's everywhere, right?

It's, it's in our phones, it's in our cars, it's in our homes. Well, it's on our streets as well. So you may live in a town, uh, if you look outside of the streetlight, it has been changed in the last 10 years into an LED streetlight.

Um, what you may not know is whether or not has a smart controller on top. So for the same reasons we got into the smart metering space, um, it basically enables communication to that asset. And why do you need a communication to a smart streetlight?

Um, it's because you can, uh, be more proactive in maintaining it. So, um, you and I, we've all seen streetlights that are out, right? And, and you're wondering, okay, did somebody call that in?

Has somebody reported that, uh, I'm not gonna bother, you know, a week later it's still out. So, okay. Clearly nobody reported it.

Let me report it. So a a, a smart technology can be introduced there to, to sort of be the virtual citizen. And as soon as there's a problem on the streetlight, an alert will go back to the operator, whoever's responsible, and allows them to be more proactive in their, in their maintenance.

So it makes us as citizens happy. The, uh, economics of the, of the city from a commercial perspective go up because people feel safe, and the quality of life is high. So, you know, the LEDs were coming in, which meant that whoever owned that light was gonna roll a truck to replace it.

So, uh, our, our solution and, and we're not the only one who offers this, was when you go out and upgrade that light, add a smart control solution, because you probably will regret not having done that. Um, and in a few years, and now you, if you have to go back to that light, the economics of that, uh, just, just don't pencil out. So that's kind of the benefits are, you know, you get the visibility improved maintenance, you can also have an improvement on the energy consumption.

So just by, by going from the older high pressure, sodium metal, metal howi streetlight technology to LED roughly gets you a 50% savings in energy. So that's good. But we're encouraging our, our customers and our install base to not stop there, add the controller, because these streetlights actually have this dimming capability.

So just like you have a dimmer switch in your home that can adjust the light level, now that functionality exists on a streetlight, why would you want to do that? Well, you could have situations where you've got traffic, the, the, the drops during the middle of the night, no one's on the street, so you can bring the lights down to still a safe level, so you're illuminating the ground, but you don't have to be bright a hundred percent. So that can get you some more energy savings beyond just the LED.

Um, so it's a combination of energy and, and maintenance cost savings that has really, um, driven this adoption of the smart lighting control solution that we've seen around the world. And we have about 4 million of those, uh, smart streetlights under management here at I. Yeah.

So there are many aspects that are improved from it, from a safety energy and cost. Um, so with that being said, are there other places that you could put this sort of smart technology besides street lamps? I just wonder how, how else can cities benefit from this type of technology?

Yeah, so I mean, the, the, the nice thing about a streetlight pole is in most parts of the world, it's energized all the time. There are some situations where it go, the energy goes away during the day and it comes back again at night. But, um, a lot of, uh, cities are looking at how do I leverage that asset for more than just this smart lighting solution, which, you know, is good in and of itself.

Um, so we have worked with, at Itron, we work, uh, with a, a, a host of, uh, technology partners who bring, uh, their solutions and we look at how we can integrate their solution with ours. So in some cases, I'll give an example where you might want to monitor the air quality on the street level, you know, at a very granular level, um, more so than what you might have using sort of federal level or government run facilities that report on, on weather or air quality in, in and of itself. Um, those can be attached to the streetlight and powered by the streetlight 'cause you've got power there.

And then you can actually leverage the smart controller to pull data from the sensor and then push that data up into the cloud, into the software that we manage, which provides now the city with a single pane of glass view of the solution that they've deployed. Um, and so more and more of these different use cases, again, tied to the challenges that the cities are facing, whether it's, you know, climate change, uh, traffic congestion, as cities are growing, um, you know, public safety of, of infrastructure, um, there's a number of different challenges. And we work with this partner ecosystem to bring together their solutions with ours so that the city sees this as a, an investment that they can leverage and build upon, right?

Starting again with the lighting and, and then adding more on there. Once you have that, now you've got, you know, you've got your lighting data, you've got data coming from a variety of different other sensors, whether it's traffic or, or air quality, then you can start bringing these data sets together and actually making some analytics at a minimum and saying, okay, interesting data. Let's do some reports or take action.

So, um, you know, for example, my, my, uh, mention earlier of the ability to dim lights, well, if you had a traffic monitor on that street or distributed on the main roads, then you would know definitively that your volumes are down. Or if there's an event all of a sudden in that part of town that normally doesn't happen, the sensors would detect your traffic levels are actually higher, so you, you shouldn't dim when you normally would. Right?

So it brings the data sets together. Um, similarly examples of, uh, I wanna manage my traffic congestion. Okay, well, as a result of managing that, maybe you deploy a smart parking solution to help people, you know, get to the parking spot as, as fast as they can get some off the road, removes the, the congestion.

But at the same time, you've got an air quality benefit too. 'cause your pollution from the vehicles has just dropped. Well, how do you know that unless you've got the data?

So it's really bringing the data sets together in that kind of cohesive manner that our, you know, that we bring to our customers through our, our software platform called City Edge. Yeah, and I can imagine by collecting that data, as you said, if there's an area of town that's more congested, um, over a period of time, they might know, okay, we might wanna have a few more police officers patrolling the area, you know, um, to help you make it more safe in that area. Exactly.

Yeah. I mean, public safety is one of the key domains that we look at, and that ranges from, you know, uh, areas where lighting isn't present, right? That presents a public safety sit, uh, uh, situation.

You may have flooding, uh, so a roadway could get flooded for, you know, heavy rains, uh, and the, and the creek or the river has overflowed that causes, you know, a public safety issue if, if the city can't get their maintenance crews out there to close off the roadway. So again, the technology has the ability to, to, to detect those situations and proactively get, uh, get the crews who need to be out there addressing that problem, uh, for the benefit of the citizens. And again, these are all use cases that the city will benefit from and, and help their city grow to the next level, which is a, is a challenge all all the cities around the world are facing.

Yeah, and it sounds like from what you're telling me, with all this energy savings, uh, on the electricity side, I know that, that the, the grid is a, a big issue for big cities. So if they're trying to build out, or even smaller cities that are trying to build out, there's only so much, um, energy available, so that would provide more, Correct. Correct.

And that's, that's, that's another area that Itron is spending a lot of our, uh, kind of research and investment is how is the, our customers, the utilities, uh, being challenged in new and different ways as more things are, are entering the grid scenario. So electric vehicles is a big area where we're, we're looking at how to help our utility customers manage the load as more and more of those appear right in, in our, in our homes and businesses. And, and again, taking it to the city level, cities are electrifying their fleets.

So all of these trucks I talked about, rolling to fix the problem on the street, on the streetlight, those are all moving to electric vehicles as well. The buses that are being run in the infrastructure are moving to electric. So what happens at the end of the day when all those fleets come back to the corporate yard, they're all gonna plug in, and that's gonna create a huge demand onto the grid, which causes instability if the city, if the utility hasn't planned for that.

So we offer solutions that can come in and help manage that load, uh, and, and supply balance, which the utilities ultimately have to be doing every day to make sure they're not having any outages or, or concerns with the, the power quality they're delivering. So it is a challenge, um, that we at Itron are, are helping our utility customers address along with the other, the other big movement to renewable energies, right? So changing the type of energy, uh, supply from what might have been a, an oil or coal base power plant to, uh, wind farm or a solar farm, which are, are great for the climate and, and, um, the overall environmental impact, but they bring a challenge in their dependability, right?

The clouds move in, solar, uh, output drops, the wind goes away. Well, your wind farm is not producing anymore. So again, this is a, a challenge that we're helping with our, our utility customers, uh, balance, uh, all of these different supply sources with the demand and helping manage the demand to match.

Yeah, absolutely. So you would say that the environmental impact is far reaching? It definitely is.

And, and again, when, when we look at, uh, kind of the market here, uh, that is a point of overlap, the climate concerns and, uh, everybody has carbon footprint reduction goals. Itron has our, our own, uh, set of goals. Uh, the utilities have theirs, the cities have theirs, um, and everybody wants to move in that direction, uh, as well as sort of the other impacts to the, the environment, uh, that are out there.

So, so utilities are being encouraged and moving, uh, uh, rapidly towards renewable. Um, but it does introduce some of those, those new challenges as, uh, as cities are growing and, and cities, uh, like the one I live here in, in the Bay area, you know, data centers are popping up everywhere, right? And thank you ai, right?

That brings, uh, all sorts of compute power that's needed. Um, which, you know, computers run on energy. So as a data center is built, you know, I look at it and go, okay, well my local utility is now gonna have a challenge of feeding that data center.

'cause that's a 24 by seven demand, right? It, it's not like us when we go to sleep and the energy drops off, that data center runs 24 by seven. Um, so they've gotta be able to, to have good visibility into their infrastructure to know, uh, how they deliver that without bringing instability into the grid.

So for a city just trying to enter into the smart city space, what advice do you have for them? What, from your experience, what challenges do you see occur when trying to integrate some sort of smart city plan, and what advice can you give? Yeah, no, that's a great question, Amanda.

I would say, you know, it starts with understanding the problem you're trying to solve. And, you know, as we go and look at these surveys, there's a number of challenges that cities face. Uh, and I think we, we we're open and, and collaborate with the cities to, to, to identify what are the key challenges they have.

Um, definitely look at, uh, solutions that have maybe a broader, uh, fit as far as, um, you, you may start with one application that addresses one challenge, but think a little bit holistically in a higher level. Um, and, and what I mean by that is, a city might have a street lighting department and they might have a water utility and they might have a, a transportation or traffic department. What we typically see is those city departments operate in silos, you know, and there's not a lot of crosstalk that happens typically now you're up at the city manager or even mayor level, depending on the size of the town or city, where somebody's looking at this across the board and saying, okay, I have a fixed budget.

How do I make the most and the bang for my buck? You know, when I make an investment choice, am I just solving the problem of the, of the lighting guy, or do I really wanna look at a solution that's gonna address something for this traffic, uh, person as well? Um, so looking at sort of a more of a holistic and homogenous, uh, deployment of technology.

And then when you get into the technology and, and the selection, you know, look at what's out there on the market, look for open standards, right? Um, it's all very important to be able to grow on top of technology, uh, rather than pick technology that might have a four or five year lifetime and then it's gone, or the company's out of business or, you know, those are sort of long-term decisions that they make. Um, the other thing is, you know, you know, it's fine to start small, it's time, it's fine to start with a pilot, decide kind of, does this give you the data you need?

Is it meeting your expectations? Um, because that is very useful and important when you go back to the city council or to your, you know, investors or whoever is funding to say, I want more money to do X, y, or Z and here's the data I got from the, the pilot I ran a year ago that shows that I'm, I'm on the right trajectory. It's gonna give me the benefits, me and my citizens and, um, that my communities, the, the benefits that that we say it will.

Um, so that that's sort of the trajectory and the journey that they should get on. When you think about the future of smart cities, do you have any ideas rolling around for new integrations or tools that you think would solve some kind of problem? Wow.

It, it's a very wide space is what I would say. Uh, Amanda, I've been doing it as I said, probably for the last 10 years. And, and what we saw is that that lighting space really grew and it, and it popped, you know, let sort of use those terms where we definitely saw an uptick.

Um, we are still kinda looking at the market and saying, okay, where's the next big pop gonna come from? And there are a number of different use cases, uh, and, and challenges that we are, uh, that we are trying to address in collaboration with the cities. Um, I think the, the key and, and maybe I'm jumping a little bit on the AI bandwagon here, but AI is all based upon the data underneath, right?

It's, it's applying logic, it's applying algorithms to data. And so, um, for us at Itron, I mean, that's sort of, that's kind of where we started with, with the metering space and the smart metering space, and then the lighting space. It wasn't just to enable the connectivity, but it was to put software in place that was collecting data.

And now you have the data, what do you do with it? You know, we often hear from customers and cities, okay, there's so much data, I don't know what to do with it, right? So I think the next level is really, uh, uplifting data into an outcome.

And, and, and, and looking at the analytics behind that, what is the challenge? What is the outcome you need? And let's not get lost in the data.

And I think, again, AI is just another way of saying, I'll give you the answer you need because I'm crunching all that data underneath. And then once you have that outcome, you can be making incremental improvements and optimizing your system. Uh, and you know, there, there's a feedback loop.

So the data will tell you, oh, that didn't work, or, yes, that did work, right? You got the outcome that you, that you needed. Um, so it's, it's just that, again, the growing need for data, uh, the growing need for the analytics to then be layered on top of that and then delivering ultimately an outcome to the customer that they don't have to worry too much about all the, uh, the sausage making as we would say underneath the hood.

I just get what I need for the benefit of my customers. Okay. Well, if there was one key takeaway you could give our audience today, what would that be?

Don't be afraid of technology. I mean, the, the, uh, depending on where you live in the world, again, I live in Silicon Valley, so it's, it's almost a foreign concept and not know what tech is. But um, you know, having worked with large, you know, fortune 500 companies and, and, and who are in a space, maybe that doesn't move as quickly.

Cities are the same, right? And there's an evolution of the staff and I think the people part has a lot to do with it. Um, and so as more and more people adopt technology, I think they should look at how they can leverage that for their infrastructure.

So when we talk about smart cities, you know, it's, it's really about the city. It's about the, the, the, the, the delivery of the services that they deliver to you and me as citizens within that city and looking at how technology can leverage that, optimize that, you know, get you more bang for your budget. 'cause the budgets are know not going up, you know, they're usually going down.

And this is where technology and automation can often come in and replace maybe a task that, um, uh, was costing you a lot more before. So that's the theme. Whichever one of these smart city use cases, you know, is the next one that that comes across as the big, the big winner.

Um, it's all gonna be based around that. So, uh, I think that's the key message and the key takeaway about um, uh, you know, not to be afraid of the technology adopted and you know, companies like Itron gonna help you navigate that journey from, from beginning to end. Wonderful.

Well thank you so much for coming on the show and sharing your insights with us today. My pleasure. And thanks again for having me, Amanda.

All right. And thanks to our audience, stay tuned. There's more.

com is the leading resource for news analysis and education on challenges facing the cybersecurity industry. com covers all aspects of cybersecurity, including data security, DevSecOps, cloud security, application security, network security, security threats and more. com has the largest selection of security content featuring breaking news, blog posts, podcasts and more.

com to learn more. com. Home of security bloggers network, This is Textron tv.

Hey guys, thanks Withrow. We're here with Nick Klowski, he's the senior research director for Ryans. And we're talking about a new study that they did with article that looks into, well just how the role the CISO has expanded and what are the implications thereof.

Hey Nick, welcome to the show. Great to be here. Mike book, I think everybody we talk to will tell us that the CISOs playing a larger role in organizations.

In fact, in some places the CISO now ones IT and security, but I don't think that's the standard operating procedure just yet. But as it's clear that security is a bigger focus, what has been the impact on all this increased responsibility on these CISOs Elements of scope creep And what we see is kind of two forks happening. On one side you have the CISOs who kind of have the scope creep, thrust upon them and are given extra responsibilities, extra functions and aren't necessarily given rewards, aren't necessarily given a bigger role and more influence within the business.

And it's just kind of like, oh, all the problems are getting dumped on those CISOs and those folks are getting burnt out fast and looking for new jobs 'cause they just want to get out. And then you have the CISOs who are being given real ownership of parts of the business, for example, the full ownership of it, given the opportunity to really have more influence at the executive level and real opportunities to grow the business and partner with other executives. And those CISOs who are getting elevated with meco creep are finding it generally very rewarding to have all those extra opportunities and extra responsibilities because it's giving them a bigger role within the business.

Is that emotional rewarding or financially rewarding? All of the above. Alright.

Because a lot of times, you know, at least in my experience, almost everybody gets added responsibilities to their job over the years. And it takes a while before anybody recognizes that maybe I have a different job but I need a different title. So will CISOs as they continue to get elevated in responsibilities, evolve into something else?

Yeah, so for example, we see this dual IT and security ownership situation where you might have a CISO and CIO kind of role. The folks who are CISO and CIO see a substantial increase in compensation, substantial increase in satisfaction, and generally speaking more of an executive role within the business. Whereas if we see CISOs who just take on discrete functions of it but don't get the full CIO function, they could own as much as half or more of it.

Their compensation stays pretty much the same as other CISOs. Their satisfaction drops and they start getting frustrated with how they're just given all the problems to solve but not given commensurate rewards. The folks who are taking on more responsibility outside of traditional cybersecurity, are there common attributes?

Are they, maybe they've got an MBA or something if they go do something different that their colleagues are not doing? We do see some commonality in that the CISOs who are getting more opportunities have diversity of experience, whether that is working across multiple industries, whether that is making an effort to get involved in risk committees or AI committees or compliance committees within the organization kind of work cross-functionally and build relationships cross-functionally. But ultimately what we see happening is CISOs are just extremely strong problem solvers.

And as businesses have bigger digital risk problems, the CISO is the best person to solve 'em. They're, they're the people who know enough about the compliance angle, enough about the technical angle and enough about the security angle to actually problem solve some of those risk issues. And they're getting thrust into those roles.

The key thing is figuring out how can I influence these decisions in partnership with other business leaders as opposed to how do I become the defacto owner of a whole bunch of risk that really needs to be owned by business units. And to that point, are they kind of, um, as they assume those responsibilities, um, are they engaging with business leaders more in terms that the business understands? 'cause I think part of the problem I've seen over the last year, and I think we've talked about it in the past, is that security people, they finally get access to the boardroom but the boardroom still doesn't know what they're talking about.

Yeah. We are seeing about 50% of CISOs report to the board either quarterly or monthly and year over year just fewer and fewer CISOs are are never reporting to the board. Many CISOs are now starting to get active on subcommittees, which is where a lot of the board work really gets done.

And then in general we are hearing more and more about CISOs getting executive exposure and looking at the rest of the C-suite as their peers. We are planning to update next year's survey, which we're probably gonna launch in late March, early April, with some really specific questions to get a sense of how frequently CISOs are meeting with executive peers and which ones they're connecting with more often When I do gain access to the board, I think one of the things that becomes quickly apparent is that the board cares a lot more about compliance than a lot of things 'cause compliance is where the fines are at. That's part of the oversight.

So is that driving more security people to take out responsibility for compliance because it's kind of like the path to the board? To an extent I think compliance can work as a forcing mechanism to get the board to care about security. But more often, or I should say more substantially, I think what we see is a lot of compliance solutions end up being security.

It ends up being figuring out how you're gonna protect and safeguard data once it's actually in your systems and where that data's living and how that data is rooting through your systems. And that just requires security input. And so security is gonna have such a big stake in solving a compliance challenge then it makes sense that security is getting a larger stake in understanding what those compliance challenges are in the first place.

One of the things that I've noticed is that sometimes when security people get exposed more to the business and the port, their appetite for risk starts to increase. They're actually start to think about things a little bit more in terms of well what is the impact of the business? 'cause the business leaders are always assessing risk and to them cybersecurity is just one more.

Um, so does it change the way the security people think? Oh yeah, we see a strong correlation in our data between exposure to the board and satisfaction with the business' alignment to security priorities. And we think two things are going on at the same time and they're kind of coming together to make that satisfaction high.

And that is board members and business leaders are becoming more aware of cyber risk and more open to real conversations. And so CISOs are starting to feel heard and CISOs are becoming more aware of business risk and understanding the priorities and what level of financial risk within cyber is tolerable and more willing to take risks than they might be if they're kind of siloed in the back office. Thinking about all the threats and worrying about what happens if there's a breach.

But come to understand, oh okay the business is aware of this risk, the business is choosing to take this financial risk. I have the backing of the rest of the organization. I can feel comfortable with this risk.

So do you think that the CISOs that understand that and have that level of conversation are arguably less stressed out than their other colleagues who are always kinda walking in every morning going, I don't know what's gonna come next, but it could be a disaster? In some ways they also tend to have a lot of executive presence and don't show that stress as much. Sometimes they actually are less stressed out or whether they're just good at showing it.

But also those CISOs tend to have been put in a position by the business to be less stressed and have more resources on the team below them so they can delegate more and focus on more strategic issues. A lot of CISOs are starting to elevate their executive presence and they're wherewithal on business acumen, but they're still forced by the way the business regards the role into kind of a back office tech function and they're working to get the business to change too. One of the other things that I sometimes wonder about is when I see senior security or even IT people for that matter get close to the business, they start to lose touch with the IT and the security teams underneath them.

So how do you kinda maintain that relationship while you're establishing these other relationships in a way that doesn't, you know, where they under underlying folks who report to you start to uh, I don't wanna say they become suspicious, but they become a little more um, negative. Yeah, in some ways this can be easier for CISOs than what I've seen in practice because generally speaking, CISOs are such curious people who care a lot about doing good meaningful work. And when when you have that attitude it's easier to kind of, you know, it's almost like we to talk CISOs into going out of being in the weeds and care about the strategic corporate stuff because they want to step in and help their teams.

They want to understand the technical problems, they want to go learn a new programming capability and they have to learn to monitor that part of their brain so they can do enough of that to stay plugged in, but not so much of that, they can't have that strategic influence over the business. Do you think there'll be more of a, a split, like when I see in the IT world there's a CIO and these larger companies frequently now there's a CTO or somebody who's involved in the actual tech. So will security teams evolve where the CISO will be complimented by somebody who is really the security operations lead or somebody like that?

I mean we see large security teams having functional department heads for SecOps, for architecture, for GRC, for ASEC product sec, like all kinds of assumptions depending on the specific needs of the org, it's really just a matter of scale. We're also seeing in those kind of dual CISO CIO roles, you might have a person who is a CISO and CIO and title who then has a head of IT reporting to them and the head of InfoSec reporting to them because it gives them the ability to be very strategic while those functional department heads are leading the technical execution, How do you perceive the relationship between CISOs and the CIOs then evolving? Because in some ways it feels like, you know, we want them to collaborate more, but you know, people start to get jealous of their resources and prerogatives.

It's gonna vary a lot from organization to organization. But what I'm seeing in a lot of healthy situations is where even if the CISO is reporting to the CIO, that reporting is almost clinical in nature. It's about, you know, getting through the reviews and the formal HR things you have to do and and a function.

They operate more as peers within the business, both with access to the executive teams and they serve as partners and they both have a shared vision for where they want it and security to go. Those two units work extremely collaboratively and they are able to have a strong healthy relationship where they are working toward common shared vision for how security and it can make the business better and deliver value. In your experience, are there courses for security people that take to kind of get this kind of business acumen?

'cause I mean there's no shortage of technical courses for security folks, but I wonder if we need something that helps them understand the business and kind of have those conversations with people. Yeah, there are some emerging programs. We at Ions have an executive competencies program where we offer coaching from recently retired or active CISOs and VSOs and a variety of asynchronous and synchronous learning opportunities to develop those business skills within a cyber context.

We see programs at universities like Carnegie Mellon or some of our faculty are professors teaching business skills to CISOs and even like always like the NACD will kind of help CISOs understand governance or help board members understand cyber. It's a growing need. And of course there's always the option of pursuing an MBA or something like that if you want to get really deep in the business.

I can't help but wonder if AI tools, whether it's chat GPT or whatever, is gonna make it easier for the security of people that understand a lot of these business rules. I mean ultimately most of what we're doing in business is not rocket science. It's been fairly well documented.

There are tons of business courses and is that all just gonna become accessible content? I think a lot of it's accessible already. The nuts and bolts side of it, CISOs are generally very smart and very curious and they can pick this stuff up fast.

Generally what we see is the tricky part is all of the soft skills that human interaction, the emotional intelligence, the relationship management and influencing where in most business functions, you gradually build those things up with other business leaders organically as you move up the ranks. Whereas in security, you're moving up the ranks due to technical achievement Often then you get thrust into this role where you're now meant to connect with other business leaders and you don't already have those relationships necessarily. You don't already have the shorthand that the business is using for specific problems and you have to go and kind of build, other people may be building up over five or 10 years of osmosis fast and that's where things get challenging.

What relationships can you build to fast track that process? One of the other things I've noticed is that more businesses today, when they all align with somebody else, there is a bigger concern about the security of the two firms and how that's gonna be maintained. Is there more of an effort to, for the CISOs to kind of collaborate with the, with other CISOs from different organizations to kinda address those business concerns and that becomes part of the job?

Yeah, we're seeing a growing trend, whether it's through ISAC or there's through organizations like us and all of the events that we host to bring CISOs together. There is a growing effort to create a stronger community among CISOs that folks can share and find that balance between how do we talk about our problems, whether it's the very real threats we're facing or whether it's the business challenges without losing the competitive advantage that we may gain through having an excellent security program. It's just finding ways to open up lines of communication in a healthy, productive way for the CISO community.

So ultimately, what's your best advice to aspiring CISOs out there? I mean there's a lot of them. I think they wind up, you know, being in the technical track, but at some point if they wanna become the next generation ciso, they need to have some sort of business experience.

So how should they go about getting that Find side projects that get you excited, that are adjacent to your technical areas of expertise, but not directly in them, whether that's an AI steering committee, whether that's a compliance committee, whether it's a customer trust initiative and volunteer for those kinds of programs. It'll get you exposure to other executives or business function leaders who will then see the kind of value you can offer and bring you in and sponsor you when business is trying to solve other problems. And then gradually you're gonna get more exposure to the business, the business is gonna get more exposure to you and you're gonna demonstrate the value you can offer to the organization beyond simply the security nuts and bolts.

Alright folks, you heard here, you wanna become a ciso, you gotta expand your reach beyond just the technical side of the job. And that all requires talking to business folks and ultimately you need to be seen. Hey Nick, thanks for being on the show.

Thanks for having me, Michael. It's great to be here. All right, and back to you guys in the studio.

Hi everybody and happy new year. Thanks for joining us for another episode of Techstrong Women where we feature amazing women doing amazing things in tech. I'm Jody Ashley, executive producer here at Techstrong, and I'm here with my co-host Tracy Reagan, creator and CEO of Deploy hub and very busy lady when working with the Linux Foundation.

I'm sure it'll come up today. Before I introduce today's guest, I wanna give you a quick update about what's happening here at Textron. com, so be sure to go and check that out.

We have a lot of virtual events happening, uh, predict 2025 is coming up. If this airs after that, you can go out and watch it on demand and I would recommend it. It's gonna be an awesome virtual event, so you wanna be sure and check it out.

com and be sure to tune in every day to Techstrong TV for great shows and interviews. Okay, Tracy, it's 2025. What's on your mind today?

So over the Christmas holiday, I, you know, I, it didn't do a whole lot, but I would still watch kind of news coming across, particularly around cybersecurity and Space Force because it's something that I'm particularly interested in right now. And this, this article came across about the DOD and, um, you know, tackling Weapons cybersecurity and it talk, it talked about, you know, that there's work that's being done to address cyber threats all the way down to like code level. But something in that really bothered me and it said, let me see if I can, I'm looking at the quote.

Um, basically it said that they know that there are, uh, vulnerabilities out there, but they're willing to take the risk not to address them. Um, I'm not sure why that would be the case. I don't understand it, uh, because it bothers me that we can do better.

And even in weapons security, I, I I feel like there is a lack of real understanding what these vulnerabilities are across the whole spectrum of cyber security. So to just say, you know, the, the risk is there, we understand it, but we're gonna move forward anyway, um, is kind of a bother. It kind of reminds me of the recent fires in, uh, California, that area Pacific Palisades that they've been, they've known for quite some time that it's a high risk area for, uh, flooding and fires.

Uh, but we did, how much did they do to, to, to make sure it something as catastrophic as a firestorm didn't happen? Or how, how prepared were they? So I feel like we've gotten into a, a place, maybe this happened in 2024 or maybe it's always happened that we're complacent when it comes to, um, predictions, right?

Predictions about what could happen in weapons cybersecurity and saying, we can take the risk even though we don't know completely what we're talking about, we're okay with taking the risk or is there something more we could do with protecting something like the Pacific Palisades from Firestorm? So it, it bothers me and as, as we go into 2025 with Gartner predicting a tripling of vulnerabilities, I feel like we've just been bombarded so much with these kinds of threats that we're just, we're numb to it. So that's my concern for 2025 and I, I feel like it's a discussion that should be had within all organizations right now about how proactive we need to be Be.

Yeah, I can imagine reading that drove you bonkers. It was kind of shocking, right? It was like, Yeah, something like Space Force Sa something like Space Force, you know, that a general would say, you know, I accept the risk without any clue of what they're, what I'm actually accepting.

I'm just gonna move forward. God knows what's gonna happen in 10 days. Well the people who are attacking us are not that complacent.

They're on their toes, right? Yeah. So we have a formidable, uh, uh, component, uh, component opponent out there that we need to be serious about.

Yeah. But we have an incoming president who wants to change his mind again and move Space Force to the state of one of his cronies. Like he, before, before the last election, he was moving it to Alabama and then Biden said, no, it's staying in Colorado.

Which obviously I pay a lot of attention to living here. And now he's talking about moving it again. So let's not focus on the secure side, let's focus on moving it and wasting a ton of tax dollars in the process.

But that's a whole nother conversation. Yes It is. All right.

Well we have a really cool guest today. I would like you to introduce you to Sal Kimmich. Is ki it Kimmi or Kimmich?

They're both good. They're both Good. You want Say I like to say it right though, so well welcome and tell us a little bit about yourself.

Yeah, actually, um, it's probably, I'd love to dive into a little bit the commentary on the DOD. Um, so I have a pretty unique background in and with open source in that throughout my career I have inhabited almost every profile of an end consumer that you can map. So I have been an consumer in a federally funded program between both the US and the uk.

I have been a machine learning engineer working within the DOD. So my first contracting role in DC was with the Missile Defense Agency. And then I moved to go work with the US Air Force, their Kessel run software incubator.

I then only left security clearance because I got married and moved to the uk and I did ask, can I work remotely for this skiff in a foreign country? And they said, no, obviously not. We read the contract, you know how this works.

Um, so I jumped into for the first time the corporate layer of open source, uh, which is generally what most people get exposure to if they're using advertising or marketing to understand it. That's the only layer that they'll ever experience. But it's really, really important and we probably should dive into why they would accept vulnerabilities, um, in the DOD specifically, uh, because it's changed a lot in the last five years.

And I think that's really positive the ways that it's changed. Um, so if you are an end consumer of open source as a federal developer, there's a couple of really interesting things. So number one, you're never going to upstream.

If you upstream once onto the thing that you were consuming in the last couple of years, you would not just immediately lose your job, you would lose your security clearance, you would never have a career again, right? And it's not one or 3% of open source consumption that is specifically in this case, federal. We're not just talking about the larger and global government consumption.

That's a different number. And that varies particularly by the European country that you're dealing with. And they've got government style OPOs in order to be able to engage with it.

But I had someone come up to me at a conference earlier this year, it was someone really early in the career and they did ask me this question, what percentage of open source consumption do you think is federal? And I was sitting at a table with a color, couple of other open source leaders and I said, Ooh, it's literally impossible to know that answer given the design of the system. But I would estimate somewhere between 30 and 35%.

And then the only reason why I'm willing to say that publicly as something slightly more than a conjecture, even though that's all it is, there's no stats. I then turn to someone who works in a major corporation that I know has not just a general osbo but a specific federal osbo. And they did not speak a word, but they did give me odd and a shrug as if that is about correct.

Right? So for every two of the developers that developers that you're thinking about consuming an upstreaming to open source, generally there's one that is consuming that information and has to have alternative pathways of communication, mainly regulation in order to make sure that those things are secure. And I think this is really, really interesting when it comes to security vulnerabilities of the supply chain.

So the first job that I ever took coming out of security clearance consumption and coming into the open and general corporate layer of production and open source was specifically sonotype. I did that because I think that they have a really, really interesting and pretty direct approach and engagement. They are really focusing on making sure that they can secure that supply chain or that end consumer class.

Now, I think in order to not be so afraid of vulnerabilities as they exist on the internet and on platforms like GitHub and GitLab, you have to understand that all of these things are built over kernels. And there are many different kernels. I've mostly studied and investigated the Linux kernel.

There are other kernels as well. And even the Linux kernel is not a single kernel. There's about seven of them that are really, really important.

There's three of them. There's like the main line, the main kernel, which most people generally use. And then there's a long term kernel of which they're very, very sincere in making sure that no vulnerabilities come into place.

And then number three, when you're dealing with vulnerabilities and open source, you have to become extremely familiar with understanding the zero day marketplace. So when there is a critical vulnerability that has been observed and been highlighted in the days and sometimes weeks before, a zero day, zero day just literally means you have zero days to patch zero days. That's what it means.

It is bad, it's immediate, and it's pervasive. Um, and so when you received a zero day vulnerability, you have to understand that all of the work has already been done to secure the critical infrastructures that you depend on. Now this is not just the DOD, these zero days and the work done before the zero day hits.

Public and corporate are protecting things like major cities, water filtration systems, those largely run on things like Kubernetes these days. So I think that's really interesting to dive into and to to consider. It's a very different world of open source.

Um, but it's increasingly important. And the nature and the style of leadership within the DOD has changed. This is not so much due to the leadership in the executive branch, they are separate, but it has changed because of one very, very specific condition.

Uh, this is the fact that generally, depending on the country that you're dealing with, it takes exactly four days of unlimited assault onto a foreign territory before you go into a condition of what is defined as protracted war. When you're in protracted war, you begin to engage in a very, very different series of process, specifically in the chains of command with DOD, so that you can be highly responsive to it. So that's had an impact on the way that open source interplays with it.

But also there's no difference in the actual nature of playing with the human gen like Titis that is open source. You have to use it because it is where the progress is made. You have to use the intelligence of the commons in order to get the right answer.

And then you have to set up additional processes to make sure that is maintained as secure. Um, so I I really enjoy watching that space and I also really am now getting to watch it from afar. 'cause I'm absolutely just engaging in the corporate layer.

Um, which doesn't typically have this kind of insight once you're in security clearance, unless you lead the country, you're probably gonna be in security clearance the rest of your career. But in the, uh, Gartner, uh, report, I, uh, I only read snippets from it, it said that 58% of they said that code level vulnerabilities would probably triple with about 58% going after government and cyber infrastructure, right? Our, you know, our utilities, our, uh, healthcare, the, the, the infrastructure, the technical infrastructure that we depend upon, that's where those vulnerabilities will be targeting.

Um, and I, you know, I it's would be a curious thing to be able to get an SBO m from every single one of those cyber uh, kind of infrastructure teams on the code they're delivering and look to see exactly what's what, what open source packages they're consuming, because those you would think would be the most then critical ones that we should be monitoring. You know, and at least minimum require for those teams to have an open SSF scorecard, right? At minimum to show that they have some commitment to adhering to security policies.

It's a, it's a, it's an interesting topic and I think that, um, there's part of us, and I'm reading this really interesting book, book called Sapiens and it talks about how we um, as our brain kind of developed, what drives us, this is a weird one, is gossip and fiction and it has for thousands and thousands of years. Sounds About right. I know.

And we will believe anything we choose to believe, right? So it's easy to say, that will never happen to me. It's easy to say I can excuse those risks 'cause I don't believe it will ever happen because we wanna believe in fiction and if somebody tells us we're okay, even though we may know the data shows differently, we're gonna believe what we want to believe.

Really interesting right now. Now Sal, you have a PhD? Uh, yes.

Interesting story. I don't, but I can explain why. So, um, so I, most of my undergraduate training was funded by, uh, the National Institutes of Health and it included both a total consumption of my cost.

So it included everything down to my rent and my healthcare. And then I was immediately positioned to do an accelerated PhD between the US and the United Kingdom, specifically working on real-time signal processing, um, for medical interventions for the human brain. So I have this great and interesting background and one quick note there, if you can get one of these unlimited, uh, government funded undergraduate degrees.

I went and I checked the contract that I was signing and it said, we will pay for all of the classes that you need to complete your degree. And I said, wait, is this limited or is this unlimited? And I found out it was unlimited.

So I in fact left my undergrad with two majors and two minors because I didn't have to pay for 'em. I could just pursue it as true education, much more European style. So I got a degree in cognitive science with a focus on neuroscience where I was doing all of my statistical work.

And I got another degree in political science with a focus on public law. I really enjoy. And I find it very interesting to look at history from the perspective of codified law because it gives you much more information about who is in power, how is that power TATed and how is it maintained or lost.

You can do that by analyzing law much better than you can by sociology. Um, but then I jumped into this accelerated PhD, so it was a three year minimum. I already had a first author paper route and if you wanna look at anything from my security clearance or my academic background, just don't search sal, search Sarah, SARA, I go by Sal because I asked mechanical Turk what three letter moniker was easiest to remember and signaled authority.

And then I used that in order to enter open source quite literally. And when I, when I look at gender and pronoun dynamics, I really, myself, personally don't care. Any pronoun said to me with respect will be treated with respect.

However, generally if it's in writing, I'm going to prefer they them because I don't want to be indexed into a specific profile that could have a bias and an algorithm. And 100% of the time, if I'm pursuing a promotion, I will request that we use he him pronouns. Not so much because I believe there's gonna be any bias from individuals that have previously worked with me.

But because it's very likely that there's an internal system that is relying on an algorithm that probably does have bias. So let me just bias it the right direction for myself. But here's why I don't have a PhD.

It's a great story and it comes from a very good mentor. So I had two different mentors. I had one at the signal processing lab at the National Institutes of Mental Health in dc technically Maryland.

Um, and then I had another mentor who was the head of the art and sciences, uh, section of the University College London, who was generally just there for life advice. And uh, I had put together a online course that taught about a thousand people how to, uh, put together a machine learning pipeline specifically for brain imaging. And if they completed that and they did a peer review style, uh, or prepared for a peer review style paper, then I got AWS open source to fund the credits for them to be able to complete it.

I got that done, I put that out online and I immediately started getting inbound requests for jobs. I turned most of them down 'cause I didn't find them interesting. But there was one job that sounded very interesting because for about four months, the CEO just kept on calling me up and we would have discussions about potentially what I would do if I went into security clearance.

'cause I was not interested and I had to be convinced. Um, the CEO was also previously a, uh, a, uh, fighter jet pilot. So very interesting because they were leading based on the profile that is impacted by the end consumption of open source, right?

Very serious. They know that if they get this wrong, right, if we mess up this vulnerability chain that will result in a death, right? So sincere and that kind of leadership style is much more available in systems outside of the corporate space.

Um, and I always look for it, but the reason why I don't have a PhD is because my advisor on the UCL side, I said, okay, unfortunately I really do think there is a job here that I am inspired by and would really like to do. And he said, Hmm, how much money are they offering you? And I said, this much money.

And he said, oh, okay. If money matters to you, I need you to know that that's more money than I'm making right now. I said, I, I think that matters to me.

And then he said, okay, you know what? Go do this. Go do this for a year, 365 days from when we stand down your PhD research.

I want you to send me an email and let me know if you wanna come back and finish. And uh, I remember the day, 'cause there were moments in and out of my first year of getting involved and stood up in federal software production where I didn't know if it was the right fit for me. And uh, but it happened to be that on day 365, I was working remotely in Barcelona that week.

So I wasn't producing code that week. I was just attending internal meetings. You cannot produce code outside of a skiff.

But I was doing a like semi vacation working on a beach in a foreign country. And I thought to myself, I can do this while making more money than I would make in literally the highest leadership position that I could ever possibly get into in open, in, uh, in academia. Uh, so yeah, it's just because money mattered to me and because I had been able to raise the signal on all the things that are important to a corporate producer or to a security clearance producer.

Can you demonstrate that you can do the work? Yes, I already had a first author paper route, so I didn't really need to wait. I had already gotten it done.

And then number three, can you excellently communicate and propagate not just your understanding of the topic, but the ability to actually do the topic to other people. Now if you have those three things, it makes it very easy to get a very, very good and interesting job because there are so few people with that combination of skill. And um, yeah, sometimes I fantasize about going back to academia, but I just cannot pull myself to do it.

'cause it used to be that I had to be in academia 'cause I needed access to supercomputers. And I really particularly love the supercomputer at NIH because if you work in this space, high performance computing of any type, you know, that our clusters are called, uh, bale wolf clusters, but not at NIH. We named them bio wolf clusters.

And I get overthinking that. I I just love that. Um, but, uh, we've Had a very interesting journey then into employment as a woman in tech.

You know, it is so many avenues and I don't think we've heard this avenue before. You know, that you yes, you basically did it. And the, um, the idea of getting, basically getting your education covered.

Mm-hmm. That's amazing. How, so how did you find out about that?

Did, did you just stumble across it or did somebody point you in the right direction? Yeah, well I had very sincere financial need. Um, so I was looking for the best opportunity out there.

And I got involved in research the second that I got to school, quite literally the first quarter of my first year as an undergraduate, I went to uc, San Diego. And uh, there was a professor there that was doing research on the cognitive design of cockpits for Boeing. And I myself am a pilot.

That's why I'm always interested and have a portfolio that keeps leaning into aviation. Um, but uh, they had shown us some transcripts that I just knew could not be correct 'cause you have to use alpha numerical when you're talking to a, uh, a control center. And I said, Hey, I think I can just correct these for you.

Um, and that was how I got involved my first year, my first week of undergrad in research. So it really, and and, and this is true. So when I, you have to be so sincere about research itself.

All of the classes that you ever take at any university that you ever take, you will never be better than everyone else in the room. And there's already gonna be 30 of you or 300 of you. But when you're pursuing research, you have the ability to see if there is knowledge that needs to be produced, go and pursue that knowledge and then share that knowledge as widely as possible.

So the first study that I was ever published on was on, uh, cockpit design of Boeing seven 30 sevens. And to this day, in my own consulting work, I use that all the time. I typically go and I'll speak to like mid-sized banks or something that has a critical service to it.

And I simply explain to them this, you now exist in a world where you had site reliability engineering and you understood that that was real time. But as we think about cybersecurity and the conditions that we have been growing into, cybersecurity is now a real time event. It has to be acknowledged in real time.

It has to be patched and as near to real time as possible. So I go in and I will teach them to use their dash bolts like a cockpit combining both SRE and cybersecurity whenever possible. But here's the second layer of that, that's really important, especially if you're paying attention to say the Cyber Resilience Act right now.

Um, there's something different about aviation than software and I think these are going to converge. We're gonna create a thing like a com, like a compliance crap. It's all gonna look the same at the end of the day as this evolves.

So if you are in a commercial aircraft, you're gonna have a black box. If the thing fails, there is going to be a perfectly preserved audit log that should allow them to understand exactly what went wrong. That is essentially the ask of the CRA.

They need you to have a verifiable and reproducible audit log of your cybersecurity methods and operations. And you should make that as automated as possible and work it into your operational design. Um, and I'm super excited to see that.

'cause I think that's really important work. And when I look at the way that compliance and regulation are evolving for software generally for open source to some degree specifically, but generally in this sector, I do think it's really appropriate to go look at the past 50 to 60 years of aviation compliance and understand how similar those things begin to look. I could not agree more.

You just just described what we've been doing at our TEUs and Deploy hub. We used to call ourself the black box of software because the problem is is that the, the, the pipeline itself for every co when we were doing monolithic, we, this argument of didn't hold as much water because everything you did in the pipeline related to that one software solution that you were delivering to end users in one big monolithic ball, right? So you could have a black box, you could see, you knew where at least what, where the logs were.

But when we're fragmented with hundreds of microservices that make up a single application, that black box is a hundred black boxes that it, nothing is, nothing is centralized. And you don't know if they're all, um, living by the same security compliance. You don't, you have, it's very hard to see that.

So centralizing this kind of data in, in the way you just described should be applied to every piece of software that we, we push out the door so we have a full view of it. And it has to be versioned. It can, it's not just for the application at the time that it's executing, it's over.

It's the history that gives us the insights. It's the change, right? It's the change that shows what went wrong.

Mm-hmm. Um, so yeah, there's so much to be done in, in software for this discussion. We recently, this continuous delivery foundation, of course I'm pushing it recently started a new SIG called the CICD Cybersecurity sig that we're really gonna look at models because pro, part of the problem of building that black box is that DevOps engineers don't necessarily have time to go figure out what they need to add to every single workflow.

And this is going to be a manual effort to build that black box. We gotta make it easy. We gotta make a, a model that people could say, here's a, an example plug in that I can use.

Here's an example command line interface that I could use to generate s bomb for god's sakes. Something as simple as that. So I'm glad that you bring that up because it is incredibly important for software as we move forward.

Now I wanna talk about your background. You said you were in Barcelona, but now you are in Italy. Tell us what you're doing in Italy with uh, awarding open source.

Okay, well first off, I think I do a lot of personal travel now because when I was on government funding as an undergraduate, the one thing they would not let you do is study abroad. They'd let you go study at MIT in the summer, but not abroad. And I wanted to see the world.

Um, so, uh, for the last three months I have been here in Kunio, Italy, which is not a well-known place, it's not a very large town. It sits on a wedge in the Alps. Uh, and it's extremely protected traditionally from uh, like land attacks.

Um, so I came here 'cause I was really interested in this place, which is well-known, two people that study sovereignty as a physical location where this city itself has remained sovereign to both political influence and religious institutional influence, which is very unique to Italy. Um, and to kind of just observe that and understand that. So I'm here 'cause I'm doing my own midlife study abroad, but, um, I'll point you right up to the ceiling real quick because you should be able to see this.

That is beautiful masterpiece. Absolutely. I know, I thought she was sitting in the Sistine Chapel for a minute when she, when she logged in.

I'm like, no, that's actually a real room. Real room. Yeah.

But, uh, that was commissioned by the family, the body family in the 17 hundreds. That's their crest right behind me. Um, and uh, I came here specifically because, you know, I've, I've got insight into the government layer, government consumption layer.

I've been working in the corporate consumption layer. Um, but there's something that everybody forgets and it's that open source is also just incredibly fun. Um, when you look at vulnerability, sustainability, maintainability, you have to recognize that these are all building blocks.

And some of them are created specifically to be supporting critical infrastructures. Those are well protected. Those are well maintained.

They'll be sitting in something like an antitrust. But there's a lot of one-offs, really interesting things that are produced in open source that aren't meant to have, have a general audience. And if they are, it's a very small audience.

So we're doing a series of awards. I'm working with Art Farrow on this and I'm waiting for whatever his videos come out to be. 'cause I said the one thing I'm not is creative.

You do that part. But, um, we're doing a series of awards based on every single Greek muse and we're gonna go find the open source, either project or committed commit, uh, community, um, that really aligns to those values. Are you working in science?

Are you working in art or music or in historical preservation? Um, if you're doing something like that in open source, I think it's really important to remember that that whole world still exists. And then to also understand this, um, it's very, very true that there's an absolute alternative to burning out in anything.

And you can call it something very simple, just call it burning in. Like stop paying attention to your retention statistics at a corporation. Pay attention, right?

If they're about loss, really pay attention to what is it that you're doing when you're doing it right. Um, and one of those things is allowing people to have and to develop their passions with technology. So I'm using this opportunity as a time to help to highlight people that are genuinely showing something that is so passionate that I find it interesting and inspiring and worth sharing.

I have one last question before, 'cause I know we're gonna run outta time, but I really have to get this question out because if there is somebody who's watching this who is an undergrad, which I hope they are, how did you find your research project and was that a government grant that the, uh, uc, San Diego was involved in? Yeah, so I, I mean honestly I started applying for funding in my first year. Whatever I could find, like, is there an associate, so You yourself were looking when you were applying for funding, where were you applying to?

So I started at the institution and then I started looking, uh, specifically into my, uh, degree program. And I started going and getting the professional level education that you need and pursuing external organizations. So two things that really helped there.

I was working with the cognitive science department and they had a bursary that was available exclusively to graduate students to support their research with training. Okay. It's not exclusive if you go and ask.

And so I went in and I got some funding to be able to pursue independent training. That's how I got connected originally with the Martino Center outside of, uh, or in Boston. And the, uh, like brain, uh, and Cognition Institute from MIT.

Um, so I went and I pursued education that was at one level higher than what was expected for me at my level because why would you wait to get it done? And then number two, I just break through whenever I see an arbitrary gate being kept closed and I will ask the questions, what are the conditions by which I can open this gate and I will ask it to the person who has the door locked. Um, one of those conditions was very important to me.

Uh, so I really wanted to join the association for the Scientific Study of Consciousness because I was studying real time interventions using FMRI brain imaging. Um, and I was told at the time that, that's great. We'd love for you to participate in our student committee, but that's for graduate students.

Now, one year later, I show up to the same person who helped me in my PhD as well. I show up to the same person who had that door locked and I said, hello, I am still an undergraduate. I have full funding, not just for myself, but for my research.

Does that satisfy the condition of being a serious researcher in this space? They said, yes. They let me join.

I was immediately working with the professionals in that field. Um, so go and look at gates, see if they're actually closed, see if you can get them open and if they are closed, make the conditions discreet, get them in writing and see if you can fulfill them. When you're fulfilling those conditions, great, you're done.

You're set. Now there is another thing that's really important. I pursued biomedical research.

So in order to do that with human subjects particularly, you have to be working under something that is called an IRB form. So the in Institutional Review board, um, I have a curd of many undergraduates old in one of those themselves under their name. But I was pursuing independent research.

I was creating my own research designs and then using the funding to get the data done and then to produce those methods. Um, and that worked. I just didn't tell myself that any of those were conditions just because they're arbitrary and they exist to satisfy a societal expectation of when you'll be ready to produce intellectual property.

And if you're pursuing open source, you're ready already. It's why you're here. Um, but one thing's really important because it's mentorship, and I know the best mentor that I ever had in life was Dr.

Lisa Iyer. Uh, uh, Dr. Lisa Eiler, uh, from the VA hospital in San Diego.

And I remember going to her early on and I was shopping around and asking every single lab that I went to, do you think I can get a first author paper done as an undergraduate? And I had some people actually laugh in my face when I said, that doesn't matter to me. That's just a closed door.

I'll knock on the next one. But I went into her office and she said like five words to me that were so powerful. 'cause I had never heard them before.

I've been well supported, well coached my whole life, but no one had ever just said about something I wanted to do. You can and I'll help you. So simple.

But that's not something that women hear. Women versus men are much more likely statistically to hear a no when making requests around funding, when making requests around promotions, all of these things. Um, and she just recognized something burning in me, the fact that I was really burning into consciousness studies and to modeling and interacting with consciousness as a computational design.

Um, and she fully, fully supported me. And I will always be grateful for that. And it's something that I make sure to say explicitly to anyone that I am mentoring, find out exactly what it is that they wanna do in life, see if I can support it, and then I do everything in my power to do that.

Sincerely, You can and I can help you. Those are very, very powerful words, right? Mm-hmm.

That's amazing. Absolutely amazing. Uh, it, you know, I, we hear, uh, the journey of women all the time and mentorship is always at the core of very successful women.

Absolutely. Mm-hmm. That's what we hear.

And it's not just mentorship from other women, it's mentorship from men as well. Mm-hmm. Yes.

Mm-hmm. Yes, absolutely. Men are part of the solution.

They're so much part of the solution. Yeah. They're also part of the problem.

But That's a yes. They yes they are. And I don't know how much time we have, but yeah, We're pretty much there.

You ladies, um, can We just ask a question? Yes. You ask your question.

Recommendation. What is a book recommendation? Tracy?

Always a recommendation. Uh, so there's two books that are super important. Um, actually I have one of them sitting right over there.

It's, uh, cybersecurity for Generative Systems. It's very good. Um, another book that you should read if you're really interested in understanding the state of cybersecurity is, uh, the Cyber Deception book.

So there's a Cyber Deception 1 0 1 book that comes out for, um, FinTech services, uh, about every two years. And it basically explains how you create honeypots and artificial systems in order to observe Adversarials attempting to get into your system without letting them do it. Um, that still is incredibly important work, and it's one of the most evolving areas of cybersecurity because now it's just agent on agent artificial intelligence.

Um, but I do wanna jump back to one thing that I think is really important to consider and think about, especially at the corporate layer for vulnerability, uh, analysis and awareness generally. There's two approaches to it that we can take. One of them is the one that most people are currently taking, and it's basically doing a scan semantic analysis and identifying either the vulnerable project or the vulnerable code snip, uh, that's incredibly computationally extensive and it may also encourage people to be pursuing a vector of comp of compound vulnerabilities always remind people that log four J in itself was not a vulnerability.

It was a compound vulnerability when in place with j and DI that made it harder to catch for a while. Um, but there's another approach to this that is entirely different, and it is using category theory in order to find those conditions. So applied category theory is a way to begin analyzing vulnerability.

Uh, and it would allow you to find categorical conditions and to avoid not just a single code snippet, but to actually be able to see, and when I say categorically, it means we have set condition, A feeds to set condition B feeds to set condition C. We now know exactly how many projects have that logical, substantial backend, and we can remove that vulnerability. Whether or not it looks the same, we can remove that logical compound across languages, across semantic complexity.

That's a direction that we absolutely have to go into. And it's not something that is a far out there idea. There are some r and d spaces that are looking into this.

And you must understand very importantly, that this is an idea that particularly the US pays attention to NIST organizational design. All of the things that it gives down for us to be compliant to are two BACT compliant. So if you're interested in this space and you wanna understand and start thinking about it, then go to the top of the supply chain.

Well, mental chain of understanding. Can we categorically provide the best solution possible? We're gonna do that with applied category theory.

It then comes down, it gets right now interpreted into a semantic language that we're communicating out. And that's where there's a lot of lossiness in communication. 'cause it's human to human communication.

But if in the next 10 years or so, and I always say apply category theory is the answer to everything, we just haven't found it yet. And that is so true. Um, but if we can close that gap, uh, we're gonna be able to avoid those conditions, not just in the current reality of production, but also moving into a quantum compute reality where they also will be able to have a much more efficient way of scanning if they're doing it as an adversarial.

So we want to make sure to categorically remove those logical conditions moving forward. And that I think is the most interesting area of cybersecurity right now. Well, thank you so much.

Um, that's a great place for us to wrap today, and we really appreciate you being here. Tracy, you got anything else before we wrap this? I'm just glad she mentioned Quantum.

Yeah, I know you're into that too. All right. Well, thank you so much for being with us today, Sal, and thanks to our audience for joining us for another, um, fun filled and very technical episode of, of Techstrong Women.

Um, we're excited you were here and as I said, keep watching Text on tv. There's a lot of lot more shows to watch today, so stay with us. Thanks again.

Have a good day. Hi everyone. Welcome to another episode of DevOps Unbound.

You know, uh, we were reminded by our producers right before going live on the show that the very first DevOps Unbound was in August of 2020. So we have been doing this now, going on four years, two months we're, we're into our fifth year of DevOps Unbound. And that to me is just mind boggling.

You know, I want to take too much time away from what we want to talk about today, but DevOps of B was originally the brainchild of, at the time the CEO of T Tricentis, my friend Sandeep Johari and myself. And we brought Mitchell in very early. And at the time it was the Tricentis CMO, Brent, and I forgot Brent's last name.

And we, we concepted out this idea of doing a biweekly video podcast series that would explore all aspects of DevOps. And even though Tricentis, and this was point of years ago, right? Tricentis obviously very focused on continuous testing and testing.

The Tricentis folks thought it was very important that we explore the full spectrum of DevOps. And over that time, we certainly have. And not only that, but we've stayed current as New forces.

And, you know, new technologies came into the DevOps space, right? We're gonna be talking about one of them today, ai, but, you know, I don't want to say attaboys or Pats on the back, but man, four plus years riding. Congratulations to our Tech strong team and our Tricentis partners who go produce this.

A special, special shout out, Jody, Ashley and Ly nor who never, never get on camera, and I'm not gonna force them to get on camera now, but these two gals, you know, week in, week out, month in, months out, they, they pick the, the topics they source the guests, they get the abstracts time, they, they make the trains run on time here. So shout out to Jody and Lin for all, all of their work over this time. Let us now though jump into today's show.

Today we're gonna be talking about AI governance, very timely topic in DevOps testing and everything else. Uh, before we jump into it though, let me introduce you to our amazing panel. First of all, she's a frequent guest on Techstrong events.

She's a friend of Tech Trunk, as a matter of fact, she's in the tech Strong. Actually, two of our folks here are in the Techron gang, but she is the CEO of Deploy hub. And, um, also an open source board member extraordinaire.

Tracy Reagan. Hey, Tracy, how are you? I'm doing great, Alan.

How are you today? It's great to be on this conversation. You know, I'm always, you know, ragged about ai, so I'm glad that you invited me to this one on this call.

I hope that we really kinda dig into the technical standards, because that's the area of interest I have, because if we have technical standards and potentially we can start building those standards into our DevOps platform. And I say that because I know that we, you know, we're, we need to build security into DevOps platforms, and we have this on our tail building some sort of transparency and accountability into the DevOps pipeline for, uh, for ai. So a huge new area, and it's great to be on this call Joining Chay and I from Tricentis.

He's, he's been on a few times before, over the last four and a half years. Martin Klaus. Hey, Martin, how are you?

Hey, Alan. I'm doing well. Thanks for having me on the call again, appreciate it.

Uh, and it was great to see you in person a few months ago. Um, yeah, so I am responsible for, uh, custom engineering at Tricentis. I've had multiple roles including product marketing, product management, and one of the things I get to do in my, in my role is to actually work with customers who are, um, defining their own AI strategy.

And so I, I'm involved in a lot of conversations where customers are trying to figure out how, how do I introduce AI in my organization? How do I make sure it's safe, it's secure that the data stays in our data center and doesn't get leaked, uh, out on the internet. Uh, and also how do we test AI technologies, uh, so that they're safe to be used in our environment?

And so that's a really fascinating, uh, conversation to have with customers of different vertical industries and different segments because they each have slightly different requirements, but they're also together all looking for the same thing. So I'm looking forward to the conversation today and talk about regulations and governance. Fantastic.

Thank you, Martin. Um, next, she's also a Text Strong gang member and frequent text strong, uh, event guest, our own Hope Lynch. Hey, hope, how are you?

Hi, Alan. Uh, so happy to be on today. And funny enough, Tracy and I had a conversation recently where we started digging into, uh, AI and some of, uh, what we saw as the issue.

So I'm so happy to be here today to talk about governance because it's, it's a critical topic. It's great to be able to elaborate on it for a lot of people. Absolutely.

Thanks, hope, and it's great to have you here. Thank you. Next, well actually last, but certainly not least, he's my co-host all for every single DevOps Unbound episode we've ever done.

He's also the CPO here Ad Techstrong, as well as VP analysts for DevOps at the Futurum Group. Mitch Ashley. Hey, Mitchell, how are you?

I'm doing real well. It's a, it's an interesting time to address this. Dora just came out with their report and kind of put the pin on the AI donkey saying that's causing us to be not as efficient, which I don't believe that's really happening.

But, you know, so getting into governance, we are trying to figure out ai, so it's a good time to be talking about governance, technical standards, regulatory, how do we do this, right? So, so folks, look, it's been, I think about two years now, right? Since chat GPT, just about two years since chat.

GPT burst on the scene, I think it was November or two years ago. And, um, and it's sucked the oxygen out of the air of every conversation we've had on tech. Sids just about it seems we're always discussing it, and we've seen the gamut of this is gonna be the greatest gift to mankind ever, right?

I'm reminded of that old movie where they have the, the book to serve how to, how to serve, or how to serve man. And it's a recipe, um, verse versus, you know, the, we must put the brakes on AI and stop it because it'll be the death of us all. AI will fight this imperfect something out of a Star Trek, be jerk kind of episode, right?

And, and seek to the d destroyed humankind. Um, I'm a firm believer that progress stops for nomad. And, and I think that's certainly been the case with ai.

It has moved ahead, full speed ahead, damn the torpedoes. However, in typical fashion, we have seen some governments, not many in the US unfortunately, but some governments have started to try to put some governance. That's what governments do.

And governance, uh, you know, around the use of ai and some private industry, public consortiums are trying to establish rules of, you know, what's ethical, what's right, what's not, right? Where does it tread on humans and human rights? Where does it help?

You know, what, what's the right, what's, what's wrong? And right here to do. Obviously, some things are clearly wrong, right?

I mean, using it to, for mal, you know, my living mal, I always mispronounce that word. Using it for bad purposes, right? Is never a good thing.

These deep fakes, and we're seeing it around election season now, you know, prevalent use, no one thinks that's the right use for it, but there are right and wrong. Um, Tracy Hope, I know you guys, both guys both have strong feelings on this hope. I'm gonna let you go first, if you don't mind.

Okay. What do you think? I think, uh, as far as governance, one of one, one thing that comes to mind as you were mentioning, um, you know, the eu, they have the EU AI Act so that they can have comprehensive enforceable standards.

There is work, I think, also happening in Canada. Australia, they're sort of looking to see what's happening, uh, in the eu, but in the US I think it's gonna take a little longer. But I do also think if there is a company that can say that they are taking these steps to have critical oversight governance, so that, um, you know, it's not a black box, maybe it's explainable, they can ensure that there's no bias.

Uh, there'd be a little bit of a first mover advantage there. I know that, uh, some financial organizations, I think Capital One is one that is taking steps in that direction, but there is a long, long way to go, uh, to get AI governance across industries and to be something that I think is pretty commonplace. Arnold thoughts?

I wanna point Out that there, there is work being done by the US government as well. We have, yes, true, true. We had the Algorithmic and Accountability Act passed last year.

Hmm, that's right. And they, and they kind of addressed many of the same things that the, you know, the EU is trying to address, which is this idea of accountability, really. Mm-hmm.

Um, you know, uh, particularly around what, and all of these go, all of this governance is really just around high risk AI applications. Mm-hmm. You know, surveillance, um, hopefully medical and warfare.

Mm-hmm. Uh, but we, the, the US is, is making some progress. But I do feel like that a lot of the governance has been turned over to the, um, European Union, uh, ever since the GDPR we're still like, you know, they're doing data stuff, so we don't have to, but we, but the US government is still, they're looking at it that that bill was passed, and, you know, it says, Hey, you've gotta make sure that you're developing, uh, fairness and, and, you know, minimizing bias and promoting transparency and mitigating any kind of discriminatory, uh, discriminatory outcomes.

Mm-hmm. Um, you know, all of this, when I think about it, and it's kind of morbid, but I think about the opening scene of robocop, right? In Detroit, And recently there was this insane article that came across and, um, tech on text, uh, crunch about Silicon Valley having a discussion of AI weapons should be allowed to kill people.

Oh, wow. Okay. You know, it's like, duh.

No. Well, that is, I think that's the first lo of robotics, right? Yes.

Mm-hmm. Yeah. Asmas lost asmas laws of robotics, And that's what that scene went from.

The robocop was right. The Rob robot goes, you know, slaughters the entire board. Mm-hmm.

So, uh, you know, so I'm sorry. Why we have, why we have, why we have governance, um, and it's only around these, these high risk systems. I feel like we have a long way to go to start trickling it down to all systems that are being written and being able to take these technical standards that's like NIST is working on, and apply them through the DevOps pipeline to start actually working to make sure that they are transparent and that some of these tools now can be applied, uh, to the process.

Mm-hmm. Opening setting of, uh, of Terminator one too, by the way, crushing the skulls of humans. But Terminator robots, um, sorry, Martin, I was just gonna jump in and, and say to me, it's fascinating when something comes along that's akin to security, it's akin to data that gets this governance, and how do we do it properly so that doesn't escape from the lab or escape from the application data doesn't leak out, um, or in this case, do harm.

I mean, you know, other software could do harm too. Algorithms could as well. You can argue whether social media does that, but AI has really jumped to the top of the list of what are we gonna do?

What do we have to do to, to secure this? And also make sure it's not used for nefarious purposes. So, Yeah, sorry.

I think it's interesting that, that you see folks like Sam Altman or even Elon Musk, uh, even asking governments to step in and help create some rules or guardrails because to see the potential of where this technology can go, I'm not sure we're quite at the turn level yet, the robocop, what have you, but because it's going to be an evolution, uh, but the evolution is happening really, really fast. And I think we're already a little bit on the back footing, uh, with, uh, some regulations as it relates to use cases outside of business. Uh, for example, you know, uh, kids in school using AI systems to do the homework for them, uh, which, you know, now puts education systems at, you know, the back footing in terms of like, how do we deal with the situation when we require essays to be written as part of like entrance, examin entrance requirements for universities.

And now anybody can just generate an essay in the voice of Ryan Gosling or whoever you wanted to in imitate and, and sprinkle in some grammar mistakes and punctuation to make it appear as if it was original authentic work. And so that's just one example of where we're thinking completely new uncharted territory because, uh, we don't know yet know how this technology can be used and applied in some cases. I thinking, especially in the business context and the kinds of customers that I'm talking to, um, the, what I'm hearing is that there's definitely a need for transparency.

And I think that's one thing that we could sort of check out very easily, because companies are asking for where is the data gonna be? What is being processed? What happens in transmission?

What about encryption? Um, who wants to the results? Who wants to, you know, the models, what models are you using?

And then how do I customize it and make it my own? Because I do not want my data to get leaked out on internet at all. But it's a very broad field, and I think the way, um, for example, the European Union on the, OR with the House, white House has also been proposing, is a good one to say.

Let's think about this from a a point of view of, of risk. What is unacceptable risk? As you mentioned, there's surveillance, um, or, uh, protected groups, uh, in, in society and other things that, uh, or medical, uh, applications.

There may be areas of unacceptable risk where, you know, we, we need to actually have some laws in place to, uh, to control those things. And then you work your way down in terms of high risk, low risk, other use cases where, you know, uh, it's, it's a less of an issue, but this is a extremely complicated, important topic that affects all aspects of, of life. I, I, I don't necessarily disagree, but you were talking to someone who's from the generation of no, no calculators allowed in the school test, right?

I, how many did, did you, were you allowed to bring calculators into your test? No, no, no, no. But kids today, they bring scientific calculators in.

They no longer have to worry about doing those equations and figuring pie and all of that. It's all there for 'em. I think we're gonna come to the same thing.

I, I think, and call me radical, but I, I think when we get to new technologies like this, our, our part of our makeup is to think, go slow. Go slow. This could have repercussions we haven't thought through.

And we know, quite frankly, that that never works, people, right? If you outlaw guns only, guns only outlaws have guns, people are, there's still gonna be a segment of the market that's gonna go as fast as they can. Um, I think almost these governance breaks, if you will, these governors on the use of these technologies is to give our society a chance to catch up, a chance to get acclimated, a chance to get comfortable with what this, what these technologies can do for us.

And I think if we recognize that, we could look at them in a whole new light. But I also think it begs another question of how do you test for AI governance? Right?

I mean, Lauren, I'll throw it at you. You're, you're zenes, you're the worldwide leaders. How do you test for AI governance?

How, how would you, you know, who's, who's minding the, who's watching the watchers here? Yeah, I don't think necessarily that it's possible to test for governance when you need to be able to, uh, well, it's more about transparency and really sort of, you know, like in the case of security, right? Like companies are used to, you know, expose and report and, and, uh, sort of showcase, uh, what sort of, uh, security policies and governance they are implementing the products.

And, uh, and we have standards around that. If you look at SOC two, and if you look at many different encryption standards, um, I, I think on the security side, we have gone off, uh, have come a long way already. I think that could be analogous to AI as well, to some extent.

Um, but the thing that I see with our customers that, that we speak with is that like, how do we effectively test an AI system? That it gives you reliable results, um, uh, in addition to how it works at the, you know, uh, is it safe to use, right? So is, is the outcome useful and applicable, uh, and safe to use as well as it technology safe to use as well?

That's where I think we're in also new territory, because if, um, AI models are passed the Turing test then, and it's a knowledge system, an expert system that, you know, uh, can involve with time, then it's going to be much more difficult to, you know, come up with prompts or tests and parameters to say, yes, today the answer I got back was acceptable. But what about next week, next month when the knowledge system has evolved and now it has been enriched with new information or with new, uh, you know, adaptations or ratings, learnings that will, will not yield a different result. And I think that's a challenge with testing that it's no longer a point in time activity.

You have to almost like have a, an ongoing monitoring system in place, um, but you also don't know what you don't know and what you should be testing for that could be exposed through some sort of loophole. And I think that's what some companies are finding out the hard way. Like if you look at some of the, you know, Canada examples, for example, where it gave wrong responses and now there's a lawsuit and so forth.

Um, and that's the challenge I think with testing ai. What we do internally is we're trying to sort of adopt this monitoring model, uh, where we are going, uh, on a regular basis, benchmark test and validate, um, and have sort of a, a red team approach as well to say like, how can we expose, um, the models in a way that, um, the results we're getting are no longer in line with what we expecting to do. Um, unless there's a new way to, uh, figure out, you know, testing of not the testing models, uh, I think, you know, that's going to be the approach that a lot of people will need to take as well.

A couple of things I would like to, uh, insert here, for anyone who's listening and is trying to, I guess, wrap their heads around, how would I start to do these things? There are tools out there, and you still have to know what you're doing, but you could look at, um, uh, Q flow, ML flow, um, they help, they can help streamline building, deploying, managing your machine learning models, actually at scale, um, open source tooling. So, uh, it definitely, you know, there's always learning curve for these things, but if you're looking for some tools, uh, that can help you get started and figure out what you should do, um, those are two pretty good ones I think that folks can use.

Then I wanna add too, on this topic, um, there's, there's, there's two levels here. We have testing and we have compliance. So if we look at what we've done through, you know, over the last five years in security, we have done everything from adding signatures to, um, repo scans to things like open SSF scorecard that looks and determines how safe software potentially, um, could be.

I really believe that these types of tools will also be created for looking and, uh, how compliant AI software is. Now the problem with compliance is that the data is often fragmented, and maybe you can look at a git repo for one particular component to see, uh, how, how compliant that component is. But it is a first step.

Compliance is a, it will be, it is important today in security in software, and it will be important in securing and making sure AI software is safe. So while testing is important, understanding the compliance levels of the code that's coming across the supply chain is not gonna change. We're just gonna have different types of tooling to make sure that the, you know, that it's, that the model is fair.

For example, how do we scan for that? Not sure. I think IBM has something called like IBM 360 that, that does some kind of fairness check.

So as, uh, you know, hope pointed, pointed out, we do have tools out there, and that's why the technical standards become more interesting, because if we can define what those technical standards are, we can also define the compliance levels that we need to achieve. And while we have seen a lot of these governance docs, um, from NIST to the eu, uh, talk about some of the basic pieces, we still haven't seen a really clear roadmap for what is compliant, and we haven't done that for, for code in general. So maybe we'll get there.

We're trying, but we have to look at the compliance question and how do we track and report compliance. So the companies who are writing software and consuming these, uh, large language models have a way to judge how safe they are. You know, in some ways this is, you know, there's compliance against standards.

I you have to have something to test against, right? Which I think is lar largely what we're saying, and there's two forms of it. One is IEEE and NIST and, and, and regulatory types of definitions of what those standards are, what what you're testing against.

Oftentimes though, compliance isn't testing against someone else's standard, it's testing against your own standard. So a lot of compliance frameworks are all about what is your policy for, for this part of security. Like if you go through a web trust compliance process, it's all about here's what our policies are to protect data, to secure this process to the handling of customer information, whatever it might be.

It doesn't tell you what the process should be. You have to define your own. Then the compliance is, we have validated that we, we have systems and processes in place to ensure that he follow those processes.

And if we are in fact following them, I, my, I have a sense that this is kind of where we're going. There'll be some things that will help guide us, um, and getting some insights too to what we should be testing against. I, I almost think we're living in this world because it's so wide open of what you could do with AI that organizations are gonna have to publish what their, kinda like their privacy policy is or what their, their, uh, uh, online community behavior policies are.

Same thing for around ai, and then do they in fact meet those? Because otherwise the requirements could be so vast, I'm not sure you could really test against everything and really know whether is this a safe system, this a safe ai. Yeah, I think Mitch, that's a great point, and sorry, um, Joe, you go, Martin.

I was gonna add one more thing. What I'm seeing already is that, um, this is not a simple question or answer already, and it's going to get even harder to answer because, um, it's going to be a more of a blended approach as well. Like right now, I think oftentimes it's pretty obvious when you're interacting with an AI chat bot on somebody's website, because that's an easy way to have some sort of customer facing interaction.

But where, uh, it's much harder to see where AI was used in process is when you have like a composite blended service where the AI tools are going to be part of the outcome. And so you can see this in art, you can see this in, in, in creative, um, uh, disciplines. Uh, you can see this even like in, you know, from a business standpoint where, you know, uh, or, you know, we were just joking about the, the after for this topic was, you know, run through an LM to say like, how, how can we clean this up and bring it down to 120 words or something like that, right?

So in those situations where AI is used as a tool, uh, uh, as part of a larger workflow, then, you know, how can you know that level of transparency, you know, be sort of transported to the, to the end user as well? Like, how do I know that the, a piece of artwork that I'm purchasing as an example, I'm not a a a dealer at all. I'm just bringing a theoretical example.

How do I know this is original work? Um, or it was used, or AI was used as a tool in the creation of, of the artwork or in an email or whatever, what have you. I, I think that's where right now the world is moving where yeah, services are more, uh, an assistant role, an agent role to basically assist with the creation of, of new work in a much more productive way.

Um, and then the, the, the notion of compliance and governance and transparency is going to be even harder to, uh, to say like, do we need to put a label on anything that, you know, has, has been touched or involved in some sort of AI tool in any creation of it? So just To correct my, uh, the name of that tool, it's an open source tool. It's called AI Fairness 360, and it is an open source.

I just Googled it, um, and it's or ML models, so something you can put in your DevOp pipeline, right? There you go. You there're any examples where things have really slowed down, right?

I mean, you think about adoption of the internet, adoption of the cloud, social media, um, I mean, the only things I can think recently around AI are Microsoft Co-pilot recall where we got ahead of our, so they got ahead of themselves and someone thought it was a good idea to snapshot your screen every, every second, but not securing of that information, the market reacted. The other is Apple's reaction to the EU AI Act, which is, uh, we don't, basically, I took it as we don't understand your regulat regulations well enough to know whether we could follow that. So we're just not gonna bring our AI to the EU until we, we feel that we can meet that compliance.

It wasn't that we don't like your standard, it was like, we just don't know what it is. Other than that, there's very few places where some, a regulation has stopped ai, uh, in, in its tracks. And, and I think one of the things we're gonna need is internal why of this is judgment call, right?

And so much of it is we're gonna need the internal board of here's how we're using ai. We're we're thinking about doing this with it. Are we delivering on the promise to our customers of safety of, uh, protecting their data transparency?

We said we're gonna be transparent. Well, are we being transparent enough? There's so much of a judgment call to this that you almost have to have some kind of an internal mechanism to say, no, maybe it's not a gate review that everything has to go through.

We can use some tools like you're talking about Tracy, to do some of those things, but it's, it's like, here's our stated policy of how we're gonna use AI and, and what we're gonna do and not do, and the line's not always clear. So how do we help clarify when we need to make those calls? I have a more fundamental question, which is what is the purpose of all this regulation and governance?

It's to build trust, isn't it? To build trust in AI and, and its use where I think the, the best way to build trust in AI is to use it and see for yourself what, what's real and not, maybe not for so good. Um, would fundamentally do having these regulations allow you to trust AI more, right?

I mean, Martin do. The, the fact that the EU has this AI act, it's not even, I mean, it's been passed, but I don't think it actually goes into effect effect until next year. Um, but by having that act, you say, oh, AI's a little more trustworthy, a little safer to use.

Now I have more trust in it, because if it doesn't, why are we doing this? Yeah, I think the, that's a good point because the, the, the what, what builds trust is obviously, uh, transparency, uh, is one way to say, look, you know, there are regulations out there, the requirements out there, here's what we are doing in order to make you feel comfortable that the technology that we're delivering, the products and services that we're offering are safe to use an environment. And so, for example, uh, just be super practically for a second, uh, tricentis, we've actually established an AI trust center that you can look up on our website where we publish, uh, our data privacy, our, the, the kinds of technology that we're using, our security, uh, uh, that we're using.

Uh, and we're also publishing the, the, the guiding principles that we use now on internal development, uh, to basically give customers that want to know, uh, a place to go to, to find out how do we, how do, what do we do internally and, and, and what do we do? And these are things that we are going to, uh, update and, uh, keep current on ongoing basis as new regulations come out. Um, I think there will still be an opportunity for like stains, like ISO and others to, uh, create more formal, uh, certifications as well that the vendors like us can, can sort of, um, uh, can achieve and publish it as well.

Just like Tracy, as you mentioned on the security side, where we have a lot of requirements already that, uh, companies can, uh, adhere to. And then also be publishing that and say, here's what we're doing and if you have more questions, let us know because we wanna work with you to understand what specific requirements do you have that we can, uh, support as well. And then it's about sort of, you know, showing and demonstrating and, um, and putting, um, you know, the proof into pudding, if you will.

I think this, this question of trust is pretty, is really important. Um, I, I, you know, I, even in Textron gang, I've said many times, I don't trust a autonomous driving car yet. I, so trust is an issue, but part of that is awareness, right?

Um, if I have, if, you know, when I was driving a Tesla, I could have put it in auto, you know, kind of an autopilot, but I never did. But I was aware that, that I was making that decision. Part of the EU act is says that, you know, if you're kind of at a minimal risk, at minimum, you need to indicate that this is, this is AI generated.

So we know in the United States right now, there's, you know, public service announcements going out about the potential of robocalls or AI generated, um, robocall that says go vote at some other location. The person that's listening to that doesn't know that that's an AI generated, uh, phone call. So, you know, the awareness of the fact that you're consuming AI data is super important.

Anything that comes across that's been generated by ai, it should have some kind of a stamp on it, some kind of a, you know, a watermark that Hologram or, or something. Yeah, yeah. So like the doctor and start check voyager or something, right?

Exactly. Exactly like that. Um, because then we can, then we are aware that we are looking at something that's been, uh, that, that is actually AI and not human.

And that's important. It's very important actually. One of, uh, one of the things that comes to mind when you say, um, do I get a sense of confidence knowing that these rules of regulations exist?

It's almost like here in Charlotte, North Carolina, let me tell you, speed limit can be 70, it can be 55, it can be whatever it wants, right? But here in Charlotte, 80 miles an hour, 90 miles an hour, pretty much everybody is doing it, and you very rarely see anyone you know, pulled over. So is the speed limit actually meaningful if there is no enforcement?

Right? I think one of the things that will give confidence is, uh, and, and not that you necessarily want to see governments going after, you know, everyone, but key stories about, uh, enforcement and how it actually has benefited people and, and made a difference, right? Having the rules is great, but understanding, um, in, in common scenarios and common use cases, real world things that have an impact, um, I think that is a big confidence builder when people can see it and connect to it in that way.

So I will just rule of thumb, so I could save someone a speeding ticket here. Yeah. In, in Florida it's a 70 mile an hour speed limit.

And the rule of thumb is they won't pull you over up to 80, anything over 80 your subject to get pulled over. Don't take that to the bank, and please don't say Alex in, In North Carolina, they often say five miles. Uh, but there, there, there are no, there, there are no, but, but You know what, bringing it back to ai, this is a perfect example where if you have autonomous driving one of Tracy Reagan's favorite things mm-hmm.

And you tell, and you tell that AI program, hey, not to exceed 75, you don't ever have to worry about it because it's not the human who's gonna go as fast as they can. Mm-hmm. If the AI is told 75, the AI is going to go 75, assuming the AI behaves as intended, and we have trust in it.

And that's the perfect example, right? Um, And if there's a bug and it goes 80 and you get a ticket, whose fault is that? This is true too.

Where, and are the tickets now automated? Because the card, you know, you're worried hooked in the law, enforce you're worried we're all worrying about a scenario that's gonna go away. If you remember back to the future, national Brown says where we're going, we don't need roads.

Didn't, we don't have the, but this gigawatts, gigawatts, whatever. Go ahead, Tracy. This brings me to another topic around governance.

There has to be industry standards. Uh, you know, financial is gonna be different than, you know, traffic control. It's gonna be different from, um, welfare or warfare or, uh, or, uh, I don't know, surveillance.

Every, I think every industry is gonna have to look at it. I think the Food and Drug Administration has done some work in healthcare, AI and healthcare defining standards. But what will be, you know, so the question is what's the, what are gonna be the standards for the industry itself, right?

The AI industry itself. So I keep going back to compliance. If you define standards and industry specific ones, uh, and then we start figuring out a way to measure the compliance of those standards.

I think we're making some progress, but bad actors are gonna do things without pur pursuing any standards as hope indicated. How do you enforce this? You know, I have a 25 mile an hour sign on my dirt road.

Do I, you think anybody's gonna ever give me a ticket on that? No. 'cause some, some neighbor put it up there and I fly by it at 40 miles hour, More than 10.

But, but you know, some neighbor put it up, Some neighbor put it up, right? But it's, so it's kind of like, you know, it's a neighbor putting it up, the EU iss putting up this sign that says you go 25 miles an hour. Um, you know, do we honor that?

And I think that most of us will try when it comes to this topic, but I do think industry standards are gonna be, uh, are gonna be as important if not more important than higher level, you know, federal level government standards on these topics. Because very different When it, when it comes to governance, now we see this, right? There're in different governance issues, score the finance, you know, what we call the highly regulated industries, financial and healthcare and, and government work and stuff like that.

And, and we've adapted to that. And as an industry, we, you know, people comply with those different vertical standards. And they're not all industry standards.

Some of them are government. Um, I, I would imagine we'll see the same thing with ai. I just, I just, you know, I have that hologram issue where ethical people will act ethically, but unethical people will almost certainly act unethically and they may not make their hologram ha the, the, the, the watermark or the, or the, or the standard, and I guess maybe this is true with a lot of governance, is the good people do their best to do good and sometimes negligently or inadvertently, they, they may miss a compliance or governor standard, but AI has the ability or the potential, AI has the potential for people who are not ethical to really abuse the system.

And I don't know if having AI acts in place, you know, we have to go to criminal stuff. I, I, I don't know what the right answer is, but, But you know, I think even the criminals are gonna have their own governance standards Really With their organizations, right? It may not be a governmental or federal standard, but these criminal organizations, some of them are very large, right?

They're, they're gonna have their own standards and it, it makes Kinda like la cosa, nostros kind of, you know, you Yes. Only go out with your wife on Saturday night, not the, not the girlfriend or whatever, you know, sometimes lack of a, a federal, well, a, a federal standard. I mean that, that's where we are now.

And states are already enacting their own laws around ai. Colorado has a lot, I don't remember the name of it, but it's basically says if you're, if you're working with ai, that is high risk, you have to, um, there's some transparency requirements and you have to have some type of review of what you're doing to make sure that it isn't endangering people. Something like that.

But that, that's Colorado. Who knows what, you know, Wyoming's gonna California or Idaho. Yeah.

So, so we're, I mean, we live in a global world, but if every state has a different policy, different law, that's a complex web to try to build products and use ai, I mean, we've seen This before. Alan, your point, your point is, is spot on, you know, this is, the whole process of governance is really around, um, uh, kind of a democratic system between the EU and between all the states and the us. Uh, everybody has to act, uh, in a gentlemanly way and, uh, agree to those standards and comply to it.

And, and that's what drives it. And if, if we don't, um, we choose not to, um, there's very little accountability or it's very hard to even find out that you're not, uh, complying to those standards. So governance is hard.

It really is, uh, especially when you're trying to define governance through a, a democratic process where everybody makes their decisions across these countries and are relying on everybody to, um, be honest, and not everybody is. Guys, we're over time, I gotta be honest with you. I apologize.

But yeah, it was an interesting conversation. I wanted to let it go a bit. I think we could all agree at this.

I think there's still a book to be written, or at least a few chapters in how AI governance is gonna take shape here and what its effects could be in a global marketplace. And it'll be up, you know, the ethical people, as we said, will act athlete plea, the trics of the world will try to give people a sense of, Hey, we could test for this, or we can, you know, given transparency, we can show you that, you know, to, you know, be the transparency that it's in compliance or what have you, test for it. But a lot of it's good, I think is the book is still yet to be written.

So we're trying, as I, I guess the, the right thing, right, is we're trying and it, and this is still evolving. Anyway, Martin, Tracy, oh, thank you guys for coming on our DevOps Unbound episode today. Mitchell, thank you as always for co-hosting.

Thank you Tricentis, for partnering with us for these four plus years. Now. We look forward to many more reminder for those folks out there.

This, what you just watched was a prerecorded version of DevOps Unbound. Every two or three versions, we do a live audience, and you get to ask the questions, and you get to make the comments, and you get to participate in this discussion. And we love having you.

So stay tuned for our next live round table of DevOps Unbound. But until then, this is Alan Hummel for Techstrong Group. Have a great day, everyone.

We'll see you soon.