Techstrong TV June 3, 2025

Hey, everybody. Are you suffering from Cloud confusion? You might not be alone.

You're watching Techstrong Day. Hey, everybody. We're back.

And I'm in Florida again. So here I am in the studio. I'm not in New York in my usual spot.

And we have some other special guests joining us for the first time ever. Keith Townsend, the CTO advisor. How you doing, buddy?

I thought I'd to wear a new hat just for their occasion. Thanks for having me. Are you gonna sign that an autographic or can we raffle it?

No, I'm, that was a firm. No, that was, no, no, no. This is a, uh, I think this one is a, definitely a keeper.

All right. I like it. It looks good.

Thanks. And on my left all the way from California, here in Florida is our own CMO advisor. Lisa, how are you?

I'm doing great, Mike. It's great to be here doing this with you in person for the first time. We've been doing this remotely for so long.

It has been a while. And, you know, and I've seen you out on the road being the host for a lot of our stuff. Yeah.

And, and that's been great, because that means I don't have to go. So this is awesome. All right.

And then of course, joining us remotely, v Schneider, who's usually in this seat with us, but she's remote in Florida today, I think. V how you doing? Oh, good.

Sorry Mike, you're audio clipped out there. But no, it's, uh, great to be here and Boca Raton, even though I'm not in studio, I am close by. All right.

Maybe we'll see you later today or tomorrow. And finally, Steven FoST, who is still in Ohio, correct? I am back in Ohio and wilder than ever.

Uh, but I did get to spend the week with, uh, Mr. Townsend last week in, uh, beautiful Emeryville, California. All Right.

And we're gonna discuss that shortly, right? In fact, we have a lot of things coming up that seem to have a lot to do with you guys in the field day. Let's join into our first topic.

There's a report out from, uh, our friends over at, uh, VMware by Broadcom. And on the one hand, I kind of looked at it and I was like, well, how much of this is a real issue? And, and, and is it a vendor issue or is it a customer issue?

And the survey points out that of the people that they surveyed, half are using the public cloud and the other half are gonna be using more private cloud. But it was a slightly, you know, three point difference. And ultimately everybody said they're using both and they're using 'em for different classes of workloads and different use cases.

And I can't help but wonder, are the only people who are really confused about this, is the vendors who were trying to compete with each other, Keith? Or do they, uh, do the IT folks have this in hand? Yeah, so as Steven mentioned, we're both at Oxide Compute last week, and Oxide is a small 75 person startup that is building a private cloud experience based on hardware.

Really interesting, uh, approach. But more importantly, we were meeting with nine IT execs, business leaders, uh, decision makers. And, uh, a couple of 'em were specifically VMware customers, one of which VMware Cloud Foundation, VMware's flagship product.

He's, uh, the, the path to it was Rocky, but they're there, they're happy. And another one, um, uh, a CTO of a big biopharm company said, you know what? VMware ain't broke.

We ain't fix. There's nothing to fix. Then a mattering and other folks who were in public cloud and a mix.

And it seems like most people, for the most part, where their workloads are sitting at, they're happy. But it is a mix of private cloud, public cloud with a couple of 'em on the extremes. But most people sell, sell, selling in that middle where they have work workloads, both in private cloud and public cloud.

Mm-hmm. Are we ever gonna get to the point where we can centrally manage all of this? 'cause it seems to me one of the issues of the day is that each of these platforms has its own control plane.

And if I have Amazon over here and VMware over here, that's two control planes. Now I add Google or Microsoft Azure three four, and then I got a different squad for every one of those control planes. So, you know, can we get to the point where maybe we can finally harmonize all This stuff?

You know what? My friends at HPE Morpheus are going to tell me Yes. And I'm going to tell you no, we will not get to the point where we can harmonize all of it.

We've tried. I think for the most part, if you're running Kubernetes, uh, and you're, you have Kubernetes running in the VM in, in Google, or you're using GKE or you're using A-K-S-E-K-S, if you don't know what any of that stuff is, well fine. Okay?

You're probably walk watching the wrong show. But if you are, uh, using, and you're all in and, and Kubernetes chefs, we'll, we we're going to get to a harmonized control plane. But if you want the value of the cloud, if you want to use AWS Lambda specifically for your short running, uh, nodes, you want to use Google, GKE for your, uh, specific runtimes, or you're using big cur, you're using these things that actually add value.

There's no way to consistently abstract that across, uh, a single control plane. So we've tried, we've failed time and time and time again. That's not, I don't think, a, a pursuit.

Most, most CTOs are, are, are under undergoing. Steven, what's your take? Do you agree with that?

Is that what you're seeing as well? Oh, absolutely. And uh, as Keith said, it's incredible to spend time not with people like us who are in the punditocracy and, uh, talking about this for a living, but people who are actually doing this for a living, this, uh, this incredible group of, of CIOs that we brought together last week, and that we, you know, folks like Keith, uh, get to talk to on a regular basis, you know, their insight is very practical.

Essentially. They're looking at this and they're saying, you know, what makes sense? I think that the thing that was kind of eye-opening to those of us who talk about things for a living was that people are not, um, universally fed up with VMware.

They're not saying, we've got a ditch VMware and we gotta ditch VMware now. Um, even though that's what you hear a lot about in the blogosphere and in the, in the press and, um, what you're hearing is people being very practical about it and saying, you know, we gotta evaluate it. We gotta decide what we're gonna do, and we're gonna do what makes fiscal sense and, um, production sense for us.

And I, I think that that matches as well some of the insights that we've gotten from, uh, the future of intelligence side. So I know that there was A-A-C-I-O insight survey that, uh, very much jived with the content of the article that we're gonna link to here in that, uh, organizations are definitely evaluating where to run cloud workloads, but the big focus is optimizing placement between public and private, not abandoning, uh, public cloud or, um, you know, going all in on, on, on, on private. Uh, I think that it really matters.

Uh, you know, at the end of the day, I think the dollars really matter. And that was another thing from, you know, Keith was talking about with this oxide solution. Um, they're not trying to knock off the small companies out there who don't spend a lot on public cloud because it just doesn't make sense for them to invest in something new.

Uh, they're going after the big fish who have big spend and need to, uh, offset that spend in some way. And I think that that's a, that's a great go to market for them. Um, and, and I think that there's other opportunities in this market for companies as well.

As Keith mentioned, you know, the, the, the, the Morpheus solution is something that's really impressed us. It, it's really a practical consideration, where can I get value? Mm-hmm.

Yeah. I don't think, to his point, I've gone to maybe seven conferences in the last five months, and every one of them has a anti VMware vibe. There's marketer up there who's up, who's saying, you know, here's an example of a customer who left VMware to come join us.

Now they always have like three or four. And to your point, I don't know if that's a survey or a real trend or not, but, you know, and some of the issues are real, some of it is, you know, if I'm a smaller company and I don't really want to pay for per processor instead of on per socket, you know, that that may impact you. Whether that's enough to anger you enough to migrate, which is an expense.

Yeah, that's considerable is varies from company to company. But, you know, as I look at the survey, and I can't help but wonder, is the marketing messages from a lot of these companies just off? Because it doesn't quite jive with what the actual customer experience is in terms of thinking.

Well, That's what I got outta the article was, is customers are speaking loudly with their voices and where they're putting workloads, they want that choice. They want the flexibility, they need to have that flexibility. They need to have the ability to be multi-cloud based on the workload, based on the needs.

Um, I think we're always going to see, um, competition between the vendors trying to get their message out. I think a lot of folks are, if they're on VMware, the migration would be such a pain and it would probably impact the business, the ability to drive revenue, the ability to drive pipeline, and they don't wanna do that. So I think we're gonna see these marketing messages continue to be a little divergent because folks are trying to compete.

It's not a co-opetition space, really. Like you were saying, we're not gonna get that central management plan. It's just not, we try to, it doesn't work.

Um, but customers are speaking with their voices and where they're putting workloads, and at the end of the day, it has to va it has to be bringing value to their business. Do you think it's getting any easier to migrate workloads? Is the cost of switching still high or, 'cause part of the thinking goes in for a lot of these companies.

It's like, well, if I'm gonna switch from one platform, and I'm not even gonna pick on VMware, it could be any of them. You know, am I, is there so much code in there that I'm kind of locked in anyway and the cost is higher? Or can we reverse engineer that code in a way that's practical?

Yeah. So this isn't surprisingly, this isn't the technology problem at all. Uh, this is a process and capability problem.

You know, we, let's pick on VMware in in two different ways. Two things can be true. A lot of people are migrating off of VMware.

While most workloads remain on VMware. Broadcom was masterful, masterful in the way that they approached their licensing of VMware. The vast majority of VMware workloads are with the large enterprise.

While VMware has something around, last time I checked 500,000 customers before the Broadcom acquisition, the, those workloads are not spread off evilly. So as you're reading the articles and people moving away from VMware and Mass, that is true, but that is by design. That is, those are the customers.

Broadcom did not want want. Uh, Proofpoint is talked to one of the major backup vendors in the VMware space, and they, they've only seen a 3% decrease in VMware workloads over the past year and, and backups of the VMware workload. So the problem isn't the, my, uh, technical migration and being able to move from AWS to GCP from GCP to prox, the, uh, I wrote about this a couple of weeks ago.

Uh, VM migration is a solved problem. The problem is around process and capabilities. One solution doesn't have NSX, another solution doesn't have vsan, another solution doesn't have, uh, Lambda.

And just not having the capability means that you have to change your processes, your people training and all of that. And that becomes the true barrier Yes. Of transformation and moving from one platform to another.

That Cultural, yeah. Change management is a very hard process undertaking. Yeah, I was just gonna add about the migration process.

It's interesting from a sustainability angle, because sometimes there, there are efforts to do a migration to improve sustainability, but the migration itself can increase the workload computationally, and that will unfortunately raise emissions. So, and sometimes from the sustainability angle, it could be catch 22. Yeah, Great point.

And Steven, these adventures are different. I've talked to three different customers and it kind of went this way. Um, smaller customers said they could get off of a platform like VMware.

It took 'em maybe a month and a half. Other companies I talked to, we and, and we're talking about the entire stack, not just the VMs underneath it, which switching to VMs is easy. I think it's their stuff that sits on top of it.

That's the problem. They were like, eh, it'll be 18 months to two years. And then I talked to another company that just finished their seven year migration effort.

So I was like, you know, this is kind of crazy. But so what drives that in anybody's mind? I mean, why not just draw a line in the sand and say, this workload goes here and that workload goes there and never the two shall mix?

Well, I think that's the problem. I, I think as you identified there, the real challenge is the operational aspects. I mean, VMware has done an incredible job of becoming the operations platform of choice for so many companies in terms of, of everything other than running virtual machine workloads.

You know, that that's really what they're doing there, is they're figuring out ways of, of streamlining the whole process of IT infrastructure and IT operations. And that is really, really hard to move away from. Ultimately, I think what's gonna, you know, spell the difference is, as Keith pointed out, the fact that Broadcom really wants to focus VMware on a certain set of customers, and they're gonna direct development in that direction.

And the customers they don't want, well, I think that there are lots of other companies out there that are hungry to attract those folks. And in fact, many of those have a really compelling story for the customers Broadcom doesn't want. And I think that that's actually good for the good for those companies as well, because what we're gonna see is people who want that customer are gonna come up with products that make it easier to migrate.

Um, your comment about somebody, uh, finishing a seven year migration was that to VMware or from VMware, because I could see, uh, somebody just finishing a seven year migration to VMware in 2025. It was actually from VMware to a different platform. But Yeah, so to just drive home this point and use a completely different technology as an example, moving from AWS if you're just consuming EC2 really easy.

But what happens when you're, you've built a whole application stack around Lambda and, uh, I am, and you're using AWS identity management to run authentication and security throughout your entire application stack. So your, uh, your database, your, uh, application servers, your storage, all rely on I am for identity management. How do you move that, if at all?

Like, there is no equivalent to I am anywhere in any, in any other, uh, solution. So that becomes the barrier and the true, uh, limit of when you're, when you're talking about migration, how deeply did you adopt the vendor's actual, and this is why we can't just abstract away the complexity. How deeply did you, uh, uh, adopt the vendor's value, unique value add?

Mm mm-hmm. All right. I'm gonna finish this one out, but I want to point out one last thing.

I was talking to somebody about this very issue and going back to VMware, they're saying, where am I gonna get all these new administrators and engineers? Am I supposed to retrain everybody on a new platform? 'cause all my people think of themselves as VMware administrators and they're not overly excited about getting a new certification and training and everything that goes with that.

So there's a lot of, uh, inertia in the system, shall we say. So I think to our point, there's gonna be a lot of platforms to manage in the future. We may not get to manage all of them together ever, but this is what we do for a living, right guys, Hey, we'll be back in a minute.

Welcome back to the Textron gang. Well, recently I had the opportunity to attend the code remix conference, the summit actually, that was, uh, based right here in South Florida. And some of the thought, thought leaders that I spoke to had some pretty interesting stories when it comes to modernization.

And one of them was Eric Costlow, who's the senior product manager of Azo. And he talked about how his team helps engineering organizations identify and eliminate unused code across legacy systems. So let's take a look at that interview and then we'll come back and I'll talk about it.

Hi everyone. I'm at Code remix in Miami, and joining me now is Eric Koslow, who is the senior product manager at Azu based in California. He's here in Florida for this conference.

Eric, it's great to have you here. Thanks for having me. It's a great conference.

Yeah. So Tell me about the work that you're doing. Um, the work that we're doing is we're helping people, uh, maximize engineering productivity by figuring out what they don't have to work on.

Many applications have built up a lot of code over the years, so as they go to modernize that code, we help them figure out what they're not using, don't have to modernize and really recapture that engineering capacity. That must be a very busy time for you. Everyone is digitizing and automating their new processes.

Is this a difficult process when people are doing this shift and how much automation is being used? A lot of automation is being used. That's the, the role of our partnership with Modern is we help provide a unique data feed of what code you aren't using, and then they help go through and make those changes to the repository so that it becomes very visible and people can save that time.

Well, that sounds like it's also helping with productivity and efficiency. Huge amount. Yes.

Can you give an example of maybe a use case that you've seen where this has really made a difference? Yeah. When we've gone through and looked at people's applications, I've seen them shipping a lot of things that they simply don't use anymore.

Uh, a few weeks ago I helped someone identify that they were still shipping, uh, what's called a soap service, which is an integration platform from Life 2002. Here it was 2025. They were still shipping that code hadn't been used in a long time.

Another group, I helped them re, uh, take a graphical interface, uh, library out of a web application wasn't used at all for other groups. We helped them minimize the amount of code that they have to actively maintain. So it works for your custom code as well.

You know, when it comes to being more green with software, that's one of the themes I've heard often is, uh, you know, not having sloppy code using code efficiency. Do you see that process kind of also keeping things more efficient in that area? Um, it just helps a huge amount.

The, the savings that we have is more about the time savings that people spend going into work every day, having to work on their applications. When you can identify what you don't have to work on, you have more time for the things that you do want to work on. You can deliver more features in terms of energy efficiency, that is a helpful thing.

The less code you have, the less things you have to do. We just help people connect on minimizing the amount of work that they have to do by focusing on what's important and what's not important. So it's been great for me to talk to a lot of people who are in these modernization initiatives of improving their applications and then pointing out things that they simply don't have to do anymore.

That's always good news. Eric Costlow, thank you so much for joining me. Thank you very much.

Alright, Steven, this, we're gonna have some more grain interviews from Code Remix coming up. Well, one of the things that we touched upon, of course, was having more green software, meaning, um, less sloppy code, less bloated code. And that was some one of the objectives, um, in the pro in the platform that he spoke of.

And also, of course, I mentioned this code remix event. And this comes, of course, timely because Azu uh, just recently, uh, uh, has a new partnership with a strategic partnership with Modern who sponsored the conference about, uh, to do with automated code refactoring and analysis. So, um, it's an interesting conference 'cause it was very hands-on to see the developers working with the solutions right there.

Um, as we discussed earlier, so this was one of the interviews that I was looking forward to sharing with everyone in the panel. Steven, does our code need to go on a diet or are our applications obese? What's going on here?

Well, absolutely, I I think that those of us who spent any kind of time in, um, application development have seen that there's a lot of cruft in there. Um, you know, there's in, in many cases, uh, you know, the, just the nature of, uh, software development, especially of custom applications leads us to having bloated software. In fact, I'll just point out that the recent CIO insight survey from future of intelligence shows modernization is a key driving decision for nearly 50% of CIOs right now.

And that's definitely what we're hearing anecdotally as well. It's one of those things where, um, they realize that modernization is a challenge. Um, they have to do it and they're, they know that it's bloated.

Other, I think drivers for modernization though, are a need to focus on cloud native platforms. And of course, concerns about security. Uh, many legacy applications have, uh, well known security vulnerabilities and unknown security vulnerabilities that companies are very, very concerned with.

That was an anecdote that actually came up last week as well. During our CIO uh, meetings. Uh, one of the, uh, one of the CIOs there was talking about an amusing, um, application vulnerability that he had encountered in his company's code.

And this is a major company that's a key part of the, uh, world financial sector. Um, and, and these things are out there. They know they're out there.

And, and, and so the idea that you can streamline code and modernize code is very attractive to these folks. Keith, we don't have ozempic for code. Not yet.

Not yet. But, but what's to be done about this? I mean, or is this just a fact of life?

No, so timely. Uh, I was in the internal FUTURUM threads, our CKO at rum. Eric shared how he completed three, no, well, not the three PR releases, pull requests for a project that he's working on completely with AI as he moved on to another project.

So, you know, I have a fairly big piece coming up on Textron ai, on vibe coding and how CTOs should be considering supporting vibe coding. And this idea, Bonnie, I would be really interested in your, uh, uh, input around vibe, coding and sustainability. But this whole idea that I am now going to apply my agents to solve some of these bug problems and basic PR bug fixes that I would actually have assigned to a junior engineer The productivity, the more I talked about sustainability with all the guests that I spoke to at the conference, productivity went hand in hand.

And being more efficient with the productivity. Exactly. I was gonna say, and the efficiency as well.

And that's key that organizations really need to employ that because when there's code bloat, it slows down performance. There's potentially customer issues with retention, with churn. So they really need to be focused on this, making their folks more efficient, more productive, but also ensuring that they're, what services and products they're delivering to the end users are what the end users are demanding and will stay using.

Do you really think that AI agents are gonna write code more efficiently than humans? 'cause they were trained using code that humans wrote. So what would make the AI agent any better?

So everyone is horrible at code. Uh, as, uh, I did an interview with, uh, AWS principal engineer Brian now, and he does not trust anyone's code. Mm-hmm.

Not his own, not, uh, not his teammates and not, and not AI's code. What you do need is systems and process to do re uh, code management, review management. And what AI is really good at is at language and communications, and may not be experts in do domains, but when it comes to structure and efficiency, I've run almost everything that I create from, uh, social media posts to blog posts to large research.

I, uh, uh, AI has proven itself pretty adept at, um, at streamlining my language. Mm-hmm. So where I can be verbose, it will absolutely streamline the meaning of it without, and it's gotten pretty good at keeping my voice without doing that.

So we're seeing that in anything that's language, uh, related, especially code. All right. I wanna engage in a little philosophical conversation about programming languages then.

And maybe I'll throw this at Steven's way first. So, Java and all these languages are abstractions that we created so that we humans could interact with machines. So if the machines are gonna write code, why are we giving them the same abstractions that we gave the humans, when the machines should be able to develop a programming language, maybe even write closer to the machine in assembly or something that is gonna be fundamentally more efficient than what we as humans did with Java.

I mean, why are we putting a, the, the human constraint on these machines? Because efficiency isn't the metric. Um, you know, we're not trying to make machines more efficient.

If we were, we would certainly all be writing in, uh, assembler for the, uh, the hardware itself. Uh, that is probably the least important consideration. Exce outside of certain tight and, and, and highly important, uh, loops of code, you know, in terms of, you know, low level, uh, matrix processing for ai, that sort of thing.

I think that, um, really what the reason that we have all these abstractions and higher level languages is, um, in search of understanding. And for me, you know, to go with what Bonnie and, and, and we discussed previously as well, uh, AI is a great way to help humans understand what this code is doing, uh, why it's doing it, how it's doing it. Uh, you know, AI assisted comments, um, are incredibly valuable.

But at the end of the day, I think that it really is going to come down to higher and higher and higher level abstractions until perhaps we're going to get to the point, uh, to, to, I don't know if I'm a fan of vibe coding Keith, but I think we will get to a point where, uh, maybe we'll have executive coding where executives can sit down and, and have a conversation with the AI about exactly what their, uh, uh, custom applications are and aren't doing in order to have greater understanding of really the goal of software, not just the features and functions and, and methodology of the software, let alone that low level aspect of it. Now, could that theoretically, I guess, uh, to Mike, uh, yeah, sure. That could theoretically evolve into a, a world where AI is writing low level code and then translating it for humans and, and, and have discussing it with us.

But, uh, ultimately I think that, um, languages, uh, programming languages are here to stay. I don't know. I think couple agents are gonna get together and go, silly humans, let's go build a better programming language, and we'll communicate amongst ourselves with that, and then we'll show it in something that the humans can understand later.

But that's how I look at it. You know, what's interesting about this, Lisa, though, if you ever see developers, they're all code shy and you know what I mean? They don't wanna show their code everybody.

Right? Right. And part of that has to do with the simple fact that they're not always sure how good their code is, and they don't wanna show it to somebody else to get criticized.

Right. Do we need to get beyond that now because Yeah. Yeah.

But that's one, from a cultural perspective, that's hard to do because they're so ingrained in their workflows and they want their workflows to be such a certain way. Um, I think there's, vulnerability is a good thing, and I think in a lot of aspects of life, I don't think it's part of the developer nature yet. It could be something down the road, but I think that's a change management challenge that an organization is gonna have.

And it's gonna differ, I think, at each organization, based on the level of code shyness that they have. Um, but people, people don't like change, and that's gonna be a challenge. Yeah.

And I, I think we're starting to see organization at the organization where 50% of the code is written by ai. This is the, the weather, we call it vibe code, ai, assistant coding. Uh, Dell announced that de Dell Tech world, how they have AI as, uh, assistant code for 20,000 developers.

It is a reality. So I think as turnover naturally happens, we're going to see more and more code within the organization. Chaired and Steven just, you know, just a little bit more pushback on the vibe coding.

Uh, it's not just entry level developers doing vibe coding or the, the, this whole executive level. You know, I'm going to describe what I wanna, I don't know anything about coding. Uh, where again, I'll, I'll point to either our CTO Eric or, uh, uh, Brian Lau from AWS talks about how they're now writing hundreds of unit tests, uh, using what we might describe as via coding or AI assisted coding stuff that they could never have done, even with a team of junior developers and churning out better code.

Now, the DevOps report from a couple of weeks ago sponsored by, uh, Google would tell you that our code is becoming less reliable. I think they surveyed the wrong users, uh, and it was the wrong organization to survey. So we do need observability from a industry's perspective to actually understand the impact culturally of, uh, what, what these code assistants are doing to our organizations culturally and getting much better feedback loop loops so we can better use these tools.

Yep. So Lisa, who's gonna be ultimately responsible for the code that's being created by the machines? Because I could easily say, you know, the machine ate my homework along with the dog.

Yep. That's a really good question. I think it's gonna be different at different organizations.

Um, we talked on this program before about, from an agent perspective, who's managing agents? Are they managed like employees? Who's responsible?

Is it the human? Is it the agent? Is it both?

Is it a shared responsibility model? I think it could be, but I think we're gonna see differences at different organizations, and hopefully we'll get some customers come into the forefront that are showing how they're doing it well in terms of it's the responsibility of the human, or it's the shared responsibility with the agents. Um, I think we're gonna see a lot of variability in that, but I, I look forward to seeing those customers coming forward who are doing it well and can prove that they're doing it well and in terms of their customers being successful with their technologies and services.

All right. Bonnie, close this out a little bit. What was your sense?

What, yeah, what was your sense of the community down there at that event? Were they excited about AI agents? Were they scared, or A little bit of both, I would Say excited.

And one of the things that, that you, you all were just talking about with, um, sharing the coding, that this was one of the more interactive types of conferences I've seen where people were on their laptops more exchanging and working as they went with what they were, what they were learning. So, um, it was a global event where people from all over the world, um, so this was, um, I would say very collaborative in the way they were interacting and using AI with the excitement of, of modern and their partnerships with DIF blue, um, and with awell that were mentioned. All right, well, everybody look at it this way, genie's out of the bottle, so there's no going back.

So maybe we can just enjoy the fact that we're gonna have more streamlined code that's more efficient than ever and let's just move forward. 'cause well, that bridge behind us is burning. Anyway, we'll be back in a minute.

Discover Techron Group, the epicenter of tech innovation. We are your go-to for reaching IT, leaders and practitioners worldwide. Our secret impactful content that sparks awareness, engagement, and top quality leads with us.

You'll access editorial websites, streaming videos, virtual events, custom content analyst research, and more. Join our satisfied clients. Let's revolutionize your tech journey.

Contact us today and tell your story to the world in the most powerful way with Textron Group. Welcome back, uh, to the Techron Gang. I'm Steven Foskett, organizer of the Tech Field Day events.

And, um, the last week we were on site, uh, with our security field day event. And of course, this was live streamed here on techron tv. Uh, I wanted to bring a couple of the takeaways from that event to the panel here on Textron Gang today.

Uh, essentially the three big takeaways from my mind, number one, uh, was a question about, uh, supply chain security, or a point that the value of supply chain security is growing. We heard this from HPE at our special event with them, uh, last month, and we certainly heard that from Dell Technologies as a big part of their pitch as well to the security space. Uh, number two, uh, this whole question of data protection and the role of data protection in security.

Certainly a lot of data protection companies have been trying to make the pivot from, uh, being backup and recovery to, uh, security companies. And of course we heard about that from companies like Veeam as well at our security field day event. And then number three, uh, we had a question really from our rum intelligence research.

Uh, our survey of CIOs and C iOS showed that consolidation on specific security platforms was very much desirable in the C-suite, but we were curious how that would translate into the, uh, professionals, the security professionals. And we did hear a bit of pushback from the delegate audience where they were saying, look, I'd rather have a best of breed point product, even if it's not integrated into an overall suite, while others were saying, you know what, I'd rather have that integration. So let's start off there.

Um, I know, uh, uh, that Mike, and, and, and folks, you might have some, uh, feelings about this third point. Let's dive in right there. What is your feeling about, uh, best of breed versus point solution?

I was shocked and appalled to discover that security team professionals don't trust vendors. I mean, I mean, holy crap, who would've thought, I mean, the issue here is nobody trusts the false positives generated by any of these platforms. So they're trying to correlate, you know, the results across three different tools to see if they're seeing the same thing before they go hunting for something that, you know, might take forever.

And security people are tired of being the, um, the, the child who cries wolf, and then everybody kinda looks at them and says, you know, thanks for the alert, but we can't find it. Or, or, or, it doesn't exist. And, um, so they've become more cautious and, and practical in certain levels.

So I, I understand why they're gun shy of platforms economically. It makes sense logically from a workflow not having a whiplash your head between different GUIs makes sense. But the fact of the matter is they just don't trust anybody yet.

So it's gonna have to be proof in the pudding. But I don't know what you're thinking or saying. MCP and AI is gonna fix all of this.

There'll never be another security issue again, because AI will save us from all of the complexity. I am half joking. Uh, I think, uh, security professionals will tell you, or a data professional, anytime you deal with mass amounts of data, there are going to be varied opinions and there is going to be varied approaches to handling that data.

And one solution does not rule them all because each area of security, just like each d area of business process, has its own unique, uh, attributes. And trying to put everything un under one roof will invariably make your organization probably less secure because, uh, you're, you're losing some level of fidelity when you go to a all-in-one product. It's just by it, it's the, it's the, the, the byproduct of that system engineering approach.

So yes, we're going to continue to, you know, you can pry my, uh, what's up gold for my dead hands of, I'm dating myself on that one. Uh, you can pry that away from my dead frozen hands. But the reality is that, uh, I would rather my security professionals be armed with the right tools to keep my organization safe than to, uh, standardize on any product.

And whether it's to save money or standardize on, uh, on data flows, there are solutions for that. Use the right tool for the right job. I think what we're seeing though, sorry, Mike, is from a messaging perspective, I've done a lot of marketing for data protection security vendors, and they want to be that platform with which everything is managed.

So that's, that's really the message that I see a lot of vendors preaching is that we can do all of this for you. So it was interesting, Steven, when you said you're seeing a mix of folks that want the point solutions best of breed to be able to, to secure the right workloads and those that want something that really helps them reduce this massive volume of security tools they have in play. But I think the messaging has been consistent for years from the vendors.

We should be your one stop shop. So that's interesting. But that's not, it sounds like that's not what the customers are saying necessarily.

It's maybe it's a 50 50 and and vendors have to get better at going, where are they best of breed for organizations? Well, It, it does seem like the customers, uh, if you ask the C-Suite folks what they want, they want a unified platform. They want to reduce the number of vendors.

Uh, that's what, that's what our, uh, the intelligence report shows. Uh, we had, by the way, uh, future and researcher, uh, Fernando Montenegro also presented during, uh, security Field Day and some of the research that he was sharing shows that the C-suite absolutely wants to reduce the number of tools. They want to have trusted vendor partners.

They wanna work with these people on a very high level and have that be sort of their, their solution. I think the skepticism comes really from the technologists who are out there saying, but, but, but as, as well as frankly from the kind of interesting solutions that they're seeing. I mean, you know, we're we're, uh, for example, we heard from C Packett and they're talking about, um, really technical stuff using, you know, deep packet inspection and being able to, uh, really see what's going on.

That's not something that necessarily is gonna be visible to the C-suite, but is very visible to the technologists and the technologists start saying, but, but, but, but, and to your point as well about data protection, I think that that was a great thing to bring in there as well. All the data protection companies wish that they could be the, the security company of choice for their customers. But, um, my concern has always been can they talk the talk and walk the walk or are they ultimately really just data protection companies?

Are they really just backup and recovery companies? You know, Keith, uh, we've been to a lot of these events and we've talked to a lot of these firms. Um, you know, are data protection companies, have they truly made that turn into security?

No. The, the, this is just, you know, if you're talking about a sim, if you're talking about, uh, getting in the network and flow of the network data packets, those security companies, uh, from a backup and data protection perspective are very good at when the data hits disk. They're not really great for when the data's, uh, in flow or in transit and stopping, whether we're talking about ransomware, some type of remote execution flaw being within the system and detecting the security events as they happen.

They don't have the complete view. Are they part of your solution? Absolutely.

Which means it's why we see at RSA, what some six to 800 sponsors on the show floor at RSA some ridiculous number that you'll never get through that. I think that is a leading indicator of whether or not we'll see consolidation of tools when end users and, and professionals stop asking for these unique variations of, so solutions. When investors stop investing in these new solutions, then we'll see.

Maybe there's, uh, a consolidation and on a one-stop view for all of their security needs. I'm gonna go back to your earlier point. I might argue that the whole debate might be moot soon because I can imagine a world that's gonna look like this very soon.

I'm gonna have an AI agent that's gonna speak to multiple backend engines for processing various analytics related to security. I'm going to use the MCP protocol, the model context protocol created by philanthropic that everybody is embracing to connect to all those different engines and those agents are gonna call 'em and invoke those agents on demand as required to deal with different security issues, including the backup and recovery when something is detected. 'cause I want that to immediately happen, should that there be an a near and dangerous present threat, and I'm gonna have all the benefits of a platform without necessarily having to standardize on a single vendor to get there.

Yeah. We're already seeing it. And not just, I, I think security is leading the, the way when it comes to data analysis and data cleanup and, uh, getting rid of that kind of first level, uh, false positives.

But let's bring in another business analogy. Campbell Soup at the AI Infrastructure Summit and that I attended a few weeks ago, and San Jose talked about how they've essentially gotten rid of, they didn't say that they got rid of their master data team, their SAP master data team, but they've gotten rid of their SAP master data team because now they're using AI to do the triage of data analyst, uh, that low level data analyst work. They were getting 4,300, you know, kind of errors a day that they would have to clean up.

There'd be a team of people doing that clean up all but maybe 5% of that is now done by ai. So there is absolutely, uh, uh, stories in, uh, SOC operations where AI is doing that first level of triage on these false positives that wrecked kind of the, the, the CISO's budget in trying to filter. So that is naturally gonna move.

And I surprisingly seeing this adoption in security more than any other area in enterprise it, I think. So you were at RSA, yes. Are the security companies fighting yesterday's war and not thinking about where the war is gonna be?

Their big focus is on cyber resilience. I talked with data protection companies, I talked with enterprise storage companies who everyone pivoted a couple years ago to, we are going to enable you to become cyber resilient, which is a journey and no one vendor has a definition of it. Um, I think that leading with that message was loud and clear at RSAC that we're going to help you become cyber resilient and, and on, on that journey.

But I think the vendors are, are evolving, they're pivoting, but I think it's around the resiliency message. And that to, to your point there, Lisa, is exactly what, uh, Fernando told us as well that, that he's hearing from his customers as well as in the future surveys. Um, this, you know, cyber resilience is a message that the C level can embrace and that the C level appreciates and the companies like that are, uh, basically using that terminology and, and, and, and driving in that direction are helping the C-level, uh, audience get on board and their unlocking budget.

And frankly, I appreciate that because one of the things, you know, even the lowest level cyber analysts will tell you is that there are going to be problems. There are going to be failures. You have to be resilient.

You can't just have a impenetrable wall that protects you. You have to be resilient when failures happen. And, and the final point that I'll bring back to, uh, from security field A was this whole question of, of supply chain and supply chain issues.

Now, again, that was a real eye-opener to hear HPE talking about how they were able to use this concept as a, a door opener for server sales, basically that by, by saying, look, we can guarantee the supply chain from one end to the other all the way to your door. You know, we're hearing about that from companies like Dell. Um, I think that reflects an overall concern among CIOs that this fragmented ecosystem is rife with potential security issues.

So this whole conversation kind of makes my eyes roll because I was on stage given a presentation at Symantec back in the day talking about the need for the integration of data protection and cybersecurity. And you would have an event and they would kick off and, and the year was 1995 and nothing changed in all of that time. And I have to wonder, you know, the security people were over here, the data protection people are the IT ops people and they don't seem to have any real reason to kind of cooperate much lately.

Yeah, but you're talking about a speaker, uh, a larger challenger enterprise. It, the, you know, I've made the argument that, uh, the rubric argument that if all of my data is on backup and I have the metadata associated with it, shouldn't my biggest partner internally be the data scientist? Because the data's already there.

I don't have to go out and search for it. It is literally right here on the secondary system that I can access the same for security and every other aspect of it. But you talk, Mike, you're talking about the complexity of enterprise IT as our good friend within the tech field day community, uh, Justin Warren likes to say scale breaks everything.

And, uh, when you start to scale your operations, uh, the, this whole ability to have interfaces from one team to another, it's just, it's a really complex people problem. As Lisa, you would understand Yes. That that's it, that's the people problem.

And that's, i I, one of my co-hosts years ago used to make stickers and, um, Justin Warren, remember Justin? Yeah, that, That was, yeah, Justin from the tech field day community, that's Yes. Yes.

So he made the sticker once and we were, we were co-anchor and, and the sticker said, and I'll never forget this, humans ruining everything since forever. And the people challenge is often, I mean, we talk about it in cybersecurity, all the weakest linker people, it also could be the strongest link, but the people problem is a huge organizational challenge in every business, regardless of industry has. I think we're gonna get to the point soon there where we can all agree that it was the machine's fault.

So we'll see how it goes. Guys, I wanna thank you all for sharing your insights. I also want to do a couple of shameless plugs coming up.

Steven, I think you have a Cloud Security Day event coming up, right? Well, not cloud security, but uh, yeah. Cloud Fuel Day.

Uh, we are, yep. This week, uh, Wednesday, Thursday this week we're going to be live, uh, with a packed two day event. It's gonna be live streaming right here on Textron, uh, tv, uh, including the over the top app.

We're gonna try to get the live stream up there if we can. But otherwise, the, the videos are certainly gonna be there, um, all day Wednesday and Thursday. We've got, uh, bunch of companies presenting all sorts of aspects of cloud, including, uh, some big names that you might recognize.

com for more about that. All right. And we're also gonna be running the RSAC virtual conference later this month.

So a lot of the video interviews that you were involved in Yep. Will be part of that as well. And we'll have some interesting keynotes.

So do check that out later this month. And finally, we have all this amazing content on Textron TV coming right up after this. So stay tuned.

And once again, thank you all for watching and we'll see you again tomorrow. Hey guys, thanks. From the throw, we're here with Wing Toe who's the CTO for Digital ai, and we're talking about the impact AI is having on coding.

And boy, there are a lot of opinions about this one wing, welcome to the show. Hey, thanks for having us. It's great to be here.

So opinions are all over the place. And let's just start with something that sounds relatively simple, but, but some people will say that the entry level coders will become more advanced more rapidly because they'll take advantage of AI to do things for them that previously they simply would not have been able to do. On the other end of the spectrum, folks will say that the senior developed folks will no longer be assigning stuff to entry level folks.

'cause the AI will do that for them. And, uh, there won't be anybody, whoever gets trained to do entry level work anymore be, and therefore will not know how things are done because everything will be done by the more experienced folks. And someday we might not have anybody who actually knows what they're doing.

So, um, on those two extremes, what's your take? Where are we and or for all I know it could be a little bit of both, right? It and it really has become, um, I think people have come maybe a bit too polarized with their thinking on this.

And um, just, just to kind of put this into a context, I thought, oh yeah, because, well, I think this is, this is somewhat in itself a repeat of history. So I thought, you know, what's the most popular programming language at the moment? Python, you know, by stretch looking all these different surveys, there's Python, I thought, I know out there there's people complaining about Python.

So I then looked up and they, sure enough, indeed there's people complaining about Python. Oh, people don't know enough about the fundamentals of memory management performance issues. Oh, it's not as type strict as some other languages.

And it's the same thing that's happening with ai because I think what's happening is that people are thinking, yep, there's gonna be a higher level of abstraction. And I think developers will end up picking up, um, different skills and they won't be able to, um, touch some of the systems as low level, or they might not in the day-to-day job, but they will, they will still understand the fundamentals of computing, or at least the, the good developer will always understand, um, the fundamentals of, um, writing software. But I do think it will help with where we're seeing technology go at the moment that in fact, the stack has got more complicated.

So I think that especially if we say you, we also hearing at the same time cognitive overload, that was the, that was always like the thing last year, cognitive overload, too much complexity, platform engineering. And I'm thinking AI could help bridge between these two things. Um, it would help reduce the cognitive overload because people will be able to, um, take advantage of ai, but there's still an awful of work out there and there's still an awful lot of code that people have to write, maintain, debug, and the senior people will just be doing slightly different things.

I don't think they'll be delegating just to agents, at least not, not in the near term. So I think it's gonna be somewhere in between. I think we're gonna be having more empowered developers overall.

The type of skills they need will be different though. Um, rather like as we've moved into high and higher levels of la uh, programming languages, they will be more abstracted from the underlying, um, hardware. And for some disciplines that's gonna be difficult.

Like, um, say gaming people need to, um, so get squeeze every ounce of performance out. Yeah, people like that need to really understand the connection between high level and low level. But for most programmers, that level of abstraction actually helps 'em be quicker.

It helps 'em do more things. And AI will also help them, um, be able to access and learn and try new, new things out quicker. But they would have to learn to use ai.

Um, uh, there there's this kind of imagination that we've gone or we've gone all the way to, like now people can just type in the record requirements and uh, I will pop the code. The reality's not quite that straightforward. Um, you still have to have enough knowledge of what you're trying to do and how to coax the right answer.

And also to understand whether you get the right answer or the right code or the right, um, uh, specification. All of that still needs to really be done by someone that has skills in defining, prompting, extracting, and reviewing. So I think it's going to be some new skillset entry and advanced programmers are going to need.

Um, now I think that's gonna be a bit of a shift. You know, I was having a chat with somebody and they said one of the things that seems to be apparent to them is that they're writing less code, but they're reading a whole lot more of it and they need to review it and understand it. And they can't necessarily outsource that to the AI agent because ultimately they are responsible for that.

Yes. And I think that's something that is going to be, um, a bit of a, a bit of a, uh, a shift or maybe an increase in what was already happening. Um, 'cause a lot of the focus has been on, um, what happens to the software developer as a, as a producer of code.

But I think what this will, um, this will cause is actually more, more requirement on having better processes and also better dis um, better disciplines, um, across the teams and across, um, organizations. Because one of the things that, um, will happen is that, um, it's, it is true now that people should be reviewing code, but it's gonna be doubly so that people will need to be reviewing code. They need to be ensuring it's thoroughly tested.

Um, they need to ensure that, um, it's, um, that is, uh, properly integrated with other systems because all of those things now become one more rapid and those are where the problems will occur because, um, AI isn't perfect. Um, and, and the, the junior developer's not perfect, um, but the rates ch the rate of change and the possibility of things creeping in will be more so. So I think there is also a shift to not only things like having to do more code reviewing, um, which one would expect if you're, um, producing code of, um, ai, but also the whole set, um, better establishment and checking of processes, which will, I think also create some element of, um, change because AI will also be used there.

Because again, there's a lot of work there. How can you make sure that, um, processes are here to, how can we, um, automate as much of that as possible? How can we use AI there?

So I think the role of the developer will not just will, will continue to shift into having to think not just about the code, but the entire, um, lifecycle and ecosystem from inception to actual delivery. And of course, the, the feedback. The other thing I do hear from developers when it comes to reviewing code is though they find it difficult to debug code created by an ai because they don't really understand how it was constructed in the first place.

They weren't really involved in it. So how do I kinda review that code? And I'm assuming at some point I'll get some help from another AI agent somewhere to review the code.

But, um, how can we debug something we don't understand? I, I think thi this is this, this will, I think this will be an interesting, um, an interesting journey for us as a, uh, as an industry because, um, uh, certainly some of the things that, um, humans do that AI certainly at the moment is not, um, so preoccupied with, although I suspect it will change, is that, um, AI generates code, but it's, it's drivers may not be the same as, um, as human. So, um, especially good, good developers, they're encouraged to do as much reuse as possible.

Um, they're encouraged to write more efficient, clean, easy to recode, but AI is designed to get the job done, um, and create the code around it. So I, I, I think it's not, I think sometimes people are overstating that it's, well, can't understand how it does it, it's, it's, it's, um, it's actually in some ways probably easier to understand because it's not doing it using more sophisticated, maybe creative ways of writing code. I don't, I think we've all had developers who are sometimes a bit too creative and a bit too ingenious in writing their code and that other developers can't understand it.

But I think the code is not necessarily as elegant, um, probably more verbose, um, not as straightforward. So maybe less interesting to read, but I don't think it's going to be, um, harder intellectually to read. Um, but it does also mean that there's gonna be a lot of it.

That's, that's the other problem of AI code. It doesn't tend to focus certainly at the moment on reuse. So I think what some of the things we're hearing is just that there's a lot of, a lot of code to review, but I think this is also where people may start needing to use AI to do that task, but not the AI that wrote the code.

So I think this is gonna be an interesting world going forward, where people have to use different systems for different parts of the software delivery lifecycle, but the developer still needs to be in control. The developer can't just assign a, uh, another AI agent to go and review the code, the ai, it has to be guided, especially if it's, if it's AI checking on ai, then there needs to be human interaction. Um, but I think that, I think the, the, the, some of the, some of the concerns about, oh, this code is unreadable.

I'm not sure whether it's quite unreadable. It just might not be very elegant or pleasing to read. Fair enough.

What is the future of a DevOps team look like? Because the more I talk to folks, the more it starts to feel like it's, you know, humans augmented by multiple AI agents, and somehow or other, um, just like humans collaborate, these AI agents are gonna have to find ways to collaborate and something is gonna have to provide some level of orchestration across those things. But, um, how far down that path are we?

Because right now I feel like we've got a bunch of individual little productivity tools, but not quite the symphony we need I I com, I com. I completely agree. I think this is going to be the next level of shift.

Um, as we think about, um, how these systems come together. I think previously everyone was thinking in terms of, um, silos, oh, how can I do, um, maybe test creation? Um, how can we ensure that there's, um, more AI driven, um, security scanning, for example, all these pieces, but they were in the silos.

And this idea that of AI agent agents or just ai um, agents is now creating the opportunity to think, how can I com? How can we start composing them? Um, but also, how can I use AI to also think about how these compositions will react depending on the industry?

How am I depend on the, um, the, uh, the processes, um, that an organization may have, um, or the policies, um, and best practice in the industry. And I think this is the way in which, um, DevOps is going to change, is what is it's going to be, how to take advantage of, because I think there will continue to be evolving agents, improving agents rather than we have already have different, um, tools in the software delivery lifecycle. Uh, now I think they're going to start being exposed as agents and, uh, challenge will be as a, um, a DevOps industries, how can we start rather than orchestrating, um, tools where we're orchestrating agents, but not just, um, statically, but how can they actually be genetically organized themselves?

And I also think, um, one of the other shifts will be not just, uh, running through the pipeline as we would normally expect, but all the tasks that people do when, as part of the pipeline approval processes, um, if something goes wrong, the remediation of that, I think those will start being agents at the orchestration system will also take advantage of, so will be a much more intelligent pipeline, um, as this industry evolves and the interfaces like, um, MCP, the agent to agent, um, all these type of, um, interfaces and standardizations are going really start making this possible. The other thing that I'm trying to wrap my brain around a little bit is the fact remains that our current DevOps workflows are very dependent upon scripts and they're very brittle. Do you think as we kinda move into the age of AI, that either A, will be easier to rip and replace scripts as needed?

Or B, will the whole system maybe just become a little more resilient? I wonder whether, um, I wonder whether script says they are today will be there in the future because, um, as same way, um, the, and this is c and this is one of the things that's, um, going to be, I think, uh, uh, Melbourne evolutionary shift been that a lot of the focus has been on, um, co-creation. Um, which clearly is, you know, is, is a great, is a great usage of ai.

Um, because there's enough structure, um, there's enough, um, there's enough rigor in, in to apply ai, um, but there's as much work involved in the rest of the delivery of software. Um, so the writing of, um, the writing of all the scripts, and especially with cloud native stack, there's, um, configuration files, there's multiple scripts, and there's scripts not just for, um, uh, one piece of technology. There's scripts for all the different layers of the, um, cloud native stack.

I think people start looking at how those can be generated using ai. So there may still be scripts underlying it, but will they be human created? Um, that sc that, that is seems to me like, um, like a prime place start applying AI technology.

And in doing so, of course it can then be reactive and more dynamic depending on changes. So as you say, they're very brutal at the moment. 'cause even though it's great because we now have much more, um, um, configuration as code, it's great, but someone still has to maintain the configuration.

Someone still has to change the configuration and there is an awful lot of it. Um, especially even just things like moving across the pipeline, moving from system to system. AI can be very well placed to actually manage and control all of that, um, so that people, humans can focus on the, what is the, I'm trying to do, as opposed to the the how.

Now, of course, all of that has to be orchestrated because the change has to be driven by, by something. So I think that's where we can start seeing, um, how these things can be more robust, um, because they may not always be, um, human written in the first place. So what should DevOps engineers be doing?

'cause a lot of them right now, you know, they're kind of like standing on the shore and they can see this tsunami of codes starting to build and they can see it's coming their way, but they're not quite clear if what to do about it. And maybe their first instinct might be just to run, Which can be understandable. 'cause I think, I think unfortunate DevOps engineers have got two challenges facing them.

Um, the fir the first is with the, with all this AI generated, even though I said, yeah, software engineers should be more disciplined, they should add more process. And, um, they, they should not fear AI because AI is just a tool. Um, but on the flip side is very possible for these, um, these, uh, developers to let things slip by and not do all the good software engineering practices.

And the DevOps engineer, especially because they're responsible for the, the, the quality of the release. They are now not only have to do what they normally do, but they have a lot of it to do. 'cause there's a lot of code being generated and, uh, have velocity.

So I think the first thing they, um, they have a challenge on it is how to ensure that any code, because it's going to have AI generated code in it, or it's gonna have some ai, um, assisted help somewhere along the way. So the first challenge is how are they ensuring that what is being provided and what they have to deliver has a, has a degree of quality and checks. And I think that that's very much, um, adopting some of the best practice that we have today around automation, um, reusable pipe, um, pipelines, golden paths, all the things that we're seeing become that we we're already discussing.

The, the industry, they're becoming even more vital because I think without, um, automation, you're not gonna get the speed without providing golden paths. You can't check that these code, this new code isn't providing. Um, so for example, uh, um, uh, IP information that usually shouldn't be, um, private data, et cetera.

So I think the first challenge is manage the code that's coming through to ensure it's got compliance and rigor and quality. Um, for example, that is all been, um, has been, um, passed through, uh, code reviews by a human, at least some human tick the box, even if they may not looked at it as thoroughly. Um, but then the second thing is I think they should, um, DevOps engineers should be looking at how they can take advantage of AI themselves because, um, the pipelines themselves can be complicated and the, um, the steps within a pipeline can also be complicated.

So how can they use AI to help them, um, understand risk of particular changes, um, so that they, that they can, they can make decisions based upon as much information, but not have to look at the information themselves. Um, again, maintaining the scripts. Um, how can, uh, uh, DevOps engineer, um, create scripts that are more robust because, um, they have AI looking for it.

I think the, um, the other opportunities they have is also, um, looking at how AI can help troubleshoot as well. I think that's ano, um, another area, especially when, because in these modern systems, especially with more coding, shorten space of time, how to synthesize all the information they're getting from all the systems and then being able to, um, actually take advantage of that, all that information. But to do that, you really need to drive insights into it if possible.

And providing AI front ends to, um, all that information is something that the, uh, um, DevOps and JU who try and take advantage of. So do you think that the quality of the applications that we're building and deploying will steadily improve as we get to ai? Because today, I mean, you know, people will complain that there are vulnerabilities in the AI generated code, but last time I checked, there was vulnerabilities in the code generated by humans too.

So, um, but as, as we get to the point where maybe we're not relying on first generation, uh, general purpose LLMs anymore, and we have LLMs that are trained for more specific tasks than we might get to better applications, I think they've really, actually really opportunity to improve the quality of this, um, software being, um, being delivered. Um, because I, I, you, you make a very good point because there, there is this, oh, AI's gonna introduce all these errors and all, all the security. I'm thinking humans are just as effective at it.

In fact, sometimes they can put in the library and then lots of people have to say the problem. Um, so I think it's really true and i, I, I think it is entirely possible to, um, take advantage of ai, especially if it's AI in combination. So, um, uh, uh, try, um, having AI write more secure code and just assuming it's more secure won't make it better using AI in part of the DevOps process to ensure that, um, we've taken advantage of, um, uh, techniques better than just pure scanning to look for vulnerabilities and also to make it easier for people to say, um, do, um, uh, software bill of materials s bombs to check whether you've got the most recent versions of libraries, very unfortunately, um, labor intensive tasks that, um, people may not do now because it's, it's, it is heavy weight, but you can do with AI because it's, it's a well structured task.

Um, it's very suited to, um, automation. Um, so it really is possible, but it relies on people actually adopting, um, best practices and robust processes. So it should improve and it can improve.

But I think part of that is the industry really, um, making sure that it adopts good software engineering as a practice and using AI to help, um, help, help, um, help strengthen that, um, as opposed to just assuming AI will do it automatically. And this may be overly simplistic, but, uh, I, I can't help but wonder, and is it coming down to the point where I'm gonna have column A and column B and column A is stuff I don't like doing. And column B is stuff I do like doing and everything in column A I'll just give to the ai, It will, it is a possibility, especially if column A is the things that, um, are, don't require the same level of creativity, but are things that need to be done, um, that tend to be, um, time consuming.

And it, and it's interestingly not always code, code writing. In fact, most of her is actually write like writing code, um, convinced them not to write code. It may be a problem there, but there's all these other tasks like, um, test, test creation, um, making sure they run, um, tests, um, filling in documentation.

Um, all there are, and I think there are a number of tasks that are, will fit into that bucket. And I think it's actually gonna be a positive thing because having those done well is actually very important. And if those things are, um, what people will consider toil work, but it's to work that has to be done well, if that can be all, because AI is effectively automating work, if it can automate work that's actually would lead to an overall better, better result and actually faster, faster software delivery, um, whether all of the things they don't like doing will be in column, mate, I'm not sure.

I don't think, I don't think the technology is quite there yet. Um, but it's interesting 'cause there will, I think there will always be a column B of the creativity and the bringing together of things that really is the, um, the things that people love doing is, is, is building things. This should really just help 'em build things more reliably and more robustly.

But right, it has to be used what it has to be used word it can't. Um, I think just, uh, just throwing stuff at it will not be the result people expect. All right, folks here heard it here.

If you have an open mind, maybe just maybe we might remember why we have the joy of software development in the first place, but we'll wait and see how it comes about. Wing, thanks for being on the show. Thank you very much for having me.

All right. And back to you guys and the student. Hey everyone, welcome back here to Techstrong tv.

You know, I, as much as I like having first time guests on the show who tell us about companies we haven't heard of before, I actually really like having friends back on the show. And this gentleman here has been my friend for, um, I don't know, kit, it's gotta be almost 10 years already, right? Eight years, something like that.

That's right, yeah, for a long time. Kit Meer Kit is the CEO of Plain Sight. Of course, I know Kit a lot longer than he's been CEO of Plain Site kid.

It's so good to see you, man. I hope all's well. How's everything?

Great to see you too, Alan. You know? Yeah, I think, I think we first met back in, I dunno, JFR days 2016 probably, right?

So yeah, almost a decade. Geez. We're uh, we're getting up there.

Yeah. Up there. You still look like you're 25, man.

Oh, Thank you. It's all the, it's the magic. The magic of ai.

Yeah. Yeah. It's all the magic.

You ain't kidding. So, kit, as I said, I knew you before, you mentioned Jfr. Give people a sense maybe of kinda your journey of how you wound up here as the CEO.

Well, you know, I've, uh, been in this software game for about 25 years, believe it or not. You know, I spent 10 at Microsoft and then, um, got to be part of the Kubernetes project at Google, which was, uh, unbelievable experience. You know, we launched it the same year as Google Glass, which I guess is back in the news.

Yeah. Um, and then, you know, after Google, uh, spent time at Jfr where I met you, but we got to, uh, you know, that was an experience of taking a company, being part of an executive team from, you know, series C through IPO, and really doing something fantastic. And then I, you know, I did another startup, uh, in the software reliability space.

Noble Nine. Yes. And I joined PlaySight about eight 18 months ago.

Um, and, you know, it's been a really interesting journey. I think the, the unifying theme for me has been about, um, building big software systems that lots of people can use and help developers and help drive efficiency, reliability, scalability. That's been my, my main, I guess, career focus.

And I've gotten to do it in a lot of different roles from, you know, writing code, uh, running teams to now being a CEO. And, uh, it's been every stage of company too. So it's, yeah, it's been a great career so far.

Hopefully just the beginning. Yeah, I hope so, too. A long career.

And it, it, I gotta tell you, as a friend, and you know, Pierre, I, it's, it's been a joy to watch that growth right at, at Jfr kinda we're running business development, corp development, stuff like that. And That's right. Um, You know, and, and it's kinda where my background is as well.

So it's always, it's always been great watching this ride. So you mentioned Plain Sight, you're there about a half a year. I don't see a year and a Half.

A year And a half. A year and a half. I'm sorry.

Yeah. I don't think you sell glasses, but I do see the eye chart behind you. But tell us what, what's Plain Sight about Kit?

Yeah, please. Site is a computer vision company, and what we really are focused on is vision, infrastructure, and what we're seeing in the world. I mean, what's changing right now is that all the cameras in the world are generating all this data that instead of being consumed by people is gonna be, and is already being consumed by ai.

And what we have in the world is all this data. And the question is like, how do we process that data? So you kind of imagine like, I have all these cameras and I have all these AI systems, how do I marry the two together?

And I can't just throw GPUs at the problem, right? It's not just a hardware problem, it's a software problem. It's a infrastructure problem.

So from, you know, my experience and my, my team's experience at Google and my CTO is at Amazon and PayPal and, and we, we both worked at Microsoft, we, we, um, we bring this kind of perspective of large software systems that we're taking to this traditionally data science problem of computer vision, right? An interesting thing is, you know, look, we know how computer vision works from a science perspective, and the software has existed for that for a long time. Open CV has been around for 20 years.

Um, is this kind of a solve science? Actually, here's, you know, the book, the, the, the, the textbook on computer vision, right? The GPUs are there, uh, the cameras are there.

And so the question's like, well, what's stopping all these benefit businesses and people and applications from getting the benefits of computer vision? And what we think the answer to that is, you know, plane sites, technology focuses on really two, two, uh, uh, areas. One is how do you define the unit of computer vision workload?

And our solution data is something we call a filter, which I can talk more about, but it's an abstraction for computer vision applications that let you build these large, robust pipelines to process visual data into structured data. And the second part is in the machine learning and training, or what we call the data supply chain. So, you know, the software supply chain, I know you know that very well.

Mm-hmm. The data supply chain is the question of how do I get my models, which are powering all these AI apps, how do I make sure that the right data, that the data is improving over time and it's sourced ethically that I know exactly where it came from, the lineage and provenance of that data all the way from the camera to a trained AI model that I can use in applications and inference. Those are the two sides of what we're doing.

And the exciting part is now Plane site has decided to announce this open source project, open filter to take the first part and make it, uh, you know, hopefully the standard for how computer vision applications are defined and run. But plain sight, I think fundamentally what we're doing is we're democratizing the ability for computer vision to happen to be cost effective, scalable, secure, liable, and fit into these enterprise environments where traditionally they've struggled to get past the prototype phase of adopting vision and AI in their, uh, in their enterprise. You know, plain sight's been around a while.

I know it's gone through some, you know, reformation, pivots or whatever, but, you know, for me it was always about, it's always been about computer vision though. Yeah. Give us an idea.

Yeah. So the, the evolution, uh, with, with, uh, plain Sight really was originally focused on, I would say, services and professional services and solution building. And there was some core IP that was developed in that.

And I think the big change, um, that, you know, strategic change that I brought to it was really about focus. And the question was, well, what should we focus on? A lot of the advice I got was focus on one industry vertical.

And, you know, you get that advice enough times. And then I thought, well, maybe this is the obvious advice, so maybe I should ignore it, right? And instead, what we ended up thinking about is how can we build a general purpose, vision capability so that we can serve many, many verticals.

And then instead of us having to go learn an industry, which by the way, that's market risk. 'cause what if we get the wrong use case that people aren't willing to pay for, right? I can't do everything.

If I pick wrong, then that's existential. Instead I thought, well, what if we could let the community decide? What if we could let the, the, the wisdom of the crowd, no crowd, right?

Crowd source it. Mm-hmm. Exactly.

So we put out the general purpose vision capabilities that can be used for all kinds of different industries. So if you think about like being able to read text, uh, in a, in a scene OCR, right? You think about, you know, object detection or other kinds of use cases.

The the problem is not that the science of doing that is actually very hard. The problem is how do I manage and orchestrate and kind of build that into a solution? This is really the insight.

In fact, um, one of the interesting things that led us to, uh, a lot of these realizations was watching what happens in the community. So if you go to Reddit, for example, the computer vision, uh, subreddit has like 115,000 people talking about computer vision apps. And when I look at that, what I see is a lot of frustration, frankly, with the state of the market for people who are trying to just get basic cast on that.

Like they're all doing the same thing repetitively over and over again. So part of our big aha was what is the sort of software infrastructure data management solution to this computer vision problem? 'cause the issue is not computer vision per se.

The issue is how do I build the infrastructure to support vision workloads? And once you kind of think of it that way, this led us to a really interesting invention, which is a new abstraction, which we call a filter. And the filter, you know, you're, everybody's seen a filter on Instagram or Snapchat.

It's an app, right? It's an app that takes, uh, a video. It gives you a, uh, AI power generally, uh, modification to that video and maybe has some action or data that comes off of it.

Now, if you take that front end concept of a Instagram filter and turn it into a backend concept, which is a filter as a, uh, workload we can do is take these apps which combine models plus code into a common a PIA common interface. And now we've moved from monolith to microservice. We've moved from, you know, VM to container in a way we've got this new abstraction for describing computer vision.

'cause generally, if you talk to computer vision people, we'll spend a lot of time talking about how do I train the model? I spend a lot of time talking about, oh, the application logic. But they don't really put those two things together.

And once you do, as you know, look at what we see in cloud computing, look at the power and the scale of these kinds of systems is because we can understand not only the work we're trying to do, but also how the work is constructed so we can manage and optimize and orchestrate that work. And that's really what Plain Sight is doing differently, is we're focused a lot on this workload concept, um, as well as, you know, helping people train really high quality models from, you know, their data supply chain. But that combination of the two really gives this powerful solution that I think can break through, um, you know, the, the impasse that a lot of people have.

And that's the evolution of the company. It's been from kind of boutique computer vision to now building this dev ecosystem, separating the, uh, the core software from the, um, the, uh, the sort of solution development. And the next stage you wanna go to actually is quite ambitious, which is to set the standard for how computer vision apps are created.

And that's why, you know, we've chosen to, uh, open source. Open source, The open filter. That's right.

Open filter is the, the new open source projects for defining computer visual workloads. You know, kit I, a couple weeks ago, I was at a conference, a company called, uh, automation Anywhere. I don't know if you've heard of, they've been around 20 years.

Much like Plain Sight, they kind of, kind of invented the RPA right. Robotic process automation industry. But they, they're walking away from RPA and into their, what they're calling a PA agentic process automation.

Because as good as RPA was, it was always missing that little something. To me. It's like putting an STP gas additive to boost my enzyme, you know, to boost my, uh, octane in my gasoline, right?

And make it a hot rod. To me, when I look at computer vision and the, the history and state of computer vision, you're at that same juncture kit where ai, I mean, and as usual, your timing's impeccable, right? Ai, yeah.

You know, ai, AI is the octane boost for computer vision because now everything we've always thought it was possible to do and could do, but you had to have resources and you didn't have, didn't have that intelligence, right? You, you either had to have a person and it got, gets quickly overwhelmed, no matter how many people you have. And the scalability issues, right?

Of, of capturing all this to try to find the patterns and do these things well with ai, you know, as they say in the, in, on the street, s**t got real, right? It's real now. That's right.

You know, and, and so now, now we could do this. Right now, there's so many things that are possible. And as you say, you don't wanna be the T-shaped one that just goes into one vertical.

You wanna be the broom shaped that we, we it scales across and let people decide, you know, I've got this great technology computer vision that I'm marrying to AI capability. The, the possibilities are endless. I, I couldn't agree with you more, but there's one big problem missing from the story you just said.

I'll tell you what that is. Good. All of this video data is running on infrastructure designed for humans.

Yes. All of this, this is the fundamental problem in the world that I am out to change. Okay.

You take one thing away from like, the big picture is what I like to call the vision internet. And I know it's an insane thing to say that we like need a new internet, but we kind of do for this purpose because look, all of the systems connecting these cameras that are streaming video, they're all designed so that it's smooth and beautiful. So you can have a great experience as a human.

Well guess what? The robots, the agents, the AI systems, all these, they don't care about that. I mean, look at how much money OpenAI is losing because of please, and thank you, by the way, I always thank Chad GBT, because now that I know it costs the money, you know, I always thank Chad GBT.

Oh, you Do. I I just did it beforehand 'cause I thought it was cool. Yeah.

'cause you're A nice, well you're a nice guy, but I, no, no, I thank Alexa too. 'cause I like to hear what she says back to me. So look, I, I thank my toaster, I think my coffee machine.

But, but my point is this, my point is this, how much money is gonna be wasted processing frames that are not adding any new information or data. So if you think about computer vision and this new vision internet concept, the whole idea is we wanna take a video and have a human never watch it, right? So if you're running, you know, you know, your, uh, uh, your security systems, your, your smart glasses, your, uh, lawnmower, your, all these different systems, right?

That are having these cameras or industrial or car tracking or people tracking, whatever it is, the goal is not to have lots of people watching all this video, right? There's just not gonna be enough people to watch it anyway. So it's all gonna go to robots, gonna go to ai.

I mean robots in the sense of, you know, automated systems. We need Netflix for robots. That's what we need to build.

We need to think about this as a different kind of infrastructure. And we have the core component to do that. If you wanna start processing this data, you start one bite at a time.

The single byte is how do I take a frame and process it using filters, using computer vision applications filters to filter that data and also to narrow the data that's being consumed. I take the raw video, I shoot it into a chat, TPT or an LLM, it's the most expensive way I could possibly do it. So when we move away from brute force, right?

We really think about this as a data compression problem. Semantic data compression, live stream of video to a little record in a database or an ERP system or A-J-S-O-N blob. That's what we're talking about.

That's where the savings gonna come. That to me is what's gonna unblock it. So the developer experience is one part.

There's a shortage of developers that know how to do computer vision. We need to encapsulate that knowledge into simple packages that anyone can run. That's what open filters gonna do.

That's what filters enable. And then the second part is the infrastructure problem. How do we make it that all these cameras have a smart and elegant and scalable and web, web scale, web friendly way to connect to the data systems, which include ai, LM, custom models, databases, rag systems, et cetera, A genix systems.

We connect those together and plain sight. And this technology we're talking about fits perfectly in the middle of those two things to enable all these new use cases that we can all imagine. And I really think, I mean, I I mean this sincerely.

I think this is the missing piece. 'cause we have everything else, right? We have all the parts.

Now the question is how do we block it? And I, I really do believe this is the, this is the problem, is that we have not built vision infrastructure for robots. We built it for humans.

I, uh, you know what? I never considered it. And it, and it just like makes instant sense.

It's like, you know, don't laugh at me, but I pay every month for a subscription to dog tv. So That my, and I won't laugh at that. I pay 11 bucks a month so that when we are not home, my dog gets to watch TV and not just any tv.

'cause dogs don't see in the colors we do. They don't, you know, they have a, their vision of the world. I mean, they, they have great vision.

They could see things that move or stuff like that. But like for instance, they see a lot of greens and yellows, not so much red and blues. That's right.

Um, so dog TV is optimized for dog's vision and it keeps them engaged. It's the same thing. It's the same Thing.

It's a great analogy. It's a great analogy. That's right.

Can CC you want to optimize the data, the, the information that we're displaying here, and again, all we're talking about here, I mean, and I hate to be like, or you know, anthropomorphize the robot, but like, it's literally just a grid of ones and zeros, right? It's like RGB values. That's Doesn't all they Say.

Yeah. It's, there's no motion, there's no blur. It's just, just r There's Love that Yeah, that's right.

So like, why would we waste our time? You know, having all this extra redundant information, uh, for somebody who literally can't experience it. Your dog TV is exactly the same thing.

What you could, you, you could do it in a way that's for humans, but it's not gonna be a great product. And the same thing here. And so as this proliferation and increase in cameras and data comes to, to the, you know, the reality, we're gonna need to consume more and more of it.

And even when people do need to see it, we can reconstruct what the human wants for that narrow subset. But the vast majority, 90% of it plus should all be processed through agents. In fact, the agents should be building computer vision apps on the fly to deal with tasks that they wanna accomplish for you.

And guess what, by having these great abstractions like filters will enable the cursors and, you know, the, the other agent platforms, you know, coming out of great ERP companies and CRM companies like Salesforce and ServiceNow and SAP and they're all building agent platforms on top of their ERP systems and CRM systems. So you're gonna have agents that can do tasks, but wouldn't it be great is that instead of checking the database, you could check the warehouse. And the way that we're gonna do that is the agent is gonna be able to see filters will be built up into skills.

Skills will be given to agents. Agents will be able to use those skills based on prompts to construct programs on the fly that they can delegate to inexpensive hardware that run in the facility and report back events based on things happening in the real world. That's a completely different extension of the age agentic world that's gonna happen.

And that's something that we're very, very excited about. And again, it all comes down to starting from the simple core. 'cause we can't define the workload at its simplest level, and we can't imagine these expansive use cases that are really gonna drive the impact.

And that's, that's really to me, this is like when I look several years into the future, this is what we're driving toward is making it so that it's very straightforward and simple so that not only a human can do it, right, and only a developer, but literally an agent can do it on your behalf. And that's the way we're seeing programming happening. You can call it vibe coding if you want, but that idea that you're gonna basically do vibe vision, right?

You're gonna say, Hey, you know, you know, Hey Mr Agent, inventory agent, go check the warehouse if I have any boxes left, hey, let me know when their delivery arrives. Lemme know when this happens. Lemme know when that happens.

You're gonna be typing those prompts in and what's gonna happen behind the scenes. You think it's gonna like just start streaming the video straight into the LLM? No, it's gonna generate a pipeline, right?

It's gonna generate a pipeline using industry standard technologies that are gonna define vision workloads. And also all those same vision workloads can be used for data collection into training for annotation, for uh, testing or ensuring that, you know, we don't have biases or ethical issues with the sourcing of data for identifying copyright infringements. All of those things can be expressed as these computer visual workloads.

It's not just about the inference, the final step. It's also about like, where does the data come from, right? If I have a camera that's watching something, how do I turn that into annotations that I can use for training?

Right? There's a whole bunch of data that needs to be processed and pre-processed. All of that can be described as vision workloads.

So this really is a comprehensive way of thinking about this lifecycle. And I'm thinking even one step further. 'cause we're just talking about vision.

There's the reason why we didn't call o you know, call it open filter and not open vision filter or something like that. This is also, uh, potentially going to be expanded, portable into a multimodal, and we started with vision 'cause it's really the hardest one and one that we saw the most immediate business value. But our, my, my request to the community is like to build on top of it and let's add audio and geospatial and other kinds of data so that we can use filters as a way of thinking about taking real world data, raw, messy, rough, real world data and processing it into structured, usable data across all these different use cases, databases, agen, X systems, et cetera.

And now we'll have a way that we can all share together. And we're not all starting from scratch. We like the code is the community value.

And that wisdom can be encoded into this, uh, this community asset. And then look, if you need help on the business side, we're here to help on a bunch of stuff and make it, you know, scalable and have somebody to call when you're, uh, you know, when it's not working and all that good stuff, right? But the, the, to me, I'm a huge community believer, you know, that I wanna see mm-hmm.

Software communities come together and build something really amazing. And software is the gift that keeps on giving. You know, like it's just such a powerful, uh, innovation in the world.

I just see that it can be, uh, huge, huge impact, Vibe, vibe, vision. It's the first time we've had it here on Tech Truck tv. I love it, man.

Vibe, vision. Let me ask a question, kid. I'm assuming open Filter's available, like on GitHub, is there a particular website that the community's gathering around a Discord server?

Something like that? Yeah, all the above. io is probably the fastest handle.

io, that's the website. It has links to everything. We have a Discord server, we've got, uh, community office hours.

There's a bunch of cool people involved in it already. One of the, one of the really cool things with Open Filter by the way, is we didn't just start it from an idea, we actually took code that was battle tested inside plane site we were using with customers. And, you know, some of our, uh, third party developers that were building Stop with it, um, basically told us we should open source data.

We were like, let's, I guess it's ready. So it's really, uh, in a really good state. There's lots to work of, work to do on it, of course, uh, as always.

But, um, yeah, it's ready to go. io. It's on GitHub, discord and Office Community Hours.

I love it, man. It's fantastic. Yeah, kid, I feel obligated to mention though, if, if you haven't, you know, if you've caught anyone here with this right by the, by the throat about, you know, the whole thing is just so fantastic.

We're doing a webinar on this on June 30th at 1:00 PM Eastern Time. com, you'll be able to register there for, we'll, we'll have it in the notes. Uh, you'll be able to register for the webinar.

Really, this is like, this is exciting stuff. I, I just feel like this, so we're on the cusp of like just reinventing so many things and disrupting so many things, but also things like computer vision that really have been not in limbo, but are about to take escape velocity, you know what I mean? Because of, of, of AI and, and the, just the whole state of where we are technology wise.

And, and it's gonna be interesting. It's, it is gonna be a great webinar. June 30th, 1:00 PM Check it out.

I'll be alright, man. Yes you will, kid. I know you're sick and you got bronchitis.

I appreciate you wasting your voice on us here today. But go rest up man. Take some lozenges.

We need you healthy for June 30th. Absolutely. Well, it's, uh, it's always a pleasure, Alan.

I really appreciate spending the time at you. You asked those great questions and uh, everybody out there hope to see you in the open source community. Absolutely.

Kit Mercer, CEO of Plain Sight here on Text Drug tv. We'll take a break. We'll be back in just a moment.

Remember June 30th, When it comes to security, the most important thing that you can do is write down everything that you need to accomplish and put little squares next to it so you can check off all of those boxes. Because once you've checked them all off, you're obviously secure. Right In this episode of the Tech Field Day podcast, compliance Does Not Equal Security.

Welcome To the Tech Field Day podcast, where we bring together a group of influential IT experts from across the enterprise IT space to talk about key concepts in the industry. This podcast features a variety of perspectives from members of the Tech Field Day delegate community, and it's often recorded in association with one of our events. Tech Field Day is a part of the Futurum Group and this podcast is published on our sister company Site Techstrong tv.

On this episode, as we head into Security Field day, we're gonna be talking about compliance and security. But before we do that, I would like to introduce our guest starting with melu. Hi everyone, I'm Mila Meyer.

I am a cybersecurity and privacy lawyer. I'll be a delegate at the upcoming security field day. And I host my MPAs socializing security, and I manage work through Compliance Counsel, which is a fractional CISO and compliance service.

So I am probably the biggest compliance nerd that Tech Field Day has invited to this podcast. Just boldly say that. Hi everyone, I'm Jack Poller.

I am an, uh, principal analyst at Paradigm Technica. I am an engineer turned marketer, turned industry analyst focusing on cybersecurity. And I'm on the exact opposite side of the fence as Milu.

Well, thank you very much for joining us. My name is Tom Hollingsworth and I am an event lead here at Tech Field Day focusing on security. Let's jump into the premise for this episode.

We've often heard that the department of security is in fact the department of, no, you can't do that. We can't expose that. But when it comes to compliance, it's often the department of, yes, we'll just put these check boxes on this list and when we address each of them and we check them off, then everything's done.

We're secure, we're not gonna be hacked. I mean, how hard can it be? However, we all know in reality that compliance does not equal security.

Alright, I'm gonna jump in with this because I think people need to understand that just because you comply with a regulation or a set of standards or something like that, that does not equal security. And I'm, I'm gonna get obviously input from everyone here. Uh, just to give you an example, uh, yeah, I totally encrypted all of that information with a des cipher, so I'm secure.

Right? Well, exactly. I think, uh, that's the key is the requirements were put are put in place.

The compliance requirements are put in place to focus on helping you secure your organization, but they are not sufficient to secure your organization. Uh, just because you encrypt something doesn't mean it's secure. If you don't use the right encryption algorithm or store your encryption keys safely, you know, if you put your encryption keys on the whiteboard behind you as you're doing webinars, then that's not really secure, even though you've complied with the requirements to encrypt all your data.

Yeah, for me it's always, I hear one, I'm pleasantly surprised that you said compliance is the department of Yes. 'cause I've never heard that as a compliance officer. I've always been told like, you guys are the police, you're the department of, no, I always thought that that was something that like IT security and compliance that we all had the same vibes on.

But I, I will say I'm a compliance officer that comes from a position of yes and problem solving. I agree with today's premise in the sense that you cannot have compliance without security. I don't think you can have security without compliance in the sense that for me, compliance is the overall framework and the structure of the program security is actually how we actually do what we say we're gonna do.

So when we're talking about doing the right thing and checking the box, I really, really hope it goes beyond just like passing an audit. Because I will agree, just doing a SOC two, an ISOD 27,001 to me is not enough. If it's actually just like a breeze of an audit and it happens once a year, then maybe you're secure for that one audit assessment that everybody scrambled for ahead of time.

But ideally, we want our system secure 24 7 365. We never wanna feel like we're only securing systems because an auditor asked us to show them something. Well, I think the auditor is a, an important part of the equation, which is, is your auditor.

And you know, we had a discussion earlier, is your auditor a CPA firm or is your auditor somebody who actually understands security and the implications of what the requirements are that you're trying to meet? Right? So if you don't understand why a requirement is put in place, what it means, and when somebody says it's not applicable in this particular case and your auditor says, I don't care.

The requirement is this, you must do this, then you have a problem, right? So does, can your auditing team and the compliance, the requirements themselves be flexible enough and bend enough to actually provide you that security that you need? So lemme ask you this question because I've had my fair share of audits that have gone sideways.

Um, and a lot of times what it comes down to is the auditors in question not being knowledgeable enough about the subject matter that they're auditing. And, and the anecdotal story is that I was in, we had installed wireless access points for a school and we had put them in the ceiling. We were running them over power, over ethernet switches.

And when the auditor was doing the audit, he wondered where the power adapters were for the access points. And that was a 25 minute conversation about how power over ethernet works because the auditor did not understand that you did not need power adapters. But on the list that he had, power adapters were listed with the access point.

And if he didn't check both boxes, he couldn't pass us for the audit. So do we run into this situation because the people who are performing the audits don't have the breadth of knowledge to understand where the importance should lie in the audit itself? Or are they just reading off of a list that somebody handed them because you're the the person on deck this week?

It really depends on the audit firm. And so for me, I always am trying to find audit firms that have really good people who know what the f**k they're doing. I have had an experience one where I used to work in cloud hosting and I used to fly around the world in audit data centers around the world.

It was like the coolest job I've ever had. I will say once, you know, we would do those internal reviews where I would basically go on behalf of the companies to basically try to break into cages and walk out with things. And so that was, you know, an internal assessment, which is a friendly auditor.

Then we would bring our external auditors, which was not a friendly auditor. I had one once come out, he had graduated from college two weeks before. He had never been to a data center before and he was there to do a FedRAMP physical security assessment of one of the best in class data centers within the US And we had to walk him out because he could not, like he was, I mean if you've ever been to a data center before, it is a really cool experience.

For the first time I was not willing to pay for somebody's first time into a data center if that was also how I needed to do my job. So we basically ended up being like, hey, like we're gonna ask your team to send out another assessment and we're gonna do it again. And fortunately, that firm did make it right.

They did send somebody else out, but it also cost us a ton of money. 'cause I was traveling out there so suddenly, like, it's wild. So for me, it's always trying to find the right audit firm to making sure that they're actually technologists and not just right, like somebody with a financial background who's done finance audits and now suddenly is doing technology audits.

That's a really steep learning curve suddenly if you're doing financial and accounting audits and then suddenly to be dropped into a data center for the first time and learn where the internet lives. So I think for me, it's really important to always be partnering with audit firms who have technologists that are also auditors who are really in the business of actually validating the controls and not just checking the box. Well, I don't, I don't wanna beat up on auditors.

I mean, I brought that up in part because we've all had those types of experiences, but I think there's also, um, uh, to some extent I have issues with some of the requirements not translating into stuff that doesn't necessarily make you secure. For instance, there's a lot of requirements for DLP data loss, data leak protection, right? And the DLP solutions, uh, up until very, very recently, the last maybe year or so, DLP was in a state where it was insufficient to be able to capture everything that people were doing with confidential data and to understand what that data is, classify it and secure it in one form or another.

However, a lot of organizations said we've, we've checked the box from a requirement we've put in place a DLP program, we get alerts, we process alerts, we do stuff. Obviously our data is secure. And that's a little bit like the, the, the somewhat facetious discussion we had at the beginning about encryption, right?

Is DLP sufficient to, for data security at this point in time? It is definitely not right? But it is a lot of what people look at when they say, I've complied with my data security requirements.

I have encryption in motion, I have encryption at rest, and I've got DLP, what else do I need to do? And it's that part of the compliance requirements that bothers me from a security perspective that it's not really providing true security functionality or utility or, you know, securing your organization. And I think that's part of why we keep coming back to this so many times is because the intent behind what we've done with security is valuable, right?

We're trying to keep data safe, we're trying to keep users safe. We're trying to make it so that everybody on the whole is better off with this than they would be without it. It's when the regulations don't really match up anymore to what we see in the real world.

And I think my favorite example of this is your average spear phishing education campaign, right? Like we tell people over and over again, don't click on links. Uh, if the URLs there type it in.

Don't just assume that whatever you're clicking on is gonna let you do that. And, and I feel like people are getting to the point where they're fairly at least aware that that's a possibility. But I also see when it's getting taken too far, I think my favorite example of that was when, uh, someone was sent an obvious email that included a link at the top that said, is this a spear phishing email?

Click here to report it. And the button was the link that spearfished you and they sent out a report. And now what you've done is you've discouraged everybody from following the guideline because, well now it's, everything's a trick, right?

You're just trying to make me look bad in front of my boss. So I'm just not gonna do anything secure at all because if I'm not gonna, if, if it's not gonna work, then why should I even bother. It's just making my life miserable.

These are the people who have a notebook full of passwords and they just change a letter and a number every time and rotate through 26 passwords because, well, the system says I can't do anything else, so I'm just gonna do it like this. 'cause I can't remember anything. Whereas if they were able to do something, you know, like, I don't know, enable pass keys or biometric uh, authentication, they'd be 10 times more secure.

But the regulation says we can't use pass keys or the reg worse. The regulation doesn't say we can use pass keys. So in order to be specific about the regulation, we must rotate passwords every 30 days.

Well, now, now, see that's where it gets really interesting is this crossover between compliance, user education, security, and what we're actually trying to do. And, you know, you bring up passwords, and I'll go to another one of my favorite examples, which is, uh, on identities as identity governance, right? And we have a requirement that we have to look at every identity in an organization and validate that that person's identity has the correct access for what they're doing now, right?

Their entitlement. So we do a review. So the way this works in real life is a line of business leader, the division manager, the department manager has a hundred employees reporting to that person.

And the IGA program sends out a list of here's your a hundred employees and here's every entitlement they have. Is this correct? Right.

So that's the meaning of compliance requirements is we've asked our, our line of business people to validate that their employees have the correct access permissions and great, that's very good. Except what does that line of business person do? I've got a hundred employees, each of which have a thousand entitlements, and every single one of these is some incomprehensible long string that I have no idea what it means.

So I'm going to say, yes, I've reviewed that, and I'm gonna hand it back and say, everything looks great. Right? So we've done everything we're supposed to do as far as governing our identities.

We've gone and we've validated everything, but nobody's actually done the real work to say, does Tom actually need access to the accounts payable system? Right? And that's, this is, this is the challenge I have with a lot of these compliance requirements because it becomes so onerous the way we've implemented the checks, it's become so onerous and burdensome on the people whose job it's not to be security that we don't get security anymore.

Okay, well, everybody's job is security. Just just so we're clear, it's just one team. It's not just compliance, security, id, it's literally everybody's job.

We all have to be secure. But I hear what you mean. And for me, I think when we're talking about is it compliance versus security, to me it's always just better if those teams are on the same page.

A hundred percent. I think from a compliance mindset, I always see as framework, the regulation, the requirement that we're going after, we're just turning into security controls, right? For me, it's the minimum.

It's really like if we do these things, we do the bare minimum and we're checking the box. When I've seen organizations be the most successful with securing their systems is when compliance and security have gotten in a room together workshop being like, these are the goals of this program. These are the requirements.

We all agree that this is the minimum that we're doing. We have to meet these things to be able to get this attestation, this goal, this thing that we're all working towards together at that time is a really, really great time for the security team to be raising any additional risks or like weak points within the system to try to figure out can security and compliance finesse those things together to try to improve the program while they're getting the budget approval spend for this new attestation? It's a lot easier for an executive team to approve a new security tool or like a new program if the compliance team, they just have easier justification being like, we cannot get this standard because we're missing these things.

And so I've seen the best security systems get implemented when compliance and security are on the same team. Compliance is coming saying, these are the rules we're trying to follow. This is, we're gonna try our best.

But if security is saying, these are the best practice things we want to do on top of these things, and then we all get approval at the same time, because that's how the whole company wins, right? If we're securing the system, the entire company wins at the same time. But it's easier occasionally for compliance to get the budget approval if there's a new attestation versus security just saying, we want this new tool.

So I will, I'll jump in here because there's, there's a trap that you've set for yourself by doing that. Um, and I, I have to reference one of my college professors, Dr. Tracy Cart, um, when she said that there's really only two ways to motivate people, fear and greed.

Well, obviously security works really well on the fear side, right? We don't want our data to get exposed. We don't wanna end up on the news.

We don't want the stock price to fall. The problem is, is that when you encourage the adoption of things through the other thing, greed, right? Like, I wanna make sure that we pass our audits.

I wanna make sure that we do these things. I wanna make sure that we do, we are compliant here. And maybe you do something innocuous, right?

Like, um, if you pass this, uh, audit, then your, your budget goes up to support these new initiatives next quarter. Or, you know, with executives, it's like, hey, if you pass the security audit two years in a row, then your compensation package is modified and you create the trap for yourself by saying, well, is the important thing that the security is in place or that I pass the audit? Because what that creates is a race to the bottom condition where I'm not looking for a rigorous auditor to come in and tell me everything that's wrong so that I can fix it.

I'm looking for somebody who's gonna come in and do a bad job of checking all the boxes to say that I've hit the number. And then that won't appear for years until we make the news. Because someone forgot about a, uh, a hard coded login in a development system.

And that's how a state sponsored actor backdoored my system and stole all my secrets. But the executive who got rewarded for passing those compliance audits for the last five years, they're long gone at this point because, well, they didn't care. All the boxes got checked.

It's tricky because this is really how the industry is set up in the sense that most organizations are not doing security or compliance because they really wanna be super secure. They should be. We can all agree on that in this room that every company should want to be secure.

Most of them are starting the approach to compliance and security because their contracts require them to, because suddenly they got a deal that requires a SOC two, an ISO 27,001 or FedRAMP in the us we don't really have technology regulation forcing down security requirements in the same way that we are seeing attestations and certifications to be doing the same yet. 'cause like GDPR, for example, in the EU only says that an organization has to follow, you know, like security. They basically get to decide what is secure for them.

Obviously, we as best practice individuals, we have certain baseline standards, right? If data's not encrypted, they're probably not gonna hit that threshold, but it's still up to the individual organization to dictate what it means for them to be secure, even to comply with something like the GDPR. So because we don't really have true regulation from a like government perspective of what it means to be secure also 'cause it's really hard for them to keep up because it's changing so quickly, it's coming through the customer demand.

And so that's why a lot of this is tied to greed in what you said, Tom, is really because it's been coming from the customers, the one demanding security. Ideally, it's amazing when cus when companies are focusing on security first, that's incredible. It's just, for the most of them, it was an afterthought.

Yeah. I I I think that's right. And I, you know, you, a lot of what you're talking about is the, the people versus process and the goal setting, right?

And if you are an organization that is simply going to do check the box compliance and security, you don't believe in security, then it doesn't matter what security doesn't matter. And, you know, this conversation has moved. You've, you've got your, your compliance and you're done.

If you're an organization that actually does believe security is important, then you need to think about how do I go, how do I become secure and compliant simultaneously realizing the two are not synonyms, right? And I, for me, that's really the key is that, that, you know, that that goal and understanding it, and it's how do you go beyond putting in place a program to do X, y, or Z to really understanding if it's making you more secure. You know, Tom brought up the great thing of security awareness training and, um, you know, my wife worked for a biotech company and she saw a spam phishing email and being a, you know, somebody who works with a cybersecurity person said, is this spam?

And I, you know, is this phishing? And I said, yes, it's clearly a phishing email. Forward it to your email administrator.

And the email administrator's response was, yes, that's security awareness training. Click on the link so you can see what happens next. So the program is actually training people to click on phishing links to tell them no, don't click on phishing links, right?

When the right response should have been, thank you for letting us know you passed the test and I'll go tell somebody to mark you manually, mark you was passed the test. Right? But so, so again, we're, it's that we're, we're paying lip service to security by saying, yes, we've got a cybersecurity awareness training in place.

Not thinking about what telling somebody click the link does. Right? And I think that's, that's the, this is what gets my goat so to speak about this, is it's really irritating when people do the wrong thing for what they think are the right reasons.

Well, the other thing that I think is important to understand is that the regulations also need to be malleable enough that we can change them when we realize we run into a problem. A good example of that is if you implement a policy for passwords in your organization, your goal ultimately is to prevent people from using insecure passwords. Right?

But what if your policy prevents the use of a more secure password? For example, a password policy that says you can't have more than two sets of two repeating characters. Okay, but what if my ultra secure 26 character password has four sets of repeating characters in it for whatever reason?

Now, technically, my password is out of compliance with your policy, so I have to weaken it to meet your policy. As someone who's on the other side of that divide, it's frustrating for me to say, why am I forced to comply with a regulation I know isn't as secure as what I'm going above and beyond to do? And creating that kind of, I don't know, friction.

Because one of the things we've learned over the years when it comes to security is it works best when it's invisible, right? I remember when Face ID came out for the Apple iPhone and everyone was complaining because, oh, well, it's not gonna be as secure as typing in a passcode. What they didn't realize was the way that they used their iPhone changed when they just had to look at it to unlock it, as opposed to typing in a code every time.

Because when you do that, people can't shoulder surf your face. So we became more secure over time simply because the security control that we were trying to prevent, phone logins disappeared underneath the phone itself, as opposed to typing a passcode or putting my fingerprint on there or what have you. So do we, are we setting ourselves up for failure?

Because we're, we're aiming at the low end to ensure compliance, but we're overly restricting the people who are not only in compliance, but maybe even possibly even beyond that. Yeah. It's always built for the weakest link, right?

Unfortunately, a lot of the, a lot of the controls are being built for the weakest link. I will say, as we're rounding out, for me, it's when we're talking about are we checking the box? Are we doing enough from a compliance perspective, or should we be doing more from a security perspective?

I would say my thought on this has always been making sure that our companies really thinking about, do they only wanna check the box? Because I agree with what we've talked about today. If you are compliant, you might not be secure.

And so for me, it's always thinking about, like, there's a misconception with a lot of organizations that they're like, oh, well, because we're compliant, we're secure. I think the conversation needs to be different of compliance is helping us set a baseline standard, but security will continue to be a best practice. What will push us forward faster and be hopefully more innovative, and honestly, maybe even a different in the market if we're thinking about security as the competitive edge that improves the organization's compliance posture.

As you can tell from this discussion, compliance isn't security, but they don't have to be polar opposites. One impacts the other and vice versa. It's very important to understand why you're trying to be compliant with certain rules and regulations, but also what it takes to make sure that you are not just checking a box.

If you ever get to the point where you're scanning down the list and checking things off, just to say that you check them off and not verifying that they're actually done or worse yet, finding yourself, opening your open up to other issues down the road, you're really defeating the purpose of both of those things. So take a step back, really understand what you're trying to accomplish, and make sure that the reason why you're holding this audit or you're meeting all of these guidelines is something that you and the rest of your team understand. That'll just about do it for this episode of the Tech Field Day podcast.

But before we go, I'd like to give our guests a chance to let you know where to find more of the content that they create, starting with melu Irun, thanks again for having me today. You can find me Ulu Meyer on LinkedIn. com.

Thanks. Uh, it was a pleasure, uh, talking with both of you. com.

And we wanna thank each and every one of you for listening to this episode of the Tech Field Day podcast. If you enjoyed this discussion, please make sure that you subscribe on YouTube or your favorite podcast application of choice so that you don't miss any of our episodes. And we would love a rating and a review if you have the time to leave one, because it lets people know what we talk about here and that we are in fact using the word premise correctly.

This podcast was brought to you by Tech Field Day, which is the home for IT experts for cross enterprise. It, it is a part of the futurum group. com, pod slash podcast, or check out all of our episodes on Techstrong tv.

Thanks for listening in. We'll see you next week. Organizations seeking to build an infrastructure stack for AI training, need to know how that data platform is going to perform.

This episode of utilizing Tech presented by Soy includes Curtis Anderson, co-chair of the storage working group at ML Commons. We are discussing storage benchmarking with ACE Stryker and learning how we can know whether the given storage infrastructure is gonna perform well enough for a given ML training environment. Welcome to Utilizing Tech, the podcast about emerging technology from Tech Field Day part of the Future Group.

This season is presented by soy and focuses on the question of AI data infrastructure. I'm your host, Steven Foskett, organizer of the Tech Field Day event series. Joining me today as my co-host is Mr.

Ace Stryker of Solid. Im welcome to the show. Ace.

Thank you very much, Steven. How are you, sir? I'm doing pretty well.

Um, this has been going great. I'm so glad to be doing this, uh, special season with solid. I'm focused on a topic that's near and dear to my heart, which is basically how we can make storage be useful.

Uh, and I guess that's kind of what you're at here too, huh? Yeah, it's been a ton of fun so far. I've, uh, uh, we've, we've had some really interesting guests so far from, uh, a lot of different, uh, corners of the industry, right?

Uh, a lot of different looks at, um, the way the data infrastructure needs are, are evolving, uh, to keep up with, uh, uh, I guess AI is the, is the, is the bright shiny object today, right? And will be for some time. It is the driver of, uh, of, uh, these, these requirements and, and these, uh, efficiency issues really coming to the forefront lately.

Um, but, uh, yeah, it's, it's been a, a great journey so far. And, and I think, we'll, we've got another great guest lined up today. Yeah.

It's one of those, one of those things that, that comes up a lot just to what you just said is basically that storage has to meet the requirements of the application. Now, that's been something that we've said forever. You know, data infrastructure, data platforms, um, you know, performance has to be well, good enough, right?

But how would we know how good is performance? That's been a challenge in the industry for a long, long time? How do you measure performance?

How do you express those measurements and how do you specify things that are good enough It turns out to be? Uh, I think a more complicated question than a lot of folks would assume, right? Um, if you as a consumer go buy a laptop, there's a number of ready-made tools, you know, you can pull off the shelf, you can run Cine bench, you can run PC mark, you can get a pretty good sense of what your, uh, hardware is capable of, uh, you know, pretty quickly.

Uh, and you can use that information to make relatively intuitive apples to apples comparisons, right? Between different options when it comes to things on a data center scale, right? And particularly as they look at things like data infrastructure and, and what are the requirements or the capabilities of the storage subsystem.

Um, we get a lot of questions about that. It, it turns out to be kind of a, a tough nut to crack, And it's the same for other aspects of the AI stack as well. Um, one of the, uh, organizations that I'm particularly fond of now, you'll recognize them from Field Day, from, uh, utilizing Tech podcast is, uh, ML Commons.

Uh, they, uh, are really focused on answering these questions. And ML Commons, as I've mentioned previously, has a storage, uh, benchmark as well. So we have decided to invite on the podcast this week.

Uh, Curtis Anderson, who is, uh, the co-chair of the storage working group for ML Commons, and is well, probably more knowledgeable about this question than anyone. Welcome, Curtis. Oh, thank you.

Thank you for having me. Um, my name's Curtis Anderson. I'm one of the co-chairs of the ML Perf Storage working group at Emil Commons.

So tell us a little bit more about yourself and what you do with ML Commons. Um, so I'm a storage guy, not an AI person. Uh, I'm learning AI as I go along.

So, uh, that's actually an exciting piece of it, is learning the new technology, um, the, uh, storage working group attempts to benchmark storage subsystems in support of AI workloads. And so I can go in a lot more detail about, but that's sort of the big picture, is I'm one of the co-chairs of that working group. Uh, so I'm here to describe what it does, how it works, and invite people to join.

Excellent. And, um, like I said, I, the, the thing that I love about, about ML Commons is that it is very practical. ML Commons is not interested in mythical angel storage numbers or, you know, you know, performance.

You know, let's see how many whatevers we can pile up. ML Commons is very interested in, like, how does this perform under, under workload? And it's the same with storage, right?

Yeah. The, uh, um, the, the benchmark emulates, uh, a workload, it imposes a workload on a storage subsystem, the same workload that a, a training pipeline would run would impose on the storage. And so you get an honest to goodness, this is how your storage product would or, or solution would perform in this real world scenario.

Curtis, can you talk a little bit more about, uh, the nature of the workload? I think in, in, uh, other episodes, we've explored the AI data pipeline a bit, and we've talked about there are discreet steps here, you know, uh, ingesting raw data versus preparing your training data set versus the training itself and the inference and so forth. So what is, what is a training, uh, workload look like, and what is the benchmark asking the storage subsystem to do?

So, zoom out in the, the bigger picture, just to set some context here. Um, uh, the, uh, a person who wants to make use of ai, they've got a, a problem statement. They have some data, they need to then start putting it together into a, a pipeline is called, um, that starts with the raw data.

Generally it's, you know, it's video or it's still pictures, or it's a audio or text. Um, they do some data preparation, which is changing the format of the data. They take the, the picture, the image, and they turn it into a numerical representation instead of an image proper.

It's not a, a JPEG or a p and g any longer. It's a, uh, a umpire array. Don't worry about what that means in a second, we'll talk about that later.

Um, so there's a bunch of data preparation, and then the training step, which involves that. That's the GPUs, the, uh, um, you train the neural network using that data. Then when that's done, it goes into inference where, uh, you say, okay, I, I now have a neural network that can tell me cats versus dogs.

I show it a picture. Is this a cataract dog? What we do in the, uh, storage working group is benchmark the performance of during training during that phase of the overall, uh, workflow pipeline.

Uh, we're working on adding the data preparation. Uh, there's a bunch of cleaning and other kinds of steps that happen there. We're working on bringing that in.

But right now, we're, we're starting with the simple, you know, the meat and potatoes, if you will, of the, um, of the workflow, which is the training step. It's very data intensive. Uh, and so it puts a large stress on the storage.

Is the, the decision to start with the training step. And, and, and it sounds like moving into data prep next, um, is that because those were sort of the low hanging fruit, the easy ones to implement first? Or are those the stages of the pipeline where you're seeing the greatest storage sensitivity?

Can you kind of walk us through the rationale there? In one sense, it is, the training is easier than data preparation because data prep is sort of unique to every different application. It's hard to develop a benchmark when there are 4,000 different ways to do something on the other.

So there is that. But, um, the, the, one of the key characteristics of the, the benchmark is, um, we measure how well the storage system performs not on the tra traditional storage benchmarks of megabytes per second and files per second. We measure on how quickly, uh, how completely the GPU can stay utilized.

If the data, if the GPU ever stars for data that, you know, the, the latest, uh, Nvidia, GPU is, the H 100 is like $40,000 a piece. You don't want that thing going idle because it's starred for data, right? And so we measure, um, accelerator utilization as the core value of our benchmark.

And can this storage product or solution keep up with, uh, this number of GPUs doing this particular workload? A uh, an image recognition workload is different from a recommender, which is different from, uh, a large language model. There's many different types of, of neural network models.

Um, and so they each impose a different workload on the story. So we measure them all separately. Um, but yeah, the, we, we are measuring can it keep the beast fed?

Can it keep the GPU busy with, with data coming in? That's sort of the core metric of the benchmark. It's a super interesting, um, choice.

And, and I think for folks who are used to benchmarks that output megabytes per second or some sort of, uh, calculated score, um, it'll be a very different look at performance, right? Um, so can, can, can the, the outputs of the test, let's say you, you stick a, a storage device or, uh, an array, you know, in the test and you run it and it says, oh, this, this storage subsystem can keep X number of, of GPUs highly utilized, swap it in for another one, and, and that other option can keep nine. It can keep y uh, uh, GPUs utilized.

Can, can that be used to make relative judgements about the suitability of storage solutions or the, the performance of, of one against another in an AI workload? I, I should say upfront that the core metric is, uh, accelerator utilization, how, you know, whether the the GPU goes I or not, but you can turn that into the traditional measures like megabytes per second and iops and all the rest of that, those informa that information's available. But the, the benchmark says, if you can't keep the device 90% utilized, then the, sorry, if you can't keep the GPU 90% utilized, then you're trying, you're overloading it, and you need to, um, uh, run this, the, the, the benchmark, again with a, a smaller number of simulated GPUs.

So that's the thing that people, uh, are gonna look at that, um, the, the person who's gonna look at the results is someone, an AI practitioner that says, oh, I know how much data I've got. I know what type of workload I'm running. I wanna know, does this vendor have a product, a storage product that will serve my needs?

Or how big of a product from that vendor do I need to purchase in order to, to serve my needs? And so that's, um, the, the, the, the practitioner also knows how many accelerators of that type they have. They have a budget given from their management, says, oh, you can buy a hundred h 100 GPUs from Nvidia.

Yeah, four, 400 grand, or no, $4 million. That's a lot. Um, so they know those things of how much data, how many accelerators, and they want to know what, how much storage do I need to buy, and that it will actually keep up with the, the accelerator count.

So that's what people vary. Is the count of accelerators that this particular configuration can support A larger config can support more accelerators. Does that make sense how those, the, there's a bunch of things going on.

But, um, the practicality as, as Steven said, the practicality is, what do I need to buy to keep my GPUs busy? And that, that's what we talked about last season on u the utilizing tech. Um, we talked to a lot of companies in the AI space, and it really boils down to that.

I mean, that's the whole ball game, basically. You're spending a huge amount of money, uh, you said like $5 million. That's a cheap infrastructure.

Yeah. Um, you're spending a huge amount of money on very expensive GPUs. Yeah.

Or, you know, ml, you know, processing asics. Yep. And you need to keep those things fed in order to make the most of that investment.

Yes. That's the thing that matters here, because if those expensive items are waiting for data, then they're not producing results, then they're not actually giving you what you bought. And, you know, that's it.

And so the question for the, the, you know, the practitioner or the person that's, that's deploying these applications that is specking these things out, they don't need to know how many 4K iops this system can handle, theoretically. Right, right, right. At q depth of eight.

You know what I mean? That they don't even know what that means. Right.

What they need to know is what you're saying, which is, I bought this many of this type and I've got this much data. Right. Will it work?

Yeah. Yes. No, right.

And, and, and that's kind of the answer you're trying to give 'em. Yes. Oh, that's it.

The, uh, the storage industry wants to know their traditional kinds of numbers. I'm a storage guy, so I can say this, right? I wanna know, uh, iops, I wanna know megabytes per second.

But the AI practitioners, they think in terms of samples per second. And, uh, in a distributed training environment, how many accelerators do I have? I use accelerator, but because I try to be non nonpartisan, it's really GPUs.

Nvidia is the, the dominant player in the market. So, um, how many GPUs do I have? And so they think in those terms, and we try to bridge the, the, the, the two sets of terminology together.

Curtis, you, you mentioned a minute ago that the test relies on emulated accelerators. Right? Which I have to imagine is, uh, um, very attractive to a lot of folks.

Uh, you know, that they can run this test without the need for a rack full of hardware, you know, running tens or hundreds of thousands of dollars. Um, can you talk a little bit more about, um, you know, whether that was a deliberate choice to kind of open up the tool to a wider audience? And are there dials in there to try out different emulated accelerators when you're running your tests?

Yep. Great question. Uh, it was an explicit decision that we made early on that most of the storage vendors they have, uh, uh, of the vendors.

'cause we also support, uh, academic research and open source and other, other potential solutions. So, um, but none of those people have the budget to go out and buy a hundred of the latest accelerators and IT, or to try it with, um, uh, you know, other vendors besides Nvidia. So, um, we needed to emulate the, the operation of one of these accelerators.

So yeah, you can, you can fire up a dedicate 10 compute nodes, uh, and your storage product or storage solution, and you'll we'll run 10 or 20 accelerators on each of those nodes, because the only thing we're doing is imposing the same, doing the, the reads and writes, uh, uh, from the storage to those nodes. We're not doing anything with the data. We're not training a neural network model.

We're just imposing the workload on the storage solution. And so, yeah, you can, um, we of course had to start with NVIDIA because, you know, they're, they're the dominant player in the marketplace, but, um, we are planning on pulling in the other vendors, uh, the startups and the, the graph cores that that servers class kinds of machines tends torn, for example, um, to bring them in as well and show, um, so that the, a customer that wants to purchase their accelerator instead of Nvidia can say, okay, here's the storage that I need to support that configuration of that hardware. Yeah.

And that reflects what we're seeing overall in the industry. I mean, first off, what we're seeing is that Nvidia is obviously the dominant supplier right now, so it makes sense to start there. But we are definitely hearing a lot of interest in alternative solutions be, you know, whether it's other GPUs or as I said, um, you know, a Asics, uh, various, you know, neural network, uh, processors.

And even as you mentioned CPUs, there certainly is a lot of excitement about CBUS that have more and more, uh, capability. And, and it's not, in terms of, and again, this kinda gets back to the question of ML perf and the, the, the mission of ML perf, it's not about the biggest number. It's about the most appropriate solution for the task at hand.

And if the task at hand is not needing absolute maximum performance, I think that what you'll find is the, the set of things that customers are looking for are varied. So they're not gonna be, you know, if if they don't need all the performance in the universe, then they're gonna start thinking about things like efficiency and cooling and, you know, environmental impact. And, you know, literally physical space.

They're gonna, of course, be looking at price. Um, are these things that, that the ML perf, uh, or the ML common storage working group, are gonna be addressing as well? Yes.

Um, uh, I personally would love to include, uh, dollars per something right in the, in there, but that's a, a delicate subject for a lot of, uh, participants in the, in the storage business. Right. And, uh, and we also attempt to address open source where there is no dollars.

I mean, there's dollars for hardware, but not for software, right. And academic researcher, uh, re academic institutions where researchers are trying to figure out how best to modify the frameworks, the PyTorch, the TensorFlow, uh, MXNet to, to do better IO patterns to match the, the capabilities of the storage system. So, um, well, I, because I'm a storage person, I'm used to include operating in dollars per something, but, uh, that's a, a much further out topic right now.

It's, uh, crawl, walk, run. And so we're, we're emulating the workloads on the accelerators. Then we'll bring, probably the next most important thing is to bring in the data, pre-training.

Um, uh, Facebook meta, uh, did a study, uh, it's actually a report on their internal infrastructure for their AI training stack about four years ago now. And they said they spend about 50% of the total kilowatt hours of electricity on data preparation, not on training. And so that's probably the second thing we'll tackle is trying to model that.

And it's in the, the workload it imposes on storage. And then, um, and how different architectures, uh, give you different results. Uh, uh, data preparation is generally CPU bound.

They don't use the accelerator for the data prep, and, um, but they're, you know, they're, they're starting to talk about it. NVIDIA's moving that direction a little bit. I, uh, I'm not sure about the other players in the market.

So there's a lot of complexity, and we're gonna keep growing the core base of the, the working group to, to handle more of the storage component. So I should throw in one more thing there that, another big picture comment about ML Commons. Uh, there are like five different types of things in the world of ai.

There's, uh, data models, accelerators, storage, and networking. You need some of each of those in order to, to get the value out of ai. ML Commons started with, uh, data and, uh, sorry, with models and accelerators about two years ago now, two and a half years ago, they added storage.

Curtis, um, the, so the, the, the roadmap for the ML perf storage test you've laid out a little bit for us, right? We're, we're focused on training today, data prep, uh, tomorrow. I, I'm curious if someone wanted to explore or evaluate the, uh, suitability of a storage subsystem for inference, for example, um, would running the, the, the higher level, you know, ml perf suite, the non storage specific tests, tell, tell a person anything about storage, are they, uh, sensitive to changes in, in storage subsystems from, you know, one run of the test to another?

Or, uh, or is that something that doesn't tend to show up on the higher level kind of, uh, system level ML perf testing? Like if you take ML PERF training, that's the benchmark for the performance of a, a piece of silicon when it's running a training task, um, that's generally compute bound, um, in, it may be more specific. The people who run that ensure that it is compute bound.

They don't want a storage subsystem slowing the performance of the benchmark of their new silicon, right? And so, um, the numbers you see in the other benchmarks at ML Commons, um, uh, won't include any impact from storage because, you know, people running the benchmark don't want that. Um, and so in that sense, they're all sort of disjoint.

But there is something we're attempting to do in the storage working group, which is, um, training will define a workload like a unit 3D or, you know, that's a, a three dimensional volume classification benchmark. Um, they'll define that, that workload, and we will run the same workload to say, oh, if you're getting, if you're running, um, ML PERF training on that workload, here is the, the corresponding information for the, the storage subsystem. We think that that has value.

Um, it's a, you know, we need to keep track of what the other working groups are doing in order to correlate those, uh, the results that way. But ML PERF also has a bunch of, uh, inferencing benchmarks. Yes.

Um, well, first off, does storage have much of an impact on those, and is there an applicability in the future? Well, we'll see ML perf storage in that area. We don't yet see a huge impact from, uh, from inference.

Generally. The, I mean, in terms of the number of inference operations that are done globally, they're almost all done at the edge on your phone, basically. Uh, or in, in some point of sale terminal or something like that.

Um, the, the storage in that environment, you, uh, a single SSD is more, you know, overkill for, uh, a, a single inference operation, right? But an SSD can be strained by a really large training operation. And so that's why we have focused on the training piece of it.

It's just a lot more storage intensive. Um, there will be points where, uh, storage will have an impact on inference, uh, but they're sort of lower down the priority stack for us at the moment. So, given the fact that, uh, yeah, as you mentioned, um, you know, that data preparation is such an important thing.

Are there standard data preparation processors or workflows that, that customers, uh, or are, are going to need to go through, uh, in order to be ready to do training? Is it as straightforward as some of the, uh, ML perf training, uh, benchmarks? Or is it, uh, a little bit different?

What we have seen is that data preparation is, um, pretty much unique to every application at every individual customer. There's, you know, um, uh, there's lots of different types of data preparation that sort of, sort of sub classifications, if you will. Um, you could take an image and remove noise from it, you know, a pixel noise, right?

That's one type of data preparation. Uh, there's, uh, in image processing, there's others that are, uh, taking image and I wanna rotate it 15 degrees, or I wanna change the color palette, or I want to, uh, keystone a little bit. A lot of different things you can do at the image manipulation level that in effect multiplies the amount of data you can hand to your neural network model during training.

That multiplication factor of, of doing that, uh, that data preparation, that's data preparation. Um, but it's unique. It's for sometimes you don't want to do rotation, sometimes you don't want to change the color palette 'cause the color is important.

You wouldn't do that when you were looking at a stoplight, for example. Um, so, uh, no, we haven't found a, we're, we're in the process of researching that question 'cause it's very important. But we haven't yet found any taxonomy that we can describe of qs, the classes of data prep that, that, that could be done.

Not Yet. Well, thanks so much for that. I think that that's a really interesting, uh, and, and it sounds accurate to me, uh, because I've seen certainly that, uh, that's how it is.

Every, everyone's basically bringing in different types of data from different sources and, um, you know, and, and it's, but, but even so, I think that you'll be able to come up with at least some standard workflows that, that represent the type of work that companies are doing on data preparation in order to make that also a relevant benchmark. Mm-hmm. Um, tell us a little bit more, I guess, to, as we, as we wrap up here, tell us a little bit more about the, uh, the, sort of the nuts and bolts here.

Um, ML perf storage, uh, just like the rest of the ml perf benchmarks, um, it happens on, on sort of a regular cadence. Mm-hmm. Um, what does that look like?

So where, where are you now and, and where does that go? And, and, and when will we see the next round of results? Sure.

Um, so I welcome anybody viewing the, the podcast to, to come join us in the working group. org and you look for storage working group, and there's a, a link there. It says, join, so you can join the worker group and show up and, and help guide the, uh, the, the project.

Um, we are attempting to have two new releases per year. One in, uh, of the benchmark, one in the spring, and one in the fall. Um, it's a struggle to get it, uh, all buttoned up, nice and tidy every time.

But we've got another couple weeks, two weeks or so. 0 is ready to be run. Uh, two months later, there'll actually be an open window where you get to submit results.

And then after the, the, the results go through a peer review process, which is private to the people who submitted in order to make sure that, uh, everything, that the benchmarks were run correctly. And all of the, the i's are dotted and t's crossed and all the rest of that. And then the results are published.

So, uh, we're, we're talking about three months from today. The results will pop out, and then we'll do the same thing again in the fall. Well, that's excellent.

Uh, I can't wait to see it. Um, I will tell you that I really look forward to the briefings, uh, before the results come out. I really look forward to combing through the results and seeing some of the stuff.

I mean, ML Commons in addition to storage, also benchmarks, uh, a lot of other areas. And, uh, you know, one of my personal favorites is the, the tiny and the mobile Oh, yeah. And the edge benchmarks that they're doing.

Yes. To show how, um, you know, not the big data center full of GPU, but all these other systems perform, uh, very, very relevant. Um, and, and very interesting as well.

So I'll definitely be keeping an eye on that as well as, of course, the storage benchmarks coming out of it. Uh, a lot of bragging rights, uh, for a lot of different companies and a lot of different solutions. And I think that's another thing that talks about the vibrancy of the storage industry overall.

Mm-hmm. We've got great solutions from a lot of different sources, whether it's open source or proprietary companies, and they're all, um, able to support various workloads. So it's, it's very neat to see that answer coming out of ML perf storage as well.

Yeah. Well, thank you so much for joining us. Before we go, Curtis, where can we continue this conversation with you and with ML Commons?

The, the best way is to join the working group. org is the email address. You can send email to it, I believe, from outside the working group.

But, um, there's lots of documents and, and presentations and things that the working group gets to see. So join. Great.

Thank you so much. Um, ACE, uh, thanks for, uh, joining me as the co-host today. Yeah, absolutely.

Thanks a lot, Steven, and thank you, Curtis. I very much, uh, enjoy the conversation here. I expect, you know, the, the question of, uh, infrastructure efficiency and keeping your GPUs, you know, maximally used is going to be relevant for, for, uh, quite a while, uh, as we look in the future of ai.

And so having better tools to, to measure that, uh, and make informed decisions, uh, you know, uh, in service of that goal, uh, is gonna be really important. So, uh, very excited by the work you're doing and appreciate your time. And of course, uh, I know that there's a lot of solid Im storage in those, uh, submitted results too, but, uh, you know, that, that's, uh, it's nice to see that too.

Ace. Yeah. Yeah.

And thank you everyone for listening to this episode of, uh, utilizing tech, uh, focused on AI data infrastructure. Uh, you can find this podcast in your favorite podcast application. Uh, you'll also find us on YouTube just, uh, search for utilizing tech or utilizing AI data infrastructure.

If you enjoyed this discussion, please do leave us a rating. Um, leave us a nice review. Uh, we'd love to hear from you as well.

This podcast was brought to you by soy, as well as by Tech Field Day, home of IT experts from across the enterprise. Now part of the Futurum Group. com, or find us on X, Twitter and Mastodon.

Yes, Mastodon at Utilizing Tech. Thank you very much for joining us, and we will see you next week.