Techstrong TV April 2, 2026

Hey everyone, we're back here live with our coverage. It's "Techstrong TV" here at RSAC. I want to introduce you to our next guest.

I met this young lady yesterday at our AI Native Dev webinar. ' I learn from it. I use it.

" And she started telling me a little bit about herself, a little bit about her company, a little bit about some conferences they do, and I want to share it with you. I think it's a great story. And I'm probably going to mispronounce her name, but I'll do the best we can.

Her name is Altshoy? Ayça. Ayça.

Altınsoy. Altınsoy. Correct.

I was putting them together. Ayça Altınsoy. Ayça works with a company called Guard6.

Guard6. And, well, you're based out of Germany now, yes? Correct.

But though you're Turkish, from Turkey. Correct. And we're going to come back to the Turkish angle in a second.

Sure. But let's talk about Guard6. Yeah.

Ayça, what do they do? So Guard6, formerly LogPoint, we've just changed the brand name and the entire brand identity, is the best ally of yourself in cybersecurity. Okay.

That's how we should start. Okay. And as the product lineage, we have SIEM and NDR in-house, and- Okay ...

we are vendor-agnostic with EDR, so this entire trilogy- Yeah ... of the visibility and telemetry data in cybersecurity. We believe that lean SecOps practices are very important, especially in the hype of AI.

So to speak, not using every appliances that are coming with some kind of AI title- Yes ... but being very focused on what matters the most and what data that we should be highlighting in the network that our products are looking at. So these are very important things.

And speaking of less complexity, this lays in the very philosophy of our product, our development, because we are so much customer focused, customer centric, and that matters to us what actually the customer wants to see in their SIEM and NDR tooling, rather than that we want to use all the AI tools out there. So this is our stance. Excellent.

Before we go further, the website? The website? The website for the company, for Guard6.

com. com. So Ayça, here's the thing.

We hear here in the US that- Yeah ... in Europe, in EU, they're taking a better stance or a firmer stance around identity, around security, around compliance. And AI is everywhere, though, because AI doesn't know any borders.

They don't care whether you're in Europe or America or Asia or Africa, whatever. AI is all over. What are you seeing in Europe perhaps that's different than what we see here in the US- Uh-huh ...

around AI and securing AI and this whole idea around SecOps and everything you're doing? Mm-hmm. I would say it's digital sovereignty, but from the perspective of not only where your data resides, it's also how your decisions are made.

Right. So digital sovereignty in the hype of AI, especially in the region Europe, it's definitely more than where the data is residing, where the data is stored, and under what legislation rules. It's about how the decisions are made.

So as Europeans, we are not letting the AI decide for us. We are focused on reducing the noise and having the simplicity in our operations, not that letting the AI decide and get the control out of us. So it is still human in the loop, so to speak.

That giving the AI the not role-based task, but mostly making AI to focus on the task, such as log enrichment, data prioritization, and triaging, but not letting AI decide on it. And also the regulations, obviously very important in EU base, EU concepts, so to speak. We are very much focused on how we are scaling AI in our organizations and for our customers.

I love it. You mentioned yesterday that your company's been putting on some events in Berlin over the last few years. Tell us about that.

Yeah. So that's actually not the company I work for- Oh ... but that's a collective I'm a member of.

Ah. Very cool. Yeah.

Tell us about it. So that's a collective from Berlin. It's called Tech Open Air.

So we have been doing this conference for some time, and I personally joined the collective in 2024. And we are technology lovers. We research about the new frontier technology.

And obviously not very surprisingly, our main theme in 2024 was about AI. Right. Yeah.

What else are you going to talk about? Yeah. We had very fantastic keynotes from the people by OpenAI, by Salesforce.

These were very fantastic crowd. It brought us a lot of perspective. It brought us a lot of community feeling.

Mm-hmm. As in today in RSA. And in 2026, this collective is bringing the discussion to Istanbul.

Wonderful. Yeah. The pearl of Eurasia.

Yeah. So the- The crossroads. Absolutely.

The bridge. Yeah. And in this time that we want to see this city as the bridge of the technology from Asia, from Europe, from US- Mm-hmm ...

and bringing all these different notions about AI into one place, which is the bridge of it. Okay. So that's kind of the idea behind.

And with that collective identity of the conference, that we want to bring so many different networks, and if that would be the practice, it'll be great to have "Techstrong" as well in the Tech Open Air Istanbul. Oh, we're going to try. We'll try our best.

Is there a website that people can get information about that? Yeah, absolutely. com.

Right. And when that's the case, then they can always like- Well, that's the event itself, but just finding out about the organization is still there. Yeah.

Absolutely. com. And also, yeah, my employer, Guard6, which is also where they can get to know about the European alternative of the hyperscaler world of cybersecurity.

I love it. Is this your first RSA? Yes, it is.

Give us some impressions of that. Well, I learned so many new things, and to me, that was the most precious aim that I came here, fly 15 hours. Uh-huh.

It's a long flight. Yes, it is. It is indeed a long flight.

Yes. But it is worth of every second of the flight because, well, the conference has just started yesterday, so the content started to flow- Yeah ... just yesterday, but the Techstrong seminar was amazing.

Thank you. I got to know a lot about, especially on the AI-infused DevOps. Mm-hmm.

So that was a great talk titles. Yes. And also, I got to know, of course, multiple different new frontier technology and open source, such as OWASP.

Yes. That was something new to me, even though they've been- Really? in the industry for some time.

Oh, I'm surprised. I didn't realize. Yeah.

Yeah, no, OWASP is huge. So, it was great to get to know new people, to get to know the new content, and yeah, that's fantastic. Today, I'm so much looking forward for some keynotes, and likewise tomorrow, there's going to be so many seminars.

Yes. So that's fantastic. I'm very happy.

I hope you enjoy. Really, it's just really getting started today. So every year, the Monday gets a little more busier.

Yeah. But traditionally, the show started Tuesday. So I think you're really going to like what you see.

And have you been to the show floor yet? Yes, I was. What'd you think about that?

Well, it's very overwhelming. Overwhelming? A little bit of...

Yes, there a lot of information. That's what I can say. So it's quite hyperbolic, but it's the melting pot, so that's the point that I like about it.

Yes, it is. And it's also very nice to hear the guardrails of the AI technology and AI-based technologies in cybersecurity. Everyone has a different kind of development.

I'm getting to know a lot of different thought schools. I'm getting to know a lot of research, so that's great. There's lots of information that one can collect and bring to home.

I got one more thing for you. Sure. And this is not for this, I'm going to tell you.

4:00 today, right across the street here at The Metreon AMC Theater- Okay ... is a film five years in the making- Okay ... about women in cybersecurity.

Okay. And it's world premiere here. It's open to all RSA folks.

I'd love for you to go see it. You know what? You're a young woman up and coming in this space, and it talks to you.

That's actually great that you touched upon it, and I just want to have one addition about that subject, which is that, your channel, your platform is great, and I've been actually getting myself ready for so many interviews thanks to you- Really? Oh ... just by watching your content.

Really? Thank you. But then, yesterday, when I was approaching you and asking if I can get an interview, that was just that notion in my mind.

No one was born as a CEO. No. No one was born as a CISO.

No one was born this fancy, shiny, career-focused person. There's always a path. There's always so much that people are giving on there, and there's nothing to shy.

No. Good for you. That's why I'm not shying away.

No, you certainly aren't. But go see that movie. I will.

I think it's perfect for you. I will. Thank you.

Hey, we got to take a break. Thank you for coming on, and you should enjoy the rest of the week. Thank you.

Don't run home. You got three more days, or it's two and a half more days. We're going to take a break.

We've got more coming your way here. We're live at RSAC. We'll be back with more.

Hey, everybody, we're here in Amsterdam at the KubeCon + CloudNativeCon Europe Conference, and I'm talking to my friend Josh Bressers, who's from Anchore. Yeah. Hey, Mike.

Hey, welcome to the show. Thanks, man. I'm excited to be here.

Everybody is kind of talking about one thing, which is open source software security is top of mind, and everybody's kind of walking around going, "Hey, it was a topic that we kind of used to nod to," but now, everybody's seen these npm packages. There's all kinds of attacks coming through, and suddenly it's all very real. So what are you hearing from folks about what they're doing about all this?

I love the question because we're literally in the middle of a supply chain attack right now against an open source scanner. It's not one of mine, thank goodness. But this is a really hard problem because we don't know what to do in a lot of instances.

So I've been doing this open source thing for 20 plus years. I joke with people that I was doing open source supply chain security before we knew it was called a supply chain. So it's been a long time.

But I think how we handle open source security is, I wish I had an answer that was just run this one scanner, buy this one product, do this one thing, but it's not one thing, right? Because when we talk about npm, the problems we see in that universe are very different from the problems we're seeing when we talk about Go or Rust or Python. And there's a certain amount of cooperation we get between these groups.

Like for example, we're here at KubeCon. Linux Foundation has an OpenSSF, and the OpenSSF has groups, for example, that brings some of these people together, where you have the package repositories starting to cooperate and communicate on malware in their infrastructure, on... vulnerabilities in the packages and how we find them and how we detect them.

And so, quite frankly, at this point, if this is something you're concerned about, this is a lame answer, but we kind of have to go back to the fundamentals of security. Is just understand what you have, scan your existing infrastructure and software for just the easy vulnerabilities. Not even the crazy stuff, just the easy stuff.

Because we're bad at that too, in many instances. And so start with the fundamentals, and then we can kind of work our way up and start having discussions about how are we going to secure the packages we're using from open source, how do we keep track of the packages we're using from open source? Because one of my favorite examples is if I asked you what is an end-of-life open source package, what would you tell me?

Maybe one that hasn't had an update from a maintainer in five years. I don't know. I mean, five's a pretty good number.

A lot of people say one or two, but there are many legitimate projects that haven't had updates in five years. And they may not have an update for another decade, not because there's anything wrong, just because they say, "It's good enough. " Right?

And so it's hard to figure this out. There are projects that will mark themselves archived or end of life, but we can't really define this. " The episode came out yesterday on Monday, where I talk to Brian Fox from Sonatype about their 2026 supply chain security report, and they do this every year.

I talk to Brian every year. I've known Brian for a million years, and I love him to death. And we talked about end of life and the fact that just defining that, it feels like it should be really easy to do, and it's almost undefinable in open source.

Which is, again, one of the super frustrating problems we have when we try to define the whole open source security universe and what's going on. Mm-hmm. I know, right?

That was a mouthful, but like No worries. Is it your sense that the adversaries are just getting smarter about how to exploit these issues, or have these issues just been around forever and they've now found the time and the tools to go and actually do something about them than when they were too busy doing other things before? So I think there's a couple of pieces at hand.

None of this is new. These are the same problems we've always had as long as I've been doing this thing. But I think it's not that the adversaries got smarter, I think they started paying attention.

Because if you look historically, a lot of the adversaries attacking organizations, backdooring components, taking over infrastructure, there was a heavy focus on a lot of the commercial software, but it's the popular commercial software, right? It's kind of like the old saying, why'd you rob banks? Because it's where the money is.

Opportunistic adversaries are going to go where they can get the most bang for their buck, so to speak. And so historically, the easy answer was often large commercial entities that had, we'll say, horrible security. But now open source is in everything.

The numbers vary anywhere between 70 to 90% of all software is open source, and it doesn't matter which number that is because it's a ridiculous amount, right? And everything is full of open source. It's all over the place.

And so this is a very opportunistic place that these attackers can start looking at to understand, what can I do here? And in some instances, it's pretty low-hanging fruit. And I'm not faulting open source in any way for this.

This is one of the challenges the open source maintainers have now is they're being essentially attacked from all angles where you have the bad guys attacking them. You have community people complaining about what's wrong, or there are bugs, or they're not fixing security fast enough. You've got enterprises using their software demanding features.

So it's coming from all over the place, right? But anyway, back to your question. So I think when we look at most attacks, they're opportunistic and it's where the money is, so to speak, right now, so it's what they're going to focus on.

But also because it's open source, there often aren't resources, right? One of my favorite examples is if we look at any of the organizations over on that floor right there that have the big booths, they have security teams with hundreds or thousands of people on them. And if we look at the data of open source projects, the vast majority, I'm talking like 90% plus of open source, actually, it's more like 99% of open source is one person.

Mm-hmm. They don't have a security team. They don't have any team, right?

It's literally them doing the work. And so I think one of the challenges the industry needs to start figuring out, and this is obviously a good place to have a conversation like this, is what do we need to do to start supporting some of this open source, some of these projects, some of these foundations? There's a lot of work to be done, and it is happening.

I'm not going to say it isn't. And especially in Europe, like we have the Sovereign Tech Agency that's been doing an amazing amount of work. The OpenSSF just got, I think it was $12 million for Alpha Omega.

Alpha Omega's been doing amazing work. I actually have an interview with Michael Wisner about that coming out next week. So I'm very excited about that, and that's really fun too.

Some people would say that relative to the mission, though, those efforts, the dollars involved are a drop in the bucket. So what do we really need to do to secure all this stuff? Because there are tons and tons of these maintainers out there, and not all of which are especially passionate about security.

Maybe. I don't know if I would say that. I struggle with this one, right?

" And they'll be, "I don't care. I hate security. " And that does happen, but I don't think it is common, right?

It gets the news because it's interesting, and we laugh at them, and we point fingers and make a big deal about it. But-I think that... So, I mentioned I had a podcast.

I talk to a lot of open source developers. It's one of my goals is to hunt them out and talk to them and understand what they're talking about, because a lot of people aren't. One of my complaints is that when you talk to a lot of organizations and even some of the foundations, I'll pick on them a little bit, is they'll tell you what open source thinks.

And then you go talk to a lot of these developers, and that is not what they're talking about, which is an interesting chasm we have to cross sometimes. But now, your question about funding, is it a drop in the bucket? It absolutely is.

But that's okay, too, because we have to start somewhere. This is one of those situations, if we would just dump more money than we knew what to do with on top of open source, it would ruin it. It wouldn't fix it.

This is like where you hear the stories of people winning the lottery and they're bankrupt a year later. So I'm not saying we should stop and say, "Oh, $12 million. Good job, everyone.

" But I do think it's a good place to start as we try to understand what can we do with this money. Because how we fund some of this stuff, what we fund, is a challenge. And I'll give you a really good example.

So in all my discussions with open source developers, there are open source developers that have, let's say, one or two small projects they work on. And they do it on the weekends, maybe one weekend a quarter at most, because they have lives and families and everything else going on. But when you have one or two small packages, basically no amount of money is going to get them more time.

Because they have jobs, they have families. If I say, "Here's $10,000," that doesn't help them, really. Maybe it'll buy them a vacation, so now they're going to miss one of their quarterly updates.

And the challenge being, these people need, if you want them to work on this, they need it to be their job. Where they're making enough money to support themselves, to support their families. But now we go from, "Oh, I'll give you $10,000," to we're talking about hundreds of thousands of dollars, or I guess euros, being the current venue.

Yeah. But you know what I mean? And so that's one of the challenges is there are developers we can easily say, "Here is a living wage.

" But then there's other developers that might have one project. Who's going to pay them a living wage to work on their one project? Do we need maybe a SWAT team that helps those one person teams, and that's their job to go help them handle this particular security update or fix this patch?

Maybe. So that has been tried in the past more than once. And in fact, that's basically what my job at Red Hat was for a long time is I would help coordinate security fixes amongst open source projects.

And there are projects you can do that with where they'll maybe accept the help, they want the help, but sometimes they won't take it because maybe the project they work on is kind of weird and obscure. But one of my favorite examples is I was reporting something to one of the font-- It was one of the TrueType font libraries one time. And I had all these reproducers that would crash the font library, and I put together a patch, and I wrote it all up, and I went to them, basically said, "Here's what we found.

This is what's going on. " Looked at my patch, and they were like, "This patch is terrible. " Because I thought it was three bugs.

It was actually one bug in some function I didn't even know existed. " Because now they looked at my patch, decided I was an idiot, and then they went and did it right. Do you think in the age of AI, it might be easier to create the patches going forward?

Because now we have AI coding tools, so theoretically, they could tell us what we need to fix. Maybe. I'm very interested in this topic.

This is something I've been looking into. So I think the AI agents give us a couple of almost dichotomies when we look at them. So they're pretty good at writing code.

But only if you're a developer that knows how to tell them how to write code. Because if you look at someone who gives a bad prompt, you kind of get crap code out. But if you give it a really good prompt, you can get okay code.

Now, there are instances where the LLMs are writing patches, and they vary in quality greatly. I know there are people building autonomous agents that look through bugs in their GitHub repository and then creates patches for them and creates test cases for them, and it's almost magical in that regard. How is the long-term quality on that?

We don't really know yet. We don't have numbers. So I'm really interested to pay attention to that.

We also have LLMs that can find security vulnerabilities. Finding security vulnerabilities as a human is really hard to do. I can probably count on one hand the people that I'd say are good at it.

And so the bar's not very high. And so I would say LLMs today are probably better than the average human at finding security vulnerabilities. And then we go back to now we have to patch them, and we have to see how the agents are going to do with their patches.

And so that might be one of the answers. I'm not willing to say yes or no yet because prove it, I guess, is where I am. I like to call myself, I'm an AI skeptic, where I want to believe, but I'm also not going to make claims that aren't true, because oftentimes that's worse than anything else.

But are we also concerned that the adversaries will be using AI to create exploits faster than ever? Because we've been lucky when you think about how long it takes them to do something. Well, maybe that's going to be now done in a few days, and they'll be launching all kinds of things that we're going to be- Yeah ...

" Hours. Not days, hours. So if you look, there are various graphs in the security world that show time to exploit.

And a couple of years ago, it was like 30, 40, 50 days. It was hundreds of days a decade ago. Now it's literally hours.

And without question, part of that is LLMs. They're really good at writing exploits. " And it does.

Well, usually. Not every time. It depends.

The answer's always it depends with this stuff, but it's pretty good. I'd say it's better than a human. I've written exploits in the past, again, when I was doing my coordination, and it would take me days sometimes to actually get to the point where...

Because you have to craft a JPEG image and then modify it in just the right way. And you test it, and then you tweak it, and then you test it, and you tweak it, and it's just back and forth and back and forth. And these things can do that faster than a human can imagine.

So, and yes, adversaries are 100% using this technology. Adversaries, when we think about the defender versus attacker scenario, the defenders always trail attackers, just by definition. That's how it works.

I know we like to pretend we can do it some other way, but it's just how it is. A wise man once told me it's a lot easier to throw grenades than catch them. Yes.

Exactly. 100%. Are the folks who write container applications maybe a little too comfortable?

" And they have to go find that container again, and they may have the same code over again, but they'll never find it because it's just moving too fast. What's reality? So I love when people ask me this because, so you've got one camp of people are saying you have to move as fast as possible.

Update every 15 minutes. Update it as absolutely fast as you can. " They're calling it a cool down period.

Because, for example, a malicious actor might have inserted a backdoor, and let's give the community and researchers time to do their investigation and understand, is there malicious code in this update? And let's say after seven days, if no one finds anything, we'll assume it's okay. And these are very different problems because you're moving very quickly, and we're telling people not to update.

But I think the reality is, if you look at the vast majority of organizations, and I talk to a lot of organizations, both big and small, most of them are not moving that fast. There are some, but most aren't. And so this is where you get back to the fundamentals I mentioned way back at the beginning of this conversation, where understanding what you have, being able to run some scans against it, understand what are the potentially dangerous spots I have in my infrastructure, and then I can focus my very limited resources on those areas.

" They're just, "We have infinite money. " And that's just not most people. Mm-hmm.

The open source community is built on trust. It's at the foundation of the thing. It was like everybody was pulling for everybody and doing the right thing.

Are we in danger of losing that? Because now you've got bad actors who are joining projects and pretending to be people who are legitimate, and they'll work on a project for six months and then insert something nasty. And do we have to kind of just assume now that not everybody is who they say they are, and their intents are not always benevolent?

That is one of the things that I talk to a lot of people about, and we don't have a very good answer. So there's two sides to this problem. What you're talking about is the XZ incident, where we had a rogue maintainer, Jia Tan, that inserted a backdoor into a project, and it is terrifying because you know what keeps me up at night, Mike?

Is that there is no way that's the only project that group went after, and we don't know who else they went after. And that legitimately scares me because I have a suspicion there is a backdoor somewhere we haven't found yet. We might someday, we might not.

It's very hard to say. Now, on the other side, there's people who say, "If you don't want to trust open source, what else are you going to do? It's 70% of your application.

You're screwed. You have no choice. " And this is something we see, where one of my favorites is you'll talk to security vendors, and they'll tell you, "Oh, you can't trust public package repositories.

You need to buy our product where we're selling you trusted open source. " There's all these things. But the reality is almost everyone is using all of the open source from all the public repositories, and there are people getting whacked every day because they're using untrusted or typo squatted packages or whatever.

But if you look at the numbers, it is almost unquestionable that the benefit we're getting from open source, even knowing some of it is malicious and some of it is dangerous, grossly outweighs the potential problems there. It's kind of like we could sit at home in our hermetically sealed houses and never get sick, or we could come to a conference knowing that there is a possibility we're going to get sick, but that's okay because the benefits we get from things like this vastly outweigh catching a cold or hopefully not worse. " If we just did this, there'd be less heartache in the world.

I wish I could say you could just do this. I wish that in a million years. Right?

Obviously, buy my product, that's the answer. But no, in all seriousness, I think the correct thing to do in this instance is the whole point of open source is that community aspect and understanding what's going on. And I'm not saying you have to be involved in every project that you use, because many of us use hundreds or thousands, so that's not realistic.

But I would say, find something you care about and try to make it better. It could be one of the dependencies you use. It could be joining a CNCF group.

It could be working with OpenSSF. There are a ton of organizations, there are a ton of projects. There's lots of projects looking for help.

Go find one that needs help and get involved. So I really think the trick to open source is being part of open source, not buying a product or setting a timeout on your cooldowns for upgrades or running a scanner. It's really about get involved and understand.

Be open source. There you go. Folks, that's the secret to all things.

Stay involved. " Yes, 100%. Absolutely.

All right. Thanks for being on the show, my friend. Thanks, Mike.

All right. And we'll be back. Hey, everyone.

We're back here live at RSAC. We're at the Broadcast Alley on Moscone West. What a crazy day.

Keynotes and this and that, all around agentic AI it seems every step of the way. But let me introduce you to my next guest. I hope I'm going to get his name right out here.

His name is Aamer Lakhani. Hey, very good. Very good at saying the name.

All right. Aamer is with Fortinet. He's going to tell us about it.

Actually, Aamer, you're the Global Director of Threat Intelligence and Adversarial AI Research at Fortinet, but that's not what you were born. Give us an idea of how you got to this place. Man, that's a handful in the title itself.

Yes, it is. I've always been interested in how technology works, but how technology breaks. I don't want people dictating how I should use technology.

I want to use technology the best way it fits my life. And that's what got me into cybersecurity is how to break technology, and really that's what I do with my job. My job is to really look at how the bad guys are using technology.

What is their motivation? What gets them going, right? And I create machine learning models to protect against those attacks, especially baselining those attacks.

That's throwback right there. That's old school. So I've been in security almost 30 years.

Back in the day, they didn't have-- Well, A, we didn't call it cyber anyway, we called it InfoSec. B, we didn't have schools that taught cybersecurity or information security. Most of my friends got into security just like you.

They liked to break things to see if they could, and then build it back better so it couldn't be broken. Exactly. Or as easily, anyway.

And that was the original hackers, that was the original InfoSec people, as we called. Hey, back in the day, I was on the BBS systems as well. I know.

Absolutely. We were all on BBS and using those things. There was no web, but it was fun times.

And that's why so many security people used to be into lock picking and that kind of stuff. Exactly. It was about the tinkering, the breaking, the putting it back better.

But in your role at Fortinet, you get to do some of that. But let's face it, this is a big company. So there's more structure around it now than when you were just hacking for fun.

Absolutely. We have a sandbox we have to stay in. Yeah.

At least most of the time. But we do like to see how far we can extend that sandbox, because that's what the bad guys are doing as well. Absolutely.

They don't have rules. They make them as they go. Exactly.

So we want to see how far they push it, and how do we stay ahead of those rules. Excellent. Let's dive in here.

We got a few other things we wanted to talk about. How long have you had this role at Fortinet? So I've had the role for about four years, but I've been at Fortinet for almost close to eight years.

Really? Yeah. Oh, good for you.

I'm ashamed to tell you, I remember when Fortinet was founded. The Z brothers had sold NetScreen firewalls to Juniper. And then they founded Fortinet shortly thereafter.

That was the birth of it. And they half invented the whole UTM category back then. But today's Fortinet, of course, is very different.

It is. But from the foundational, it was always about more than just protecting against packets, because that's what everyone did in the past. They had a packet filter and access list.

Yep. And that worked, but until you understood what was the intent behind those packets. What was really the actions those packets were taking?

And I think that's where Fortinet made a name for themselves. And obviously, that's extended well beyond just network security. With agentic AI, with cloud, with the OT space.

I'm always reminded of a quote from Muhammad Ali. I love boxing, and Muhammad Ali had a famous quote. " Yes.

It's all about having that visibility, and I think that's what Fortinet does, is it gives you that visibility. ExcellentAll right. I'm going to switch gears a little bit.

You guys recently came out with a threat landscape report? Yes. Tell us about it.

Well, so we do this every year, is look at all our telemetry, look at all our data, look at our research, and we try and get a baseline on what's happening in cybersecurity. What are the trends telling us? What are the bad guys doing?

And, what do organizations need to do to stay ahead of that? And so, that's kind of where it starts off at. Absolutely.

So this is an annual report, and what's nice is this is your own data. This is not, you hired some research company who maybe talked to people, maybe didn't, but submitted data. This is really, real-world data that you've anonymized and are able to pull in on.

I always like to ask, what are the three key things in this report that you think people need to know about? Yeah. So, good question.

And, when you say it's our own data, I do want to point out that we're kind of lucky that we have a lot of network data, as well as endpoint data, as well as cloud data. So we get a lot of different sources. And from all those sources, I think the things that we're seeing, of course, is, it shouldn't be surprising to a lot of people, but maybe some of the details behind why they're happening may be surprising, is that, attackers are getting smarter.

They're using AI. Yeah. The time for exploitation is pretty much zero now.

They're not waiting for vulnerabilities anymore. One of the things that kind of surprised me is when I was looking at the data, I saw scanning went down, brute force attempts were going down. " I see a decrease in attacks, but exploitations went up.

And it wasn't until I started thinking about it, it's like, well, attackers, they're just being much more efficient these days. Yes, they are. Right?

They don't need all the stuff they used to do, even a year ago, now to be much more successful attacks and stay under the radar, be undetected. Yeah. So we're seeing this in ransomware, and across the board.

How much of that you think is AI? So there's a lot of it that's AI. AI used to be helping attackers basically get around- Better phishing ...

exactly. But now it's totally different. There's tools out there such as FraudGPT or WormGPT, which are basically the hackers' versions of ChatGPT.

Right. And these tools will kind of craft an entire attack. They'll scan an organization.

They'll tell them, "Hey, these are the vulnerabilities they already have. These are the exploits that are already available, and this is how you target those exploits. These are the people that you have to go after.

" Don't send them a phishing attack. Join this Facebook group that they're already part of, and post something to them. And put it in there.

It's a hard world, man. I want to stay on this report a second. There's always something in the report that you're like, "You didn't see that coming.

" Like, maybe I thought it was there, but I didn't realize it was that widespread. What in this year's report kind of, maybe surprised you? Well, I think once you're working with it every day, there's things that are not really surprising, but things that always stand out.

For example, hackers are not hacking you anymore. They're simply logging into your networks, right? I mean, just the amount of breaches that are occurring from stolen credentials, credential stuffing, combo lists, that's amazing.

You'd think, as security people, we're kind of surprised by that because, of course, like every security person, we probably have a separate username and password and two-factor authentication everywhere. Yeah. Normal people don't have that, by the way.

Well, because they think it's a pain in the- Exactly ... it's cumbersome. Hopefully- That's a better word.

Cumbersome. Yeah, hopefully there's solutions around. Passwords are the weak link, and- They have been, but you know what?

So let's get rid of passwords. Let's get rid of passwords. I've been hearing that line for 15 years, and here we are.

What's your password, okay? I don't know if we ever get rid of... It's like 100 years ago, I started a security company, and we did network access control, vulnerability management, intrusion prevention.

And back then, all the rage was, we're going to have a password list, an agent list, agentless vulnerability management, agentless NAC. Right? No agent, no...

And people were amazed. " Well, we just put a little code on your computer. But it's not an agent, it's just a little code.

Yeah, b******t, it was an agent. Right? It was.

I don't care what you call it, it was an agent. It wasn't agentless. I think it's the same thing.

If you're substituting password for something else similar that's unique to you, other than maybe some biometrics, it's not passwordless. You're still using a password. It's just a different form of a password.

Right. And so, over the years, I've seen a lot of companies say, "We're going to get rid of your passwords. We've got pass keys.

We've got two-factor authentication. " But I think you're bringing up a very interesting point, that something that we're starting to see is, when you get to the heart of this problem, it's about managing identities. Yes, it is.

And, identities are not people identities. There's machine identities. Machine identities.

Agentic AI agents, those are all identities now, and needing management. It's going to blow it up. We're not ready to scale that.

We're not, and people have kind of... IAM, like identity and access management, it's never been a sexy solution, right? People have always gravitated towards something else, but it's going to be like, we have to go back to these foundations.

It's the heart of it. Yeah. Exactly.

Not only is it the heart of this whole agentic problem, it really at the end of the day is the closest thing we have to cloud security, right? I grew up, I came up in the moat-and-castle era, right? We had the big firewall, UTM, and big boxes at the perimeter.

There's no more perimeter. Like fortifying your networks? Like fortifying it?

I'm telling you, I remember when it was launched. But really, in today's world, IAM, identity and access control, IAC-Is where the action is, right? Look, in the last month, we've seen AppSec get spun on its head with all of this new AI-based scanning.

I don't know if it's ever going to be the same. I think AppSec has fundamentally changed from finding bugs to fixing bugs. Attackers are not doing that.

So they used to. You know what happened is, a vulnerability came out, attackers used to scan the internet. And- Well, now- ...

every security guy you ever talk to, they always said the best thing you could ever do for security was patch. Right. Anyone that ever said that never had to patch systems on a large scale, right?

But today, attackers don't care. They're continuously scanning. They're indexing targets as soon as a vulnerability is released.

And now they're using AI to write those exploits, and so it's zero time to exploit. AI is scanning it. Yeah.

They're using AI to scan it at a speed- Yeah ... and depth. You've read the same stories I have, right?

Finding 600 vulnerabilities in a day in open source, 120 something in Firefox alone. What we did yesterday, what we did last year, isn't going to scale under the weight of this kind of- You can't keep up with the volume and the velocity of attackers. You've got to use AI.

Yeah. It's the only way you're going to be able to keep up, really. If you're going to be attacked at machine speeds, you have to defend against machine speeds.

At machine speeds. And I know that kind of sounds like a marketing line, but it's so true because there's no way to do it without AI. No, it's logic, right?

You need fire to fight fire. Hey, let's pivot a little bit. Let's talk about RSA.

What's Fortinet's story at RSA this year? Well, Fortinet's story, this year we're looking at our agentic AI solutions, our FortiSASE solutions, our FortiAI SOC. So we have a lot of different things.

So if you're at RSA, come by the Fortinet booth. We have a lot of demos. But I think what we're really trying to show you is, now we have the tools to combat the speed of attacks.

It's like the first time in my career, well, at least in a very long time, that I actually feel like there's more hope than we've ever had before because it's always like a cat and mouse game, or- Yeah ... hey, we're behind the curve. But AI is kind of changing a little bit of that, at least right now it is.

I don't know how long this window is going to be, but because of the investment AI takes right now, the good guys have a little bit of an advantage that the bad guys don't, and that we can maybe utilize. But of course, we know that's going to change, right? Yeah.

So we might as well try and get ahead. I think, like they say in football, right, on any given Sunday- Yes. On any given Sunday- Great movie, by the way.

Yes, it was. But on any given Sunday. " I said, "Look, you do the best you can.

You start with a zero trust kind of posture. You isolate it, you limit it. But at the end of the day, we're in uncharted waters here.

" You have to have zero trust for your agents, and you also have to look at how you're using LLMs and AI in your own environment as well, because there's a lot of attacks that I think we talk about, but unless people start seeing them or experiencing them, they don't really understand- No ... what does data poisoning really do long term, right? What does model theft really mean to an organization?

What does your- Look, we're going to find this. So here's the thing. Over the next six months to two years, we're going to find all of these things out in spades.

Because we're going to learn it the hard way, because that's the only way we've ever learned it, unfortunately. People like you, we know it's coming. But you're not going to convince mainstream USA to do what we need to do until, unfortunately, we hear some bad stories.

Yeah. The thing is, some of the tools for mitigating those risks are right here. They're here.

We're demoing them at the Fortinet booth right now. I learned this lesson in security. You know what?

About 30 years ago, I started a company in security, and I realized how hard it was to sell security. " We'd wind up in jail, of course- ... but they'd buy our stuff.

And nothing's changed. It seems to get religion about security, you got to suffer some sort of calamity or some sort of incident. And then all of a sudden, they're looking for God, right?

They're an easy sell. I'm afraid it's the same thing here. Yeah.

I hope we can kind of start changing the hearts and minds. People get smarter. Yeah.

Yeah. " She's like- I can see ... " Not a programmer.

" Right. That's how that goes. Yeah, there's no other choice.

It was lawyers or, yeah, it was, for us. Exactly. Or lawyers.

Yeah. " Like, "I understand- Yes. It got- "...

" ... it went mainstream. Yep.

We'll see. We could hope. Yes.

We could hope. Hey, man. Hey, if my mom's discovering it, I think there's a hope.

You know what? My wife's family still doesn't understand what I do for a living, but it's okay. Anyway, best of luck to you- Thank you ...

and the guys at Forti, and folks at Forti. We're going to take a break. We're here live at RSAC on Broadcast Alley.

We'll be back. Okay, thank you. Hey guys, thanks for the throw.

We're here with Vic Chenoweth, who's the CEO at Tempo Software, and we're having a little chat about how to keep software development projects and teams on track because, well, there's this thing called drift, and it happens to the best of us. Vic, welcome to the show. Thanks, Mike, for having me.

How does this problem kind of manifest itself from your experience? " And then over time things happen, but how do we kind of prevent that drift from kicking in? Great question.

I think people have been trying to solve for this for a long time, and progress improves it along the way, and then other things happen that impede it. The latest being AI, and what's probably the biggest impedance we've seen just because of the pace that innovation can occur, both inside of a company, inside of a team, as well as with competitors. I think that's the core of the issue is that people, businesses, teams, individuals have to respond to signals and changes in the market much faster than they have in the past, as well as changes inside of the company because of the progress and speed they're able to achieve, even with just copilot constructs, let alone agentic AI.

Sometimes I feel like maybe we're overly wedded to our project management applications, and we decide that this is the outcome that we're aiming for, and so no matter what happens in the rest of the world, we kind of just keep coming back to that, even though the project we may have launched four months ago is no longer relevant. I think that's absolutely true. One of the things that I would suggest is true of any plan is it's wrong.

Just depends how wrong it is, and if it's a time wrong, if it's a sequence wrong, if it's an amount wrong, whatever the case may be. And so, I think you're exactly right. I think what we've gotten to is much more a continuous planning construct where the most effective organizations, or at least that's what the most effective organizations are adopting.

So what's your best advice to folks about how to have these conversations? Because I think part of it too is everybody kind of has their weekly meeting and they do the check-in, but it's not clear to me that the check-in is actually anything that's coordinated with a meaningful KPI for the business. So I think where we've gotten to today is that you need data to support those check-ins.

Those check-ins are pretty oriented towards a team or a set of teams. You might even get to a program level with those check-ins. " So on and so forth.

And so it's really a data element, and it's a real-time data element. So basically planning an orchestration at the speed of AI. And another thing there is just drip dropping, probably knocking down the historical walls that occurred and the processes associated with this historical walls.

And what I mean by that is product organization relative to the engineering organization. They work closely together throughout time, I would suggest, at least the most effective have. They're just more in real-time synchronicity relative to what they've done historically.

I feel like there's a lot of dependencies that exist between different projects, but not all those dependencies are maybe obvious or even well explained, and it's not uncommon to see folks working on some project only to discover that some dependency that they were counting on isn't ever going to show up. So, how do we kind of have this conversation in a way where we understand if a team decides to abandon a particular project, the downstream implications of that decision? Great question.

So first off, making dependencies visible and discussing them openly is paramount. The second piece there is really, again, a construct of what you have to do, and at the speed of innovation and development today that we maybe didn't have to do historically. So if I go up a legacy planning approach, hey, let's look at our strategy, reassess things, think about a strategic roadmap annually.

Let's go execute that, keep working things quarterly, more like at the team and function level, and then step back at a strategic perspective annually. That cycle doesn't work anymore. So I think companies, the most effective in near real time, are able to connect planning and execution, including dependencies at the execution level up to strategy level in near real time.

It's not quite real time. Certainly, the project level is. But if it was an annual planning cycle, it's at least that it looks at strategy and the connectivity.

It's at least quarterly and some are probably monthly. " So how do we get to that ability where maybe the information and the insights we're looking for just kind of get automatically generated by the tools we're already using? Great question.

And that is what we're trying to do at Tempo, as you probably know. And so what we do there is we want to connect all of the signals, which includes all the delivery capacity, time, dollars, you name it, and then systematically connect that so that everybody can see it in real time as well as when they want to step back for a planning process and orchestration. It requires commonality really in data.

So one of the things we don't want to do, because it's really hard to do, is enforcehow people are going to work and in what tools they're going to work. What we have to be able to do is uniform that or standardize that data and bring it together so that program management can see it, you can see it at a portfolio level, and teams can see it. Because let's say, Mike, you're on one team, I'm on another team, and you have a dependency on me.

You need to know how am I doing that or on that, and is that going to be ready in time for you, or do you need to redirect? It has to be done at that data level. It can't be done word of mouth, which probably worked effectively with some teams and some organizations in the past.

That just doesn't apply anymore. How smart will all this get? And I'm asking this question from this perspective.

" Great question. You probably asked that question today. It'll give you an answer.

How accurate it is, I can't tell you, but it'll probably certainly get you thinking. I mean, AI is advanced materially, and it keeps advancing daily, if not hourly to some degree. And so, leveraging it just to ask some of those what-ifs is something we can do today.

What we're trying to do, I think what's really important for engineers and product organizations, really delivering any product or service as we're innovating, is accelerating the planning and orchestration of the work. Because that hasn't naturally or been the first thing out of the gate to apply AI to. It's really been, how do we develop faster?

How do we innovate faster? But now that we've done that, how do we orchestrate and plan work more effectively so we can actually get the outcomes and optimize the deployment of whether it's the combination of human engineering and agentic engineering. So what do you see organizations doing these days still that makes you shake your head a little bit and go, "Geez, folks, I think we need to be a little bit smarter than that"?

I think the most common thing that I see, and I've seen this at Tempo, and I had to go, "Hey, we've got to think about this differently," is, "Hey, I have a mindset of I'm going to keep doing it the way I've always done it. " And the folks who are making the most progress, doing things, getting the most value of AI at the speed of AI, so to speak, step back and re-engineer those processes. They go, "Look, I need to have an AI-centric view of the world versus my legacy-centric view of the world," and go with AI.

Give you an example. Strategic roadmaps is a functionality that exists in a SPM umbrella. " Well, we should be able to just ask an application to go, "Here's all the things we know.

You understand what a strategic roadmap is because here's my last one. " It's a different way of looking at the world, including how you even build that strategic roadmaps functionality, just using that as an example. Do you think that teams, each of which is likely to have their own AI agents at some point, will enable those AI agents to negotiate with each other?

Because that's what we humans do, right, when we have projects and things change, we discuss, and we come up with some sort of optimal plan, or sometimes also known as a compromise. How will these AI agents articulate that conversation on our behalf? Great question.

I think folks already have AI agents working with each other, and therefore, they're having to negotiate with each other. I think the big question is going to be is, how much, I don't know, decision-making authority do we give to the AI agents versus the humans that inform those AI agents and guide them? I don't know the answer to where that ultimately ends up.

It's probably the multimillion-dollar question that folks have, one of many. But it is something that we're already at. We already have AI agents talking to each other and negotiating with each other.

And it's really going to be the question of how do we leverage that, but in such a way that we don't lose the control and ultimately the ownership of what we want to get at the end of that day. I don't think it's much of a dirty secret, but very few projects in corporate America actually show up on time and under budget. So, do we have any hope of actually achieving that goal someday?

I guess the question is, is the goal to be over 50%, over 30%? I don't know. What is that goal?

I think the way we look at it is for every individual customer we work with, we want to go, how do we improve upon what you're doing today? And I think that's what every customer wants to do, or every enterprise out there wants to do. With AI, the opportunity's going to be to accelerate all those things.

Question's going to be whether our expectations align with that new speed or not. I don't know what the right answer is. I think at the end of the day, we shouldn't deliver every project on budget, on time, so on and so forth.

Because if we do, we may have aimed a little bit too low on average. What are the right balances there? I can't tell you.

It's probably certainly over 50%. But again, if you don't aim high enough, you're going to hit your budget and your time every time. So it's a balance there, quite frankly.

Ultimately then, this is an exercise in change management. So what's your best advice to folks about how to navigate that? Because it's not just a technology issue, it's a matter of often humans and political capital and will.

Great question. I'd go back to something you mentioned earlier, which is visibility to dependencies. To me, it's like, let's provide visibility and transparency to what we're trying to accomplish and why.

So, a lot of folks have been afraid, and they continue to be a little bit fearful of, "Oh, AI is going to displace me. I'm not going to have anything to do," so on and so forth. And if you come at it from a fear of your approach is to kind of be protectionist of jobs, that's going to be a hard way to win at the end of the day.

However, if you go, "Hey, this technology is here. It's going to move forward. " So on and so forth.

The way I always think about it is I want to automate myself out of a job anyway because there's always something more important to do. And so, I think people just have to have alignment on why and what we're trying to accomplish so that you can actually get everybody participating in it. Because without everyone participating or the majority, it's really hard to move it forward.

And in my experience, if people understand the why and have had a chance to contemplate it, internalize it, discuss it, they'll own the what, the when, the how. And so I think that it's really a transparency, and I would suggest that's probably the most important part of any sort of change management initiative, is making sure that folks understand visibility to why we're doing this change. All right, folks.

You heard it here. Hey, even in the age of AI, it's still true that if you want to shoot for the moon, maybe aim for the stars. Hey, Vic, thanks for being on the show.

Thanks, Mike. Appreciate it. All right, and back to you guys in the studio.

Hey, everyone. It's Alan Shimel from Techstrong. We're going to continue with this fantastic series we're doing of sessions between some of the leaders at Microsoft, as well as analysts from the Futurum Group.

In this next session, we're lucky to have Clay Wesner. Clay is the Partner for GPM Power Apps Studios at Microsoft, and Futurum analyst, Keith Kirkpatrick. In today's session, it's really a customer success story where Clay, joined by Keith, are going to delve into a real-world customer story, in this case, Wells Fargo, offering a blueprint for leaders ready to scale success in the age of intelligent apps.

You're going to see how Power Platform is being used to modernize complex, regulated workflows with Copilot Studio agents and Power Apps. This session will highlight architecture, business impact, and lessons learned from deploying intelligent apps at scale. I think it's really a great session you're going to enjoy.

Let's go to Clay and Keith. Hi, I'm Keith Kirkpatrick, Research Director with the Futurum Group. I cover enterprise software and digital workflows.

Hi, my name's Clay Wesner. I look after our low-code developer experiences on the Power Platform. Well, thanks for joining me today, Clay.

Maybe, Clay, you could talk to me, though, a little bit how Power Platform can actually help these organizations balance that agility to handle these types of scenarios with their compliance needs that often come up when you're dealing with things like banking or insurance or any one of these regulated types of processes. Yeah, absolutely. And within the product, we sort of refer to this as managed platform because it is very much a feature of the platform of how you can govern at this scale.

And this has come from not just us deciding exactly what's going to be in there, but really folks and customers leveraging low code over the last 10 years and evolving to have a really, really strong governance. Because I think we learned very early on in the journey that if those guardrails are not there, people are just inclined to want to turn it off. And I use the word guardrails deliberately, and a lot of the things we do in a managed platform is focused around how do we give you the right controls so you can still enable these types of tools, whether it be building apps, building automations out at scale, but do it in a way with the right sort of controls and guardrails on it.

And so examples are things like data loss prevention. So I can set rules around what connectors and what data you can access versus someone else. And so I can also say how many people you can share an app or a workflow or something with.

And so I can sort of mitigate the risk that you might be able to have working in low code versus someone that's received more training or onboarded to the platform. And so typically, what we see customers do is sort of implement this zoned approach of their zone one is everyone in the organization, and they say, "You can build apps for personal productivity. You can connect to your office data.

You can sort of work with those well-known sources. And you can go and share apps and flows and agents with up to 10 people," as an example. "But once you want to go beyond that, we want you to engage a little more with IT.

We want to make sure things are supported. " And then what typically happens is we'll then have a zone two, which is potentially some more sensitive data, potentiallyYou know, ability to share with more people within the organization. " And then that final zone would be your IT, your dev center, who's working with your really critical data around things like finance and HR.

powerautomate, and start building, and they're not going to fall into a trap there. They're going to fall into the pit of success because we've put those right guardrails on what they can access and what they can do. If you look at not just if we're talking about, let's say, agentic technology, but just everything.

If you look at the development of the smartphone, everyone expects sort of a consumer-grade experience throughout all facets of their life. And I guess that, do you see that sort of pushing or helping to evolve kind of what customer success might look like not just now, but into the future? Again, earlier in the low-code journey, it was always IT departments, development teams that were looking at the low-code platform.

And more and more these days, as we're talking to customers, it'll be their employee experience team. It will be folks responsible for actually healthy and productive employee experiences. And that's what I mean.

It's not just about cost saving, but it's about bringing the right tools in. There's the SNCF, the French railway, are actually a really good example. They run Power School, which is an onboarding school for the whole Power Platform when any new employee starts.

And this is becoming a really, really common practice that more and more folks are, as they join an organization, they're getting training on these tools, not as something they have to use to do their job, but as a benefit to them to be able to do their job in a more productive way. And I think, again, the consumer push and acceleration of AI is just accelerating that within the enterprise as well. Right.

When you're talking about human in the loop, you raise a really good point, because ultimately this is still new technology, and you want to make sure that particularly in, you're dealing in a commercial environment, that you don't want this agentic technology to sort of run wild or unchecked. So I'm just curious if you could talk a little bit about, have you seen other examples where customers have actually deployed their sort of checks, and balances to make sure that their technology does what it's supposed to? Yeah, absolutely.

And there's a couple of ways we're seeing folks doing that. One is just in how we define and build the agents and the tools themself. While that agent has the ability to issue refunds, it can only do them up to 100 pounds.

So it has very specific guidelines built into it that once it goes over certain criteria, loop in a human, send them an approval workflow so that they can approve this, review the details. So that first one is just very structured, giving the agent details. The other side, and this is where we've sort of really seen how apps have evolved in the last couple of years.

If you've been looking at what we've done with Power Apps, we've introduced this concept of an agent feed, which is really about in the same UI that you would come into the app and do your day-to-day work, you start getting this feed of activity from the agents that are in your digital team effectively. And so you can start seeing where they're completing actions, where they might need assistance, or where they're getting blocked. And so what we're starting to see there is even our UI patterns of what we traditionally thought an app was, is starting to bring in this agentic behavior to give, you know what I mean, that human in a loop and that oversight capabilities.

So I still want someone to have a really clear view of what tasks are being completed by the agents, what's being completed by AI. And in that view, be able to get into the reasoning, understand the logic and sort of the thought process that the agent followed behind it. So it's not a mystery of why something progressed or why an action was performed.

But as the human responsible managing that team of agents, I can effectively go in and see why it did something that might be then become a teaching moment for the agent where we correct that behavior or change it for future cases as well. Well, it's really interesting you mentioned sort of the generational shifts that are going on. We're seeing the entry of these, I guess you'd call them AI natives, coming into the workforce where they don't know anything other than a world with AI.

And I guess that kind of begs the question, we've heard so much about AI in the past, particularly the last couple of years. Can you talk to me a little bit about what role can AI actually play within customer success? Because it's a wide, AI has so many capabilities, but I'd just be curious to see if we could boil it down to this function.

Yeah. And I think it honestly depends on the customer and how they're approaching it. One of my favorite examples of, I think, sort of scale and pace, PG&E here in the United States, they're a big Power Platform user, and we talk about scale.

I think they estimate since they started their journey in 2021Along the lines of like $38 million in savings that they accrue to the power platform like huge in terms of scale. But so much of actually what they've implemented is not just cost efficiencies they introduced an agent called Peggy and they actually have a nice little avatar for Peggy that they introduced across the organization. And Peggy now handles, it's between 30% to 40% of their IT help desk calls.

So built in Copilot Studio, Peggy has access to their knowledge base, all their policies and documentation. And just Peggy, one agent, they estimate saves them about $800,000 a year. Wow.

And it's absolutely transformational. And so even with the savings they were getting on the power platform between apps and automation, there is a limit. Mm-hmm.

There's a limit to how much productivity that can drive agentic, you know what I mean, tools and what people are able to build in Copilot Studio has really just broken through that barrier. And you look at, again, someone like PG&E, when they implemented Peggy, it was very simple, looking over knowledge bases, access to information. It helped a large volume of sort of tickets that would come through the help desk that used to be a human replying to an email or replying to an IM.

Those humans now are actually providing much higher quality support on more technical cases. They're not helping someone log into Citrix for the first time or point them to something that's really well documented. Peggy's able to do that.

But then they've also continued to evolve it over time. And so, again, one of my favorites that Peggy can do is getting folks that get locked out of their SAP accounts. One of the most common things that IT, apparently happens thousands of times.

And now Peggy, using an integration between Copilot Studio and Power Automate, can actually open up SAP and go and unblock that person's account for them after they interact with her on Teams. And so this was something that was critical to an end user to get unblocked really, really quickly. Peggy's able to do that for them fast, but it wasn't high value from an IT support team and what they were really providing, them going and opening up an account and unchecking a blocked checkbox.

And so I feel it's a really good example of where they started simple. They focused over sort of knowledge base examples. They evolved it into actions.

But it's something where they've gone for a high volume cost inefficient area. They've applied agentic AI to it, and that's something that go back three or four years ago would have been an extremely expensive tool to go and implement. Leveraging LLMs and leveraging Copilot Studio, they've been able to do all that in low code, which is super impressive.

I'm curious, one thing, Clay, that you alluded to earlier is if we think about how apps were previously developed and rolled out, it was IT who kind of managed that. Now, it sounds like what you're saying is we're getting to the point where business leaders or even folks who are working within departments may be able to actually launch apps or launch agents, obviously, with that human in the loop and with those specific guardrails. Are you seeing any kind of patterns emerging in terms of customers who've successfully scaled this agentic automation from more of a grassroots approach as opposed to springing from IT?

Yeah, you're absolutely right. We sort of see an approach from both directions and some- Mm-hmm ... customers very deliberately approach it from one or the other to start with.

I actually feel like all the examples I've sort of talked about today do quite well balancing both spectrums, both ends of the spectrum, sorry. And I think that's where you start getting the real value multipliers. PG&E, great example.

I talked about Peggy earlier. That's an IT or centrally led tool. It was about optimizing a process within the IT team.

But at the same time, they have thousands of developers across their organizations. And when I say developers, I mean low code citizen developers that are enabled to go and build apps, to go and build agents, to go and build automation across their team. And they've sort of very deliberately focused their center of excellence, their digital transformation team on a few core objectives.

So that's the team that sets their governance policies, makes sure it's scalable, and then they also support and train those different sort of divisional leads across the company. PG&E actually, again, I think they're on the spectrum, the end of the spectrum, where they're doing this in a really amazing way. They have a conference every year called Level Up Now, where they actually get together all their citizen developers, and those divisional leads from across the company to come together, share stories, share learnings, and sort of explain new technology.

But it starts becoming a real cultural tool in that they're enabling people to go and solve these problems, make themselves and their teams more efficient. And there's reward that comes from that. They're getting folks together.

They're getting a lot of learning. And so I think, while lots of companies are enabling citizen development, the ones where we see it's truly being successful, they're bringing this level of evangelism to it. To help organizations address all of these issues.

There's obviously the human, the technology component. I would also say there's just the practices and sort of learnings. We actually have some good documented platform guidance out there of what are the best practices in thinking about this zoned approach that I was talking about and in how people can sort of apply different levels of control to different parts of the organization.

I would say then we start looking at the specific technology. One, a lot of those guardrails just light up directly in the product. So as a new citizen developer, as a maker, when I go land at any one of the power platform tools, I can get welcome guidance with links to internal learning, explanations of where I can go to support.

I get routed to my own personal developer environment. So I actually have a sort of controlled, dedicated environment for me to go explore in, to experiment in. I'm not sort of working in prod.

I have the ability to be controlled. And then things like pipelines, which effectively are a low-code ALM tool, so that once I do build something, I can either use it for myself and my personal environment, but if it gets to the point where it does make sense for it to be deployed somewhere centrally, leveraged by others, I can go through an automated deployment process where the right checks go. I have an AI advisor that reviews my code, makes sure my apps are secure and performant and accessible, and then get the right approvals before that gets deployed.

And it's really that mix of, we want to democratize, we want to make these things available to everyone across the organization, but then have these right built-in tools so that you don't have to go read a wiki to find out what's the process that you should follow. It's built-in to the developer tool. So I kind of just, as I start building, get guided to the right environment.

I get guided to use the right data. I get guided to share it and deploy it in the right way. And all of that we bundle up and sort of leverage within that managed environment, which gives the admins, the IT, the central digital teams, that control centrally to sort of set up those tools and that content that they want available across the organization.

It sounds like all of these tools really underscore what you were talking about before, which is this culture of trying to utilize technology in a way where it's deployed at the right time, in the right space, and with the appropriate guardrails, but while still fostering a culture of experimentation and ensuring that people feel empowered to use these new tools. You're absolutely right. Like the cultural, I think when we talk about your first question about what's the new definition of customer success, I think it's the customers that have implemented the right culture, and it feeling like it is a culture of empowerment and experimentation, not, you know what I mean, not something that they have to fight really hard to get access to.

Because that's where a lot of these examples where we have customers turn around, they've built something that's ended up saving them millions of dollars, it came from the expert that was involved in the business process. It didn't come from a central team. And to get that creativity and get that ideation, you need to give people access to these tools.

" And I think so will users, so will makers, they will find a way. And to restrict these tools, to hide them, folks will go find a tool on the web that can help them be more efficient in their job. The companies that are doing this right are making it part of their culture to provide those tools and just really enable people from the get-go.

The technology is probably going to be more accurate over time if we're talking about trying to really assess images and differences between them. But one of the other things I'm really curious about is how can agentic AI and all of this technology be used in regulated industries? I'm thinking in particular financial services, banking, insurance, where there's a lot of complex process, but you also have to be mindful of all of the regulations that are attached to those industries.

Yeah. And it's actually quite surprising, I think, in this technology shift with AI, compared to when we moved to the cloud, compared to the internet, compared to a lot of the others, I think actually the regulated industries have actually been quite a lot of the front runners on this. EY, for example, built PowerPost, which helped them with their financial processing, sort of end-of-month processing.

They built this as a sort of typical low-code application. They're already looking at how they bring agentic checks into it to make sure that things are being posted in the right period, that they have the right information. Again, time-consuming sort of manual checks.

Wells Fargo have rolled out agents to more than 4,000 branches. You know what I mean, a huge, huge number. And they targeted a process that was around their branch forms and procedure management.

And this is something that was particularly time-consuming. So if you went into a branch and said, "I need to set a power of attorney," or "I need to open an account," under maybe a non-traditional circumstance, there's a huge amount of internal documentation around those procedures, the right forms, the right information to collect. Going to find the right form.

So a heavily regulated scenario, but also really impacting a customer who's literally standing in front of you waiting, maybe on their lunch break, trying to get through the bank really quickly. And so they rolled out an agent, across all their branches to actually manage that forms and procedure scenarios. And so that now in the branches, those employees are jumping straight onto an agent, talking about the scenario that the customer has and working with this agentic AI to basically get guidance on the right forms, the right procedures to follow.

Even in these regulated industries, they're seeing the value in AI, and I think it's more about how they do it, making sure they have the right checks in place, making sure they have the right guardrails rather than what they probably would've done five years ago, where they just tried to turn it off. We talked a little bit about potential friction there, but are there any other sort of potential hurdles that organizations need to be wary of? Uh, and what's sort of your take on a solution?

Like most things, we talked about human in the loop. Making sure you introduce this technology in the right way to organizations is really, really important. I mentioned EY earlier.

They were really, really successful in after building PowerPost, which helps them manage their sort of end-of-month financial processing. It simplified it. It brought in some agentic behavior to validate quality.

And they had huge gains in efficiencies in both. I think it was 70% in sort of the time, or 95% in lead time to get things posted, and about a 35% cost saving for them. So real sort of impact to the efficiencies of their users.

But what they did really well was once they built that tool, they told that story, they evangelized it. And so they helped people understand that this is how this technology was helping them, this is how it was implemented. And that not only made, obviously, people a lot more receptive to onboard and leverage the technology, but it also started driving this ideation of other things to go improve within the organization and using similar technology.

A lot of these companies are not coming in and doing a full low code approach of apps and agents and automation and reports all on day one. Where we're seeing folks be really successful is they're leveraging the composability of the platform. They're starting with, for example, they might have a legacy application that's inefficient for a user.

So they go and use an app, they build more efficient, streamlined UI over the top of that. That's an incremental solution they can deploy, they can get out to their users and start seeing benefits. Then on that same app, they can go and add automation.

They can start getting approval workflows. Then they can start bringing in agentic AI, getting that automation and that AI behavior incrementally building these solutions over time. And it's very much intentionally how we've designed the platform in that these are not all or nothing solutions.

And back to our earlier conversation, pace is extremely important these days, and people don't want to go do a 12-month waterfall project of every requirement met. They want to find ways to incrementally build. And by leveraging a platform that has common governance, these tools are designed to work together, apps with automation, with agentic behavior integrated into Copilot with that unified platform.

So essentially you're setting up a framework to enable organizations to really drive these best practices in terms of making sure that, yes, you are implementing new technology, but you're doing it in a thoughtful way where you have the right checks in place and you really are making sure there's other things that you need there. You need the audit trails. You need to make sure that when you do a project, you're going back and you're actually assessing does the technology achieve the goals that we set out to when we deployed it.

Exactly. And I think it's that there's two parts to it. One is that being proactive.

So as you're releasing a new app or a new agent to the organization, do you have the right controls around it, the right guardrails from the beginning? And again, our goal is let's have the right framework, the right tools, the right guidance to go really enable that and let an organization tailor those guardrails to sort of accommodate their level of risk, what they're comfortable with doing. But then on the flip is make sure we just have the right visibility, the right auditability, so that as you're leveraging AI more and more within the organization, it's really transparent- Mm-hmm ...

about what it's doing. I think one of my favorite things with Copilot Studio, and Pets at Home is a great example of this, as it's interacting with customers on customer service, you can go into any step through any sort of run or action the agent has performed and understand its thought process. Why did it do this particular step?

What were the inputs? What were the outputs? What were the reasoning?

Um, and not just understand it, but then also help teach it to handle sort of moments, in a different way in the future. And I think having those sort of tools from a governance perspective just built in, again, we talk about it being unified for the developer, unified for the end user, but also for the admin. So that they're doing in this sort of a central and controlled way.

And even then, whether you're building an app, an automation, an agent, you've got that composability across the platform. But I don't think admins really want a super composable admin story. They want that to be a lot more unified and controlled.

So, it's bringing the blend of those worlds of let's bring together multiple technology, multiple tools, but make sure then you sort of have one central view of how it's all coming together. If you want to really drive the use of new technology, you need to do it in a very stepwise fashion, using a platform that allows you to unify people, processes, technology. It doesn't make any sense to try to do it in a very disjointed way.

You won't have the governance required to do it safely. You'll confuse people in terms of: Which tool should I use? Which approach should I use?

Ultimately, it really does matter to make sure that you have a unified way of approaching the implementation of new technology. It's also really critical to make sure that as you go about your journey, whether it's implementing low-code processes, implementing agentic technology, to have a clear understanding of your business goals. What outcomes do you want?

How are you going to measure them? And then how are you going to take all of these different learnings and then streamline it so you can actually apply it and scale it over the enterprise, not just for today, not just for tomorrow, but well into the future. And finally, I think the most important thing that resonated with me today is you need to look for a trusted partner, trusted technology partner, to help you through this journey.

Agentic technology is very new. Low code, yes, it's been around for a while, but there are still quite a few pitfalls that can be out there. To go it on your own can be very, very challenging because you have all of that risk of potentially opening yourself up for errors, missteps, and of course, there's that, we talked about it a little bit today, regulatory concerns.

It makes a lot of sense to partner with a company that has experience working with other enterprises to deliver these types of benefits using that new technology. In between the hype and the haters are a lot of people experimenting with AI as a tool to help them in their day-to-day work and life. That's the topic of this episode of "Utilizing AI," featuring Dave Graham from MLCommons, Brian Martin from Signal 65, and myself discussing our own hands-on implementation of AI tools.

Welcome to "Utilizing AI," the podcast focused on practical applications of artificial intelligence from the Futurum Group. Every Wednesday, we explore news and use cases of the ways in which AI is transforming enterprise IT and the industries it serves. I'm your host, Stephen Foskett, President of the Tech Field Day business unit here at Futurum Group.

Before we dive into this discussion, let's meet who's on the panel today. Hi, I'm Brian Martin, AI Data Center Performance at Signal 65. You can find me on LinkedIn.

And I'm Dave Graham. I'm the Director of Marketing for MLCommons Association, and you can find me on LinkedIn. Go figure.

And as I said, I'm Stephen Foskett. It's funny, I'm not just the President of Tech Field Day business unit, I'm also an active user of this stuff. And that's why I am really excited to get the three of us together to talk about this because we have a lot of conversations about AI, and a lot of them are theoretical.

A lot of them are based on, I believe that AI could do this. I believe that OpenClaw might do that. I believe that this is a use case.

But Brian, Dave, and I are actually doing it on a daily basis. Let me just get this straight at the beginning. I am a believer in the potential of AI as a tool to make me more efficient.

AI gives me superpowers. I use it all the time, but I am a skeptic of overblown statements, and I believe that people have, in many cases, the wrong impression of what this stuff is capable of, simply because they're basing their impressions on rumor and legend and myth, as opposed to actually running the stuff. So Dave and Brian, like me, are actually running this stuff.

I don't know, raise your hand if you have installed OpenClaw. Right? Raise your hand if you used a LLM to do something today that would've taken you an hour and it took you 30 seconds, right?

Absolutely. Right? Right.

We're doing this stuff, and there's this gulf. So I'm going to throw this to Brian first. As you said, you're an engineer.

You work for Signal 65, which is a part of the Futurum Group, which does actually the testing behind much of what we're talking about here. You've had your hands on this stuff more than anything. Talk to us a little bit about your own experience with these tools.

So thanks, Stephen. One of the interesting things I've done recently, I spent the last, I would say, eight or nine days trying to create an AI version of myself as a performance engineer. I gave it a task.

We had a high-level project. I'm working on a Dell R770, doing some performance metrics. Started scaffolding up a set of experiments and was reminded very quickly of an old saying from the storage industry.

There's two kinds of people: people who've lost data and people who will. If you use these AI tools enough, you're going to find out those situations where you didn't think of something, it didn't think of something, something got accidentally deleted. In a lot of ways, it's amazingly like working with a very talented junior engineer.

They don't necessarily have the depth of context and experience that I do. But they're very earnest, they work very hard, and they're very clever. And that has been a very interesting Experiment with, I'd say, some very promising results and a few painful hiccups.

Painful hiccup. You talked about scaffolding, Brian, and I think I have more scaffolding around this joint than I have anything else at this point. So, a lot of what, similar to yourselves, I approach things from a couple different perspectives.

Obviously, I live in the marketing space, but I come from the technical side, right? So- We know you're a nerd ... yeah, it's kind of a distant second.

Technical marketing engineer, I guess you could call me at the end of the day on this stuff. But a lot of this is I have a pragmatic need for using AI for anything, right? So how do we integrate tools that enable us to communicate more effectively?

As a sufferer or a useful idiot when it comes to the side of my brain that's ADHD focused, I have to use tools to keep me on track and task, right? So I spend a lot of time figuring out what's my personal information data store, right? So it's kind of building that out, and so using tools to augment that.

Similar to yourselves, again, Brian, I've spent all weekend and most of the last week doing characterization of Andrej Karpathy's auto research stuff, right? Sitting down- Right ... and designing experiments around the experiment itself, right?

Which, as you know, can blow itself out into large amounts of trouble and turmoil. But part of this is it's necessary for what I do. And also on the research side, I have a half academic brain here as well.

As I started my PhD, one of the things I was interested in was the role of data in society and using AI. And that's on pause for the time being, but as I kind of rejoin that world again, it's sitting down and looking at the documentation and looking at these things. How does it augment?

How does it enable, Steven, to your point earlier on, how does this enable us to do more? More, better, faster, quicker. " Exactly.

Yeah. And that's the thing that I see here too, is that I was blown away right from the beginning. That's why we started utilizing AI and utilizing tech in the first place back in 2020, because I was finally able to do things faster, better, more flexibly than I could without once we got these tools.

We actually launched this podcast, by the way, before ChatGPT was introduced. And so it was more theoretical at first, but we were already starting to use LLMs and deep learning and so on in theory. Mm-hmm.

And I saw that things were rapidly improving. Once the GPTs came, it became very clear that this stuff could be extremely useful. And I use it every day, like you've said, I use it every day in 100 different ways- Mm-hmm ...

to make me process text better, process audio better. I will say that one thing I use, not just on a daily basis, but multiple times a day, is AI-based transcription of audio and video files. We produce video at "Techfield Day" at an alarming clip.

At "Futurum," we've got daily news programs, daily podcasts. Sure. I'm feeding those things into a local model that's running on my Mac Studio on my desk.

It's able to do a phenomenal job of transcribing that. I can interact with that transcription. I can find out things that I need to know in order to properly handle the content that we're creating.

And it's the same with external people. So for example, Dave, if you share a link with me on LinkedIn about, "Hey, I was on this podcast," the first thing I'm going to do is take that YouTube video and run it through- ... on MacWhisperer and Parakeet, and then I'm going to talk to you on the podcast using an LLM in order to figure out what's the interesting angle there.

Yeah. And this is transformative and cool. And it's also very cool because it's running locally.

But my goal of running locally has not been met beyond things like transcription, because frankly, I have not had any success running AI models locally. I literally, on Friday, took this out of a machine because I found that it was not a useful way to run any kind of useful tools locally, and I'm still spending money in the cloud for that. I think you have more hardware than me.

Have you managed to run anything locally? Go ahead, Dave. Brian, you go.

Yes and no, right? So I just recently upgraded to the new M5- Mm-hmm ... Max, right?

So it has 64 gigs of memory, whatever, and 40 GPUs and whatever is in this MacBook Pro. I do video, I do audio, I do the things, right, similar to what you do. And a lot of what I've built is that planned obsolescence plan.

I know in three years or four years, since that's when my M1, which is sitting behind me somewhere, that one's going the way of the dodo at this point. So yeah, I have that. I have a Mac Mini behind me.

I have an AMD workstation card here. But I've found I don't have a good small language model. I don't have an SLM that's here- Yeah ...

that's really characterized. I have data repositories and curated data sets, which is great. We all have our Claude MD files, we all have our representative curated who and what we are in the world.

But increasingly, this goes to the Google methodology behind Workspace, why Workspace was created, and all this kind of stuff. Increasingly, my life is almost entirely fixated or located ... off board.

It's not here locally as much. I have a NAS, I have all that kind of stuff, but really the things that I operate with and in tend to be around APIs that are interfacing with a, well, my current favorite right now, which is RunPod. Mm-hmm.

They're interfacing with Google services. They're interfacing with... Yes.

You know what I mean? Like that's it. It doesn't- Right ...

end up sitting here. So my need for discrete hardware that's highly optimized, tweaked, and tuned, and like that MSI Armor card I think that you were just showing, I don't need that so much anymore. Right.

And I'm also on a Mac, so I couldn't use it anyway if I tried. But, that tends to be the limiting factor. So a lot of what I'm interested in is actually human interface to the technology that I'm doing, much less than I am interested in the hardware interface to my humanity or my digital persona.

Brian, over to you on that one. Yeah. So I've got an AMD Threadripper system with an RTX Pro 6000, which lets me run like GPT's OSS 120B locally.

I've been pretty happy with that. 5 models locally. I'm in this constant quest for something that feels close enough to the frontier models that I instinctively trust it or I subconsciously trust it.

And I've come to accept that there are going to be those scenarios where things go sideways, and I just have to account for that. But as I get deeper into OpenClaw and its many derivatives, I really want to have a model running locally that is going to be in charge of things that I care about. Like as I start to share- Mm-hmm ...

the things that are my personal accounts, my personal documents, my email accounts, those are scary things to put out in the cloud right now. I don't want those models running outside of here yet. But someday, maybe.

But I also feel that the current crop is, well, on one hand, the current crop is very promising, but the frontier models keep moving forward. 6- Yeah ... which is unfair in one respect, but it is what's available.

Yeah. And then, as Dave said, spending thousands of dollars learning and finding out is part of what we do. It's the nature of the work.

But also finding ways to be more balanced, like pushing these things to their limits. I'm advising an aircraft safety startup, 550 Aero, and the guy there has found a way to bring these agents together. He's building a data science team purely from AI.

And he's been able to automate, and accelerate studies across thousands and thousands of experiments, digital twins flying in virtual reality, to test his system, all driven by AI. So the expertise, I think you mentioned, Steven, the what we can do to augment ourselves, accelerate ourselves, is absolutely showing up in a powerful way. And as long as I'm aware and we're aware that it does have edges, it's not perfect, we keep pushing forward.

Yeah. And I think that's one of the things that separates people from people who are more experimental from either the haters or the lovers. And that's that it's not a question of hating, it's not a question of loving, it's a question of seeing this as a tool that could give us a special capability.

And so, for me, that's really kind of been a quest for me. And like you both, I think, I've been trying out different models based on the hardware that I have. The reason that I chucked this thing is, at first I was under the impression that a GPU could assist in the processing.

Ultimately, it didn't. Ultimately, my Ryzen 5 12-core CPU is actually faster than trying to make use of an older GPU. On the flip side, like you, Dave, I also have a Mac Mini.

I've got M4 Pro Mac Mini, unified memory. That has allowed me to run more models locally. I'm very excited with what you can do with the unified memory.

I'm excited with what Apple has been able to do with rolling out NPUs. I don't yet have an M5, but I'm actually kind of excited about it because I believe that it actually has four times as many NPU processors as the M4, so it ought to be noticeably faster. But I've been trying to run various models.

Again, with OpenClaw, I've been trying to run the Qwen models, the Llama models, because I feel like they're a pretty good system. But unfortunately, I haven't been able to make it run effectively, and I'm back to using online cloud-hosted APIs of based models for basically everything, simply because it just doesn't generate the tokens fast enough, and it doesn't have the context window to handle any kind of data, really, in order to make any kind of effective use of local resources. Now, that being said, I am excited about what we might be able to do once we get past 64, 128 gigs.

Basically, so my implementation of OpenClaw is using 10, 15 gigs just for the key value store, which means that my system doesn't have all that much power. And so I've been thinking about, well, maybe I could have an AI offload, another machine that's actually running the model. Mm-hmm.

But that hasn't been something that I want to do because it just means buying more hardware and deploying more hardware. Have you had any more success than that, Brian? I was going to jump in and say yeah, so part of the luxury of the job I have is we have testing labs, and when those systems are idle, I can deploy on those.

So I occasionally have the luxury of running on an 8 by H200 or an 8 by MI300. Well, I've got one of those too, I just haven't started using it. Oh, yeah.

Well, watch out for the power drain. They're loud and expensive to run. Just waiting for National Grid to drop in the three-phase for me.

So, I mean, hey. Perfect. We're just bringing online a liquid cooling lab, which is a conversation for another time.

But having access to those for comparison and to play with. 5, these are trillion-parameter models. Some of them take 16 high-end GPUs to run.

Now we're getting what feels like frontier model quality in the, well, data center, not quite in the office. But it's local. I can run it there.

We can batch it and get incredible response time with incredible accuracy. So it's fun to watch that sort of, what do you call that, burst out. Trying to...

Yeah. I think there's the things, so going back to the experimentation stuff that I've been doing. Now I'm looking at eight different data sets, and there was the original client mix one from Andre on Hugging Face, and started adding in FindWeb and a couple of these bigger semantic data sets that contain the world plus dog in terms of information, but all in structured formats and stuff like that.

And so one thing I experiment on this stuff, and it's one-- And I'm watching my RAM get progressively eaten up. I have an instance on an MI300X right now running in RunPod that's out of the 192 gigs on there, I think is chewing up 128 gigs of memory on that thing, which is ridiculous. Right.

But, this is a corpus of data. This is a lived experience of billions of people, when it comes down to it. It's a summation of human thought and thinking across a, whatever, a wide range of texts and whatnot.

And so, Stephen, you mentioned this marching onward, and we can talk about flash economics right now, though I'm sure Ryan Charette would want to be a part of that conversation when it comes down to it. Yeah. Though, when we start to look at this, there's this kind of weird inflection point of economy, right?

I bought my M5 because I knew RAM pricing was going up. I got my wife an AI395 from Corsair, right? One of the little mini PCs with 120 gigs of memory right before the memory bubble hit.

It was around Christmas time. And the reason being, it was I saw this kind of trend in here. And I wonder, and part of my wife would actually be a great person to have on this as well, because she's looking at AI usage and digital health, and looking at these kind of phenomenology of what happens when you start to inject these things into daily living.

And a lot of what we're talking about, Bryan, you're testing from an infrastructure standpoint, the art of the possible. I'm looking at a pragmatic approach to it for marketing, but also in what we can do with it in order to make our lives. Stephen, you're using it to live your life and make your business more opportunistic.

And I think this is one of the really incredible things that we're being offered right now, whether it be on-prem or on-premise. You're going to go after me if I say it wrong. Whether it be on the cloud or here locally, right?

This principle is we're watching that kind of march from it has to be on a hyperscaler. It has to be in a neo cloud. It has to be there to something where am I able to do enough locally to solve, like Bryan, you want something that's going to run your life.

Stephen, you want something that's going to run your transcription service. Am I able to do enough here? Like just enough LLM, right?

Like getting into that space. And I find that fascinating because this is where we start to see when we need to do new house builds or new house starts, right? We start to see our infrastructure surround this type of concept.

What happens when you have a smart home that integrates small language models that are able to do the proof of concept stuff that we talk about here theoretically, and start to pragmatically put this in here. That's the thing that excites me at the end of the day, and what I'm hoping my testing experimentation leads to more understanding and knowledge. But, yeah.

Yeah, and for that note, that's been a really interesting thing for me is that the more I've tried to run things locally, the more I've realized that it's much more practical just to run them in the cloud. And so with my workflows, for example, I have tried to do local language models to do even simple tasks like summarization. And I've found that it's actually not just easier, but more cost-effective even to just run them using, for example, Gemini on Google.

Because they have a really well-supported API, and I'm able to just hit that. And it costs not even pennies to do a simple task like summarization on using Google, OpenAI, Anthropic, that sort of thing. I've also been looking at, I have not yet got it up and running, but for example, Cloudflare has Workers AI.

5. 5 running in Cloudflare on Workers, and I've been thinking now that's a really cool idea because, yes, it's still pay as you go. Yes, it's still...

But it's going to be sips, it's going to be inexpensive, and it's going to be highly scalable and reliable because I've been using Cloudflare Workers for many things in the past, but not so much for AI models. Mm. What about that-- It seems like there's sort of a dichotomy in people's minds that either you're running it locally or you're running it on specifically OpenAI or Anthropic.

What about that space in the middle of service providers that are able to provide models that are a little different? So Steven, you get to bring back the hybrid cloud phrase because- Yeah ... this is going to be the new hybrid in AI.

Yeah. What do I run locally? What do I run in a Cloudflare or a RunPod?

What do I run on the frontier space? And I think part of where we're going to see tools like OpenClaw and the derivatives step up is helping to make those decisions, right? We're going to have a local router to figure out what goes where, how to optimize, and it's going to be an ongoing process.

I can almost envision, a set of virtual sliders. Like, okay, try this, try that. Okay, I think we're settling in about here.

And then of course, new models drop and everything gets, the perturbations happen, and then we settle out into a new norm, and then more new models drop, and maybe it shifts. Yeah. That's exactly what we've been doing, right?

So one of the things we talked about at GTC this past week, it was only last week, was this idea of testing endpoints now all of a sudden, right? So the old model, again, small plug, but for what MLCommons does, but, a lot of what we were attempting to do is benchmark and characterize infrastructure. So AI infrastructure, from training to inference, to storage, to whatever.

And that's all well and good, and we operated in kind of fixed cycles about every six months, alternating between all these major themes. One of the things we kind of determined along the way, and credit where credit's due, some of our competitors or other analysts within the space were hinting at this and putting out data was that because that wheel turns so quickly now, because of the nature of everything, everything is becoming an endpoint. So again, that hybridization, it's not just local, it's not just a Neo cloud, but it's also that API that sits in the middle of it that you're wanting to test so that you get- Mm-hmm ...

5 when it launches, or you get what Cloudflare is offering or whatever. So it's this idea that we have to now start testing. It's probably more advantageous for us or one of the more forward-looking things, start testing those endpoints themselves, right?

So you get greater model velocity, you get greater engagement with what people will ultimately end up using, right? Because not everybody's going to want to sit down and provision or RunPod thing. I'm using Jupyter Notebooks, which is enabling my deployments to be a lot quicker, but I'm not using any of their endpoint API stuff at this point, right?

And so it's that kind of constant re-envisioning and recycling of, there's both, and then there's the and. Let's use them both together in order to accomplish something. So a hybridization model of OpenClaw plus open router to determine where data is going to be shunted or the calls that need to be made, or Brian, the stuff that you're testing in your lab, if it's running on your RTX 6000 Pro, but then you need to augment it with an 8 by V300, which you probably don't have at this point.

I can bounce out to Scaleway or CoreWeave or some of these other things and be able to hybridize this thing together. The end cost is probably less than you trying to consume the services directly yourselves in that case as well, which is an important concept. We're privileged because we have the income business opportunity in order to do this, but when we start to look at the ubiquity of using these things, the hybridization model probably makes the most sense for that, putting the reference to Altman, but that universal basic compute concept that Altman tossed out there as a cheeky aside to UBI, which is inherently more useful by the way, don't do UBC.

But that kind of concept, so. Right. Computers eat tokens.

People don't eat tokens. Yeah. And they certainly do eat tokens.

If Jensen, like Jensen said, if he has a 500K staffer that's not spending 250K of tokens per year, they're not doing their job. I mean, Jensen, I urge you to give me those $250,000 worth of tokens, and I will absolutely exploit that to the ends of the earth. Well, yeah, that's actually a real good point that you make because even though this is still pretty cutting edge, and even though some of these models are still moderately expensive, I am hard pressed to spend that much money on tokens in the cloud, and that's actually been one thing that's holding me back from local models, in that, I look at it and, my budget is not many thousands of dollars a month.

In fact, I would be surprised, I don't actually know because I need to look at that. But I would be shocked if I'm spending more than a few hundred dollars a month on tokens, despite the fact that I am aggressively trying to deploy these tools, and that's made me kind of question whether I need to go out and run out and buy an M5 Max with 128 gigs of RAM or whatever it is, to run this stuff locally, because frankly, it's still pretty cheap to run it in the cloud, even though it does eat up tokens. Yeah.

I mean, I had heartburn going from the Claude $20 a month plan to the $100 a month max plan. Mm-hmm. However, I looked at my usage patterns.

I said, "You know what, by the way, by the time I figure all this out, if I wanted to use this stuff," I mean, Cursor was a great example. I used Cursor a lot last year to the point where I blew $500 in a month on Cursor, and I went, "I'm not a developer. I should not be spending-" But you're not blowing $5,000 a month.

No, I'm not. No. Right.

Thank God. My wife and other people in my life would absolutely pillory me for that one, and for good reason. But it was one of those situations where the tipping point, to your point, Steven, was what am I using this for?

Is there a comfortable kind of offset where $100 a month actually makes a lot more sense, where it's not all you can eat, but it's enough that I can eat, in terms of the things I do, and I've looked at my usage patterns over the past month on Claude and Claude Desktop and the stuff that I'm doing, and I'm hitting 25% utilization of my credits per day or my usability window, and that's goodAnd I'm seeing the outputs of that, and I'm able to work within that particular space. So I have no need to do the $200 a month plan or any of these type of things. I think the flip side of that, and not to be negative about it, but the flip side of that is I've turned into a SaaS monkey when it comes to actually- Oh, yeah ...

spend. com and all that kind of fun stuff. There still is that argument to be made, and I think there still is, we're talking about it loosely here, there is that rollover where, at some point it becomes more tenable to run in cloud or more tenable to run locally depending on what you do.

If I had Brian's hardware, I'd be running locally all the time. So. Not that I'm jealous or anything.

But the problem is that, Brian, your hardware is not your hardware. It's the lab, and so you can't use it all the time, right? Exactly correct, Steven.

This is the challenge, right? I really wanted to like, okay, I want the quality of a frontier model. I would trust my version of Kimi2 running my stuff locally.

It's not locally, I can't count on it, so I need to find something else in that space. Dave, you just need to be running more in parallel. One of the things that starts to uplift, I watched my cloud, I spent over $500 one month on cloud last year, and that's when I went to the $200 max plan for all you can eat when I was driving it hard.

But as we start spinning up multiple agents in parallel, and I think Jensen's really looking at, in the data science space. If you look at what Andre was saying, he's running these experiments, he's running them in parallel, he's running hundreds of them, thousands of them. The ability to go wide then starts to tax me is how much can I keep track of?

Where do I start losing operational control of what's going on? And then the next step is how do I learn and embrace delegation for that? And that comes incrementally with trust.

So that's the next frontier for me is delegation, distribution, and starting to build an environment there that is reliable. Yep. And that's the real challenge, I think, is that unfortunately, too, I'm looking at what I'm doing and some of it looks, if you'll forgive the phrase, pretty janky.

I've got some frameworks that are not exactly bulletproof. And you're talking to somebody who was very deeply into business process automation and so on before the AI boom hit. I'm a Zapier power user.

I like to say I have features in Zapier named after me because I pushed their product so far to the limit that they came back to me and said, "Man, okay, we'll develop something that will meet your need," back five years ago. Now here we are in the future, and, a lot of that stuff is still running pretty good, and my AI stuff is still just duct tape and bailing wire. Because I just haven't gotten to the point of maturity.

Dave, take us home here a little bit. How effectively are you using AI right now, and when do you suppose you're going to be able to say, "Yeah, I've got some good AI-powered processes helping me develop superpowers"? Yeah.

As the cautious optimist or the pessimistic optimist, I'm not a full Luddite. Again, I come out of a background being social work and background being psychology and background being therapy. I came out of a very humanities-focused kind of space, right?

That was my ontology for decades. Still I'm in that space, right, but now, kind of leaning in it. I think, I've given over some of the easy stuff, I will call it.

" Things that I would devote a lot of brain power to. I'd still love, just like you like watches, I like pens, right? I still have my pen and paper- ...

that I jot notes down. And it's good because it's an analog kind of... There's a tactile feel to doing this stuff.

But when I look at it, the summation of what I do, a lot of what I'm spending time on is, are these digital assets where I'm building things out, like market requirement stocks or looking at the kind of grants, how I communicate this stuff. Those are, I won't say they're easy, but they're easy enough that you can frame it, you can hit that 90% good enough, I can evaluate, and then I can pass on. I think, Brian, you mentioned trust.

This is where I'm at. Yeah. I'm still building that basis of trust.

Trust but verify, right? It's that kind of basis. When I flip into my academic side, the paper that I'm writing with Patrick Hughes right now, the idea is everything that gets reported back, I'm having to double-check the data.

Because why? Because I don't inherently trust what is going on and what's being reported back. I need to see, and I need to understand the basis for that.

As my understanding grows, as my model usage grows, as my understanding of those characterizations grow, I think that's where it leads me to integrate more and more of it into my life. I drive a car that has AI in it, which I disable a lot of those features at this point, but, again, I characterize, I try to understand. That is my job, my day job with ML Commons is that idea of characterization.

It's understanding what it is and what we do, right? And so I think that's the baseline. And then joining stuff with you all here at Tech Field Day stuff, I'll be doing the AI Field Day in May and participating in that kind of stuff.

And, again-You can pay attention. I try to make everything open and audible for what I do because, again, because I'm trying to engender that trust that I expect from the models back to me, and then from me to the greater community out there as well. Got it.

Great. Yeah, for me, just trying to keep pushing the limits forward as I can look at what runs locally, what runs balanced, how far we can push this system. A lot of work lately with Dell and Dell Infrastructure.

In that vein, you can see me at Dell Tech World, or find me on LinkedIn, and let's keep pushing the limits of AI. Yeah, and thanks for that. Dave, Brian.

Dave, I'm looking forward to seeing you at AI Field Day. Brian, we just saw you at AI Infrastructure Field Day. Thank you for hopping out for that.

com, you'll be able to see these. This podcast lives as well on the Techstrong side. ai, you can see the home of the podcast, and you'll be able to see a lot more of what we're talking about and doing here in terms of video.

But thank you both for joining us so much. This has been a lot of fun conversation. I wish we could talk, well, we probably could talk for hours on these subjects.

Let me know if you find anything that really, really works. And to the listeners as well, I hope that you hear what we're saying, which is not that these tools are just magical, not that these tools are evil, just that these are tools. And we're figuring out how to use them.

We're figuring out how to integrate them with our workflows. Mm-hmm. And I think a lot of our listeners are in that same situation.

They see the power of agentic AI, they see the power of generative AI, and they see that these are things that can help them do their jobs. And now the question is more, when and where and how should I use it, and how can I make this most effective? And for me, that's the whole ballgame.

" This podcast is available on YouTube, which is the best place to subscribe, or you can find us on your favorite podcast application. And of course, we would love it if you would give us a comment, a rating, a review. This podcast is brought to you by the analysts and experts from the Futurum Group, where insights meet AI.

ai, the "Utilizing AI" YouTube channel, or Techstrong TV app on your streaming device. Thanks for listening, and we'll catch you next week.