Techstrong TV – March 3, 2025

Hey, everyone. Happy Monday. You know what?

We're just throwing more money on that AI bonfire. You're watching Textron Gang. Hey everyone.

Happy Monday. It's Alan Shimel here for Techstrong and Techstrong gang. Welcome to another week of as AI Turns, or as AI burns as in money.

But we'll, we'll get more into that in a second. Let me, let me introduce you to our fantastic gang. We've got assembled today to talk about, as usual three different topics that we'll be heading on today.

Um, first of all, down Texas Way from Austin. He's the, uh, visible impact, uh, uh, CTO co-founder as well as future and analyst. It's our friend, but he's originally from the Bronx.

Our friend Guy Currier. Hey guy. How Guy Currier.

Excuse me, I'm sorry, guy. Uh, uh, doing great. Really happy to, uh, to be here again.

It's actually been a minute. Um, and, uh, the weather remains completely unpredictable here. I'll just say that much.

I miss the Bronx. Yeah. I almost made you sort of part of the Curry basketball family there.

Del Curry, Stephen Curry, I forgot Stephen Curry's younger brother. I would be honored to be in their company. I, I can't touch, I don't know if you quite measure.

I've never seen you shoot three, so I'm not going to, You never will. Yeah, not gonna go there. Uh, moving over from Texas to New Mexico, though she is our open source champion and Linux Foundation participant, as well as the CEO founder of Deploy Hub and many other things.

Our friend j Reagan. Hi, jc. How are you?

I'm doing great, Alan. Thanks for having me as always. I always love being on these calls.

Lots of fun. It's okay. ai and Gestalt it text strong.

It SM we've got some news coming with that one. Uh, sag Soha. Hi, Sagner, how are you?

Hi, Alan. I'm good, thank you. How are You?

Good Sag. Is it Soha or Saha? I always mess it up.

It's Saha Saha with the a I'll try to remember. That's Saha. All right, good.

I'm Gu Sagner. It's great to have you here. And we'll go from there.

Up up to, uh, Harrison, New York. You don't know where Harrison New York is? It's upstate.

Well, for people like me from Long Island, Harrison's upstate, other people may say, ah, it's just Westchester, but he's our chief content officer, Mike Vizard. Hey, Mike, how are you? I'm well.

10 miles from White Plains is upstate. Huh? Okay.

How old? I was wondering that too. Alan, You're also 10 miles from the Bronx.

Oh, no. Hey, look, I, I was on the south shore of Long Island. That's a far ride to Harrison.

Um, anyway, On Rochester right now, throwing stones. That's all I gonna say. Let 'em, all five of them.

They're freezing there right now. Um, anyway, it's good to have you, Mike. So, I I, I teased it in the, in the opening.

You know, I, I also, I discussed this on my Shimmy says LinkedIn Live last Thursday, and it's on our Techstrong, uh, TV show from last Friday. The replay of that LinkedIn Live, you could probably get it on Techstrong TV or on YouTube as well. You know, more money being thrown at the AI and AI data center investment bubble irrational exuberance.

If I ever saw it, uh, in my, in my, uh, presentation, I even used a little flip chart and I actually hand wrote down some numbers. It's pretty good. 2 trillion.

I did include this 200 billion, that meta rumor has meta may be looking at over the 65 billion I think they already committed to. And, um, you know, of course meta at this point is still denying this 200 billion. But Mike, what do, what do you got here?

To your point, there's just a ton of money pouring into this space. And the issue though is Microsoft is kind of like making some signals here that people are having a hard time trying to figure out and read. They're saying that some of their models are more efficient.

There's conversations about them pulling back on their leases. Now, whether they over-provisioned in the first place is hard to know, but, um, they might be signaling the folks that maybe we are not gonna need as much hardware as we go forward here with ai. And, um, tip of the hat to both, uh, Alan and, uh, Mitch, who's not on the show, but you guys have been banging that drum for a while saying that, you know, we'll get more efficient, but Guy, what do you think is going on here?

Is this just one big giant pumble about the Burst? Because NVIDIA's stock took some wild rides just on a numbers that said there was, you know, awesome, and then everybody punished them. Well, I don't know if I'd put it quite as dramatically as either Alan with his irrationality remark or you with big giant bubble would, but, um, I, I think I've been saying on this show for a while, um, along with, uh, Dave, Nick and our colleague Dave Nicholson, that, um, that, uh, there is gonna be a reckoning at some point.

Uh, my feeling has been that capacity, uh, power capacity specifically and maybe cooling a little bit capacity, um, will be that reckoning. Um, and it's, it's, I feel like that this is kind of how it starts. You start to see these big wild plans and then counter currents like, like, like what happened with Nvidia.

Nvidia beats its number, but drops in, uh, in, you know, in, in like, was it eight, 9% in, in, uh, market cap? Uh, there are just so many things going on. It's a little bit hard to, um, figure all of it.

Uh, I, I do think that, um, the exuberance around this is not necessarily irrational. The meta story in particular, it makes me think, Hey, wait a minute. Uh, maybe meta facing, you know, from its cash cow, Facebook facing sort of a, a a a relatively stable and starting to decline user population, um, is chasing after, you know, AI and AI services llama and whatever else they produce as, as a new, you know, more promising long-term revenue stream.

But I wanna point out one thing, the macroeconomic environment here, um, this did not occur to me until just like, as I was, as I was preparing for, for this show. We have to remember, or it's helpful to remember that all these tech companies have been sitting on wads and wads of cash for over 10 years now. They've been building up a cash with nowhere to invest it.

I mean, that's why they do it, right? Didn't Apple had a trillion dollars or something like that at some point? Or, I, I don't remember how much it was.

That's probably more than needed. So, so when AI comes along and generative AI and this gold rush comes along, cash is cheap for a lot of you. They don't need to borrow.

They just, they, they can just spend. And much like on a, an earlier edition of this show, I was saying, you know, this is like the, the, the proverbial, you know, decision that, uh, you know, is gonna always be approved by upper management. Let's invest in ai.

You're gonna get that budget to spend. Um, in the same way I think in these tech companies, um, real smart product folks coming in and saying, here's the 10 things we can do with ai. We just need more capacity.

They're gonna be told. Yes. So in that way, I guess it is a little bit irrational.

I think there's been a lot of money slashing around in these tech companies, and now all of a sudden there's a place they get invested that, you know, everybody's gonna like until the other shoe drops, which is kind of inevitable. I, I'd like to comment, with all due respect to my colleague, Guy Currier, When you, when you are talking about two plus, it's two plus trillion with a T of dollars pledged, what, what kind of telethon are we running here? Right?

I mean, let, let, let's be, let's be realistic. Let's look at the GDP of the top. And, and this is all on my shimmy says you could go check it out.

Let's look at the top 20 companies, or the top 20 nation states. GDP for 2024. The us I I, I think the US was 30 something or 38 or 39 trillion.

China was 28 or 29 trillion. And then there's a pretty big drop off. 5.

Japan is about 4 trillion, uh, UK's about three and a half trillion. 1 trillion. And then it drops off again, right?

So you go through, let's say, you know, those six countries, India, maybe at number seven around there. And then there's a huge fall off down to Italy and Mexico and, you know, some of the, uh, Canada, you know, still highly, highly industrialized first world, you know, big economies. You are talking about more money there than the GDP of some of these major industrialized nations.

Where does the money get spent for every, you know, a billion here, a billion there? Before you know it, you're talking real money. But where does every out of every, excuse me, out of every billion that gets spent or supposedly going to get spent on this AI and data center, boom, about half of it, they estimate half of it goes into semis, into chips.

I don't, I mean, that's a damn good business for Jensen, Wong, and Nvidia, right? If, if 2 trillion is gonna be spent and half of that goes to chips, holy merrow, that's an awful lot of GPU right there, right? A trillion dollars in GPUs.

com days, right? I used to be out, well, worked for a company not far from you, Mike, and purchase New York. And we had data centers all over the world, and we, we partnered with some really upstanding companies, WorldCom, MCI, if it rings a bell, um, the good folks at Enron, the big e down in Houston, we know what happened to them.

Level three in, in, uh, oh, right outta Boulder, Colorado. Interlochen, Interlochen, remember that guy? Level three was there, there was a whole bunch of fiber and all, and it was a very similar thing.

We're going to need more fiber. We've gotta do the last mile, but we need more backbone. We need more backbone.

We were using T one T threes at the time. Right? You, you, you didn't mention Akamai, that that was their Yeah, well, Akamai was creating A-C-D-N-I, I tell you a funny story about Akamai, my teen investment sense, I picked ink to me.

But anyway, um, I knew a founder of Ink to me. Anyway, go ahead. com time that basically that dark fiber as we called it.

'cause we never lit it up because we over, over capacitated, over capacitated our fiber needs. That dark fiber stayed dark. com bubble burst.

I don't think we lit a lot of that up until two, until 2005, 2006. And then even after the great recession, there was still dark fiber that now we're lighting up. Are we gonna do a similar thing here?

Are we just gonna build so much excess capacity that it's gonna take 10 years for that to be soaked up in, into the, into the, into the marketplace? And what's that gonna do to Nvidia stock? What's that going to do to all of this?

I I do think it's gonna spur electric, uh, innovation in, in generating electricity, right? Hopefully clean, sustainable. But it's gonna have to, it's gonna have to, yeah.

No, not, not in this present administration drill, baby, Jill. Okay, let's just wait, wait. I just wanna get back to the, this their stock.

You know, it is chaos because when you have an earnings report that says that you are, you know, earning $40 billion a year at a growth rate of 78% year over year, and your stocks go down, we have, we have a different thing happening in the market. I believe that there is a lot of volatility right now because of the current administration and just in people are feeling insecure. And maybe some of those are small investors.

I would love to know who is, you know, selling off their NVIDIA stock when they have a, an earnings report that looks like that. They, to me, it's insanity. They buy an anticipation of a blowout earnings Well, and then try to sell it and, and make quick money.

Or people who, short, short, there are people who are shorting stocks. That's what they do for them, right? It's short, right?

They're shorting. Yes. Haven't we always seen that?

But still, when you have, if you, to me, that's just insanity right now. It really is. It's insanity that a company could be doing that well, and, and, and an and investors are gonna start dumping the stocks.

Tracy, You see this right here? You know what that is? Yes.

I get, I do. That's The violin playing harsh flowers for Jensen and all his cronies at Nvidia. And you know, The other thing we haven't really talked about is the CHIPS Act, right?

Is this, is, is, you know, the potential of pulling some of that funding away from the CHIPS Act, is that having an impact? The, The, the problem is the CHIPS act, Well, they replaced it with small potatoes. Starburst, Right?

Small to this. They Yeah, but they replaced it with Starburst also. That's a half Stargate.

Stargate Stargate. I, I really don't care. No, no, they didn't.

So here's the deal. The CHIPS Act is US government money that they're giving the private industry to, to build chips and data centers here. Stargate is private money that's being invested into AI and data centers.

We have a similar thing going on, by the way, in China. Again, this is all on my shimmy says thing. The Chinese government has, I forget how many Juan, a trillion Juan or whatever it is, comes out to about $138 billion that the Chinese government is earmarking for AI and AI data centers for the big Chinese companies and smaller Chinese companies do, to be fair.

But, you know, the 10 cents, the, the, the Baidu, the Alibabas on top of that, some of these Chinese companies, Alibaba, for instance, announced, uh, I think it was 35 billion of their money. So you are seeing both governments and private companies come in here over the top on this. I mean, I look, I don't think a fraction of that will be spent in actuality when push comes to shove.

And I think we're gonna wind up, I think it is irrational exuberance guy with all due respect. And, and the airs are gonna come out of this balloon, especially if we can't point to real wins with AI. Real quick, I if we're gonna really get into it, I, I do need to point out that, that, um, that those announcements are not really commitments.

Their announcements or their Pledges, you know, sort of their pledges ideas are good thing. It's like writing a telethon. Yeah, no, I get It.

Saying because meta denies, uh, any $200 billion. No, but they do acknowledge the 65 billion. Yeah.

Okay. But also Alan, uh, comparing GDP, which is a yearly annual figure spending over the course of a year with pledges that are gonna, gonna take a, a couple years, at least probably three years to play out, Even if you took three years guy. I mean, this is, come on.

It's, it's, it's crazy Money. But it's funny because the exact example of building out the, the, the, the fiber backbone because of the internet is what that instantly came to mind for me as well. And I, I, I think we're just really arguing about degree, and I'm picking on you for word choice, which is not even fair.

Um, overinvestment seems so likely here. Uh, I if not, if if only for the macroeconomic, uh, I, it's probably not even ma the economic condition I talked about before, which is these are companies with tons and tons of cash that until this AI boom were, they were mostly holding under 'cause they didn't know what to do with it. Now they have something to do with it that looks promising, and yeah, they're probably gonna overdo it.

That's, you know, that's, that's the boom bus cycle. I agree. Um, Yeah.

So there's this also f the funding frenzy that's been going on, like with OpenAI and tropic HuggingFace, they have all reported, uh, substantial fundraising, uh, rounds and even, uh, for AI startups. There is this cold rush of funding offers coming in from venture capital firms. And this started back in, uh, 2022 when OpenAI heated up the AI conversation with the Chad Ity.

And, uh, now they're plowing all this money into this up and coming, uh, AI companies, um, no hold spot really. And that shooting up their AI valuations like overnight, like OpenAI and tropic are, have been, uh, beneficiaries of this too. Like open Eyes valuation probably reached over $150 billion this year.

And that's a lot more than a lot of companies in, uh, that have been around a lot longer. And, uh, anthropic two, uh, hit I think $20 billion. So, uh, that there may be definitely some overspending over here going on because, um, it's really like the, it's, it's really like the AI rush sort of, uh, thing that's going on.

Companies are really, uh, you know, enticed by this new, uh, shiny new toy sort of a thing. And, uh, this ties nicely into, uh, this, uh, future research finding, which, uh, was released a while back, which state that, you know, ai, um, readiness of companies is really lagging behind. And, uh, they're not really, um, at par with the expected ROI returns that, uh, that companies thought there would be.

And, uh, a lot of things can be, uh, there, there are a lot of factors really that, uh, go into explaining why that is. There are cultural conflicts, technological lags, and the general lack of readiness of companies. So, and a lot of companies are also using IS corner cases, and there is also this AI washing that's going on.

So, uh, there is definitely, I would say that, uh, there is some overspending going on, but also it'll take a while before things start to fall in place. And, uh, companies start to see their, uh, expected returns coming in. Ney, you're not implying that companies are using AI as rapidly as possible without really knowing how it works, are you?

Yeah, that can't be, This is Outta my cousin video. Absolutely not, your Honor. There's Lot of waste out there.

Yeah. This is the report from the Futurum group, right? That you're referencing, I think, right?

Mm-hmm. Yeah. Yeah.

My colleague, Olivia Blanchard led that effort. He's, he's our AI specialist. Mm-hmm.

So, I mean, what you're seeing is the trickle down, right? If you get 2 trillion here, it trickles down. So it's, it's sucking up all of the, soaking up all of the VC funding that's out there, because everyone wants to ride that.

Quite frankly, it sucks up our attention here on Textron Gang, two out of every three segments. So it's somehow AI related, right? It, this, this is rippling throughout at certainly the tech economy, if not the world economy in general.

You know what I find interesting? One last thing, Mike, when I went through the GDP numbers and my shimmy says, every one of these companies is all in on ai, right? They've all made announcements except Russia.

You don't hear anything from Russia on ai. Now, are they just not saying anything 'cause of sanctions? They're not allowed to, are they just being really secretive?

Or are they too busy fighting wars? Or, or why buy what, what you can steal, right? So who knows?

Well, they're all stealing. Look, there's no angels here. Every, this is, we, we have cast safety, morals, ethics to the side.

It's a full out race to, to be the king of ai, right? And they're all, we're all stealing from each other. There's no doubt in my mind.

All right, well, we have mentioned this in the past, but it's worth mentioning again, we do not give stock tips on this show. But I guess we could remind folks that, you know, Joe Kennedy got outta the market before the great crash when his shoe shine, boy gave him some stock tips. And then you figured out, well, maybe everybody's in this a little bit too hard, so do what you like Folks, but, and he did go into the bootlegging business.

But that's a whole nother story. Um, yeah, let's take a break here on text on Gang Joe Kennedy. We'll come back with more.

You're watching Text Drug Gang, Discover Textron Group, the epicenter of tech innovation. We are your go-to for reaching IT, leaders and practitioners worldwide. Our secret impactful content that sparks awareness, engagement, and top quality leads with us.

You'll access editorial websites, streaming videos, virtual events, custom content analyst research, and more. Join our satisfied clients. Let's revolutionize your tech journey.

Contact us today and tell your story to the world in the most powerful way with Textron Group. 2, outta three, uh, segments. This is the second segment.

So we're talking about AI and its impact on software development. 'cause there's just a lot of conflicting reports out there. com, there's two stories talking about how, uh, AI is not really gonna do every software engineering task all that well, and that the quality of the code is somewhat suspect at the same time.

There are reports out there that says, uh, I think it's from the Fed in St. Louis. That demand for developers, is that an all time, uh, pre covid low?

So we're not quite clear what that means either. And finally, though, we see people like Mark Benioff at Salesforce, you know, they have a bad quarter and they're telling Wall Street, they're gonna get rid of their software engineers to make it seem like they're gonna be more efficient and maybe buck up their, uh, stock price. So Tracy, what's your read on what's going on here?

So what, you know, e every time we go through a kind of a shift in how we develop applications, we kind of see this, it feels like, uh, the job that, you know, the jobs in some sectors or jobs in some skills will, will start disappearing, and jobs in other areas will start, uh, becoming in demand. And I kind of feel like that that's what I mean. We have to remember that the technology sector is still growing.

Um, according to the Bureau of Labor Statistics, it's like a 10% increase since last year. So it's to say that, you know, one article about potentially jobs on indeed, uh, being less, like 30% less this month and maybe a couple of months ago, I don't think is a good reflection of the real job growth in the tech sector. There's a, there's quite a few jobs that are, uh, openings around, you know, AI people who, who understand large language models.

People are just certified in it. Uh, security is a big to topic. So I think we're seeing a shift, and I think some software developers who may have been doing, you know, JavaScript all this time might be not as much in demand.

Um, but I don't think that we can really read into, uh, we can't read too much into what Indeed is seeing. And there is a shift away from, uh, a lot of, uh, developers are moving away from thi uh, platforms. Like, uh, indeed, to be quite honest.

There's other, you know, like Stack Overflow. A lot of people I know, they go to Stack Overflow, they go to, uh, GitHub discussions because platforms like Indeed are starting to become a little bit annoying, to be quite honest, in the way that it used. They use AI to find, uh, candidates.

You have to have this perfect match. So I think developers are starting to move away from some of these sites, uh, and go to some of the sites that other developers are at. They're actually looking that they know that they have openings on their teams.

So maybe there's a little bit of a shift in the way developers are looking for work as well. I just don't see a, a massive decrease in the number of software developer jobs. I really don't.

Alan, how much of this has to do with something called the economy? Stupid? I mean, yeah, no, I obvious.

Well, look, in the last two months or so, we, we have seen economic numbers trending down, unfortunately, both on unemployment and a bunch of other, you know, kinda leading indicators. But, you know, even like Trace, you mentioned Stack Overflow, that's where developers used to go to, to, to code and get tips in coding and, and to commiserate with other developers and figure out how to do things. Guess what?

AI has hit Stack Overflow hard. Their numbers are down. I the last, I think we discussed this on a gang a couple weeks ago.

Stack Overflow is down 40%, 40% of their traffic is, is missing, or, you know, less than their peak. Um, to me, that's an indication of something. People are using AI instead of Stack Overflow to get code snippets and learn how to do things, perhaps.

But that's a real number. And that's, that's indicative. Here's the deal.

I'm sorry, go ahead. No, Benioff, as you said, said, yeah, we're gonna do less. Uh, when Zuckerberg, when, uh, you know, usual dark sweep parade out here, uh, Zuckerberg said that they were going to eliminate some mid-level engineering jobs this year because of ai.

Other people are saying we're gonna eliminate some, you know, fresher kind of jobs, engineering jobs because of ai lower level folks. No one says they're eliminating the, you know, the higher level developers. But if you were a young person coming outta school, I, I got a call a couple weeks ago from a recent college grad.

He had a psychiatry, psy psychiatry degree bachelor's, and he said, you know, I'm really thinking about going to a code bootcamp and becoming a software developer. If you're that kid, is that what you want to do? Or do you go to an AI bootcamp and become an AI engineer or an ai, whatever the word is.

If it was your kid, what would you tell him? I'd tell him to go into data science. I told them that too, by the way.

Okay. Um, yeah, data science engineers are probably gonna be a much, far more in demand, but a lot of people don't like that kind of work. It can be pretty tedious.

Development's a lot more fun. But if you really wanna make sure you're gonna have a job in the future, data science engineering is, or platform engineering. Mm-hmm.

So, so long then, let me ask you about this. 'cause you tracked this AI pretty closely, but some of this is a moment in time, and I think some of the execs, when they say they're gonna eliminate this, that, and the other accounting on the reasoning capabilities of the AI models to become more advanced than they are today. So what is your sense of how smart will the LLMs get, and will they be able to take on more of these developer software engineering tasks that today maybe they can't?

Um, yeah, so I feel, I think that AI coding assistance are definitely getting better with each generation, and they may be able to absorb, uh, a good amount of the work that, with the coding and stuff. But, um, also because of that, companies are less keen on hiring junior developers especially. So the apprenticeship is fading, which means that, uh, it would be increasingly trick year to train the next batch of, uh, junior software architects.

Um, also the no-code, low-code, uh, applications, they have a role to play in this because, uh, more employees are now writing applications using ai. But also it's important to remember that the overall tech industry and AI evidently has a role to play in this, but the industry is emerging from volume hire to hiring that matters. So instead of recruiting a large number of, uh, people, companies are now choosing to hire, uh, more experienced people who can do more and can do better, and then ramping up their output, uh, using ai.

So senior software architects and developers are now using AI generated code in many cases and using AI to debug or, you know, and testing codes. Um, but so largely experts believe that the size of development teams is going to shrink over the years, and it feels like that has already begun. But to what extent that is going to be, and definitely, I don't believe that it's going to wipe out developers and AI is going to consume all of that, uh, all of their responsibilities anytime in the future.

I, I just think it's a shift. It's a shift. I mean, think about when we went, you know, we go back some years, we think about when we went from the mainframe to distributed platforms, you could say that there was a decrease in the number of cobalt developers who were being hired during that period of time, which was totally true.

It did. Many of those COBOL developers, me being one of them, said, Hey, I'm gonna go learn to be to, you know, write a presentation manager, um, application and learn a database manager and become an OS two bigot. So it's about pivoting for a lot of these developers.

And they have, they have to start pivoting and if they wanna stay, uh, relevant, and the ones who aren't are gonna find that they don't have a lot of job openings. But just a point, um, just in personal experience, you know, we have in our open source community, um, we have college kids that show up, uh, and, you know, want to get experience through the open source process. So far, all of them have gotten entry level coding jobs.

All of them have gotten in entry-level coding jobs on applications that are using ai. Now, are they an AI developer? I can't say, I wouldn't call them an AI developer, but they are entry-level jobs teams that are working on AI applications, and there was an entry-level spot for them.

Interesting guy, Alan pointed out this, pointed out several times that, um, in the industrial revolution, there was this widespread feeling that people were gonna lose jobs. And what happened instead was people became, workers became more productive. I think this is a little bit different.

I, I'm not sure exactly how, but it's kind of like giving everybody, it's not just coders. It's, I mean, think about doctors and lawyers, right? Um, part of the fundamental skillset now, um, and going forward is going to be the use of AI assistance for whatever it is you're doing.

And I don't know about you, but my general experience with, with, with coding and, and with programming is that, um, for almost any application or service, it's almost a bottomless pit of features and enhancements and, and, and so forth. I mean, the, the, the, the paradigm between perfecting and releasing, um, in code development, uh, that's why I've been saying since the beginning that, uh, the, the, one of the big benefits of AI is quality. Because you can get, move, maybe move that needle closer to perfecting while still releasing thanks to your AI assistant.

The thing is that so far the AI assistance, uh, uh, in coding seems to be about an inch deep, but it appears by its nature, by the way it was trained, it is, it simulates depth. So that's what we all need, need to learn. But Alan, you know, you could be right.

We could just be doing more coding better with the same number of folks in the end. You know, I, I, let me give you a real world example. I did an interview last week with a developer advocate from a company called Postman.

Uh, you could go look it up on text, on tv. It's not, this isn't about Postman per se, it's about him. He, he was a, he's been a developer for, I don't know, 15 years and years.

We transitioned to developer advocate and we were talking about this issue of developers and ai and he said, you know, my sister called me last week and she's opening a bakery, and I'm her brother and I'm her IT person, as most of us in the IT world. Are we, we are the IT person for our family members who are non it, right? We're tech support.

And, um, my sister said, I need a website for the bakery. Do you know anyone I could hire? Or could you do it for me?

I know you're busy. He said, well, let me, let me, let me ask you a couple questions. You know, what color scheme are you looking for?

What's the vibe you want? What do you want to say? 7 of Claude Sonnet came out.

He went into sonnet. He said, build me a website. And it was for GoDaddy was the host, build me a website for GoDaddy.

This is, this is this, this, that, and that. He said a half hour later, her website was up and running, doing e-commerce even, right? You could order cookies online.

And he said, I, if you would've told me five years, 10 years, 15 years ago, that that's what it was gonna take to put up a website. And granted, this isn't an enterprise application, but it was a damn nice website and it was everything this woman wanted. And it was in 20 minutes from inception to, to, to production.

That's the world we're living in right now. You are talking to someone who went from being a lawyer to doing websites back in 94, 95 whenever it was. And it was, you know, it was revolutionary.

Right? Now, look, this is, this is baby stuff, baby stuff. Here's, Here's, here's the bias.

I keep hearing in all the conversations that I'm having with people, if I'm a senior developer or engineer, they go, great, I don't need all these junior guys, and I'll just automate all this function and I'll do that myself. Then you go talk to the junior guys and they go, great, we don't need these senior guys. 'cause I can do all this stuff myself.

So I can't figure out which end of this thing is gonna wind up being the front of this thing, This thing. Well, the senior guys to the senior guys might wanna get rid of the senior guys 'cause they're more expensive. But on the other hand, if it's left up to the senior guys, they'll get rid of the junior guys because they want to keep their jobs.

Yeah, that's, so, uh, just just to point to our, our, the listeners here. If you are looking for a job, there are two things that you should really think about. Make sure that your LinkedIn profile looks awesome.

Um, make sure you have everything on there that you have in terms of skills. I would even say if it's a skill that you know, but you have not actually used it in a job, list it and make sure your GitHub profile is awesome because more employers are looking at the, your GitHub profile as much as they are your LinkedIn profile. So resumes and cover letters are all well and good, but when it, it boils down to evaluating somebody before you get the phone call.

They're gonna look at your LinkedIn page, they're gonna look to how, how you, uh, brand yourself and they're gonna look at your GitHub profile. So interesting. Keep that in mind.

Great advice that is really, Hey, except it's not a person reading it, it's an AI agent that's looking at it first. Eventually there's a person looking at it there. The AI agent only looks at your, your resume and that's the problem.

Alright, let's take a break here on text on Gang. We're gonna come back and, you know, the speaking of old and new and senior and junior developers is the dividing line between, do you use Rust or see in your Linux? com is the leading resource for news analysis and education on challenges facing the cybersecurity industry.

com covers all aspects of cybersecurity, including data security, DevSecOps, cloud security, application security, network security, security threats, and more. com has the largest selection of security content featuring breaking news, blog posts, podcasts, and more. com to learn more.

com. Home of Security Bloggers Network. Hey folks, we're back.

And one of the great things about open source and one of the bad things about open Source is everybody can see everything and everybody's very transparent. So in the land of Lennox kernels, there's been this kind of debate going on where, uh, Linus Torville who runs that project, pretty much wants everybody to use Russ so they can have more memory safe code in the operating system. And this is a good thing from a cybersecurity perspective, but there's a little pushback in the community and, uh, Linus is busy trying to rail everybody back in.

But this conversation seems to be going on elsewhere. Everybody is trying to figure out, well, how do I get rid of these languages that are deemed to be unsafe? And that includes Java C and a lot of this old stuff in favor of what they call memory safe languages like Rust.

Um, Alan, what's your take on what's going on here? But it seems to me that, um, there's resistance to change. Hi, boomer.

Uh, Okay, and we're done. That was a good, That, that's what this is coming down to, to me folks, right? I'm a boomer, so I I get it, I get it.

But you know, the, the old guard is digging in their heels saying, Hey, we, we've been working in sea forever, our whole careers. We know every nook and cranny. We know how to avoid that memory leak.

We know how to avoid, make it more secure. We know everything we need to know about. See, I'm so damn comfortable in it.

Why would we think it's changing in the next generation? Sorry. And Rus is harder to learn too.

Rus is not an easy language to pick up. It's hard. I've looked at it and said, I'm glad I'm not coding 'cause I wouldn't wanna learn this, but maybe it's not a, a one or the other option, right?

I mean, most of the kernels written in sea, they're not gonna throw that all out and start rewriting the entire kernel in rust. Why not? It would be a huge undertaking, and I don't think They're gonna throw out working code.

But over time, as you replace Over time, new modules, new modules, I'm guessing that they're probably gonna move to Rust. Uh, I, you know, why not? I mean, that would, that makes sense to slowly move over to Rust if that's the direction they choose to go.

You could have kept stuff in COBOL or web or, or assembly language or, you know, or look, time marches on computer languages March on, right? There was a time where the lamp stack was PHP, now everything's Python. Um, Russ Golan another, you know, go Golan, another, uh, modern kind of, uh, uh, languages.

Um, c was not perfect. C Sharp was perfect, was also not perfect Java for, you know, the tremendous success in what Java's done for code mobility and everything. A lot of security issues there.

You know, over time I've come to trust Linus Valli's kind of guts on these things. And clearly his gut is telling him moving the rust is a smart thing for the Linux kernel. Um, I I'm going to give him the benefit of the doubt and say, if he feels that strongly about it, there's probably something to it.

But I think you're touching on the problem there, Alan. It's a meta problem. It's not a technical one so much I think, which is, um, how what, uh, Lin's, uh, four vault's influence is, is outsized.

And I mean, he deserves it, but it's outsized in a corporate decision. So let me, let me just back up one second. It's really hard to make a change.

It can be really hard to make a change. And I sympathize with the desire to keep things simpler in such a large project. One language, um, more developers who know it and understand it, it's what they're used to.

But then you're ossified in terms of, in terms of being able to move forward sometimes and a corporate decision or a committee decision or whatever. These things get considered and debated and then a roadmap and a plan is put together and all that other sort of thing. That's not how the foundation is running because of Al's outside influence.

He can't really, he neither does he, can he, nor do I think he wants to dictate anything, but when he advocates for something, it creates, you know, a a lot more stress then in a more, I don't know, egalitarian or corporate kind of process. And I think that's kind of what's going on here. I think if you pulled anybody out and said, um, should, should the language in which the lice Colonel, uh, uh, is programmed, never, ever, ever, ever, ever, ever forever change.

Should it always be c forever, they would say probably no. Very few people would say that there should be no room for change. So then it's not a question of whether, but when and how you do it, You know, I can't help, but I'm sorry, go ahead, Mike.

This is, and I'd love Tracy's opinion on it. 'cause I think this is a change management issue writ large, right? So if somebody will sit down and say, we should write this in Rust, and then they'll look around their squad and go, who here knows Rust?

And then you hear crickets, so then they just keep going back to Java to guy's point. But what does it take to get, uh, development team to learn a new language? How long does that take?

I mean, you're talking about significant training and that might take years to make that transition. Tracy, what do you think? Well, to be quite honest, the, the truth is there's not a lot of developers that no see anymore, right?

So Sea Development has, you know, dropped off Java, replaced it. So if you're looking at, uh, a group of sea developers who are maintaining the kernel then, and you want to take advantage of the security, uh, features of Rust, then you would think that those seed developers would be okay to move to rust. But as Alan points out, they do get, they really develop, you know what software languages is like religion.

You're asking them to change the, you're asking them to change religion, and it is extremely difficult for them to change religion. It really is. It's so hard for developers to pivot.

They get so stuck in the mud. But with regards to the, the benefits to the kernel itself, if we are building a more secure kernel, it's easier to maintain because of some of those, the memory management that it has. I can't imagine why they would continue writing it and see, I really can't.

But we have, they have to change religion and the culture of developers is very odd. They, they really dig in. They love their code, they really, really love their code.

And to see it move to something else, um, is hard for them. But new modules, Hey guys, hey ladies, whoever you are working on that, I think that you're gonna be coding and rest very, very soon if you're gonna continue on that project. Look, resistant, the change, there's always resistance to change.

Change is hard, right? Especially in software development where it is direct, right? Tracy, it's akin to religion, right?

I've seen the, the, the, the squabbles between Java and C folks back in the day. net? Oh, you were less than human, right?

And, And yeah, they are, we are less than human. I'm a T net programer. I'm, yeah, actually, you know, so an alien in disguise.

This, this is visit You guys. Yeah, That's software. That's what makes it fun though, right?

It, you know what's interesting is when you go to large enterprises and say, what, what, how many in what languages do you guys support over here? The average large enterprise probably supports seven to 12 different computer languages. And they have different teams for every language.

And that's not the most efficient way of doing things, I'll tell you that. So I am, I'm, I'm going through my mind. It's to see if I can think of a single instance in history where there was a mass religious conversion that was peaceful and I'm coming up blank.

So, well, we might, we might be seeing it with Elon in Doge. I'm only kidding. Hey, we gotta call a wrap on this version of the Gang.

Guys, it's been real. It's, it's been a, a really great discussion on some really great topics here. I hope you've all enjoyed it.

We have a full text on TV schedule following this immediately, so stay tuned wherever you're watching this, unless you're just watching this as a podcast or something, go, go find, go to Textron tv. Go to our YouTube channel, go to any of our sites, check it out. Guy Tracy Sagner.

Mike, thanks for joining us today. For now this Alan Shimel. We're out.

Hello and welcome to the latest edition of the Textron AI video series. I'm your host, Mike Zern. Today we're with a tins who is CT for Galileo, and we're talking about a leaderboard for AI agents that they've created.

And well, a lot of these technologies are innovating so fast that everything seems to leapfrog each other on the leaderboard, but some, no one's quite sure who's in best in what at any given moment, but at 10 you guys are trying to figure that out. How do you determine that? How do you track that?

I mean, and, and for that matter, how did you guys get in this business? Yeah, so, um, so for those who don't know, uh, I'm atten, I'm one of the founders of Galileo. And the reason we got into this space 'cause was 'cause uh, from my personal experience, uh, I kind of spent much of my career in the, in, during the growth phase of Uber and very early AI at Apple.

And we saw evaluations being a very cumbersome task and kind of the primary reason why AI systems fail in production is because of lack of robust evaluations. So we kind of went down this path and, uh, ended up, uh, choosing language modeling as our sort of strength and, uh, for it down, down that path. And, uh, for us, uh, of course, you know, the whole industry is kind of caught on and, uh, started using LLMs for industry use cases.

And one of our goals was always to focus on benchmarks, uh, for a lot of these LLMs. 'cause we saw an explosion of, uh, of LLMs being a real thing in the, you know, first couple of years of, of, um, you know, the the chat GPD moment. Um, and, uh, we knew that, uh, often in ai people focus on academic benchmarks, you know, just to show strengths and weaknesses of models and they don't really do good justice to industry use cases.

And, uh, that was one of the reasons why we built, uh, what's called the hallucination index. And we published a couple of versions of those in, uh, the months prior. And more recently because agents and agent workflows have become, uh, very standard and, uh, it's still early days, but people have started adopting building agentic applications.

We sought to build an agent leaderboard, which kind of uses a lot of the evaluation techniques we used for the hallucination workflows and rag workflows when we published our hallucination index in that we essentially curate, uh, high quality data sets across multiple industry domains. In the case of agentic leaderboard, we looked at over 25 common industry use cases and domains, and then really used our own metrics and scorers, which we publish as part of the Galileo platform, kind of like dog feeding the Galileo platform to see which models, uh, perform well for certain kinds of agent tasks and don't for others. 'cause it's always this game of, oh, which model should I use for my application?

Uh, is one of the first questions any practitioner would ask is that, and our goal has always been to kind of publish industry benchmarks and kind of show how these models perform. But in the light of more practical industrial use cases, and a lot of the insights which are, uh, pretty surprising in the agent leaderboard, they kind of point to, uh, the fact that, you know, a model can do great in an academic benchmark, but the, the rankings can be completely different when it comes to the industry. How much difference is there in capability for these various AI agents versus what it might cost to implement them?

It seems like if I read the report, um, there's not a huge gap between them, but the pricing is kind of fundamentally different. Actually, that was one of the most surprising, uh, if I were to rank the three top three surprises from the leaderboard. One was this, uh, the this, uh, uh, sort of tussle between cost and performance.

'cause amongst the top three ranked models, which included Gemini and uh, as well as OpenAI models, uh, the performance difference between the best and the worst in the top three was 4%, which is minimal marginal, uh, and the cost difference is 20 x. So, you know, as an industry practitioner, if you really care about cost, you're kind of in luck. 'cause you might find a model that performs equally well for your use case, uh, as any other at one 20th, the cost.

How much are people kinda evaluating an AI agent today initially and going with one? And then are they gonna swap those out over time based on pricing or is this kinda like, I make a decision once and I stick with it? It is more the former where you would kind of expect, uh, a developer or you know, a maintainer of angen system to really go with what's best.

'cause on one hand there's this massive reduction of the cost of intelligence where a small model, uh, 100 the size of say a super large, very intelligent reasoning model can do as well, if not better for their use case. And uh, that combined with the fact that every day there's newer models, newer tools, newer frameworks, which are emerging, it's only natural for you to kind of do a, you know, a cost benefit analysis. And, um, uh, that's why a lot of, uh, uh, on the tooling side of things, uh, many companies are kind of building these AB testing tools internally as well as using from open source, which kind of allows them to almost swap out models, uh, in their production applications, uh, and quickly do a test of, you know, if I swap out anthropic with something else, you know, what's the variation in performance will my system incur?

Um, and and that's kind of part of the story that Galileo also intends to solve where we wanna provide a platform and tooling to be able to do that very efficiently and do that in real time. Yeah, that was my next question is how dynamic can this be? Because can I have a scenario where LLMA is what I use in the morning and LLMB is what I'm invoking in the afternoon because some of the pricing structures changed on you in on that LLM per se.

I mean, just how, uh, disposable are these models gonna wind up being? That's a great question. Uh, I do think that there's a lot of variance, um, kind of in, in the era in which we are in, um, uh, but that, that variance will sort of go down as the tooling, uh, matures.

And there's um, there's, there's minimal difference between, uh, different l lms but also in terms of cost. Um, but also I think, uh, one prediction that I have is, um, in probably the next 12 to 15 months, what we'll see is this clear, uh, segregation between, uh, hey, here's the class of reasoning models, which, uh, are expensive, but they solve one particular kind of task very well, whether it's, you know, deep research or, uh, you know, things that require you to think a lot and reason about a lot. And they are very expensive for a reason because behind the scenes there's, you know, if you go to the technicalities of it, there's multiple forward passes which are happening.

And uh, the amount of GPU operations, it's really like the cost is kind of driven by the GPU operations. It's a lot more for those kind of models versus the second class is, hey, you're building some sort of an industry application, whether it's a chat bot or a summarizer or q and a system. Um, there's a, a list of maybe a dozen or two practical use cases and there's a swath of much cheaper models which help you achieve that.

Um, uh, that, that's kind of on the LLM side. Uh, one of the other things is with agents, it's not just the LLM, it's different functions are in the mix now. There's different tools and uh, the, the ecosystem has become a little more complex and spirited with my myriad of different operations.

So that also and of causes additional variance where, you know, you might have stability in the LLMs and you're like, alright, this particular model Gemini, for example, is great for my agent application, but today I'm using auto gen for my tool orchestration and tomorrow maybe land graph or something else might, you know, just perform more deterministically for my use case. I, I do see for the foreseeable future that as the tooling and tool ecosystem matures, there will be this, uh, challenge of a lot of variability in the different components. 'cause everything's now predictive and, uh, sometimes something may work for one use case and may not work for another use case.

So what you really want to build is like this system that allows you to multiplex between different models, between different frameworks and uh, that's kind of your, that would be my advice to anyone who's in the early days of building agent systems. Do you think at some point we may even have, um, I don't know, an AI agent or AI model that is optimized to figure out all that multiplexing logic? Absolutely.

In fact, we are already seeing, um, this at least in the private, uh, private models ecosystem with, with OpenAI Swarm and various others who have built in function calling as well as tool selection capabilities. Uh, the, the truth is that there will be innovation on both fronts. Uh, like the large model providers are also in the game of not only building great systems for academic purposes, but also, uh, they're big businesses now and they, they're trying to cater to industry, but there will also be open source models and, uh, open source tooling which will kind of, you know, race one-to-one with, uh, a lot of the innovation that OpenAI does.

Uh, speaking of open source, one of the other great findings from the agent leaderboard was that, uh, Mistral is probably the top open source model, which came out in our rankings with a very high tool selection score. So if you're a fan of open source, Mistral is certainly a great starting point for you. Do you think we'll see some segmentation along the lines of, I may use open source for most of my functions, but then I'm gonna use proprietary models for specialized purposes that maybe they're better able to reason across?

I mean, will there be segmentation kind of like the difference between, I don't know, buying a Honda and a Ferrari? That's a good point. Um, I would say that the jury is still out on, uh, whether that would be the sort of segregation factor between open source and proprietary.

Deep seek is a great example of that where it was able to compete and outcompete on certain benchmarks. Oh, one a level models by, uh, by OpenAI. So you might as well, you know, not use OpenAI and use deep seeq or try out deep seek flavor of models for your use case.

And that is open source. And, uh, uh, so, so there is this push towards having open source reasoning models, which are equally powerful, if not more. The race, I think is in optimizing the costs.

'cause the lower the costs go, the more available it is for, for usage. And people will only pay for, uh, usage, whether it's open source or whether it's, uh, using an OpenAI model or any proprietary model. Uh, it'll only be worth the dime if they're able to solve industry specific applications, uh, for the rest of us who are tinkering around and just doing, you know, simple q and a.

Of course, there is that, you know, uh, reason to pay for just directly using these models, but real like that, the real monetary value kind of comes from industry. What's that one thing you see people doing over and over again that kind of makes you shake your head a little bit and go, folks, we need to be a little smarter about this than that. It's really how people are still playing whack-a-mole with the different models and different frameworks.

Uh, there's almost tool fatigue and tool noise at this point around which model or which framework to use for your specific application. Uh, um, I, I think the right way to go about in, in an environment where there's so much options to choose from is to, uh, really build a system, whether it's an internal tool or, you know, go for a, some kind of a, a platform which allows you to multiplex between the different choices and quickly get to the decision of that this is the combination that works for my summarization use case or some other, uh, q and a system that I'm building. Uh, Galileo, just to throw a shameless plug in there, is kind of one of those systems which allows you to make these decisions pretty quickly.

Uh, but then that, that's half the story. Uh, then because once you build your application with, say, you know, the ideal combination of LLMs and prompts and frameworks, that is bound to change, uh, based on the real time performance that you would get in production. So you need another system that allows you to monitor that and, uh, make those decisions on the production side of things.

Uh, and there I think the real differentiator is your production application will constantly have newer forms of data to deal with, whether it's new questions or new kinds of ways of querying your application that will likely determine the performance. So you need a way to measure it and then kind of come back to the drawing board and, um, redo your entire experiment that, Hey, here's a new kind of data that my chat bot received. Clearly, you know, frameworks A, B, and C are not working for me.

Here's a quick test that I'm gonna do with d, e, and F and throw that back into the production. And you need a system that allows you to do this in with minimal downtime. Folks, you heard in here to quote one great philosopher, don't worry, be happy because well, the cost of switching between AI agents and LLMs is not nearly as high as you thought.

Hey, add, thanks for being on the show. Thank you so much, Michael. It was a pleasure.

All right, and thank you for all watching the latest episode of the Textron Do AI video series. You can find this episode another on our website. We invite you to check them all out.

Until then, we'll see you next time. Good morning, good afternoon, wherever you are. I hope you are starting a really wonderful new year for everybody out there.

Um, this is 2025, and one topic that is at the top of everybody's mind still is ai. Now, the thing about AI is that you simply cannot ignore it. Whether you are a technology leader, whether you are a business leader or in any of the other functions like HR or legal or finance, AI is at the top of the mind.

And you in your leadership position are asked multiple questions around ai. You are asked to build a strategy around it, you're asked to build projects around it, and then drive those projects to conclusion and to successful conclusion in spite of all the ambiguity that is around this technology that is around this topic. So the, in the next 30 minutes or so, we are gonna try and give you a path through this confusion, we are gonna share some of the ways in which we have accomplished success in this area.

I'm gonna share some of, uh, the things that worked out for us and also some of the things that did not work out for us, so you don't have to repeat those mistakes. So let's go. I think it's going to be fun.

30 minutes. So the thing about AI is that a lot of people talk about, and in the media, when you hear about it, they talk about the upside of it. Oh, it's going to be fantastic.

It is going to change everybody's life for the better. Uh, there's going to be a massive increase in our GDP or the GDP of the world, and the productivity go up and most of them are very possible and are actually true. There can be unimaginable amount of growth when it comes to ai, but at the same moment, and in probably the same measure, we also have to think about the change that it is going to drive.

And that change could be on the environment side, it could be on how businesses are done today. It could be the change that might impact your job, my job, everybody else's job. So it's very important that when we think about ai, we think about the potential of unimaginable growth, but we should also take care that we are ready for the indiscriminate change that it is going to drive.

With that out of the way, the first question you must be asking, who are you and why should I listen to you? So my name is Debil Bakhari, I'm the chief technology and product officer for Extreme Networks. Extreme Networks, if you are not familiar with it, is the largest pure play enterprise networking company on the planet.

Um, and you should listen to me or to Extreme because we have now successfully productized multiple generations of ai. And the latest one we recently announced is our extreme platform. One that really takes AI and narrates it with a platform, an enterprise wide platform, um, so that you can actually unlock the real potential of it.

Now, AI as a product and AI as part of a platform, that is an entirely different topic, which we are not gonna talk about today, but something maybe, uh, for future. So this, so all the things that I'm gonna talk about today are really based on our experience as we product ties and brought to market extreme platform one. Uh, so I'm not gonna stand here, well, in my case, sit here and pontificate.

I'm gonna share some real world experiences with you, some of them very positive, and some of them, uh, which will be ous, right? Okay. So let's go.

Each one of us wants to be successful in ai. And in order to wrap our heads around it, it's sometimes easy to put something as an equation. So here's the equation that we have learned over our experience of productizing it.

There are multiple components in it. The first and the foremost is user experience and the trust gap. And they are conversely, um, or there they're not directly proportional to it, uh, themselves, or they are inversely proportional as they say.

So the better the user experience, the more success you can have, but the bigger the trust gap, you know, the less success you will have. And then there are a couple of other components, which is technology and also willingness to pay. And willingness to pay is something that people usually don't talk about it.

So I'll talk about that towards the end and make sure that we actually understand what we mean by that. If you notice in this technology is not the biggest factor in there. Uh, and that is one of the biggest problems that happen with AI projects.

The moment somebody asks you, Hey, uh, so and so you need to come up with a AI project for our company. And the first thing that people go, they look out and they go out and they start searching for the best LLM or the best model to use that. That's just an absolute wrong position or wrong place to start with the first and the foremost thing is user experience and trust gap.

So that's where we will start with. Now, one thing to remember when it comes to trust is if your people do not trust it, they are simply not going to use it. And it doesn't matter how good or bad your AI product or technology or agent is, if people don't use it, it's not going to be successful.

So think about trust upfront. Now, the thing about trust is that trust and value, they kind of go hand, hand in hand. So the more trust people have in an AI product, the more valuable they consider it, or conversely, the more valuable it is that could drive more trust in it.

But there is a thing called complexity in there. And typically the higher the value of the ai, um, you know, unless you are at the very, very start, uh, it's going to directly relate to the complexity of the project. Okay?

So when it comes to trust, there are two, three things that you need to consider. Number one is pick a use case that actually matters. So the first thing that we are gonna talk about is how do you pick a use case to which you are going to apply ai?

That's number one. Number two, how are people actually going to experience? So what is the UI for that ai?

And the last part is culture. So these are the three things that feed directly into this trust and value, um, in an equation that we talked about. So the first thing, how do you pick a framework or how do you pick, um, an experience that you're gonna apply AI to?

How do you pick your AI use case? How do you pick your AI strategy? Different companies call it differently, but a lot of the times what happens is that you get a call from your boss or from your board and they ask you like, Hey, we need an AI strategy, and off you go to the races.

But I think the best way to do it is to take a step back and think about what are the use cases, current use cases inside your company to which you wanna apply it. And there's a very easy or simple way to wrap your head around it. And we call it the ARC framework.

Um, and this is accelerate, replace, and create. So simply speaking, it is like, what are the experiences in your, um, domain, in your work environment or in your company that you want to accelerate? What are the ones that you want to replace and which are the ones that you want to create afresh?

And this is in your context. If you are doing AI for your team, you can run the ARC framework in the context of your team. If you're doing it for your entire enterprise, you can do it in the context of the enterprise.

And if you are productizing it and you are bringing it out to an industry, then you can do it in the context of that industry. But the framework nonetheless, stays the same. What are you going to accelerate?

What are you going to replace and what are you going to create? So here, let's just take a few examples. These are simples very simple examples will make sense to everybody when it comes to acceleration.

A lot of companies start with their customer support experience. Why? Because nobody likes to sit on that call listening to that, you know, fun elevator music and waiting for a customer support agent.

Or when you get to that agent, nobody likes to start from like, is the light blinking and have you turn it on? And is the power plugged in and stuff? People want to get to their resolution very quickly.

And a lot of companies, a lot of use cases on applying AI to that. And what are we doing here? We are accelerating an experience that is already present there, right?

And it is very powerful. That's an example of accelerating a use case. Now, let's think about an example of perhaps replacing a use case.

And we are in the networking technology domain, so I'm taking a lot of examples from there because these are things that we have already done in our portfolio. So this experience is the acquisition of knowledge. Now, what do I mean by that?

It could be finding information inside the documentation. It could be finding information in real time for a product that you're using or a problem that you're having, or even things like explaining something that the product is showing you. You remember those error codes.

Now, good thing we don't have error codes and SaaS applications, but sometimes information that is displayed, um, in those dashboards and in you, in those you nice pretty, uh, graphs, you might wanna know what they actually mean. Now, having a chat bot that is right there and giving you that information in real time, that is expo that is really replacing the experience of having to go out digging through documentation and websites. So again, very simple example.

Now in terms of creating a new experience, uh, this is something for example, right now, at this point in time, um, especially in the networking world, if you want to pick up information from various different pluses in your network in real time and create this, what if scenarios and stuff, it's something that is very, very cumbersome to do, to a point where I would say that they are not really something that everybody does in their daily life. So it's almost that that experience does not really exist. So this is with the use of AI creating an experience where you can acquire the information, you can actually, uh, structure the information and you can build insights on top of it all within the same product, within like minutes or so.

So this is creating an experience that is brand new. Now, this is just the example. You have to apply it to your space, to your domain.

But think in terms of what are you going to accelerate, what are you going to replace or what are you going to create? The chances are you're probably gonna start with the a part first. Now, as you're picking this up, one example, or one thing that I will share with you is do not pick a use case that nobody cares about because you are going to get some money to actually go this AI use case, but once you do it, you are only going to get more money if that use case were valuable.

So go pick a use case that is valuable from day one. Don't go for the low hanging fruit, not when it comes to ai. Okay?

So this was the part of picking an experience. So once you have picked up the experience and you are trying to build something around it, the second thing that you have to think about is how the users of this AI application, what would be their experience? Because as different experiences are developed during ai, the trust or the adoption for that is not linear.

You know, if somebody is used to a conversational AI and they will start adopting it, but when you actually, um, introduce them to an interactive or a collaborative AI or to a agentic ai, the adoption is gonna drop down before it comes back up. So consider these things as you build the UI for your program. So how did we build a UI for the ai?

Um, in platform one, we essentially went with three different ways of interacting with ai. So conversational, interactive and autonomous agents. And my feedback, um, or my, um, sharing my experience with you, um, that don't go with just one, because if you just put a chat bot in your product or in your project, uh, very soon you're gonna find that there are use cases that do not lend itself very well to a conversational interface.

So always invest in and think about various different interfaces, very different user experiences. So here, let's just take a quick example. This is something that is going to be very familiar to all of you.

This, by the way, is extreme Platform one. Um, of course this thing is gonna look a little bit different based on whatever product you are using, but the idea here is conversational. So you have, think about that you have an expert sitting right next to you or another colleague sitting right next to you who happens to be an AI chat bot, and you are talking to that chat bot.

And as you are talking, you are learning, as we talked about knowledge acquisition. Uh, you are doing things, but this is a conversational interface and this is something that people are a lot more familiar now in 2025. So this is the base experience that you can provide in your project or in your product.

But as we move forward, um, there is the second interface, which I think will become more and more valuable as we go past 2025. Um, and that is really the experience of creating real time dashboarding. It's really a canvas.

We call it an AI canvas. And the idea there is that there's a lot of information that is present in the products, but not always in a way, shape or form that is actually valuable. And that is something that you can really, truly use.

I'm talking about the users. So giving the ability to the users to interact with the system and create a new dashboard altogether. So in this example, what happening is that the user is actually working with that conversational ai, and it is asking the AI to give it or give the user multiple different pieces of information.

But while the user is doing that, the user has the ability to take that information and put it on a canvas. And as the conversation continues, it continues to put more things on the interface or on the canvas, and in the end it ends up with essentially a real time dashboard, right, that they can now create and then now they can actually use it whenever they want. This is an absolute powerful way of allowing people to use AI to create something that was not present there.

And, uh, this has worked fantastically for us, and this is something that I highly encourage people to think about and use when you are doing your AI projects or your AI products, right? And we'll talk about the difference between a project and a product in a little bit. So if you go to the next slide, um, you will see that the next big thing is obviously agent ai.

Still, I briefly talked about, you know, the conversational as well as collaborative ai. Uh, but this is 2025 and everybody should be thinking about agent ai. And if you are not, then I'm pretty sure you're getting peppered by everybody to think about it and come up with a strategy around it.

Now, let's take a swing back on the agents, because before you go into the technology, it's kind of important and interesting to understand what an agent does, because when you talk about agen AI in the industry, there, there are two connotations in which we talk about it. One is, um, you know, running AI agents behind the scenes do all those different things, right? So there's a master agent, and then, you know, there's a bouncer agent and there's an agent that distributes the query and so on and so forth.

But if you think about a curtain, and on this side of the curtain is the user, and on this side is the system. Agentic AI in the system is one thing, but then exposing those agents to the user side or user visible agents is something a little bit different. And now here we are talking about the user visible agents.

So this is agents that user would interface with. So some of the characteristics of it, the whole idea behind agent AI or AI agents is that they're autonomous in nature. Now, autonomous doesn't mean that there cannot be a human in the loop.

You can have both kind of agents out there. Um, but the key is that it also has memory. So it has context memory in, in AI words is really context for the user.

So it retains the context. Context, it is reactive, or it can be proactive. It is very specialized in doing a few things.

Um, and remember the last part, it can interact with humans or it can interact with other agents. So this is kind of a rough definition of agents as they appear to the user. Okay, so those are the agents, but how should I think about agents?

And you should think about agents the same way as you think about humans. So I always like to give this example that, um, how do you create teams, right? You find various people, there's definitions of their roles, they're expert in different things, and then they interface with each other, they interact with each other, and that creates a human team.

Now, if you take it to the other extreme, what happens is that you have a bunch of agents. Now imagine that these functions are being done by AI agents, and these AI agents need to be organized the same way as you organize human teams, and then they talk to each other. Now, I wanna see these are two extremes of the spectrum.

A fully human team, which is not only just the extreme, but is also the norm today. But in the far out world, we could or far out time, we can think of, you know, teams that are just a hundred percent based on AI agents. But what's the reality?

The reality is going to be that it'll be a combination of human and digital workforce. So when you are building an AI agent, when you are thinking about an AI agent, keep this context in mind that your AI agents should behave towards the human users as humanly as possible. So with this context, uh, let's just give you an example.

So this is how we have done agents in extreme platform one. Now it, it, the video will do a bunch of different things, and of course you can rewind it, you can pause it, you can look at it. But generally what's happening is that when you have agents, you have a lot of AI agents, uh, you need to do the following things.

Number one, there needs to be some sort of a catalog where you can find an AI agent that does a certain thing. This is pretty similar to where you have a job description and you go out and you try to find somebody that can actually do that job. So you have to be able to find AI agents, then you need to be able to go and configure them.

This is kind of, uh, teaching a new person how to do their jobs. This is like about, um, now on the AI side, you actually go and configure that AI agent. Um, and then once you have configured it, then you need to do lifecycle management, and you run this.

And in that configuration, it, that agent might be interacting with other AI agents or with human people or the human workforce out there. So that's just the example of how to think about AI agents. So when you think about AI agents, start with the context of that this is digital workforce and it has to work with humans, and then start thinking about how to organize them, how to make them discoverable, and how to make them configurable.

Now, once you do that, um, then only think about the technology. So I can spend a lot of time about technology behind the scene, but, um, what I would tell you is that most of the times, um, what our experiences, and we have hundreds of thousands of e you know, users out there, like 50, 60,000 big customers all the way from mid-market to, you know, fortune 10 out there. And what our experiences that as people go into building these agentic ai, most of them will probably fail.

But that's not a bad thing, that's a good thing because that's experimentation. Uh, and that just helps with that AI literacy across the organization. But once you have done that experimentation, you are gonna be, um, dealt with a question, the age old question.

Uh, are you gonna build it yourself or are you gonna buy that as a product? And I'm not gonna tell you to lean one way or another. You can build it yourself, you can buy it.

Uh, but that is a decision that you have to take before you dive into the technology. Now, of course, we are a product company, so it's not like we can actually buy it, so we have to build it ourselves. And as we built it, these are the experiences, um, that we had as part of that.

Now, there is so much technology, there is technology overload essentially when it comes to AI out there. So it just to organize it, and I'm gonna build out this slide just to organize it. We think of it in three big buckets at the, at the bottom of it is the data hub.

Now e you can call it data hub. You can call it data pipeline, you can call it data fabric, whatever you call it. But the idea there is this is the layer in which you are collecting data, you are organizing data, you are indexing it, you know, and you are putting data compliance in it.

This is where who can access what resides. This is where, uh, you know, your data residency and data governance pools, that that is where all of that sit. Then a layer above that is what we consider as AI services.

This is where, you know, your model management sits in. This is where your multi, um, you know, your rag architecture sit in. This is where if you are multimodal, that's where it set in.

So how do you organize all of these AI services together? So that's, think of that as a second layer. The third layer is perhaps the most important, uh, in the actual usage of ai, and that's the safety and guardrails.

This is the fact that, you know, how do you deal with hallucinations? How do you ground your models? What is your resiliency?

How do you build a human in the loop? Kind of an architecture and stuff. So lots and lots of technology around there.

And the reality is that I'm not gonna go ahead and give you recommendations because by the time I finish this presentation, there's gonna be new advancements in this area. But what I can do is share some of the experiences and some of the learnings that we have in this process. So starting from the bottom right in the data hub, uh, don't go with like, I need all of their data in the world to be able to do this.

The most important part in the data is the data governance. Start with the data sets that you already have, but make sure that you think through the data governance, um, as one of the most important thing in your data layer. The second thing, which is really interesting is that, um, where is that data coming from structured, um, and unstructured.

And if you are producing that data, then push down to those teams that are producing that data the right way to do it. I'll give you one example where we realized is, um, that a lot of the documentation that is being written, the simpler the English, or I would say the language simpler, the language that is used in that documentation, the easier it is for an AI system as opposed to when you're writing it for a human consumer, in which case, you know, you want to add more, uh, context and perhaps, you know, a little bit more flowery language and stuff. So things like that become really important in your data layer, in your AI layer.

Um, quite frankly, uh, rag, uh, or some variation of it, like metadata base rag and has rag whatever rag you're using, because that is what will really help you when it comes to, uh, trust and stuff on top of it. Um, so at this point in time, rag architectures are really, really, really, really critical. Now, when you come to safety and guardrails, quite frankly, I would tell you don't go on this journey on your own.

Um, make sure that you have some really good strategic partner for us. Microsoft and Amazon are two really big partners, and we have learned a lot from there. Um, and we have incorporated a lot of their technology.

But one thing that I will point out is that fact checking mechanism, which are really that, you know, how do you deal with hallucination? How do you deal with, how do you ground your models and stuff? So that fact-checking mechanisms are really important because remember, it doesn't really matter what your AI does.

If people don't trust it and don't use it, there is no value to it. So these are some of the things that you should consider if you are building your own project. And these are also the things that you should consider if you are buying AI from somebody.

And at that point, this becomes questions that you should ask, okay? Now I said, we'll come towards willingness to pay. And you might be thinking to yourself, Hey, I am building an AI that is for my own team, or my own organization, or my own company.

Well, this willingness to pay doesn't really, um, you know, come into, into play here. But what I will tell you is that it does, because if you are building a product, then willingness to pay is very simple to understand that is something that your customer is willing to pay you for it, but if you are building an internal project, that that willingness to pay really shows up in the, in, in the form of investment in your project. So how much your organization or your company is willing to invest in it, that's really also willingness to pay as well.

So with that context, how should you think about willingness to pay? Now this is very simple graphs, and these graphs, I've kept them super simple on purpose, uh, because these topics can be complicated, that they can be presented at least at the very start of it very simply. So on, on the X axis, you have a done adoption, and the YH as you have willingness to pay.

So willingness to pay actually increases that adoption, but there is a tipping point unless you have reached that tipping point, um, your ability to charge for it, either to your company or to your customers is not gonna be that high. And then once that chasm of adoption, um, is jumped or, or your users on the other side of it, then the willingness to pay is directly tied to the clarity of the ROI. And that ROI is based on what is the value of that ai.

Now, if you apply it to the art framework, generally speaking, this is not a hard and fast rule. There could be exceptions, but the more you are, um, the higher up or the farther out you are in the ARC framework, the higher the willingness to pay attached to it. So if you're just accelerating an experience, which is really truly automation and efficiency game, a productivity game, there's a certain willingness to pay as opposed to when you are creating a brand new experience, which might actually start a new brand new revenue stream there, the willingness to pay would be very different.

So these are some of the kind of contours of this conversation that you should be, uh, familiar with or you should keep in mind. So now in what we have seen in our efforts to productize this is that conversational and interactive ai, as we described earlier, they lend themselves better to just being part of the subscription. So what do I mean by that?

They're just part of the product, they make the product better. Or if you're an internal project, then it makes your project better, you know, uh, cheaper to run, or people love it more and they adopt it more. But this is the willingness to pay is embedded into the product or the project.

But as you go towards autonomous agents, the autonomous AI agents, that is where each agent can have a very clear determinable, demonstrable, ROI. And at that point in time, your willingness to pay internal or external should be attached to that. So what is that?

What is that ROI? Now this is an area that I would say is at the very start of its progress in there. So I'm gonna leave you with a few thoughts around this, um, as you determine these out.

So how do you price this? And then remember, if you're building a product, it is pricing. And if you are building an internal project, and this is about how much investment you're gonna ask, so it's, it, it works for both.

Are you gonna base it on the current cost of delivering agents, right? You say like, Hey, it takes me X number of dollars, you know, to build this AI agent, so I'm gonna price it that way, or that's the money that I'm gonna ask for this project. Um, well, you could do that.

That's a very simple way of doing it. Uh, but the reality is that the cost of delivering this agents is going to, uh, crash over time because of the economies of the scale and the cost of the AI is coming down, uh, pretty, uh, tremendously over time. Um, so that might not be a good way to do that, or would you do it on the future cost of delivering this agent?

Well, you don't really know what the cost of that agent is, or would you do it at the current cost of the task that the agent is going to produce or to deliver. So think about it this way, I said that, think about AI agents as more like human agents or human workforce. So typically what you do is this, this is a job description, and in the market, this is the pay associated with that, which is a generally accepted cost of doing the task that the job does.

So you can take that model, um, you know, in in view as well as you're thinking about it, um, or you can also think about that. What is the value of the outcome that it is still delivering? So, you know, what is the task that the AI agent is doing, and how do you value that?

I would tell you that there is no easy way to do that, you know, so these are just some of the questions that you have to think about. But what I would tell you is that do not leave it for later when you are going into the ai. You have to think about all of these things in one go, and we'll just do a quick summary of it.

In order for you to be successful in ai, you have to think about user experience and trust. These are the two main components underneath this user experience and trust is how do you select the use case for which you're gonna apply AI and use the ARC framework or any other framework that works for you? ARC has worked really well for us, it works very well for our customers out there.

Uh, pick use cases that you wanna accelerate, you want to replace or you want to create. Um, and then that's a good way to start getting started on the use cases. Uh, pick a use case that actually matters.

Don't go for, you know, low hanging fruit and then consider what are the, what is the UI or the user experience you have to deliver on those use cases to help people get over that trust gap. So you start with conversational, you know, you start with human in the loop, then you maybe take them towards that collaborative interactive where some of the functions are done by the humans, some of the functions are done by the ai, and then eventually taking them towards these egen AI that can be very, very independent. But the key is that this is a journey.

So think about user experience and journey. Now, the two other things, obviously technology, technology. I kind of shared with you some of the things that you need to consider.

Remember, how are you gonna manage your data is going to be really critical. And then how are you going to build those safeguards and guardrails on the top? And then only the technology.

Uh, I know I'm, I'm A CTO myself, but I'll tell you, uh, that when it comes to ai, technology is comparatively the least important part of the equation for the success in ai. It's important, but the least important. And then lastly, think about willingness to pay.

It could be pricing if you're building a product, or it could be investment if you're doing an internal thing. Look, uh, I hope this was a useful conversation, um, because this is based on years of years of experience in productizing ai, successful productization of ai. And we believe that the path to the future is going to be brighter and better with ai, but there are hurdles on the way.

Uh, but if you're cognizant of that, and if you are willing to think about that upfront, then we can actually be very successful. I wish you all a great start to the new year, and may your AI projects be successful, and may you be, um, you know, on at the end of this year thinking about all the great journeys that you have been on with that. Thank you very much.

This is Nabil LRI signing out. Enjoy the rest of the conference. Hi everybody.

Welcome. We're glad that you've joined us today for another episode of the latest greatest cloud transformation late great cloud transformation. We're talking about really sort of the next generation of how we think about the cloud and the things that we're doing with it.

We're talking about security today, about safeguarding innovation and, uh, strengthening that security where we jumping into app app security and a lot, and a lot of things here. But, um, before we get too far down the road, thank you for joining us for this video series. Uh, the, the last great cloud transformation is sponsored by CloudFlare.

We're glad to have them, uh, on board with, with us working on this, uh, helping input with some topics and things like that. And obviously participating on, on our, uh, live editions, which we do on a monthly basis, as well as these recorded episodes. So thank you for being here with us.

My name is Mitch Ashley, I'm VP and practice lead with futurum Group, analyst firm, uh, heading up the analyst area for DevOps, DevSecOps, application development, AppSec, et cetera. So kind of right in, in vain with this, uh, my co-host Alan Shimel is, uh, unattainable, uh, de detained or whatever the word is the phrase is. And, uh, so I'll be, I'm, I'm hosting both parts of the chair today.

Uh, you know, it's a little bit of a coup, but he'll be back next time. We'll see him on our next episode, I'm sure. So let's get to our conversation, to our topic.

Um, let's first start by doing some introductions. I know Chris has been with us on a few episodes here and some different topics. He'd been on other webinars with me and talking a lot about application security and, and, uh, cloud Chris Blask, introduce yourself.

Oh, I've been a forest company my way through the security industry for 30 something years. Uh, I inflicted an early firewall in the markets, and they called border ware, uh, in the early nineties and ran Cisco's firewall business, the turn of the century. I've been following this inevitability curve and my new series on here on Textron, um, from one spot to another, from firewalls into, uh, sim and network management.

From that, you know, the obvious next step is threat intelligence. So I, uh, chaired an IAC for a while, and, uh, supply chain has been my focus the last five or six years, you know, so, you know, software, bill of materials, hardware, bill of materials. How do we connect all these things, which a, a and, and currently, so currently I'm, my main role is I'm vice president of strategy for sbe, which is involved in the BUM space.

And I've been, uh, co-chairing several, uh, cisa uh, working groups on SBO sharing. So we're currently have a group looking at ISACs, um, as SBO distributors. How does that know in the middle start taking this information and, and propagating it, SBOs software bill of materials?

Absolutely. Great. Thank you Chris.

Um, Catherine. Catherine, welcome. Glad to have you on, I think the first time we had you on the show.

Catherine Newcomb with CloudFlare, please introduce yourself. Yeah, great to be here. I'm excited to talk about application security.

Um, my name's Catherine Newcomb. I live in Denver right now. Um, I've been in cybersecurity for about five years at this point.

Um, and I started in the network firewall space, um, an encryption, and now I'm a product marketing manager for CloudFlare, um, for their application security business, uh, where I focus on their web application firewall product, um, our software supply chain product, as well as our encryption and certificate lifecycle management products. Very nice. And, and I do like to say full disclosure, Textron is a customer of cloud flares.

We do use their services. Enjoyed very much so thank you Catherine, and team for that. Uh, last but not least, another newcomer to our show, Kurt Hendle, who's with, uh, Teradata.

Tell us about yourself, Kurt. Yep. So I've been working in security probably eight or nine years at this point, uh, but in the software industry for close to 15 years now, anywhere from development, uh, into business analysis, product management, even, uh, doing a little bit of red teaming myself.

But, uh, I am currently the chief security architect at Teradata. And so I've been focused on architecture mostly for the past six, seven, possibly eight years, and really kind of a generalist. So AppSec is where I spend the least amount of my time where we focus on the architecture, the requirements, threat modeling, um, especially compliance.

We do a lot of the, the major compliance frameworks at Teradata. So we've been pushing that recently. Um, and I'm based in the Pacific Northwest, up in the Seattle area, and happy to be here.

Very nice. All the weather and fires and it's cold and I'm just glad we all made it. Maybe it's 'cause we didn't have to travel anywhere, so, so I hang tight.

I'm glad we're all here. And you know, our, our thoughts go, our hearts go out to the folks dealing with the fires and, and, uh, some weather down south and southeast, et cetera. So, um, let, let's kind of jump in this way.

Um, it, it, it's a big topic when we talk about sort of the kind of current state of the cloud and where it's moving to. Um, but I don't think it's too much news to everyone that application and app APIs, API first kind of design into applications, you know, it isn't just things that sit at the edge anymore. We think about also the security of the AppSec and the kind of, uh, software we're creating, the innovation that we're making, um, as maybe as part of the cloud.

'cause sometimes application lives within it, you know, like a, like a provider like CloudFlare or certainly at the edge or at the core as well. Maybe Catherine, if you wanna start us out with how do you, you're, you're, you're managing, doing product management in this space. How do you look at this, uh, sort of this problem or this space and define it?

Um, so looking at application security, um, when we're talking about this at CloudFlare, we're mostly talking about web application and API security. So if you're an OSI person, layer seven model, um, and you know, when people are accessing these external facing web applications, they're doing it from a ton of different devices and in a ton of different ways. So they're accessing from things like mobile, uh, desktop, laptop, and they're accessing these AppSec that could be hosted anywhere.

So on-prem, in public clouds, private clouds, hybrids. Um, so as we're securing, we need to think about how can we secure, um, all of these users and the end servers as they're sort of accessing these web AppSec, right? So how do we make sure that, um, mobile traffic is protected, user data is protected, um, and sensitive data is not, you know, leaving an app.

And then how do we make sure that a web app server itself is protected? Um, so at a very high level, that's about what I think, that's what I think about when it comes to application security. Um, some new things we're thinking about in this space.

Um, I talked about software supply chain. This is increasingly becoming, um, an area of interest as people create more complex AppSec with more third parties in them. Of course, API first development has also meant we've had to adjust our thinking a little bit around application security as well.

Kurt, how about you as a, as an architect, security architect, you may, maybe you don't get into the innards of applications per se, application security, but traffic over the, obviously our networks or heavily API driven, um, you know, when you think about the security architecture, where does this fit into your purview? I think it, it fits in really everywhere, right? So we're, we're building these huge applications, sometimes small applications.

We, we do all sorts of scale at Teradata. And in my previous roles, I've, I've worked with pretty simple AppSec all the way to super complex microservices architectures. And so, like Catherine could have said, you have the mobile aspect, you have the server, there's application code literally everywhere, including on the person's device.

And so how do you secure it as best you can, um, within reason, right? Because if it's too secure, it doesn't work. If it's not secure enough, well, you end up in the Wall Street Journal and you're in trouble.

Um, so we, from an architecture standpoint, we really try to focus on all different aspects of it, where the biggest threats lie, um, and then implement controls and use technologies to, to simplify the implementation and streamline it without making it overly complex. And so it's, it's just becoming more difficult given that, um, the, the kind of classic perimeter is gone, right? I'm sure you can relate to that, Chris.

Oh yeah. Well, it was easy back in the day, right now you had to get on the internet and you needed a firewall. Get a firewall, right?

And I'm thinking as Kurt and Catherine, your comments remind me of these transitions we go through. Like there was mainframes before our time, but you know, I, I'm old enough to have seen the end of that where all of your capabilities are just to keep one computer running and run terminals and printers and things off that. And then we get into, you know, sort where I came in, where we're starting to build networks, fractally more complicated, just, you know, how do we do that with, when all of our resources were just keeping one computer running, we figured it out, you know, now we're here, we're talking about web APIs, Catherine, you know, you know, the data going in and out and with being stored 30 years ago, you couldn't have that conversation.

Now we're saying, alright, what do we do in this case? And it's very complicated and I think in, and Catherine you mentioned the supply chain. This is, I think we're filling in the dots.

Security has been, is not, i is not new, right? Been saying you should know your inventory for a long, long time, and we gotten away with not knowing it. Now we're starting to fill it in, need to actually know where the software is, where the data is, and we're working through that.

So it's exciting times, but it's not different in type than other transitional periods. Certainly is an evolution, right, of what we've gone through. And to think about, you know, from the bastion host days early, early on for a free firewall.

Um, Well, firewalls used to be a million dollars a year. I think when I got involved, you know, at least as I tell the story, there were a hundred in the world and they typically were seven computers and a team of people. And my argument at the time was my mom needs one.

Yeah. You know, and so we're at this stage where what used to take so much time in here in the API, uh, world has to take less time a lot. It, it, so let me, let me throw out this hypothesis here.

I think it may be pretty obvious, but maybe it isn't, is I think we live in a world, you know, now we we're thinking about things as zero trust, right? Of, of you, you know, anything is susceptible, being compromised and could compromise other things. How do you pro protect all parts of the network applications, the infrastructure?

But we're also living in a world where if so much is determined by what our applications do, not just connecting users to AppSec, but applications really utilizing the network, being part of the network. It's a dynamic world, right? It, it isn't a good set of firewall rules and an application firewall, and we're all good, kind of set that up.

And it isn't the old days of I've got a pizza box in, in my rack for every function that I need, and they're all doing their thing. I'm good, right? We need it.

It's a much more dynamic environment. So I'm not saying we're reconfiguring our security all the time, but a security has to adapt to, you know, what's happening in the application because we may distribute it to a different part of the edge tomorrow with Kubernetes, or we may, you know, uh, acquire business and suddenly our network has looked much different than it did, you know, three weeks ago. I'm, I'm curious, Kurt, as a practitioner, you know, how do you think about that of, you know, you mentioned microservices and all the things that are being created, you know, in the groups that you're working with.

Um, we, we hate for security to be sort of the last thing to be thought of, but you wanna be in the conversation so you can prepare as well as react when you need to react. Well, I think what you just said is, is really important that you wanna be in the conversation. You don't wanna be doing this retroactively.

And so when you're, when you try to tackle security retroactively, it is infinitely harder to accomplish than if you do it from the beginning. So I have, I do it both ways. I have teams that we work with proactively where they bring us in at the very start and we're building the design with them shoulder to shoulder, drawing the picture in doing security by design or by default as we like to say now.

Or we have legacy applications, which you're doing retroactively and they're quite a bit higher in terms of risk because they've been neglected for so long. Or you find out about something after the fact and it's like, well, how did this get out there? Well, there's shadow IP in a lot of the world.

And so it's, it's hard to, to really kind of put a, a recipe together that successfully achieves it. And then with the, the rapid pace of technology today and how the cloud has just kind of blown this wide open where people can deploy new applications in a hundred different ways faster than ever. How do you keep up?

So you have to implement tooling within reason without doing, without having too much sprawl. You have to have the right personnel partnering with these teams, uh, to ensure that you have coverage and that you, you're really architecting things from the start. Um, and not just kind of using band-aids in bubble gum per se, to, to secure your environment later on.

Catherine, appreciate your thoughts on this because, you know, I remember the days of networks for speeds and feeds and points of presence and connecting A to B and kinda looked like this nice diagram that you stitched together and that was a network and you secured it, now it's overlay on top of overlay and it's changing and mm-hmm. You know, it's, it's multiple pieces that, uh, much more complex to, to secure. How do you, how do you have this conversation with people?

Yeah, definitely. So as you were sort of talking about this, you know, obviously there's a need for responsiveness and customizability and security, but I actually also wanna make the argument for unified policy management in application security. This is something that I've seen actually, for example, um, we have some customers who have protected their SaaS AppSec, like what is traditionally more of a network firewall or zero trust type use case with the same policy they're using for their web applications.

And by doing this, they're able to do things like make sure that zero day exploits aren't able to exploit their SaaS AppSec, you know, as well as their, um, web AppSec. And we see a lot of value out of these unified policy managements. I was talking earlier about, you know, how we have all these AppSec hosted in different places.

We see a lot of customers, for example, will host, um, you know, an app across multiple clouds for like a resiliency use case. If they're worried about outages, you'll, you'll certainly see that, um, for example. But then how do you have to, you know, actually secure an app that's stored in multiple places?

Do you write different policies for, for wherever those are stored? Um, do you write different policies for APIs versus, you know, traditional AppSec? Um, so we see a lot of benefit out of like a unified policy for all of those disparate sort of endpoints and all of those disparate, um, locations that they're stored.

Uh, for CloudFlare in particular, how this sort of works out is our WAF is like the backbone, the architectural backbone of the rest of our application, um, security portfolio. And this works out really well because you can do things like have a WAF and an API like positive security model protecting your APIs. Um, so you could do things like detect zero days and volumetric attacks, which are, you know, APIs can also be susceptible to as well as, you know, do the things like Ebola and, and all those API specific attacks all within sort of one, um, control plane, which we find a lot of people get a lot of value out of because of this really, really disparate environment.

Okay. Chris, I saw a lot of hand waving head nodding, I jumped outta your chair on this one and so I kind of have feeling you might resonate with this. No, um, I, I gotta throw out there, I was gonna, uh, before Catherine got into the, the policy thing that's swear been a my time, but yeah, the concept of an SBO m the software bill of material for the current release version of Adobe Acrobat as opposed to an sbo om for, as we're look talking about here, some ephemeral web app that one time for five seconds exists in the cloud.

You know, think about that. How do we, how do we deal with that? And I, and, but I think policy is, is the answer all hacking?

All hacking is policy hacking. I will figure out how you do things and I will figure out where the gaps are. I don't wind you near that gap.

And we live in a world right now where we generally have no idea what policy applies to any of us anywhere with few exceptions. And in this topic, and because I'm used to the supply chain topic, imagine I needed to get the, the SBO M or custody information about a piece of software on his phone right now, I could get it in between five days and six months. Today, I need to get it in a half a second.

That means I need to read the policies between me, the person who bought the phone and the first time the company I bought it from, and like their relationship, their contracts, their policies, you know, upstream all the way. And we have to get that done in the next decade. So without unified and, and, and adaptable, you know, transparent policy frameworks, none of this technology is gonna make a difference.

So I think we, we will do that. And there's interesting things going on down that path. It's kinda interesting in a way, just connecting dots between what you said, Catherine, and you were talking about Chris, there's your own unified policy management, right?

Of what you're doing. So, you know, you're, you, what you're applying where and how you're applying it. And then that's how that interconnects or interrelates with the people you connect with, work with, use their service product, whatever that is too.

And I, and I appreciate what you said Chris, about, think about just serverless technology like a lamb to kind of service, right? That, you know, it's there now, it's gone tomorrow may not be the same thing. It was a second ago when it, when it ran.

Um, so it in some ways, Catherine, it's all sort of a dynamic unified policy management, right? It can't be a static thing. Am I, am I on base here?

Yes, of course. You know, you do have to be responsive to the environment, um, you know, threat landscape. Um, this is one, one area where I strongly advocate for actually ML driven, um, detections and policy.

Uh, this is a thing where, for example, if you have a really large data set, uh, you can train your ML models. Um, how we do this at CloudFlare, just 'cause I think it's a little easier if I give an example and it's, uh, we will score each request on a scale of like one to 99. And if something is less than 30, that means like it is very likely to be an attack.

And because we have, um, hundreds of terabytes of requests, or, sorry, hundreds of millions of requests every single day, um, we have so much data we could train this on and say a little blog in Malaysia gets attacked by a new attack we've never seen before. Suddenly, because that tiny blog in Malaysia got attacked that gets feed and fed into our ML model. We don't have to rely on a security engineer to like go and find and analyze that attack and turn it into a regular expression like firewall rule.

Um, the ML will basically just say, okay, like, since it matches something like this, um, we will just automatically block it. And this is why I'd say ml um, sort of combined with that traditional, um, you know, security analyst looks at the traffic and writes a rule that matches it and then blocks traffic. Um, you gotta combine I think these types of approaches.

So ML is a really, really great application, um, when it comes to being responsive to the threat landscape. And we have some data around this as well. Um, we recently, not that recently, like half a year ago released our annual application security trends report.

Um, and we found out that, uh, for example, like zero day vulnerabilities, um, we probably wouldn't have been able to find this out with just security engineers analyzing it. But with our ml, we were able to detect, um, and exploit 22 minutes after the, uh, proof of concept was posted online. So, um, really, really great applications there.

A lot of interesting stuff going on for sure. Well, that doesn't make the case for dynamic security. What does, right?

Um, I, I'm curious, Kurt, how do you, is, is someone, you know, applying these things, applying security? Are you, are you looking at things like ml are you doing in via yourself? That's something you look for in the vendors, the partners that you work with.

How do you leveraging either that or the kind of technologies to help shorten that cycle between when things change and how you can account for it and secure it? Right. The, I think the ML piece of it is, is hugely important because I mean, humans, we're slow.

The, the technologies we use, the, the computers and I, each servers process all of this far faster than the human brain and I ever could. And so we need to augment ourselves with this technology. So anytime we're evaluating new solutions and bringing them in, like I'm currently in the process of implementing a big one right now that focuses on platformization and ai, ml, it's all part of it because in humans with eyes on glass, like it's great to have those guys in the sock, but they'll get overwhelmed very easily with the speed at which things happen today.

And so we need to leverage technology and machine learning enables us to do this faster than ever. And it's only getting better, right? And so augment the human with that technology and you can very quickly pare down all of that information to what matters most and focus on real attacks like Katherine was just talking about.

I wonder, you know, there's so much activity around ai, of course, a lot of it because of gen generative ai, um, Chris to do security engineers have to become machine learning experts to be able to do this stuff. What does it take to really leverage it? No, but knowing, knowing something isn't gonna have, um, uh, causing any problems.

But, uh, I, I just gonna agree more with, with both, uh, with Kurt and Catherine. 'cause you know, and, and you're point Kurt, it's all about time, time the transparency. How, how long, and again, I've seen this over and over in my, my career where we get to these points where what we're mostly doing is sharing the war stories.

You know, I have no idea it was 72 hours. None of us slept. There was caffeine.

And, and my my question always is, okay, if there was twice as much, what would you do? Because obviously that we're at the limit, we can't possibly work any harder to stay awake any longer. And, and this, yeah, ai, ml, Oracles, whatever we call it this, um, my, a big has been a big part of my, uh, my focus on supply chain before it would, you know, AI became, you know, uh, uh, general, um, uh, generative, what the hell we call it?

I'm sorry, I forgot. Yeah. Ative ai.

Yep. Generate ai. Yes.

Uh, too many terms to Throw Around. Yeah, because again, we need to, you know, just for supply chain things, I need to read the contracts. I mean, I can literally call someone up, you know, it's not a security engineer, but it's some administrative person to the company, and I had to get them on the phone and get them to pull a PDF and read the contract and find out if the clause allows me to get the information I need.

That's not worth a human's time. I mean, that's the kind of stuff that computers can do really well, and they're just beginning, but that's obviously the direction we're going. And if you can't see your policy environment five years from now, by various definitions, your competitors will be so much faster than you are that it won't matter anymore.

Kurt, I'm, I'm curious, without giving us too many specifics about Teradata and not asking you for that, but what's your sense of, what are the, what are the new priorities that are on your Yeah, on your horizon or things you're dealing with now and that you've kind of added in the last year or so? What's changed about how you're thinking about security and that you've gotta address now? I think there's, there's always classic problems that we, we have to deal with and tackle.

Like, we can't forget things like identity and network security and the rest of it. But the, the prevalence in the emergence of generative AI and putting AI and machine learning in everyone's hands has meant that security teams have to be hyper aware more so than ever because these new technologies, people are latching onto them without considering the risks. They're like, that's awesome.

I can speed up everything I'm doing. And suddenly you see a news story about, well, what was it like Samsung engineers leaked their code through a generative AI solution or whatever. So you're, you can quickly lose intellectual property or put it at risk.

And so we have to think about securing our environment for those solutions, or putting the guidance out for people to use AI and machine learning. Um, and I mean, getting visibility of all of this, and another big one that's been getting pretty popular and we're seeing a lot from different vendors and acquisitions and whatever, is data security, posture management. Where is my data?

Where is it moving? How secure is it? Because at the end of the day, that's what the attackers want.

They don't wanna sit in your network and use your resources to, to launch attacks as much as they used to. They wanna grab your data, steal it, monetize it. So need, we're, we're focusing on data security big time in, in the more recent years, especially, um, forward looking because we have more data than ever.

Interesting. Catherine, from your perspective, you know, communicating with so many companies, what are some of the changing priorities from your, from your viewpoint? Yeah, I mean, certainly the gen ai, um, piece is something we're seeing a lot.

Um, everybody wants to put in an an LLM on their web application. Um, and of course that means that you have to think of that as like a data security concern as well. Um, because you wanna make sure your LLM is not gonna like accidentally leak somebody else's social security number, because that's certainly happened before.

Um, and so at, at CloudFlare we're thinking about this of like, basically how could you basically just put a WAF in front of an LLM, um, from that perspective, how could you prevent it from exposing sensitive data to the end user? Um, but then, you know, you gotta think about these more complex issues as well. Like, how do you prevent somebody from poisoning the model?

How do you prevent, um, you know, some of these other, like, how do you prevent it from hallucinating? Uh, these are all, you know, sort of adjacent to security concerns. But, um, but nonetheless, we see some security teams focusing on this, um, increasingly.

Um, additionally we also think about, you know, the, the LLM sort of security use case as a little bit of a just, um, increased API security use case since a lot of times, um, people are not building these LLMs themselves and hosting them themselves. They're often, you know, bringing in LLMs from third parties, which, uh, necessitates, um, APIs, right, for integration. So how can you make sure that these APIs are staying secure and not leaking them back to the host and whatnot.

Um, so that's definitely something we're seeing as well. Um, I would say additionally, one thing I've been hearing a lot lately is, uh, software supply chain security. Um, I think Kurt mentioned the beginning, um, sort of securing code that lives on the client device as well.

Um, this is something that we've been hearing a lot about, especially as it comes with the PCI four, um, compliance, which is gonna be mandated at the end of March, um, in a couple months. Um, PCI four has a new compliance requirement around client side security and securing, um, the client side, like software supply chain. Um, so this is something we've been getting a lot of questions and inquiries lately.

Um, you know, how much are organizations responsible for, um, the code that loads on their end user's devices, uh, when they visit their websites? Um, this is something we are seeing a lot of people trying to actually actively get control over, um, and make sure that they're not, you know, serving, uh, code to the client devices that could do things like download a crypto mining software onto their phone, which, um, believe it or not, we have seen somebody's trying to make, you know, personal laptops part of a crypto mining network, which is pretty crazy. But, um, so yeah, I would say the client side component is, is something I've been hearing a lot lately as well.

I, I just have to say, I, I love living in a world where we can use the term, uh, you know, hallucinating artificial intelligence in a conversation like this. Seriously, just, we, we understand about that. It's not a sci-fi movie.

It's real. Oh, It's, it's real. Yeah.

Hey, so I've, I've kind of a left field question for you, Chris. So if this, if I throw you too far off the track, I'm guessing you're thinking about this though, is, is there an SBO m in our future for LLMs and s SLMs and all of these things? 'cause in a way, this is a whole nother part of the software supply chain, right?

We're handing off to something that's doing inferencing, either on a chip on our handset or in the cloud, all of the above. How does that fit into, do we need to be thinking or at least wondering how we're gonna solve this problem And not only in not left field and that's, that's right in the middle of the, the track. So in short, yes.

You know, there's ano there's another assistant working group, um, Dmitri Rayman, uh, my colleague CTO at at SBE is, uh, a co-chair now on, on AI bomb or an AI bomb has been talked about for a long time. So what does that even mean? You know, so AI is code.

So there's this, you know, same sort of standard SBO stuff about that, but there's also the training data and the models are produced, right? And this sort of goes back to my last comment about ephemeral, ephemeral SBOs. You know, we start with the idea that I am a software provider and every 16 years I release new code and I carve a new SBO m you know, on purist graphite.

Um, but we live in a world where code gets compiled and used all over the place. You know, how do we even look forward and say that I can commit to a policy that says I will, if asked, provide the contents of this code without, um, actually going out and printing or saving or producing quadrillions of SBOs forever, you know, in, in exabytes storage. Uh, so this AI is, you know, what we're currently calling AI is just another forcing function of the level of complexity we're at.

So we need to be able to provide the answers to live up to the policies that we've agreed to, um, which is, you know, you know, in the SOM case we're talking about a software inventory that I will be able to tell you what code that was running or you know, what data set was used, and we have to get there. And, and, and it's, it is reasonable progress down that path. It's a, it's a complicated one that is very similar patterns to how we'll do other things of similar complexity.

Um, Kurt is, is that on your radar yet at all, kind of thinking about security of, from a supply chain for LLMs and AI and ML algorithms and all that kind of stuff? No, I mean, it's, it's certainly jumped up on the radar, especially since the whole SolarWinds thing happened. Um, as Chris Blask talking, it got the wheels turning in mind of, well, if we're gonna be kind of, we're moving towards leveraging a AI in the sense and dynamically generating SBOs and things, is this another attack vector we potentially have to watch out for?

Is how do you weaponize that and, and protect against it? Because I mean, as we see attackers evolve their tactics and techniques faster than ever, they're coming up with new creative ways that defeat the traditional approach in microseconds. And so how, how do you stay ahead of that curve now?

And so I obviously, I don't have the answer right now, but it's, it's really interesting as Chris talked to start thinking about this, this new sort of problem that we're facing. And again, it all falls back to the rapid evolution of technology. Yeah.

Speaking of that evolution, uh, just in the last week or so, uh, Satya Nadal, the head of the Microsoft was talking about the death of SaaS, meaning that's kinda the clickbait liner. The, what I think he was really talking about is e evolving nature of software architecture that I would describe it as. Today's microservices or backend code are tomorrow's AI agents, right?

We'll see more and more parts of AppSec built through, you know, with or through or maybe completely with AI agents. And it reminds me of going into the, uh, cloud native era of, oh, how do we secure microservices now that we're gonna do that kind of thing? That's kind of the, that's the next edge that we're, we have to work on and think about how, uh, there are different things we have to do for securing AI agents.

How are the orchestrated? Is it Kubernetes or it's some other thing that's managing all those things. And, uh, given that we're putting AI agent building capabilities in everybody's hands, in many cases, it, uh, could make for interesting.

I use that in a nice way, uh, interesting environment to try to secure and manage. So in some ways, the future is bright, but it may be, uh, pretty intense at the same time, same time. Well, and I think kind of building on that too is the, the technology behind ai, it's backed by machine learning.

Like you're, you're making technology autonomous, right? So it's not as predictable anymore. So how do you secure what, when you don't exactly know what turn it's gonna take next, Non-deterministic, right.

Well, I, I gotta add a note, a no of hope though, because it's easy, you know, to your point, uh, Kurt, the short answer is yes, because there's a new attack vector. Oh, yeah. Um, but, you know, you know, throughout my career I've been arguing this one, it's like, we'll probably keep the lights on.

It's like, no, no, if we don't do this and that, then, you know, we will, you know, the, the, we're on this, we're doing this call right now. We've managed to figure out everything else up to this point. And not only that, but I think that we're, we've been mowing the lawn, and I think, you know, what we need to do, generally speaking in cybersecurity has been known maybe forever, certainly 50 years, but we haven't gone around to doing the vast majority of it yet.

'cause we haven't had to. But as we do, and I, I will take a risk and, and put a lot of my, my faith in policy, you know, in, in real policy transparency, you know, in, again, in this decade, it gets harder to be an adversary because, you know, these are the happy World War II fans out there, you know, or no fans, you know, but the, the ubo wars, right? There was the happy days when you could just have a U-boat and sink shipping all day long.

You know, that's kind of most of the world, most of the, the history of the internet to date. It's not necessarily gonna just stay that way, that long forever, where there's always a new attack service, and there's always a, a, a new way when the last one is, is locked. I think we will, we'll keep it running.

We will all be fine. And I think over, you know, at least over a period of decades, being an attacker will become much, much more difficult. I mean, I might argue it already is becoming more difficult.

It 'cause the, while, while the, the technologies we use as practitioners are getting more advanced, that helps make it more difficult for the adversaries of the world. That's not to say that they can't employ similar technologies, right? So now we're kind of, we're creating that chicken and egg problem all over again and playing a game of cat and mouth.

It's kind of the next arms race, if you will, as technology evolves, everybody has access to it. Well, let's do this. I appreciate all the conversation, and we brought up a number of topics, um, just as a kind of concluding thought.

Uh, we, we've been talking about what are the things we need to be thinking about? Maybe they're newer, maybe they're on the horizon, maybe already working on this today. Um, if you had to say, there's one thing you'd really want to emphasize this, if you were, you know, somebody who's listening to this and maybe making a few notes, the thing that sort stands out to you as something really important to be thinking about in the next, let's say, six to 12 months, if not today.

Um, Kurt, do you want to give us your thoughts? And then Kathleen, if you would, and Chris, you can wrap it up for us. Sorry, did I say Kathleen?

I mean, Catherine, excuse me. Kathleen. I work with a Kathleen.

Sorry if I've been doing that. Oh, good. Okay.

Yeah, I, go ahead, Kern. I mean, it's, we wanna avoid that situation where everything is a priority, so nothing's a priority, right? I think we, throughout this conversation, we've highlighted the importance of esmo is we've highlighted the importance of application security and how it's, it's becoming more important than ever because our application code is, is literally going everywhere.

And that's, that's kind of the gateway for a lot of the attacks we're seeing in the world today. And so I think the, the emphasis is on application security, but it's also to say, let's not forget the rest of it, because all of the, the other parts of cybersecurity are hugely important. And we still need that visibility.

We still need the coverage, and we need to be thinking about ease of use as well, and avoiding the sprawl. So I know these aren't necessarily specific cybersecurity things, but they, they help you simplify your approach and, and focus on what matters. And that depend that that changes everywhere you go.

Every enterprise or company has different priorities. And so I think focusing on those things help enable us to, to focus on what matters for where we're at currently. Good.

Catherine? Yeah, so I mean, like Kurt said, you know, we wanna make sure that we're not making everything equal priority. So I think when it comes to application security, which is of course, my area, what I would say is most important in this space is visibility.

Um, the attack surface is getting more complex, applications are getting more complex. Um, you know, where they're hosted is getting more complex. So how do we actually have visibility into our entire, entire application attack service?

How do we have visibility into the APIs developers are creating so we can actually secure them? How do we have visibility into the software they're adding, um, to these AppSec? Uh, that I would say is probably the most important thing for application security and also one of the most challenging things.

Excellent. Chris, Uh, you know, Kurt and Catherine both want exactly where I'm going, so I'll just build on that. You know, do do things that save you time to transparency.

You know, if you, you know, don't panic, nothing's on fire. And, and when things are on fire, panic less, right? Just take your time and, uh, getting visibility, you know?

Yeah. Look at how long it takes you to figure out. And anytime you find a, a, a way, you know, in this, in this topic we're talking about here, to spend less time to figure things out, you have all that time back to do things.

And it's easy to just, you know, particularly in transitional periods, to just do more and more and more of what you've been doing, you know? But, uh, understanding the environment you're in so you can apply your resources appropriately is, is everything. And there are lots of ways to do that these days.

You know, there's, there's a lot of Russian panic, and there are a lot of, and you know, I will say it, AI and things like that out there who will actually make your life easier, give you some of your time back. Mm-hmm. And you'll point, feel better knowing what's going on, make and make better plans, a better strategy.

Yeah. To that point. Exactly.

Chris, and, and Catherine mentioned it around, uh, ml, some of the things that I'm really excited about AI is actually just the understandability of what's happening. You know, Kurt mentioned about as things ramped up or, or you did, uh, uh, in, in the, if the tax doubled, right, how would we handle that if we're already maxed out? So some of it is just handling the volume of things that are happening.

But I think one of the things that I think is most exciting about generative AI is it's also so complex. No one person can understand the full system, right? Or maybe even understand truly what's going on in a case of an attack or where you have vulnerabilities.

And generative AI is starting to make some inroads and helping us understand systems and, and giving us some insights to some of the complexity. We may not be able to fully get into our head all at once. So, for example, I've been doing some work around how do you modernize mainframe applications?

Well, nobody was around that built those things. Well, maybe people that built the network aren't even around, right? So help us understand what really is happening with all this data that we've collected.

And the natural language interface through that is, is a great aid. And I think it's just a real practical thing that we can start to begin to use today. So don't think of AI as just as the next, you know, it's gonna replace all of our software and it's all gonna be different.

And what do we do? There's things today that is already helping us with. So, you know, there's some real things too, not just what's on the horizon.

Well, thanks to all of you. It's been great, Catherine. Uh, we appreciate your perspective, and Kurt, you're bringing, um, your experience and perspective.

And of course, Chris, always good to be chatting with you and your connections into the security world. And some of the folks are working, collaborating together, which by the way, is another superpower we have in security. And that's the fact that we work together and collaborate on, on these things.

We're not going at it alone. So thank everybody for their good work that we're doing to help advance. We hope this has been a helpful conversation for you in thinking about the, the last great cloud transformation, what we're doing differently and thinking about, uh, as we move forward.

So as we've got our heads down, getting stuff done, getting our priorities done, getting our plans in place, and executing for 2025, but also kind of thinking a little bit about what's next and what we might be considering and learning from others that are working in our space. So thanks to all of you. Thanks everybody for joining us today.

And thank you to the Cloud four team for, uh, for sponsoring, um, our show today. And we look forward to joining us either on another recording or be sure and check the calendar for one of our live events where folks can ask questions and engage with us in a similar kind of conversation. We have many of those coming up.

We'll talk to you again soon. Take care, everybody. All right.

Well, thank you guys so much for coming to my talk. I'm talking about, uh, when Agile doesn't work, um, and just replies to common objections to agility. Um, so I've been doing work with software for about 12 years.

Um, I see here I said primarily in agility. 'cause I've been an agileists, a scrum master for, uh, several of those years, but I've kind of dipped in and out of several different roles. Um, I've been in product ownership for a while, a little bit here and there.

I've been a business business analyst for a while. I've done some software development, so I've kind of been all over. Um, and so that's kind of where I've been.

Majority of my work is in agility. So with that, um, that's where a lot of my, um, experience comes in. I have some, some stuff to say about, um, you know, how agility works.

So, um, and this is a picture of my wife and my child, just so you guys kind of know who's talking to you a little bit, a little bit of connection there. My, um, my child is here, he is, uh, three or four, I think, but he's, uh, seven now. Um, just about ready to turn eight.

So he is gotten a lot bigger, but I just really love this picture. So I just wanted to share that with you all. Uh, so yeah, there you go.

So what I wanted to hear today is, um, like I said, I've been here, uh, several years, and especially in agility, and I have met a lot of people that have a lot of stuff to say about agility and why, you know, it doesn't work or why, you know, it might be good generally, but hey, it doesn't work here. And, um, and so really walking through those arguments that I've heard, and then kind of some things that I've experienced along the way that kind of serve as a counter argument. Um, so, so I've got several slides that kind of have that same format, just the, the argument and then kind of counter arguments against that, so to speak.

But then it's the next part is how to engage with people. Um, because when, um, when I first gave this presentation several years ago, that's one thing that I didn't really include. And I feel like now I really need to include some of that, because as I was going back to these slides, it felt a little bit like, Hey, here's why I'm right and why you're wrong, kind of thing.

Like, Hey, agility works. You're wrong. Here's some counterarguments to prove that it works.

And like, no, that's not really the point. That's not, I'm not here to try to be right. I don't think anybody here, hopefully is here to be, right?

I think we're trying to do right for our business. Everybody cares about the business. People that don't like agility, they care about doing things well.

Um, and we care about doing things well, um, as agileists. Um, and so really just wanna make sure, like as an agile analyst, I, I believe agility is correct. I believe it's the right way of going about doing things.

Um, I wanna just talk about how can we engage people in a way that's effective to help you under help 'em understand the benefits of agility. So, um, but yeah, first talk about the things I've heard against agility, so we can kind of walk through them. So, um, I'll go ahead and start down that path here.

So one of the things that I first heard when I was starting my career was that Agile isn't really known for quality. We really need high quality. And again, some of these things you may have heard and then other things you may be like, no, I haven't even heard that, because it, it just doesn't seem true at all.

This is one thing I have heard, and just as a caveat, this is really something I heard when I first started out. Uh, and the company I started out at was a logistics company. Um, and they were smaller company, but they were hungry.

They really, really wanted to just go after growth and go after all these new clients and grow as fast as they can, as strong as they can. And, um, and so they had a backlog of a ton of things that they really wanted to get done on our, on our products. They wanted to add a lot of new features, um, a lot of really good features we were all excited about.

Um, but they wanted to move so fast, and they were like, Hey, we gotta move faster. Like, let's, let's try this agile thing. Like, let's get into this whole agile thing.

Let's really step that up so we can go faster. And so, um, a lot of the dev team that I was on, they kind of equated, oh, agile is that thing where you just push so fast and so hard, you just neglect quality along the way. And so this is what I heard at that point, well, agile isn't known for quality.

We really need high quality. We can't skimp on quality as we build this thing. And, um, you know, again, the the counter, and again, all these things, as I mentioned earlier, I wouldn't just say, here's the counter, like, this is why you're wrong.

But I want us to understand from our perspective, um, kinda what the counter is in ourselves so that whenever we do engage with people, we know what's on, what's going on. So the counter that I have for this is that without exception that I've seen, like, this is true when I gave this talk, and it's, it's also true now. Um, but I first get this that is, is that without exceptions, teams that I have personally seen, do Agile have been of much higher quality than the ones that have stuck to Waterfall, um, just due to, uh, various factors.

Um, and I mentioned Scrum as well. I here as if you're using Scrum. Uh, so if you're using Scrum teams can specify some quality gates in the definition of done.

So those of you that use Agile, most of the time you are using Scrum. Um, if you don't know the difference, uh, agile is a very broad term. Uh, there's, there's a manifesto out there that has just four things that describe what Agile is, and then 12 principles, and that's all there is.

You can read that in three minutes and you'll, you'll understand to some degree what Agile is. Whereas Scrum is kind of one layer above that. And it's a framework that is a little bit more intense.

It's not, it's not fully intense, it's only 22 pages, but it's definitely more than one page. So it's a little bit more than Agile. And, uh, if you're unfamiliar with the Scrum is, although I can't imagine that because it's so widely used, but Scrum is where you kind of take a specific measure of time.

Like two weeks is pretty typical. You say, for the next two weeks we're gonna make a plan for these next two weeks we're gonna do that plan. And then at the end of that two weeks we have this thing, we've built this piece of software that gets integrated into our app and that piece of software we're gonna test, we're gonna, we're gonna review that with people, we're gonna review how we did our work, and then we're gonna start the whole process over again.

You're gonna keep doing that time after time after time, um, until you build something great, you know, incrementally, uh, one iteration on top of another. Um, so when you do that, uh, one thing is really important is when you get to the end of that, of that time box, so to speak, and at the, the end of that two weeks, you wanna have that thing that you've built quality. You wanna say, okay, we cannot call this thing done.

We can't call this thing done unless it meets our criteria for what done even is. There's a bunch of things that we have to have in place if we wanna call that done. And, uh, that's called the definition of done in, in Scrum.

And so if you have that definition of done, you can add anything you want into it that you and your team know to be a good quality gate. So, for example, you can be like, I don't want this thing released until it's fully tested, a hundred percent code coverage. I don't want this thing out until QA has given, uh, has done a full aggression test.

I don't want this thing out until, um, our, our, um, our PO has, our product owner has done their due diligence and looked over the whole thing. Um, and whatever else there is, uh, out there that's gotta go into your definition of done. So you can make sure you say that thing can't pass our, our can't pass into the, into, um, into the next stage until it meets that definition of done.

So in that way, yeah, that the quality is a very big part of Scrum at that level. And then the last part I just wanted to to touch on is that quality is actually, so it's something that's a good practice, whether it's agile or Waterfall, it's not necessarily that like, well, because you're agile, you're less quality, or 'cause you're waterfall, you are less quality. Like these quality gates are important, you know, whatever you do.

Um, I think it pairs better with agility though, as you see there at the bottom. And the reason I say that is, is really because of the iter iterative nature of agility. Like if you're spending two weeks doing something and then testing it, and then even if you release it within that two weeks and then you release these small chunks of work, um, you only find a few bugs at a time and they come out iteratively and incrementally.

Whereas if you have waterfall, you'll spend months and months and months building something. I've seen this been months and months and months building something, release it, and then a plethora of bugs come in. And that just doesn't seem quite as tolerable to users that I've seen.

As you release something, a few bugs come in, but you fix 'em in a day or two, and that's like, okay, that's fine. Waterfall usually something after years, and then it's just buggy. And people are like, well, these guys built a buggy app.

So, um, anyway, that's why I think it pairs better with agility. But either way, uh, quality is not something that I would say is anti agile or Agile is not anti quality. It's the opposite, um, one I can see.

So that's the first one. Uh, another thing that I've seen a lot, or I've heard people say a lot is that, um, you know, we developers know best. You know, we've built the app, we've done the coding.

We, we know what this app really needs, but, uh, we lose our voice. We give it to the business. The business just continually just in Agile.

There's a product owner. The product owner tells us everything that we need to do, and we have no voice in that. Um, and uh, that's absolutely not what Agile's about.

Um, in fact, agile, one of the 12 principles of Agile said that one pager that kind of tells everything that Agile is one of those 12 principles, is that business people and developers must work together. Um, and I, I'll keep on reading some of these things here, is if the product owner is too focused on one side, so if they're too business focused or if they're just a really technical product owner and they're just really focused on the technical side and they don't really prioritize some of the stuff that the business is asking for, um, then that's a good opportunity for you Scrum Master to get them, get involved and, and, and help the product owner and say, Hey, let's come up with some kind of a balance here. How can we make this thing work for everybody?

Um, and the really key things are the bottom two, being open and building trust. So as a dev team, you don't wanna cannibalize all the features. Kinda like I was saying earlier.

There needs to be some kind of a balance there. I have seen both sides. I have seen, uh, the business come in and say, Hey, you really need to get our features done.

This is important. We, we really need to get these done. And devs like, no, we can't get these done.

We really gotta work on these quality things. And then the dev side actually gets to the point that somehow they're bullying the other side and saying, no, we are, we are doing all these refactors and you're getting nothing. It's really been kind of toxic and there wasn't a lot of trust built, but that's what's really key.

Building trust, being open product owners need to understand the value of the technical stuff you're doing and being able to understand the value of the business stuff you're doing, and figure out which one's are true priorities, um, and keep a good pulse on that. So that's, that's one very important thing. Okay.

Uh, next we have, uh, something that I hear quite often. I've heard it pretty much everywhere I've worked, which is that what do you do when you finish your sprint early? You know, so they say here, sometimes we finish a sprint early, what do we do now?

It just seems like wasteful to do nothing. We're just sitting there. We, we planned, uh, a sprint, so we're gonna do this work and two weeks and we're gonna take two weeks to do it.

Well, we got it done in a week and a day. We have four days left over. So now we're just sitting here.

Well, the first thing I just, I just wanna get this outta the way. I didn't even have this as my first point, the first time I gave this talk, but I really, I added it because it's, it's, uh, something I feel like just needs to get outta the way, which is you don't have to do nothing and you shouldn't just do nothing like that. Just that, that is a little wasteful.

So you're free to get a head start on the items in the backlog. So just because it's not in the sprint, you can go to the backlog and find some things to do and, and get started on those. The only thing I would say is a caveat is one of the benefits of having a, uh, a sprint is that you all talk about it, you plan together, uh, you know everything there's know, uh, about it 'cause you've planned together.

Um, and there's, there's conversations that happen along the way, but generally you have an idea of what you're doing. Um, so if you do just wanna do something in the backlog, make sure that you talk it over with whoever needs to be talking to make sure that the, the product owner knows you're doing it. That, um, that it has all the details.

Maybe there's something that the product owner wanted to put in there, didn't get around to yet, or whatever. Just make sure that there's some clarity that like, I'm doing this, okay, I got, let me, let me go ahead and get started on this. But generally, you're free to get ahead, start the point of the, of the, of the time block of the, of the sprint, um, isn't to lock out things that you, it's just to, to plan what you want to deliver.

So if you've gotten that done, great, let you know, do other stuff. That's all good. Um, but I would go on down the next point here is what is waste?

So in my earlier career, I did a lot of work with, um, with Lean Six Sigma and in the Lean Six Sigma world, waste is an incredibly well-defined, strictly defined term. Like if someone were to say, what is waste? From a lean perspective, you can go, ah, I know what, I know exactly what waste is.

In fact, there are eight different kinds of waste. And they form a, uh, an acronym. They form the acronym downtime.

You have defects over production, um, waiting, um, not engaging employees, and you can keep going down. And, and the point of that is that you can, uh, and this is from a manufacturing standpoint, if you can walk into like a manufacturing facility and you can just look around and you can say, oh, there's inventory over there just waiting, that's waste. Oh, there's this person over here kind of just leaning against, that's waste because you trained to kind of use your eyes to see waste.

And it's a little bit different for software. Um, we, we don't exactly, uh, have the ability to kind of just see inventory in a corner. Um, but we do, uh, as a, as a kind of a, a caveat, we do see like Kanban boards and stuff that is, is waiting in a Kanban boards, you can see some of that inventory in a digital stance, excuse me.

'cause every, every ticket that you have in your board, uh, whatever column it's in, whether it's to do in progress, those tickets, um, they represent some work that went into it. So people met to talk about those tickets, right? The business or the product owner met with some stakeholders and met with, I don't know, business analyst, whoever it is, I don't know.

But, but somebody met to build out that backlog and put a lot of effort into that ticket that is just to most people, just little couple lines on a board. But that represents a lot of time and effort. And when it's just sitting there on a board, it, it provides no value.

The only way that that value is realized is when that ticket gets developed and makes it into production, and then they can start realizing the value was created for, until then it's not producing any value. So there is legitimate waste in the sense of it's sitting there producing no value, but work went into making that. Um, so that's true, but within the context of a product, waste is a bigger problem out there.

And that bigger problem is what if you spent all this time building the wrong thing, right? Like, what if we had the product owner, the, the business analyst, the developer, all these people meet, discuss, and build out the requirements or the tickets or whatever for this thing, and they went into the backlog and it went into development and someone spent a lot of time developing it. Then it went into code review and someone spent a lot of time code reviewing it, and then it went into, uh, qa and someone spent a lot of time QAing that.

Then it went into user acceptance and, and your pos and a lot of time, you know, you're doing that, and then it was released and all this work and all, who even knows you were to just measure the amount of touches that that went through and the amount of, uh, of billable hours that went into that. Who knows how much that would have cost to get that through. And then you get into production and nobody uses it, or it provides no value, or it didn't do anything near what it was supposed to have done.

That is the biggest kind of waste, which is building something that's just not valuable, um, building the wrong features the wrong way, the customer doesn't want it, whatever. And that is something that Agile is really focused on, is particularly have, scrum is heavily product focused. It can, and it, it is, it does tolerate a little bit of process waste, you know, like, it, it, it basically says that the most important thing is making sure you're delivering well, right?

That you're iterating, you're getting feedback on stuff, and then feeding that back into your product and then building the right thing. Um, so that is something that Scrum is heavy, heavily focused on. Um, and maybe there's a little bit of process waste in there and that you have to wait a little while here and there, but considering what it's trying to prevent, that's a tolerable, in my opinion, um, a tolerable, um, thing to, to take on.

But if it is a major issue, if you're like, I can't tolerate any process waste at all, no process waste, total waste free. Um, there are other things besides Scrum, like we have Kanban, like you're from a Kanban, it's like Scrum in some ways, but that has got, um, uh, there is usually you have like a, like a board with like to do and progress and different kind of columns, but you don't have a sprint. You just kind of pull whatever's next on the top of the, of the lift and you do it that way.

So whatever's the top is good. So if it truly is something that's important, um, you know, there's always the ability of using something like Kanban to manage your processes. All right, next we have in our business, things change so much we can't really commit to the work for a full sprint.

I've heard this one, um, actually pretty often at several different companies. And it, it does seem strange to me because, um, the company that we've started that I started with, like I said, they, they really kind of adopted Agile. Um, but when I got there, it started, you know, their agile journey, um, from like a more waterfall way of working.

Um, and they were saying, well, we can't do Agile because it's too, things are too chaotic here. That seems strange to me because wouldn't that pair more with Agile than Waterfall? The things are chaotic, but regardless, they were like, we just can't do the sprinting thing.

We can't, we can't create a sprint. Two weeks is too, too, uh, is too short of a time box or too, too big of a time box. We need, uh, we need to just be free flowing.

And um, initially my thought, my thought before I really put a lot of this together was, okay, um, shorten your time box. If you have a two week sprint, you spend two weeks building something and then you start over and build something another two weeks. So on just shorten it to one week.

Um, and I thought about that for a little bit and I was like, you know what though? Um, if you can't get, if you can't, two weeks seems just like a lot. Um, so how do I wanna say this?

Um, if you can't plan two weeks, it's probably a bigger problem. Like there's a fine line, there's a fine line between being agile in the sense of like, we want to respond to change, and then just saying we're chaotic. You know, if you're responding to change, if you have a short change window, okay, that's tolerable.

You're using agile, that's okay. If you're chaotic, you're like, I can't even plan two weeks without something going crazy and blowing up in our face. That's probably something you need a little bit more problem solving around.

Like, that's probably a legitimate problem. So like, um, I'm just looking at my time to see if I have time for this story. Um, yeah, so when I was, again, earlier in my career, one of the big things, um, somebody come up to us and said, Hey, we just, we just sold this thing, like we have to start building this today or tomorrow, like soon we gotta get on this.

And I was like, well, well, we gotta have a little bit of conversation around this. Like, let's plan a little bit more. And, and, you know, uh, the guy who was selling it, uh, who happened to be kind of in the, uh, executive, um, area was like, no, we need to do this now.

He pulled a developer aside and said, Hey, what are you working on? Like, oh, I'm doing this. Like, not anymore.

You're doing this now. And, um, just, just, you know, totally blew it up. And, um, well, long story short, we didn't build the right thing.

And there's a lot of problems around that. And, um, whenever you dig into the actual root cause of that, um, sales was really struggling. Basically.

Uh, they weren't meeting the numbers they really hoping for, and they were just throwing as many hail Marys as they can, like trying to close these deals, promising things that they, they're just, they're like, I don't know if we can deliver it, but we gotta get this thing done. Like really just working, working themselves to death, trying to get these sales closed. And so we were kind of in the middle of that.

And so that was a bigger, deeper problem. It wasn't just, you know, well, we need to do better within it, or, you know, lock people out and say like, no, you can't do that. It was like, we need to fix our sales problem as a company.

So that was a big problem solving issue for us. And so, um, you know, just kind of goes to show that sometimes the problems aren't always when they're just right in front of us, they can go a little deeper than that. And so that does need a little bit of problem solving.

Um, and again, just like I said last time, if this is truly is an unavoidable issue, if you're like, you know what, we can problem solve all we want, but this is just the nature of what we do as a business. I, I don't really know a good example, but there may be businesses that are just like, we are chaotic. That's part of what makes us us, and there's not really a way around it.

Okay, scrums probably not the right framework, and you can use Kanban, and Kanban is also agile. It's just a different way of being agile. So I would just recommend looking into that if that's, uh, truly a problem.

Okay, next we have, developers don't want to give us timelines, but we really do need them. With Agile, we're having trouble getting these. So I heard this a lot, uh, again, just, uh, really throughout my career is just, you know, well, one of the big things about Agile is that it always goes longer than the timeline, which isn't necessarily true.

Uh, waterfall also goes away farther than the timeline. But, um, but one of the things, the misconception I think is that you just throw timelines out the window. With Agile, that's not really the case.

What you do is you respond to change with agile. So Agile's not anti timeline, it's just pro responding to change. So you can build a timeline and then just make sure that you're, uh, adaptable with it and you're able to change and willing to change if and when things change out there in the world.

Uh, if you're stakeholders just, you know, understand that there's something else that they need or, or whatever. So as you kinda see here, it's really a function, a lot of just the kind of team you have. Um, I've been on newer teams and um, you know, timelines are a struggle.

There might be teams, they're like, oh, we're so new. We don't really know the tech stack very well. Um, I think this will take like two days, but just to be safe, let's call it a week.

'cause we're so new and it takes like two or three weeks. Um, so that, that's how bad these things can get sometimes. But once you get more mature, I do see that there's a lot more comfort with, uh, with these, with these estimates.

Once you know the tech stack, once you know what's what goes on, it gets a little better. You also have data as you get more mature, you can be like, how long is this going to take? Well, we had something similar that happened, you know, six months ago, and that one, um, took about, you know, three months.

So it'll probably take about three months to, um, you know, we may buffer it a little bit, maybe four months, but generally we're probably gonna be good around three months. And that, that's a little bit more of an accurate statement. As you get more mature, you just get better at understanding what your, um, uh, you know, what your timelines are gonna be.

Um, but then down there at the bottom, like I I said, it kinda goes back to what we're saying earlier, timelines or estimates. And they should be reviewed and adjusted and adjusted in the sprint review. Like the reason that people want timelines, there's a lot of reasons people want timelines.

One of them may strictly just be for budgeting. It's like, you know, we, we, we think we're gonna make this much money from this feature. Um, we won't spend that much on it.

So we have a good ROI, whatever, you know, it's just stuff that people, uh, people want just for that reason. And, um, and so when we get to our sprint review, so again, with the sprint review, we spent two weeks building a portion of this feature. We got to a place that that thing has at least some part of it done, and we're going to review that with our stakeholders.

When you get to the table with that, you can say, okay, here's the features we built. If they're like, oh man, I just think maybe I see it now and I, I know we said this, but I'm seeing it now and I think maybe that over here is better. Can we adjust it to be that?

Well, you can pull off a timeline and go, cool. Um, yeah, that's gonna take a little bit longer. So maybe this is a three month project now it's gonna be a three month and a week, or it's three month and two weeks, you know, adding more sprint onto it.

Is that okay? And, you know, maybe they'll be like, yeah, yeah, that's good. And you, we just adjust the timeline based off that.

Or maybe they're like, no, honestly, we, we kind of have a deadline. We really gotta be done in three months. We can't do that.

Okay. Okay, cool. That's, that's perfectly fine.

There are other things in this timeline that we can probably remove to make room for this other thing you're asking for. Is that okay, can we remove this thing here? And maybe at some point in all that time, you'll get to some kind of good agreement.

But having the timeline, there is a, it's really good to have a artifact that'll look at and, and kind of understand where things are landing, uh, as far as, you know, the plan and, and how and, and where, where you're gonna get done and what's gonna be in there and things like that. So yeah, generally use the timeline's fine. Nothing wrong with that.

Um, if, if your customer doesn't want a timeline, okay, great, no need to. But if they do want one, you know, no big deal. You just gotta make sure that everybody kind of agrees that there's adaptability there, um, in case you want something different.

So, um, this leads to kind of the last portion of this, which is what to do in meeting resistance. This is, like I was saying earlier, like a lot of these things as I've been going through them, it might be like, well, yeah, like these are, these, these are legitimate things people believe about Agile It, why it doesn't work. Um, and, you know, I just, it just feels like it's very argumentative to say, well, here's why it does work.

You're wrong. And that, that's not the goal. It goes for us to understand, like I was saying earlier, for us to understand why agility works in the face of when people say it doesn't, or we know it does.

Uh, here's some ways we can kind of look at that. But as far as you're talking to other people, if you're talking to other people and they're telling you these things like doesn't work as X, Y, and Z, the best thing you do is just, you know, ask them things like, Hey, well would you be willing to try this? Would you willing to do an experiment?

Like not telling them, here's why you're wrong. Just being like, well, I hear what you're saying. Um, let's just showing them that it works.

Like what kind of experiments can you run to show people that this little thing right here that we wanna try to put into place might actually work? And, and just helping them kind of get there. Um, and the next thing is just really important, and this was added since the first time I added, I did this talk as well, which is showing empathy.

So again, you're not there to be, right? Like, we're all here to succeed together. Like you're trying to help people, you're trying to empower people to do the best that they can.

And agility is kind of the tool we have in our tool belt to help people do that. So, show empathy, like when someone's like, I just, this isn't gonna work. Like a lot of people have been hurt by Agile, if you wanna call it that.

Um, Agile's been used as a weapon in a lot of ways, you know, and not the agile that, not true agile, but that, that there's some things that people call Agile. Um, like maybe someone was like, you know, agile to us was that we had to use this tool. We had to use this exact template.

We had to do this exact way, we had to meet these exact standards if we didn't, we're not meeting our Agile quality, you know, controls or whatever. And it was just dumb, you know? And that's their experience of Agile.

And so they're coming to you saying, no, we don't like Agile. It's really bad. It doesn't work.

And you know, some of the, some of what you do is you say, I'm sorry, you know, I'm sorry that Agile didn't work for you. I'm sorry you experienced that, that wasn't the right thing to do. Um, but I think you have an experience of Agile that's not right.

Like I think maybe if we work together, we can figure out something better than what you've experienced in the past. And that's important too. And, um, and if you can meet them at that level, like understand where, where their hearts at and their head's at with this agile journey and meet them at that level, that'll go a lot further than just trying to dogmatically say, here's why I'm right.

So that's a good way of kind of meeting people where they are. Um, but the last thing you wanna do is when Agile really doesn't work. So other situations where people are right, where they're like, agile won't work here because this, and, um, I put here when change is expensive.

'cause that's the whole point of Agile is that you, you are able to change direction a little bit based on what you, what feedback you get from your products and from other places. So if you can't change very easily, if change is very expensive, then Agile doesn't work very well. Now one thing I'll say is that in my, the first time I gave this talk, there's a lot of places I would've said, oh yeah, like, here, here and here.

But as I've gained more experience, I've seen Agile work in places, I would've thought it wouldn't have worked very well. Like, I would've said something earlier in my career, I would've said something like, well, agile won't work very well in like a highly regulated place like the government or the bank. And I'm seeing a lot of Agile teams in the government.

I'm seeing a lot of agile teams in the bank. I worked at a bank, now we were Agile. And, um, you know, the reason I would've said that they wouldn't work is 'cause of all the government regulations.

Like, well, you can't release this thing until you have all your paperwork in place. And that could take forever. And believe me, I was at Bank, it did take forever, but we were able to do it iteratively too.

And so it, it actually, we were able to problem solve a lot of those things and get to a better place. And so that list of places that I would say Agile probably won't work is, is shrinking to the point that now I can't really think of a place that I would say, oh, agile won't work here. Like, even if you're building a building or something, um, it's always a good pro like good thing to do, to occasionally bring the client into the building and have them look around so they can tell you things like, well, I don't know.

I don't, I don't really like that plug over there. I want you to move that outlet over the other side of the room. It's better to know that now than when the building's built and you gotta undo a bunch of stuff.

So even there, I would say probably Agile's a good practice. It may not be as, as effective as other places, but it is still a good practice. So yeah, I would say most places, if, if it won't work at the very least, you can glean a lot from it.

So I would still be very cautious to say it doesn't work, uh, more so now than when I first gave this talk. And that is all I got. So again, I wanna thank you guys for choosing to, um, to come to my talk or to listen to me and having me here.

Um, yeah, thank you guys so much. I appreciate your time. See ya.

Hey everyone. Happy Monday. You know what?

We're just throwing more money on that AI bonfire. You're watching Textron Gang. Hey everyone, happy Monday.

It's Alan Shimel here for Techstrong and Techstrong Gang. Welcome to another week of as AI Turns or as AI burns as in money. But we'll, we'll get more into that in a second.

Let me, let me introduce you to our fantastic gang. We've got assembled today to talk about as usual three different topics that we'll be heading on today. Um, first of all, down Texas Way from Austin, he's the, uh, visible impact, uh, uh, CTO co-founder as well as future and Analyst.

It's our friend, but he's originally from the Bronx. Our friend Guy Currier. Hey guy.

How Guy Currier, or excuse me, I'm sorry, guy. Uh, uh, doing great. Really happy to, uh, to be here again.

It's actually been a minute. Um, and, uh, the weather remains completely unpredictable here. I'll just say that much.

I miss the Bronx. Yeah. I almost made you sort of part of the Curry basketball family there.

Del Curry, Stephen Curry, I forgot Stephen Curry's younger brother. I would be honored to be in their company. I, I can't touch, I don't if you quite measure, I've never seen you shoot three, so I'm not going to, You never will.

Yeah. Not gonna go there. Uh, moving over from Texas to New Mexico, though she is our open source champion and Linux Foundation participant, as well as the CEO founder of Deploy Hub and many other things.

Our friend Tracy Ragan. Hi, jc, how are you? I'm doing great, Alan.

Thanks for having me, as always. I always love being on these calls. Lots of fun.

It, yes. Okay, moving from Tracy up to Ohio, where, uh, she resides and editor of Textron AI and Gestalt It Textron it. SM we've got some news coming with that one, uh, sag.

So, hi Sagner, how are you? Hi, Alan. I'm good, thank you.

How are You? Good Sagner. Is it Saha or Saha?

I always mess it up. It's Saha Saha with the a I'll try to remember. That's Alna.

Saha. All right, good. I'm Guil Sagner.

It's great to have you here. And we'll go from there. Up up to, uh, Harrison, New York.

You don't know where Harrison New York is? It's upstate. Well, for people like me from Long Island, Harrison's upstate, other people may say, ah, it's just Westchester, but he's our chief content officer, Mike Vizard.

Hey, Mike, how are you? I'm well. 10 miles from White Plains is upstate.

Huh? Okay. I was wondering that too, Alan, You're also 10 miles from the Bronx.

Oh, no. Hey, look, I, I was on the south shore of Long Island. That's a far ride to Harrison.

Um, anyway, on Rochester right now, throwing stones. That's all I gotta say. Let 'em, all five of them.

They're freezing there right now. Um, anyway, good to have you, Mike. So, I I, I teased it in the, in the opening.

You know, I, I also, I discussed this on my shimmy says LinkedIn Live last Thursday, and it's on our Text Strong, uh, TV show from last Friday. The replay of that LinkedIn live. You could probably get it on text, on TV or on YouTube as well.

You know, more money being thrown at the AI and AI data center investment bubble irrational exuberance, if I ever saw it, uh, in my, in my, uh, presentation, I even used a little flip chart and I actually hand wrote down some numbers. It's pretty good. 2 trillion.

I did include this 200 billion that met a rumor has meta may be looking at over the 65 billion I think they already committed to. And, um, you know, of course meta at this point is still denying this 200 billion. But Mike, what do, what do you got here?

To your point, there's just a ton of money pouring into this space. And the issue though, is Microsoft is kind of like making some signals here that people are having a hard time trying to figure out and read. They're saying that some of their models are more efficient.

There's conversations about them pulling back on their leases. Now, whether they overprovision in the first place, it's hard to know. But, um, they might be signaling the folks that maybe we are not gonna need as much hardware as we go forward here with ai.

And, um, tip of the hat to both, uh, Alan and, uh, Mitch, who's not on the show, but you guys have been banging that drum for a while saying that, you know, we'll get more efficient, but guy, what do you think is going on here? Is this just one big giant pumble about the burst? Because NVIDIA's stock took some wild rides just outta numbers that said there was, you know, awesome, and then everybody punished him.

Well, I don't know if I'd put it quite as dramatically as either Alan with his irrationality remark or you with big giant bubble wood. But, um, I, I think I've been saying on this show for a while, um, along with, uh, Dave, Nick and our colleague Dave Nicholson, that, um, that, uh, there is gonna be a reckoning at some point. Uh, my feeling has been that capacity will, uh, power capacity specifically and maybe cooling a little bit capacity, um, will be that reckoning.

Um, and it's, it's, I feel like that this is kind of how it starts. You start to see these big wild plans and then counter currents like, like, like what happened with Nvidia. Nvidia beats its number, but drops in, uh, in, you know, in, in like, was it eight, 9% in, in, uh, market cap?

Uh, there are just so many things going on. It's a little bit hard to, um, figure all of it. Uh, I, I do think that, um, the exuberance around this is not necessarily irrational.

The meta story in particular, it makes me think, Hey, wait a minute. Uh, maybe meta facing, you know, from its cash cow, Facebook facing sort of a, a a a relatively stable and starting to decline user population, um, is chasing after, you know, AI and AI services llama and whatever else they produce as, as a new, you know, more promising long-term revenue stream. But I wanna point out one thing, the macroeconomic environment here, um, this did not occur to me until just like, as I was, as I was preparing for, for this show, we have to remember, or it's helpful to remember that all these tech companies have been sitting on wads and wads of cash for over 10 years now.

They've been building up a cash with nowhere to invest it. I mean, that's why they do it, right? Didn't Apple had a trillion dollars or something like that at some point, or, I, I don't remember how much it was.

That's probably more than needed. So, so when AI comes along and generative AI and this gold rush comes along, cash is cheap for a lot of you. They don't need to borrow.

They just, they, they can just spend. And much like on a, an earlier addition of this show, I was saying, you know, this is like the, the, the proverbial, you know, decision that the, you know, is gonna always be approved by upper management. Let's invest in ai.

You're gonna get that budget to spend. Um, in the same way I think in these tech companies, um, real smart product folks coming in and saying, here's the 10 things we can do with ai. We just need more capacity, they're gonna be told.

Yes. So in that way, I guess it is a little bit irrational. I think there's been a lot of money slashing around in these tech companies, and now all of a sudden there's a place they can invest it that, you know, everybody's gonna like until the other shoe drops, which is kind of inevitable.

I, I'd like to comment, with all due respect to my colleague Guy Currier, when you, when you're talking about two plus, it's two plus trillion with a T of dollars pledged, what, what kind of tele are we running here, right? I mean, let, let, let's be, let's be realistic. Let's look at the GDP of the top.

And this is all on my shimmy says you could go check it out. Let's look at the top 20 companies, or the top 20 nation states. GDP for 2024.

The us I I, I think the US was 30 something or 38 or 39 trillion. China was 28 or 29 trillion. And then there's a pretty big drop off.

5. Japan is about 4 trillion. Uh, UK is about three and a half trillion.

1 trillion. And then it drops off again, right? So you go through, let's say, you know, those six countries, India, maybe at number seven around there.

And then there's a huge fall off down to Italy and Mexico and, you know, some of the, uh, Canada, you know, still highly, highly industrialized first world, you know, big economies. You are talking about more money there than the GDP of some of these major industrialized nations. Where does the money get spent for every, you know, a billion here, a billion there?

Before you know it, you're talking real money. But where does every out of every, excuse me, out of every billion that gets spent or supposedly going to get spent on this AI and data center, boom, about half of it, they estimate half of it goes into semis, into chips. I don't, I mean, that's a damn good business for Jensen, Wong, and Nvidia, right?

If, if 2 trillion is gonna be spent and half of that goes to chips, holy merrow, that's an awful lot of GPU right there, right? A trillion dollars in GPUs. com days, right?

I used to be out, well, worked for a company not far from you, Mike, and purchased New York. And we had data centers all over the world. And we, we partnered with some really upstanding companies, WorldCom, MCI, if it rings a bell, um, the good folks at Enron, the big e down in Houston, we know what happened to them.

Level three in, in, uh, oh, right outta Boulder, Colorado. Interlochen, Interlochen, remember that guy? Level three was there, there was a whole bunch of fiber and all, and it was a very similar thing.

We're going to need more fiber. We've gotta do the last mile, but we need more backbone. We need more backbone.

We were using T one T threes at the time. Right? You, you, you didn't mention Akamai, that that was their Yeah, Well, Akamai was creating A-C-D-N-I, I tell you a funny story about Akamai, my teen investment sense, I picked ink to me.

But anyway, um, I knew a founder of In to me. Anyway, go Ahead. com time that basically that dark fiber as we called it.

'cause we never lit it up because we over, over capacitated, over capacitated our fiber needs. That dark fiber stayed dark. com bubble burst.

I don't think we lit a lot of that up. Two two until 2005, 2006. And then even after the great recession, there was still dark fiber that now we're lighting up.

Are we gonna do a similar thing here? Are we just gonna build so much excess capacity that it's gonna take 10 years for that to be soaked up in, into the, into the, into the marketplace? And what's that gonna do to Nvidia stock?

What's that going to do to all of this? I I do think it's gonna spur electric, uh, innovation in, in generating electricity, right? Hopefully clean, sustainable.

It's gonna have to, it's gonna have to. Yeah. No, not, not in this present administration, Joe, baby Joe.

Okay, let's just wait, wait, I just wanna get back to the, this their stock. You know, it is chaos because when you have an earnings report that says that you are, you know, earning $40 billion a year at a growth rate of 78% year over year, and your stocks go down, we have, we have a different thing happening in the market. I believe that there is a lot of volatility right now because of the current administration and just ins that people are feeling insecure.

And maybe some of those are small investors. I would love to know who is, you know, selling off their NVIDIA stock when they have a, an earnings report that looks like that. They, to me it's insanity.

They buy in anticipation of a blowout earnings. Well, and then try to sell it and, and make quick money. Or people who short there are there who are shorting stocks.

That's what they do for them. It's short, right? They shorting Yes.

For haven't we always seen that, but still, when you have, if you, to me, that's just insanity right now. It really is. It's insanity that a company could be doing that well.

And, and, and an and investors are gonna start dumping the stocks. Chase. You see this right here?

You know what that is? I Get, I do. That's the violin playing horse and flowers for Jensen and all his cronies at Nvidia.

And you know, The other thing we haven't really talked about is the CHIPS Act, right? Is this, is, is, you know, the potential of pulling some of that funding away from the CHIPS Act, is that having a, an impact? The, the, the problem is the CHIPS act small.

Well, they replaced it with small Potatoes. Starburst, right? Small to this.

Yeah. But they replaced it with Starburst also. That's a Half trillion Stargate.

Stargate Stargate. I, I really don't care. No, no, they did it.

So here's the deal. The CHIPS Act is US government money that they're giving the private industry to, to build chips and data centers here. Stargate is private money that's being invested into AI and data centers.

We have a similar thing going on, by the way, in China. Again, this is all on my shimmy says thing, the Chinese government has, I forget how many Juan a trillion or whatever it is, comes out to about $138 billion that the Chinese government is earmarking for AI and AI data centers for the big Chinese companies and smaller Chinese companies do, to be fair. But, you know, the 10 cents, the, the, the Baidu, the Alibabas on top of that, some of these Chinese companies, Alibaba, for instance, announced, uh, I think it was 35 billion of their money.

So you are seeing both governments and private companies come in here over the top on this. I mean, I look, I don't think a fraction of that will be spent in actuality when push comes to shove. And I think we're gonna wind up, I think it is irrational exuberance sky with all due respect.

And, and the air's gonna come outta this balloon, especially if we can't point to real wins with AI. Real quick, I if we're gonna really get into it, I, I do need to point out that, that, um, that those announcements are not really commitments. They're announcements or they're pledges.

That's pledges are good. It's like running a telethon. Yeah, no, I get it.

Saying because meta denies, uh, any $200 billion. No, but they do acknowledge the 65 billion. Yeah.

Okay. Al but also Alan, uh, comparing GDP, which is a yearly and annual figure spending over the, a course of a year with pledges that are gonna going to take a, a couple years, at least probably three years to play out. Even if you took three years guy.

I mean, this is, come on, it's, it's, it's Money. But it's funny because the exact example of building out the, the, the, the fiber backbone because of the internet is what instantly came to mind for me as well. And I, I, I think we're just really arguing about degree, and I'm picking on you for word choice, which is not even fair.

Um, overinvestment seems so likely here. Uh, I, if not, if only for the macroeconomic, uh, I, it's probably not even matter, the economic condition I talked about before, which is these are companies with tons and tons of cash that until this AI boom were, they were mostly holding under 'cause they didn't know what to do with it. Now they have something to do with it that looks promising, and yeah, they're probably gonna overdo it.

That's, you know, that's, that's the boom bust cycle. I agree. Um, Yeah.

So there's this also the funding frenzy that's been going on, like with OpenAI and tropic HuggingFace, they have all reported, uh, substantial fundraising, uh, rounds and even, uh, for AI startups. There is this cold rush of funding offers coming in from venture capital firms. And this started back in, uh, 2022 when OpenAI hit up the AI conversation with, uh, Chad Ity.

And, uh, now they're plowing all this money into this up and coming, uh, AI companies, um, no hold spot really. And that's shooting up their AI valuations like overnight, like OpenAI and anthropic are, have been, uh, beneficiaries of this too. Like open air valuation probably reached over $150 billion this year.

And that's a lot more than a lot of companies in, uh, that have been around a lot longer. And, uh, anthropic two, uh, hit I think $20 billion. So, uh, that there may be definitely some overspending over here going on.

'cause um, it's really like the, it's, it's really like the AI rush sort of, uh, thing that's going on. Companies are really, uh, you know, enticed by this new, uh, shiny new toy sort of a thing. And, uh, this ties nicely into, uh, this, uh, future research finding, which, uh, was released a while back, which state that, you know, ai, um, readiness of companies is really lagging behind.

And, uh, they're not really, um, at par with the expected ROI returns that, uh, that companies thought there would be. And, uh, a lot of things can be, uh, there, there are a lot of factors really that, uh, go into explaining why that is. There are cultural conflicts, technological lags, and the general lack of readiness of companies.

So, and a lot of companies are also using IS corner cases, and there is also this AI washing that's going on. So, uh, there is definitely, I would say that, uh, there is some overspending going on, but also it'll take a while before things start to fall in place. And, uh, companies start to see their, uh, expected returns coming in.

Lna you're not implying that companies are using AI as rapidly as possible without really knowing how it works, are you? Yeah, that can't be, this Is look out of my cousin video. Absolutely not.

Your Honor Report. There's a lot of waste out there. Yeah.

This is the report from the Futurum group, right? That you're referencing, I think, right? Mm-hmm.

Yeah. Yeah. My colleague, Olivia Blanchard led that effort.

He's, he's our AI specialist. Mm-hmm. So, I mean, what you're seeing is the trickle down, right?

If you get 2 trillion here, it trickles down. So it's, it's sucking up all of the, soaking up all of the VC funding that's out there, because everyone wants to ride that. Quite frankly, it sucks up our attention here on TechOne Gang.

Two out of every three segments are, it's somehow AI related, right? It, this, this is rippling throughout at certainly the tech economy, if not the world economy in general. You know what I find interesting?

One last thing, Mike, when I went through the GDP numbers in my shimmy, says, every one of these companies is all in on ai, right? They've all made announcements except Russia. You don't hear anything from Russia on ai.

Now, are they just not saying anything 'cause of sanctions? They're not allowed to. Are they just being really secretive?

Or are they too busy fighting wars? Or, or why buy what you can steal, right? So who that's, Well, they're all stealing.

Look, look, there's no angels here. Every, this is, we, we have cast safety, morals, ethics to the side. It's a full out race to, to be the king of ai, right?

And they're all, we're all stealing from each other. There's no doubt in my mind. All right, well, we have mentioned this in the past, but it's worth mentioning again, we do not give stock tips on this show.

But I guess we could remind folks that, you know, Joe Kennedy got outta the market before the great crash when his shoes shine. Boy, he gave him some stock tips and then he figured out, well, maybe everybody's in this a little bit too hard. So do what you like Folks, but, and he did go into the bootlegging business.

That's a whole nother story. Um, yeah, let's take a break here on Text and gang. Joe Kennedy.

We'll come back with more. You're watching Textron Gang, Discover Textron Group, the epicenter of tech innovation. We are your go-to for reaching IT, leaders and practitioners worldwide.

Our secret impactful content that sparks awareness, engagement, and top quality leads with us. You'll access editorial websites, streaming videos, virtual events, custom content analyst research, and more. Join our satisfied clients.

Let's revolutionize your tech journey. Contact us today and tell your story to the world in the most powerful way with Textron Group. 2, outta three, uh, segments.

This is the second segment. So we're talking about AI and its impact on software development. 'cause there's just a lot of conflicting reports out there.

com, there's two stories talking about how, uh, AI is not really gonna do every software engineering task all that well, and that the quality of the code is somewhat suspect at the same time. There are reports out there that says, uh, I think it's from the Fed in St. Louis.

That demand for developers is at an all time, uh, pre covid low. So we're not quite clear what that means either. And finally, though, we see people like Mark Benioff at Salesforce, you know, they have a bad quarter and they're telling Wall Street, they're gonna get rid of their software engineers to make it seem like they're gonna be more efficient and maybe buck up their, uh, stock price.

So Tracy, what's your read on what's going on here? So what, you know, e every time we go through a kind of a shift in how we develop applications, we kind of see this, it feels like, uh, the job that, you know, the jobs in some sectors or jobs in some skills will, will start disappearing. And, and jobs in other areas will start, uh, becoming in demand.

And I kind of feel like that that's what I mean. We have to remember that the technology sector is still growing. Um, according to the Bureau of Labor Statistics, it's like a 10% increase since last year.

So it's to say that, you know, one article about potentially jobs on indeed, uh, being less, like 30% less this month and maybe a couple of months ago, I don't think is a good reflection of the real job growth in the tech sector. There's a, there's quite a few jobs that are, uh, openings around, you know, AI people who, who understand large language models. People are just certified in it.

Uh, security is a big to topic. So I think we're seeing a shift, and I think some software developers who may have been doing, you know, JavaScript all this time might be not as much in demand. Um, but I don't think that we can really read into, uh, we can't read too much into what Indeed is seeing.

And there is a shift away from, uh, a lot of, uh, developers are moving away from thi uh, platforms. Like, uh, indeed, to be quite honest. There's other, you know, like Stack Overflow.

I, a lot of people I know, they go to Stack Overflow, they go to, uh, GitHub discussions because platforms like Indeed are starting to become a little bit annoying, to be quite honest in the way that it used. They use AI to find, uh, candidates. You have to have this perfect match.

So I think developers are starting to move away from some of these sites, uh, and go to some of the sites that other developers are at. They're actually looking that they know that they have openings on their teams. So maybe there's a little bit of a shift in the way developers are looking for work as well.

I just don't see a, a massive decrease in the number of software developer jobs. I really don't. Alan, how much of this has to do with something called the economy?

Stupid? I mean, yeah, No obvious. Well, I look in the last two months or so, we, we have seen economic numbers trending down, unfortunately, both on unemployment and a bunch of other, you know, kinda the leading indicators.

But, uh, you know, even like Trace, you mentioned Stack Overflow, that's where developers used to go to, to, to code and get tips in coding and, and to commiserate with other developers and figure out how to do things. Guess what? AI has hit Stack Overflow hard.

Their numbers are down. I the last, I think we discussed this on a gang a couple weeks ago. Stack Overflow is down 40%, 40% of their traffic is, is missing, or, you know, less than their peak.

Um, to me, that's an indication of something. People are using AI instead of Stack Overflow to get code snippets and learn how to do things, perhaps. But that's a real number.

And that's, that's indicative. Here's the deal. I'm sorry, go ahead.

No, Benioff, as you said, said, yeah, we're gonna do less. Uh, when Zuckerberg went, uh, you know, usual dark sweep parade out here, uh, Zuckerberg said that they were going to eliminate some mid-level engineering jobs this year because of ai. Other people are saying we're gonna eliminate some, you know, fresher kind of jobs, engineering jobs because of ai lower level folks.

No one says they're eliminating that, you know, the higher level developers. But if you were a young person coming outta school, I, I got a call a couple weeks ago from a recent college grad. He had a psychiatry, psychiatry degree, bachelor's, and he said, you know, I'm really thinking about going to a code bootcamp and becoming a software developer.

If you're that kid, is that what you wanna do? Or do you go to an AI bootcamp and become an AI engineer or an ai, whatever the word is. If it was your kid, what would you tell 'em?

I'd tell 'em to go into data science. I told them that too, by the way. Um, Yeah, data science engineers are probably gonna be a much, far more in demand, but a lot of people don't like that kind of work.

It can be pretty tedious. Development's a lot more fun. But if you really wanna make sure you're gonna have a job in the future, data science engineering is, or platform engineering.

So, so long then, let me ask you about this. 'cause you tracked this AI pretty closely, but some of this is a moment in time, and I think some of the execs, when they say they're gonna eliminate this, that, and the other accounting on the reasoning capabilities of the AI models to become more advanced than they are today. So when is your sense of how smart will the LLMs get and will they be able to take on more of these developer software engineering tasks that today maybe they can't?

Um, yeah, so I feel, I think that AI coding assistance are definitely getting better with each generation, and they may be able to absorb, uh, a good amount of the work that, with the coding and stuff. But, um, also because of that, companies are less keen on hiring junior developers especially. So the apprenticeship is fading, which means that, uh, it would be increasing ni trick year to train the next batch of, uh, junior software architects.

Um, also the no-code, low-code, uh, applications, they have a role to play in this because, uh, more employees are now writing applications using ai. But also it's important to remember that the overall tech industry and AI evidently has a role to play in this, but the industry is emerging from volume hire to hiring that matters. So instead of recruiting a large number of, uh, people, companies are now choosing to hire, uh, more experienced people who can do more and can do better, and then ramping up their output, uh, using ai.

So senior software architects and developers are now using AI generated code in many cases and using AI to debug or, you know, and testing codes. Um, but so largely experts believe that the size of development teams is going to shrink over the years, and it feels like that has already begun. But to what extent that is going to be, and definitely, I don't believe that it's going to wipe out developers and AI is going to consume all of that, uh, all of the responsibilities anytime in the future.

I, I just think it's a shift. It's a shift. I mean, think about when we went, you know, we go back some years, we think about when we went from the mainframe to distributed platforms, you could say that there was a decrease in the number of cobalt developers who were being hired during that period of time, which was totally true.

It did. But many of those COBOL developers, me being one of them, said, Hey, I'm gonna go learn to be to, you know, write a presentation manager, um, application and learn a database manager and become an OS two bigot. So it's about pivoting for a lot of these developers.

And they have, they have to start pivoting and if they wanna stay, uh, relevant, and the ones who aren't are gonna find that they don't have a lot of job openings. But just a point, um, just in personal experience, you know, we have in our open source community, um, we have college kids that show up, uh, and, you know, want to get experience through the open source process. So far, all of them have gotten entry level coding jobs.

All of them have gotten in entry level coding jobs on applications that are using ai. Now, are they an AI developer? I can't say, I wouldn't call them an AI developer, but they are entry level jobs teams that are working on AI applications, and there was an entry level spot for them.

Interesting guy Alan pointed out, has pointed out several times that, um, in the industrial revolution, there was this widespread feeling that people were gonna lose jobs. And what happened instead was people became, workers became more productive. I think this is a little bit different.

Um, I'm not sure exactly how, but it's kind of like giving everybody, it's not just coders. It's, I mean, think about doctors and lawyers, right? Um, part of the fundamental skillset now, um, and going forward is going to be the use of AI assistance for whatever it is you're doing.

And I don't know about you, but my general experience with, with, with coding and, and with programming is that, um, for almost any application or service, it's almost a bottomless pit of features and enhancements and, and and so forth. I mean, the, the, the, the paradigm between perfecting and releasing, um, in code development, that's why I've been saying since the beginning that, uh, the, the, one of the big benefits of AI is quality. Because you can get, move, maybe move that needle closer to perfecting while still releasing thanks to your AI assistant.

The thing is that so far the AI assistance, uh, uh, encoding seems to be about an inch deep, but it appears by its nature, by the way it was trained, it is, it simulates depth. So that's what we all need, need to learn. But Alan, you know, you could be right.

We could just be doing more coding better with the same number of folks in the end. You know, I, I, let me give you a real world example. I did an interview last week with, uh, developer advocate from a company called Postman.

Uh, you could go look it up on text, on tv. It's not, this isn't about Postman per se, it's about him. He, he was a, he's been a developer for, I don't know, 15 years.

And we transition to developer advocate and we were talking about this issue of developers and ai and he said, you know, my sister called me last week and she's opening a bakery, and I'm her brother and I'm her IT person, as most of us in the IT world. Are we, we are the IT person for our family members who are non it, right? We're tech support.

And, um, my sister said, I need a website for the bakery. Do you know anyone I could hire or could you do it for me? I know you're busy.

He said, well, let me, let me, let me ask you a couple questions. You know, what color scheme are you looking for? What's the vibe you want?

What do you wanna say? 7 of Claude Sonnet came out. He went into son and he said, build me a website.

And it was for GoDaddy was the host, build me a website for GoDaddy. This is, this is this, this, that, and that. He said a half hour later, her website was up and running, doing e-commerce even, right?

You could order cookies online. And he said, I, if you would've told me five years, 10 years, 15 years ago, that that's what it was gonna take to put up a website. And granted, this isn't an enterprise application, but it was a damn nice website and it was everything this woman wanted.

And it was in 20 minutes from inception to, to, to production. That's the world we're living in right now. You are talking to someone who went from being a lawyer to doing websites back in 94, 95 whenever it was.

And it was, you know, it was revolutionary. Right? Now, look, this is this baby stuff, baby stuff.

Here's, Here's, here's the bias. I keep hearing in all the conversations that I'm having with people, if I'm a senior developer or engineer, they go, great, I don't need all these junior guys and I'll just automate all this function and I'll do that myself. Then you go talk to the junior guys and they go, great, we don't need these senior guys.

'cause I can do all this stuff myself. So I can't figure out which end of this thing is gonna wind up being the front of this thing. Well, the senior guys to the senior guys might wanna get rid of the senior guys 'cause they're more expensive.

But on the other hand, if it's left up with the senior guys, they'll get rid of the junior guys because they want to keep their jobs. Yeah, that's, so, uh, just just to point to our, our, the listeners here. If you are a looking for a job, there are two things that you should really think about.

Make sure that your LinkedIn profile looks awesome. Um, make sure you have everything on there that you have in terms of skills. I would even say if it's a skill that you know, but you have not actually used it in a job, list it and make sure your GitHub profile is awesome because more employers are looking at the, your GitHub profile as much as they are your LinkedIn profile.

So resumes and cover letters are all well and good, but when it, it boils down to evaluating somebody before you get the phone call. They're gonna look at your LinkedIn page, they're gonna look to how, how you, uh, brand yourself and they're gonna look at your GitHub profile. So keep that in mind.

Great advice that is really, Except it's not a person reading it, it's an AI agent that's looking at it first. Eventually there's a person looking at it there. The AI agent only looks at your, your resume and that's the problem.

Alright, let's take a break here on Texture and gang. We're gonna come back and, you know, the speaking of old and new and senior and junior developers is the dividing line between, do you use Rust or see in your Linux? com is the leading resource for news analysis and education on challenges facing the cybersecurity industry.

com covers all aspects of cybersecurity, including data security, DevSecOps, cloud security, application security, network security, security threats, and more. com has the largest selection of security content featuring breaking news, blog posts, podcasts, and more. com to learn more.

com. Home of Security Bloggers Network. Hey folks, we're back.

And one of the great things about Open Source and one of the bad things about Open Source is everybody can see everything and everybody's very transparent. So in the land of Linux kernels, there's been this kind of debate going on where, uh, Linus Tovo who runs that project, pretty much wants everybody to use Russ so they can have more memory safe code in the operating system. And this is a good thing from a cybersecurity perspective, but there's a little pushback in the community and, uh, Linus is busy trying to rub everybody back in.

But this conversation seems to be going on elsewhere. Everybody is trying to figure out, well, how do I get rid of these languages that are deemed to be unsafe? And that includes Java C and a lot of this old stuff in favor of what they call memory safe languages like Rust.

Um, Alan, what's your take on what's going on here? But it seems to me that, um, there's resistance to change. Hi Boomer.

Um, Okay, and we're done. That was a good, That, that's what this is coming down to, to me folks, right? I, and I'm a boomer, so I I get it, I get it.

But you know, the, the old guard is digging in their heels saying, Hey, we, we've been working in sea forever our whole careers. We know every nook and cranny. We know how to avoid that memory leak.

We know how to avoid, make it more secure. We know everything we need to know about c I'm so damn comfortable in it. Why would we think it's changing in the next generation?

And Rus is harder to learn too. Russ is not an easy language to pick up. It's hard.

I've looked at it and said, I'm glad I'm not coding 'cause I wouldn't wanna learn this, but maybe it's not a one or the other option, right? I mean, most of the kernel's written in, see, they're not gonna throw that all out and start rewriting the entire kernel in rust. Why not?

It would be a huge undertaking, and I don't think They're not gonna throw out working. No, but over time, as you replace Over time, new modules, new modules, I'm guessing that they're probably gonna move to Rust. Uh, you know, why not?

I mean, that would, that makes sense to slowly move over to Rust if that's the direction they choose to go. You could have kept stuff in COBOL or web or, or assembly language or you know, all look, time marches on computer language is March on, right? There was a time where the lamp stack was PHP, now everything's Python.

Um, Russ Golan another, you know, go Golan, another, uh, modern kind of, uh, uh, languages. Um, c was not perfect. C Sharp was perfect, was also not perfect Java for, you know, the tremendous success in what Java's done for Code mobility and everything.

A lot of security issues there. You know, over time I've come to the trust, Linus Valli's kind of guts on these things. And clearly his gut is telling him moving the rust is a smart thing for the Linux kernel.

Um, I I'm going to give him the benefit of the doubt and say, if he feels that strongly about it, there's probably something to it. But I think you're touching on the problem there, Alan. It's a meta problem.

It's not a technical one so much I think, which is, um, how what, uh, Lin's, uh, four volt's influence is, is outsized. And I mean, he deserves it, but it's outsized in a corporate decision. So let me, let me just back up one second.

It's really hard to make a change. It can be really hard to make a change. And I sympathize with the desire to keep things simpler in such a large project.

One language, um, more developers who know it and understand it, it's what they're used to. But then you're ossified in terms, in terms of being able to move forward sometimes and a corporate decision or a committee decision or whatever. These things get considered and debated and then a roadmap and a plan is put together and all that other sort of thing.

That's not how the foundation is running because of to Wall's outside influence. He can't really, he d neither does he c can he, nor do I think he wants to dictate anything, but when he advocates for something, it creates, you know, a a lot more stress than in a more, I don't know, egalitarian or corporate kind of process. And I think that's kind of what's going on here.

I think if you pulled anybody out and said, um, should, should the language in which the line Colonel, uh, uh, is programmed, never, ever, ever, ever, ever, ever forever change. Should it always be c forever, it would say probably no. Very few people would say that there should be no room for change.

So then it's not a question of whether, but when and how you do it, You know, I can't help, but I'm sorry, go ahead, Mike. This is, and I'd love Tracy's opinion on it. 'cause I think this is a change management issue writ large, right?

So somebody will sit down and say, we should write this in Rust. And then they'll look around their squad and go, who here knows Rust? And then you hear crickets.

So then they just keep going back to Java to guy's point. But what does it take to get a development team to learn a new language? How long does that take?

I mean, you're talking about significant training and that might take years to make that transition. Tracy, what do you think? Well, to be quite honest, the, the truth is there's not a lot of developers that no C anymore, right?

So C development has, you know, dropped off Java, replaced it. So if you're looking at, uh, a group of seed developers who are maintaining the kernel then, and you want to take advantage of the security, uh, features of Rust, then you would think that those seed developers would be okay to move to rust. But as Alan points out, they do get, they really develop, you know what software languages is like religion.

You're asking them to change the, it you're asking them to change religion, and it is extremely difficult for them to change religion. It really is. It's so hard for developers to pivot.

They get so stuck in the mud. But with regards to the, the benefits to the kernel itself, if we are building a more secure kernel, it's easier to maintain because of some of those, the memory management that it has. I can't imagine why they would continue writing it and see, I really can't.

But we ha they have to change religion and the culture of developers is very odd. They, they really dig in, they love their code, they've really, really love their code. And to see it move to something else, um, is hard for them.

But new modules, Hey guys, hey ladies, whoever you are working on that, I think that you're gonna be coding and rest very, very soon if you're gonna continue on that project. Look resistant, the change, there's always resistance to change. Change is hard, right?

Especially in software development where it is direct. I see it's akin to religion, right? I've seen the, the, the, the squabbles between Java and C folks back in the day.

net? Oh, you were less than human, right? And Yeah, they are.

We are listing human. I'm a T net programer. I'm, yeah, actually, you know, so an alien in disguise.

This, this is Visiting you guys. Yeah, That's software. That's what makes it fun though, right?

And you know, what's interesting is when you go to large enterprises and say, what, what, how many in what languages do you guys support over here? The average large enterprise probably supports seven to 12 different computer languages. And they have different teams for every language.

And that's not the most efficient way of doing things, I'll tell you that. So I am, I'm, I'm going through my mind. It's to see if I can think of a single instance in history where there was a mass religious conversion that was peaceful and I'm coming up blank.

So, well, we might, we might be seeing it with Elon and Doge. I'm only kidding. Hey, we gotta call a wrap on this version of the Gang.

Guys, it's been real. It's, it's been a, a really great discussion on some really great topics here. I hope you've all enjoyed it.

We have a full text on TV schedule following this immediately, so stay tuned wherever you're watching this, unless you're just watching this as a podcast or something, go find, go to Textron tv. Go to our YouTube channel, go to any of our sites, check it out. Guy Tracy Sagner.

Mike, thanks for joining us today. For now is Alan Shimel. We're out.

Hello and welcome to the latest edition of the Textron AI video series. I'm your host, Mike Zern. Today we're with a tins who's CTO for Galileo.

And we're talking about a leaderboard for AI agents that they've created. And well, a lot of these technologies are innovating so fast that everything seems to leapfrog each other on the leaderboard, but some, no one's quite sure who's in best in what at any given moment, but at 10 you guys are trying to figure that out. How do you determine that?

How do you track that? I mean, and, and for that man, how did you guys get in this business? Yeah, so, um, so for those who don't know, uh, I'm Atan, I'm one of the founders of Galileo.

And the reason we got into this space 'cause was 'cause uh, from my personal experience, uh, I kind of spent much of my career in the, during the growth phase of Uber and very early AI at Apple. And we saw evaluations being a very cumbersome task and kind of the primary reason why AI systems fail in production is because of lack of robust evaluations. So we kind of went down this path and, uh, ended up, uh, choosing language modeling as our sort of strength and, uh, forward down down that path.

And, uh, for us, uh, of course, you know, the whole industry is kind of caught on and, uh, started using LLMs for industry use cases. And one of our goals was always to focus on benchmarks, uh, for a lot of these LLMs. 'cause we saw an explosion of, uh, of LLMs being a real thing in the, you know, first couple of years of, of, um, you know, the, the chat g pity moment.

Um, and, uh, we knew that, uh, often in ai people focus on academic benchmarks, you know, just to show strengths and weaknesses of models and they don't really do good justice to industry use cases. And, uh, that was one of the reasons why we built, uh, what's called the hallucination index. And we published a couple of versions of those in, uh, the months prior.

And more recently because agents and agent workflows have become, uh, very standard and, uh, it's still early days, but people have started adopting building agent applications. We sought to build an agent leaderboard, which kind of uses a lot of the evaluation techniques we used for the hallucination workflows and rag workflows when we published our hallucination index in that we essentially curate a high quality data sets across multiple industry domains. In the case of agent leaderboard, we looked at over 25 common industry use cases and domains, and then really used our own metrics and scorers, which we publish as part of the Galileo platform, kind of like dog feeding the Galileo platform to see which models, uh, perform well for certain kinds of agent tech tasks and don't for others.

'cause it's always this game of, oh, which model should I use for my application? Uh, one of the first questions any practitioner would ask is that, and our goal has always been to kind of publish industry benchmarks and kind of show how these models perform, but in the light of more practical industrial use cases, and a lot of the insights which are, uh, pretty surprising in the agent leaderboard, they kind of point to, uh, the fact that, you know, a model can do great in an academic benchmark, but the, the rankings can be completely different when it comes to the industry. How much difference is there in capability for these various AI agents versus what it might cost to implement them?

It seems like if I read the report, um, there's not a huge gap between them, but the pricing is kind of fundamentally different. Actually, that was one of the most surprising, uh, if I were to rank the three top three surprises from the leaderboard. One was this, uh, the, this, uh, uh, sort of tussle between cost and performance.

'cause amongst the top three ranked models, which included Gemini and uh, as well as OpenAI models, uh, the performance difference between the best and the worst in the top three was 4%, which is minimal marginal, uh, and the cost difference is 20 x. So, you know, as an industry practitioner, if you really care about cost, you're kind of in luck. 'cause you might find a model that performs equally well for your use case, uh, as any other at one 20th, the cost.

How much are people kinda evaluating an AI agent today initially and going with one? And then are they gonna swap those out over time based on pricing or is this kinda like, I make a decision once and I stick with it? It is more the former where you would kind of expect, uh, a developer or you know, a maintainer of an agent system to really go with what's best.

'cause on one hand there's this massive reduction of the cost of intelligence where a small model, uh, 100th the size of say a super large, very intelligent reasoning model can do as well, if not better for their use case. And, uh, that combined with the fact that every day there's newer models, newer tools, newer frameworks, which are emerging, it's only natural for you to kind of do a, you know, a cost benefit analysis. And, um, uh, that's why a lot of, uh, uh, on the tooling side of things, uh, many companies are kind of building these AB testing tools internally as well as using from open source, which kind of allows them to almost swap out models, uh, in their production applications, uh, and quickly do a test of, you know, if I swap out anthropic with something else, you know, what's the variation in performance will my system incur?

Um, and and that's kind of part of the story that Galileo also intends to solve where we wanna provide, uh, platform and tooling to be able to do that very efficiently and do that in real time. Yeah, that was my next question. How dynamic can this be?

Because can I have a scenario where LLMA is what I use in the morning and LLMB is what I'm invoking in the afternoon because some of the pricing structures changed on you in on that LLM per se. I mean, just how, uh, disposable are these models gonna wind up being? That's a great question.

Uh, I do think that there's a lot of variance, um, kind of in, in the era in which we are in, um, uh, but that, that variance will sort of go down as the tooling, uh, matures. And there's, um, there's, there's minimal difference between difference, uh, different LLMs, but also in terms of cost. Um, but also I think, uh, one prediction that I have is, um, in probably the next 12 to 15 months, what we'll see is this clear, uh, segregation between, uh, hey, here's the class of reasoning models, which, uh, are expensive, but they solve one particular kind of task very well, whether it's, you know, deep research or, uh, you know, things that require you to think a lot and reason about a lot.

And they are very expensive for a reason because behind the scenes there's, you know, if you go to the technicalities of it, there's multiple forward passes which are happening. And, uh, the amount of GPU operations, it's really like the cost is kind of driven by the GPU operations. It's a lot more for those kind of models.

Versus the second class is, hey, you're building some sort of an industry application, whether it's a chat bot or a summarizer or q and a system. Um, there's a, a list of maybe a dozen or two practical use cases and there's a swath of much cheaper models which help you achieve that. Um, uh, that, that's kind of on the LLM side.

Uh, one of the other things is with agents, it's not just the LLM, it's different functions are in the mix now. There's different tools and, uh, the, the ecosystem has become a little more complex and spirited with a my myriad of different operations. So that also kind of causes additional variance where, you know, you might have stability in the LLMs and you're like, alright, this particular model Gemini, for example, is great for my agent application, but today I'm using auto gen for my tool orchestration and tomorrow maybe l graph or something else might, you know, just perform more deterministically for my use case.

I, I do see for the foreseeable future that as the tooling and tool ecosystem matures, there will be this, uh, challenge of a lot of variability and the different components. 'cause everything's now predictive and, uh, sometimes something may work for one use case and may not work for another use case. So what you really want to build is like this system that allows you to multiplex between different models, between different frameworks.

And, uh, that's kind of your, that would be my advice to anyone who's in the early days of building agent systems. Do you think at some point we may even have, um, I don't know, an AI agent or AI model that is optimized to figure out all that multiplexing logic? Absolutely.

In fact, we are already seeing, um, this at least in the private, uh, private models ecosystem with, with OpenAI Swarm and various others who have built in function calling as well as tool selection capabilities. Uh, the, the truth is that there will be innovation on both fronts. Uh, like the large model providers are also in the game of not only building great systems for academic purposes, but also, uh, they're big businesses now and they, they're trying to cater to industry, but there will also be open source models and, uh, open source tooling, which will kind of, you know, race one-to-one with, uh, a lot of the innovation that OpenAI does.

Uh, speaking of open source, one of the other great findings from the agent leaderboard was that, uh, mytral is probably the top open source model, which came out in our rankings with a very high tool selection score. So if you're a fan of open source, MRL is certainly a great starting point for you. Do you think we'll see some segmentation along the lines of, I may use open source for most of my functions, but then I'm gonna use proprietary models for specialized purposes that maybe they're better able to reason across?

I mean, will there be segmentation kind of like the difference between, I don't know, buying a Honda and a Ferrari? That's a good point. Um, I would say that the jury is still out on, uh, whether that would be the sort of segregation factor between open source and proprietary Deep seek is a great example of that, where it was able to compete and outcompete on certain benchmarks, oh one, uh, level models by, uh, by OpenAI.

So you might as well, you know, not use OpenAI and use deeps seek or try out deep seek flavor of models for your use case. And that is open source. And, uh, uh, so, so there is this push towards having open source reasoning models, which are equally powerful, if not more.

The race, I think is in optimizing the costs. 'cause the lower the costs go, the more available it is for, for usage and people will only pay for, uh, usage, whether it's open source or whether it's, uh, using an OpenAI model or any proprietary model. Um, it'll only be worth the dime if they're able to solve industry specific applications, uh, for the rest of us who are tinkering around and just doing, you know, simple q and a.

Of course there is that, you know, uh, reason to pay for just directly using these models, but real like the, the real monetary value kind of comes from industry. What's that one thing you see people doing over and over again that kind of makes you shake your head a little bit and go, folks, we need to be a little smarter about this than that. It's really how people are still playing whack-a-mole with the different models and different frameworks.

Uh, there's almost tool fatigue and tool noise at this point around which model or which framework to use for your specific application. Uh, I, I think the right way to go about in, in an environment where there's so much options to choose from is to, uh, really build, uh, system, whether it's an internal tool or, you know, go for a, some kind of a, a platform which allows you to multiplex between the different choices. Then quickly get to the decision of that this is the combination that works for my summarization use case or some other, uh, q and a system that I'm building.

Uh, Galileo, just to throw a shameless plug in there, is kind of one of those systems which allows you to make these decisions pretty quickly. Uh, but then that's, that's half the story, uh, then because once you build your application with say, you know, the ideal combination of LLMs and prompts and frameworks, that is bound to change, uh, based on the real time performance that you would get in production. So you need another system that allows you to monitor that and, uh, make those decisions on the production side of things.

Uh, and there I think the real differentiator is your production application will constantly have newer forms of data to deal with, whether it's new questions or new kinds of ways of querying your application that will likely determine the performance. So you need a way to measure it and then kind of come back to the drawing board and, um, redo your entire experiment that, Hey, here's a new kind of data that my chat bot received clearly, you know, frameworks A, B, and C are not working for me. Here's a quick test that I'm gonna do with d, e, and F and throw that back into the production.

And you need a system that allows you to do this in with minimal downtime. Folks, you heard in here to quote one great philosopher, don't worry, be happy because well, the cost of switching between AI agents and LLMs is not nearly as high as you thought. Hey, adding, thanks for being on the show.

Thank you so much, Michael. It was a pleasure. All right.

And thank you for all watching the latest episode of the Techstrong AI video series. You can find this episode and others on our website. We invite you to check them all out.

Until then, we'll see you next time. Good morning, good afternoon, wherever you are. I hope you are starting a really wonderful new year for everybody out there.

Um, this is 2025 and one topic that is at the top of everybody's mind still is ai. Now the thing about AI is that you simply cannot ignore it. Whether you are a technology leader, whether you are a business leader or in any of the other functions like HR or legal or finance, AI is at the top of the mind and you in your leadership position are asked multiple questions around ai.

You are asked to build a strategy around it, you're asked to build projects around it and then drive those projects to conclusion and to successful conclusion in spite of all the ambiguity that is around this technology that is around this topic. So the, in the next 30 minutes or so, we are gonna try and give you a path through this confusion, we are gonna share some of the ways in which we have accomplished success in this area. I'm gonna share some of, uh, the things that worked out for us and also some of the things that did not work out for us so you don't have to repeat those mistakes.

So let's go. I think it's going to be fun. 30 minutes.

So the thing about AI is that a lot of people talk about, and in the media, when you hear about it, they talk about the upside of it, oh, it's going to be fantastic. It is going to change everybody's life for the better. Uh, there's going to be a massive increase in our GDP or the GDP of the world and the productivity go up and most of them are very possible and are actually true.

There can be unimaginable amount of growth when it comes to ai, but at the same moment, and in probably the same measure, we also have to think about the change that it is going to drive. And that change could be on the environment side, it could be on how businesses are done today. It could be the change that might impact your job, my job, everybody else's job.

So it's very important that when we think about ai, we think about the potential of unimaginable growth, but we should also take care that we are ready for the indiscriminate change that it is going to drive. With that out of the way, the first question you must be asking, who are you and why should I listen to you? So my name is Nabil Bukhari, I'm the chief technology and product officer for Extreme Networks.

Extreme Networks, if you are not familiar with it, is the largest pure play enterprise networking company on the planet. Um, and you should listen to me or too extreme because we have now successfully productized multiple generations of ai. And the latest one we recently announced is our extreme platform.

One that really takes AI and narrates it with a platform, an enterprise wide platform, um, so that you can actually unlock the real potential of it. Now, AI as a product and AI as part of a platform, that is an entirely different topic, which we are not gonna talk about today, but something maybe, uh, for future. So this, so all the things that I'm gonna talk about today are really based on our experience as we productized and brought to market extreme platform one.

Uh, so I'm not gonna stand here, well, in my case sit here and pontificate. I'm gonna share some real world experiences with you, some of them very positive, and some of them, uh, which will be ous, right? Okay.

So let's go. Each one of us wants to be successful in ai. And in order to wrap our heads around it, it's sometimes easy to put something as an equation.

So here's the equation that we have learned over our experience of productizing it. There are multiple components in it. The first and the foremost is user experience and the trust gap.

And they are conversely, um, or they're not directly proportional to it or themselves, or they are inversely proportional as they say. So the better the user experience, the more success you can have, but the bigger the trust gap, you know, the less success you will have. And then there are a couple of other components, which is technology and also willingness to pay.

And willingness to pay is something that people usually don't talk about it. So I'll talk about that towards the end and make sure that we actually understand what we mean by that. If you notice in this technology is not the biggest factor in there.

Uh, and that is one of the biggest problems that happen with AI projects. The moment somebody asks you, Hey, uh, so and so you need to come up with a AI project for our company. And the first thing that people go, they look out and they go out and they start searching for the best LLM or the best model to use that.

That's just an absolute wrong position or wrong place to start with the first and the foremost thing is user experience and trust gap. So that's where we will start with. Now, one thing to remember when it comes to trust is if your people do not trust it, they are simply not going to use it.

And it doesn't matter how good or bad your AI product or technology or agent is, if people don't use it, it's not going to be successful. So think about trust upfront. Now, the thing about trust is that trust and value, they kind of go hand, hand in hand.

So the more trust people have in an AI product, the more valuable they consider it, or conversely, the more valuable it is that could drive more trust in it. But there is a thing called complexity in there. And typically the higher the value of the ai, um, you know, unless you are at the very, very start, uh, it's going to directly relate to the complexity of the project.

Okay? So when it comes to trust, there are two, three things that you need to consider. Number one is pick a use case that actually matters.

So the first thing that we are gonna talk about is how do you pick a use case to which you are going to apply ai? That's number one. Number two, how are people actually going to experience?

So what is the UI for that ai? And the last part is culture. So these are the three things that feed directly into this trust and value, um, in the equation that we talked about.

So the first thing, how do you pick a framework or how do you pick, um, an experience that you can apply AI to? How do you pick your AI use case? How do you pick your AI strategy?

Different companies call it differently, but a lot of the times what happens is that you get a call from your boss or from your board and they ask you like, Hey, we need an AI strategy, and off you go to the races. But I think the best way to do it is to take a step back and think about what are the use cases, current use cases inside your company to which you wanna apply it. And there's a very easy or simple way to wrap your head around it.

And we call it the ARC framework. Um, and this is accelerate, replace, and create. So simply speaking, it is like what are the experiences in your, um, domain, in your work environment or in your company that you want to accelerate?

What are the ones that you wanna replace and which are the ones that you want to create afresh? And this is in your context. If you are doing AI for your team, you can run the ARC framework in the context of your team.

If you're doing it for your entire enterprise, you can do it in the context of the enterprise. And if you are productizing it and you are bringing it out to an industry, then you can do it in the context of that industry. But the framework nonetheless, stays the same.

What are you going to accelerate? What are you going to replace and what are you going to create? So here, let's just take a few examples.

These are simple, very simple examples that will make sense to everybody when it comes to acceleration. A lot of companies start with their customer support experience. Why?

Because nobody likes to sit on that call listening to that, you know, fun elevator music and waiting for a customer support agent. Or when you get to that agent, nobody likes to start from like, is the light blinking and have you turn it on? And is the power plugged in and stuff?

People want to get to their resolution very quickly. And a lot of companies, a lot of use cases on applying AI to that. And what are we doing here?

We are accelerating an experience that is already present there, right? And it is very powerful. That's an example of accelerating a use case.

Now, let's think about an example of perhaps replacing a use case. And we are in the networking technology domain, so I'm taking a lot of examples from there because these are the things that we have already done in our portfolio. So this experience is the acquisition of knowledge.

Now, what do I mean by that? It could be finding information inside the documentation. It could be finding information in real time for a product that you're using or a problem that you're having, or even things like explaining something that the product is showing you.

You remember those error codes. Now, good thing we don't have error codes and SaaS applications, but sometimes information that is displayed, um, in those dashboards and in you, in those, you know, my pretty, uh, graphs, you might wanna know what they actually mean. Now, having a chat bott that is right there and giving you that information in real time, that is expo that is really replacing the experience of having to go out digging through documentation and websites.

So again, very simple example. Now in terms of creating a new experience, uh, this is something for example, right now at this point in time, um, especially in the networking world, if you want to pick up information from various different pluses in your network in real time and create this what if scenarios and stuff, it's something that is very, very cumbersome to do, to a point where I would say that they are not really something that everybody does in their daily life. So it's almost that that experience does not really exist.

So this is with the use of AI creating an experience where you can acquire the information, you can actually, uh, structure the information and you can build insights on top of it all within the same product within like minutes or so. So this is creating an experience that is brand new. Now, this is just an example.

You have to apply it to your space, to your domain. But think in terms of what are you going to accelerate, what are you going to replace or what are you going to create? The chances are you're probably gonna start with the a part first.

Now, as you're picking this up, one example, or one thing that I will share with you is do not pick a use case that nobody cares about because you are going to get some money to actually go this AI use case, but once you do it, you are only going to get more money if that use case were valuable. So go pick a use case that is valuable from day one. Don't go for the low hanging fruit, not when it comes to ai.

Okay? So this was the part of picking an experience. So once you have picked up the experience and you are trying to build something around it, the second thing that you have to think about is how the users of this AI application, what would be their experience?

Because as different experiences are developed during ai, the trust or adoption for that is not linear. You know, if somebody is used to a conversational AI and they will start adopting it, but when you actually, um, introduce them to an interactive or a collaborative AI or to a agent ai, the adoption is gonna drop down before it comes back up. So consider these things as you build the UI for your program.

So how did we build the UI for the ai? Um, in platform one, we essentially went with three different ways of interacting with ai. So conversational, interactive and autonomous agents.

And my feedback, um, or my, um, sharing my experience with you, um, that don't go with just one because if you just put a chat bot in your product or in your project, uh, very soon you're gonna find that there are use cases that do not lend itself very well to a conversational interface. So always invest in and think about various different interfaces, very different user experiences. So here, let's just take a quick example.

This is something that is going to be very familiar to all of you. This, by the way, is extreme Platform one. Um, of course this thing is gonna look a little bit different based on whatever product you are using, but the idea here is conversational.

So you have, think about that you have an expert sitting right next to you or another colleague sitting right next to you who happens to be an AI chat bot, and you are talking to that chat bot. And as you are talking, you are learning, as we talked about knowledge acquisition. Uh, you are doing things, but this is a conversational interface and this is something that people are a lot more familiar now in 2025.

So this is the base experience that you can provide in your project or in your product. But as we move forward, um, there is the second interface, which I think will become more and more valuable as we go past 2025. Um, and that is really the experience of creating real time dashboarding.

It's really a canvas. We call it an AI canvas. And the idea there is that there's a lot of information that is present in the products, but not always in a way, shape or form that is actually valuable.

And that is something that you can really, truly use. I'm talking about the users. So giving the ability to the users to interact with the system and create a new dashboard altogether.

So in this example, what happening is that the user is actually working with that conversational ai, and it is asking the AI to give it or give the user multiple different pieces of information. But while the user is doing that, the user has the ability to take that information and put it on a canvas. And as the conversation continues, it continues to put more things on the interface or on the canvas, and in the end it ends up with essentially a realtime dashboard, right, that they can now create and then now they can actually use it whenever they want.

This is an absolute powerful way of allowing people to use AI to create something that was not present there. And, uh, this has worked fantastically for us, and this is something that I highly encourage people to think about and use when you are doing your AI projects or your AI products, right? And we'll talk about the difference between a project and a product in a little bit.

So if you go to the next slide, um, you will see that the next big thing is obviously agentic ai. Still, I briefly talked about, you know, the conversational as well as collaborative ai. Uh, but this is 2025 and everybody should be thinking about agenda ai.

And if you are not, then I'm pretty sure you're getting peppered by everybody to think about it and come up with a strategy around it. Now, let's take a swing back on the agents, because before you go into the technology, it's kind of important and interesting to understand what an agent does. Because when you talk about agent AI in the industry, there are two connotations in which we talk about it.

One is, um, you know, running AI agents behind the scenes do all those different things, right? So there's a master agent, and then, you know, there's a bouncer agent and there's an agent that distributes the query and so on and so forth. But if you think about a curtain, and on this side of the curtain is the user, and on this side is the system age agentic AI in the system is one thing, but then exposing those agents to the user side or user visible agents is something a little bit different.

And now here we are talking about the user visible agents. So this is agents that user would interface with. So some of the characteristics of it, the whole idea behind agent AI or AI agents is that they're autonomous in nature.

Now, autonomous doesn't mean that there cannot be a human in the loop. You can have both kind of agents out there. Um, but the key is that it also has memory.

So it has context, memory, and in AI words is really context for the user. So it retains the context. Context, it is reactive, or it can be proactive.

It is very specialized in doing a few things. Um, and remember the last part, it can interact with humans or it can interact with other agents. So this is kind of a rough definition of agents as they appear to the user.

Okay, so those are the agents, but how should I think about agents? And you should think about agents the same way as you think about humans. So I always like to give this example that, um, how do you create teams, right?

You find various people, there's definitions of their roles, they're expert in different things, and then they interface with each other, they interact with each other, and that creates a human team. Now, if you take it to the other extreme, what happens is that you have a bunch of agents. Now imagine that these functions are being done by AI agents, and these AI agents need to be organized the same way as you organize human teams, and then they talk to each other.

Now, I wanna see these are two extremes of the spectrum. A fully human team, which is not only just the extreme, but is also the norm today. But in the far out world, we could or far out time, we can think of, you know, teams that are just a hundred percent based on AI agents.

But what's the reality? The reality is going to be that it'll be a combination of human and digital workforce. So when you are building an AI agent, when you are thinking about an AI agent, keep this context in mind that your AI agents should behave towards the human users as humanly as possible.

So with this context, uh, let's just give you an example. So this is how we have done agents in extreme platform one. Now it, it, the video will do a bunch of different things, and of course you can rewind it, you can pause it, you can look at it.

But generally what's happening is that when you have agents, you have a lot of AI agents, uh, you need to do the following things. Number one, there needs to be some sort of a catalog where you can find an AI agent that does a certain thing. This is pretty similar to where you have a job description and you go out and you try to find somebody that can actually do that job.

So you have to be able to find AI agents, then you need to be able to go and configure them. This is kind of, uh, teaching a new person how to do their jobs. This is like about, um, now on the AI side, you actually go and configure that AI agent.

Um, and then once you have configured it, then you need to do lifecycle management, and you run this. And in that configuration, it, that agent might be interacting with other AI agents or with human people or the human workforce out there. So that's just the example of how to think about AI agents.

So when you think about AI agents, start with the context of that this is digital workforce and it has to work with humans, and then start thinking about how to organize them, how to make them discoverable, and how to make them configurable. Now, once you do that, um, then only think about the technology. So I can spend a lot of time about technology behind the scene, but, um, what I would tell you is that most of the times, um, what our experiences, and we have hundreds of thousands of, you know, users out there, like 50, 60,000 big customers all the way from mid-market to, you know, fortune 10 out there.

And what our experiences that as people go into building these agent ai, most of them will probably fail. But that's not a bad thing, that's a good thing because that's experimentation. Uh, and that just helps with that AI literacy across the organization.

But once you have done that experimentation, you are gonna be, um, dealt with a question, the age old question. Uh, are you gonna build it yourself or are you gonna buy that as a product? And I'm not gonna tell you to lean one way or another.

You can build it yourself, you can buy it. Uh, but that is a decision that you have to take before you dive into the technology. Now, of course, we are a product company, so it's not like we can actually buy it, so we have to build it ourselves.

And as we built it, these are the experiences, um, that we had as part of that. Now, there is so much technology, there is technology overload essentially when it comes to AI out there. So just to organize it, and I'm gonna build out this slide just to organize it, we think of it in three big buckets at the, at the bottom of it is the data hub.

Now e you can call it data hub. You can call it data pipeline, you can call it data fabric, whatever you call it. But the idea there is this is the layer in which you are collecting data, you are organizing data, you're indexing it, you know, and you are putting data compliance in it.

This is where who can access what resides. This is where, uh, you know, your data residency and data governance pool, that that is where all of that sit. Then on layer above that is what we consider as AI services.

This is where, you know, your model management sits in. This is where your multi, um, you know, your rag architecture sit in. This is where if you are multimodal, that's where it set in.

So how do you organize all of these AI services together? So that's, think of that as a second layer. The third layer is perhaps the most important, uh, in the actual usage of ai, and that's the safety and guardrails.

This is the fact that, you know, how do you deal with hallucinations? How do you ground your models? What is your resiliency?

How do you build a human in the loop? Kind of an architecture and stuff. So lots of loss of technology around there.

And the reality is that I'm not gonna go ahead and give you recommendations because by the time I finish this presentation, there's gonna be new advancements in this area. But what I can do is share some of the experiences and some of the learnings that we have in this process. So starting from the bottom right in the data hub, uh, don't go with like, I need all of their data in the world to be able to do this.

The most important part in the data is the data governance. Start with the data sets that you already have, but make sure that you think through the data governance, um, as one of the most important thing in your data layer. The second thing, which is really interesting is that, um, where is that data coming from structured, um, and unstructured.

And if you are producing that data, then push down to those teams that are producing that data the right way to do it. I'll give you one example. What we realized is, um, that a lot of the documentation that is being written, the simpler the English, or I would say the language simpler, the language that is used in that documentation, the easier it is for an AI system as opposed to when you're writing it for a human consumer, in which case, you know, you want to add more, uh, context and perhaps, you know, a little bit more flowery language and stuff.

So it's things like that become really important in your data layer, in your AI layer. Um, quite frankly, uh, rag, uh, or some variation of it, like met database rag and has rag whatever rag you're using, because that is what will really help you when it comes to, uh, trust and stuff on top of it. Um, so at this point in time, rag architectures are really, really, really, really critical.

Now, when you come to safety and guardrails, quite frankly, I would tell you don't go on this journey on your own. Um, make sure that you have some really good strategic partner for us. Microsoft and Amazon are two really big partners, and we have learned a lot from there.

Um, and we have incorporated a lot of their technology. But one thing that I will point out is that fact checking mechanism, which are really that, you know, how do you deal with hallucination? How do you deal with, how do you ground your models and stuff?

So that fact checking mechanisms are really important because remember, it doesn't really matter what your AI does. If people don't trust it and don't use it, there is no value to it. So these are some of the things that you should consider if you are building your own project.

And these are also the things that you should consider if you are buying AI from somebody. And at that point, this becomes questions that you should ask, okay? Now I said, we'll come towards willingness to pay.

And you might be thinking to yourself, Hey, I am building an AI that is for my own team, or my own organization, or my own company. Well, this willingness to pay doesn't really, um, you know, come into, into play here. But what I will tell you is that it does, because if you are building a product, then willingness to pay is very simple to understand that is something that your customer is willing to pay you for it, but if you are building an internal project, that that willingness to pay really shows up in the, in, in the form of investment in your project.

So how much your organization or your company is willing to invest in it, that's really also willingness to pay as well. So with that context, how should you think about willingness to pay? Now this is very simple graphs, and these graphs have kept them super simple on purpose, uh, because these topics can be complicated, but they can be presented at least at the very start of it very simply.

So on on the X axis, you have a done adoption and know the y axis, you have willingness to pay. So willingness to pay actually increases that adoption, but there is a tipping point unless you have reached that tipping point. Um, your ability to charge for it, either to your company or to your customers is not gonna be that high.

And then once that chasm of adoption, um, is jumped or, or your users on the other side of it, then the willingness to pay is directly tied to the clarity of the ROI. And that ROI is based on what is the value of that ai. Now, if you apply it to the art framework, generally speaking, this is not a hard and fast rule.

There could be exceptions, but the more you are, the higher up or the farther out you are in the ARC framework, the higher the willingness to pay attached to it. So if you're just accelerating an experience, which is really truly automation and efficiency game, a productivity game, there's a certain willingness to pay as opposed to when you are creating a brand new experience, which might actually start a new brand new revenue stream there, the willingness to pay would be very different. So these are some of the kind of contours of this conversation that you should be, uh, familiar with or you should keep in mind.

So now in what we have seen in our efforts to productize this is that conversational and interactive ai, as we described earlier, they lend themselves better to just being part of the subscription. So what do I mean by that? They're just part of the product, they make the product better.

Or if you're an internal project, then it makes your project better, you know, uh, cheaper to run, or people love it more and they adopt it more. But this is the willingness to pay is embedded into the product or the project. But as you go towards autonomous agents, the autonomous AI agents, that is where each agent can have a very clear determinable, demonstrable, ROI.

And at that point in time, your willingness to pay internal or external should be attached to that. So what is that? What is that ROI?

Now this is an area that I would say is at the very start of its progress and there, so I'm gonna leave you with a few thoughts around this, um, as you determine these out. So how do you price this? And then remember, if you're building a product, it is pricing.

And if you are building an internal project, and this is about how much investment you're gonna ask, so it's, it, it it works for both. Are you gonna base it on the current cost of delivering agents, right? You say like, Hey, it takes me X number of dollars, you know, to build this AI agent, so I'm gonna price it that way, or that's the money that I'm gonna ask for this project.

Um, well, you could do that. That's a very simple way of doing it. Uh, but the reality is that the cost of delivering this agents is going to, uh, crash over time because of the economies of the scale and the cost of the AI is coming down, uh, pretty, uh, tremendously over time.

Um, so that might not be a good way to do that, or would you do it on the future cost of delivering this agent? Well, you don't really know what the cost of that agent is, or would you do it at the current cost of the task that the agent is going to produce or to deliver. So think about it this way, I said that, think about AI agents as more like human agents or human workforce.

So typically what you do is this, this is a job description, and in the market, this is the pay associated with that, which is a generally accepted cost of doing the task that the job does. So you can take that model, um, you know, in, in view as well as you're thinking about it, um, or you can also think about that. What is the value of the outcome that it is still a rig.

So, you know, what is the task that the AI agent is doing and how do you value that? I would tell you that there is no easy way to do that, you know, so these are just some of the questions that you have to think about. But what I would tell you is that do not leave it for later when you are going into the ai, you have to think about all of these things in one go, and we'll just do a quick summary of it.

In order for you to be successful in ai, you have to think about user experience and trust. These are the two main components underneath this user experience and trust is how do you select the use case for which you're gonna apply AI and use the ARC framework or any other framework that works for you? ARC has worked really well for us, it works very well for our customers out there.

Uh, pick use cases that you want to accelerate, you want to replace or you want to create. Um, and then that's a good way to start getting started on the use cases. Uh, pick a use case that actually matters.

Don't go for, you know, low hanging fruit and then consider what are the, what is the UI or the user experience you have to deliver on those use cases to help people get over that trust gap. So you start with conversational, you know, you start with human in the loop, then you maybe take them towards that collaborative interactive where some of the functions are done by the human, some of the functions are done by the ai, and then eventually taking them towards these genic AI that can be very, very independent. But the key is that this is a journey.

So think about user experience and journey. Now, the two other things, obviously technology, technology. I kind of shared with you some of the things that you need to consider.

Remember, how are you gonna manage your data is going to be really critical. And then how are you going to build those safeguards and guard rails on the top? And then only the technology.

Uh, I know I'm, I'm A CTO myself, but I'll tell you, uh, that when it comes to ai, technology is comparatively the least important part of the equation for the success in ai. It's important, but the least important. And then lastly, think about willingness to pay.

It could be pricing if you're building a product, or it could be investment if you're doing an internal thing. Look, uh, I hope this was a useful conversation, um, because this is based on years of years of experience in productizing ai, successful productization of ai. And we believe that the path to the future is going to be brighter and better with ai, but there are hurdles on the way.

Um, but if you're cognizant of that, and if you are willing to think about that upfront, then we can actually be very successful. I wish you all a great start to the new year, and may your AI projects be successful, and may you be, um, you know, on at the end of this year thinking about all the great journeys that you have been on with that. Thank you very much.

This is Nabil RI signing out. Enjoy the rest of the conference. Hi everybody.

Welcome. We're glad that you've joined us today for another episode of the latest greatest cloud transformation late great cloud transformation. We're talking about really sort of the next generation of how we think about the cloud and the things that we're doing with it.

We're talking about security today, about safeguarding innovation and, uh, strengthening that security. We'll be jumping into app, app security and a lot, a lot of things here. But, um, before we get too far down the road, thank you for joining us for this video series.

Uh, the, the last great cloud transformation is sponsored by CloudFlare. We're glad to have them, uh, on board with, with us working on this, uh, helping input with some topics and things like that. And obviously participating on, on our, uh, live editions, which we do on a monthly basis as well as these recorded episodes.

So thank you for being here with us. My name is Mitch Ashley, I'm VP and practice lead with futurum Group, analyst firm, uh, heading up the analyst area for DevOps, DevSecOps, application development, AppSec, et cetera. So kind of right in, in vain with this, uh, my co-host Alan Shimel is, uh, unattainable, uh, de detained or whatever the word is the phrase is.

And, uh, so I'll be, I'm, I'm hosting both parts of the chair today. Uh, you know, it's a little bit of a coup, but he'll be back next time. We'll see him on our next episode, I'm sure.

So let's get to our conversation, to our topic. Um, let's first start by doing some introductions. I know Chris has been with us on a few episodes here and some different topics.

We've been on other webinars with me and talking a lot about application security and, and, uh, cloud Chris Blask, introduce yourself. Oh, I've been for company my way through the security industry for 30 something years. Uh, I inflicted an early firewall in the markets, something called border ware, uh, in the early nineties and ran Cisco's firewall business, the turn of the century.

I've been following this inevitability curve and my new series on here on Textron, um, from one spot to another, from firewalls into, uh, sim and network management. From that, you know, the obvious next step is threat intelligence. So I, uh, chaired an IAC for a while, and, uh, supply chain has been my focus the last five or six years, you know, so, you know, software, bill of materials, hardware, bill of materials.

How do we connect all these things, which a a and, and currently, so currently I'm, my main role is I'm vice president of strategy for sbe, which is involved in the BUM space. And I've been, uh, co-chairing several, uh, cisa uh, working groups on SBO m sharing. So we're currently have a group looking at ISACs, um, as SBO M distributors.

How does that no, in the middle start taking this information and, and propagating it SBOs software bill materials. Absolutely. Great.

Thank you Chris. Um, Katherine, Katherine, welcome. Glad to have you on, I think the first time we've had you on the show.

Catherine Newcomb with CloudFlare, please introduce yourself. Yeah, great to be here. I'm excited to talk about application security.

Um, my name's Catherine Newcomb. I live in Denver right now. Um, I've been in cybersecurity for about five years at this point.

Um, and I started in the network firewall space, um, in encryption. And now I'm a product marketing manager for CloudFlare, um, for their application security business, uh, where I focus on their web application firewall product, um, our software supply chain product, as well as our encryption and certificate lifecycle management products. Very nice.

And, and I do like to say full disclosure, Textron, you as a customer of cloud flares, we do use their services. Enjoyed very much so thank you Catherine, and team for that. Uh, last but not least, another newcomer to our show, Kurt Hendle, who's with, uh, Teradata.

Tell us about yourself, Kurt. Yep. So I've been working in security probably eight or nine years at this point, uh, but in the software industry for close to 15 years now.

Anywhere from development, uh, into business analysis, product management, even, uh, doing a little bit of red teaming myself, but, uh, I am currently the chief security architect at Teradata. And so I've been focused on architecture mostly for the past six, seven, possibly eight years, and really kind of a generalist. So AppSec is where I spend the least amount of my time where we focus on the architecture, the requirements, threat modeling, um, especially compliance.

We do a lot of the, the major compliance frameworks at Teradata. So we've been pushing that recently. Um, and I'm based in the Pacific Northwest, up in the Seattle area, and happy to be here.

Very nice. Well, all the weather and fires and it's cold and I'm just glad we all made it. Maybe it's 'cause we didn't have to travel anywhere, so, so I hang tight.

I'm glad we're all here and you know, our, our thoughts go, our hearts go out to the folks dealing with the fires and, and, uh, some weather down south and southeast, et cetera. So, um, let, let's kind of jump in this way. Um, I, it's a big topic when we talk about sort of the kind of current state of the cloud and where it's moving to.

Um, but I don't think it's too much news to everyone that application and app APIs, API first kind of design into applications, you know, it isn't just things that sit at the edge anymore. We think about also the security of the AppSec and the kind of, uh, software we're creating, the innovation that we're making, um, as maybe as part of the cloud. 'cause sometimes applications lives within it, you know, like a, like a provider like CloudFlare or certainly at the edge or at the core as well.

Maybe Catherine, if you wanna start us out with how do you, you're, you're, you're managing, doing product management in this space. How do you look at this, uh, sort of this problem or this space and define it? Um, so looking at application security, um, when we're talking about this at cloud, we're mostly talking about web application and API security.

So if you're an OSI person, layer seven model, um, and you know, when people are accessing these external facing web applications, they're doing it from a ton of different devices and in a ton of different ways. So they're accessing from things like mobile, uh, desktop, laptop, and they're accessing these AppSec that could be hosted anywhere. So on-prem, in public clouds, private clouds, hybrids.

Um, so as we're securing, we need to think about how can we secure, um, all of these users and the end servers as they're sort of accessing these web AppSec, right? So how do we make sure that, um, mobile traffic is protected, user data is protected, um, and sensitive data is not, you know, leaving an app. And then how do we make sure that a web app server itself is protected?

Um, so at a very high level, that's about what I think, that's what I think about when it comes to application security. Um, some new things we're thinking about in this space. Um, I talked about software supply chain.

This is increasingly becoming, um, an area of interest as people create more complex AppSec with more third parties in them. Of course, API first development has also meant we've had to adjust our thinking a little bit around application security as well. Kurt, how about you as a, as an architect, security architect, you may, maybe you don't get into the innards of applications per se on application security, but traffic over there.

Obviously our networks are heavily API driven. Um, you know, when you think about the security architecture, where does this fit into your purview? I think it, it fits in really everywhere, right?

So we're, we're building these huge applications, sometimes small applications. I mean, we do all sorts of scale at Teradata. And in my previous roles, I've, I've worked with pretty simple AppSec all the way to super complex microservices architectures.

And so, like Catherine could have said, you have the mobile aspect, you have the server, there's application code literally everywhere, including on the person's device. And so how do you secure it as best you can, um, within reason, right? Because if it's too secure, it doesn't work.

If it's not secure enough, well, you end up in the Wall Street Journal and you're in trouble. Um, so we, from an architecture standpoint, we really try to focus on all different aspects of it, where the biggest threats lie, um, and then implement controls and use technologies to, to simplify the implementation and streamline it without making it overly complex. And so it's, it's just becoming more difficult given that, um, the, the kind of classic perimeter is gone.

Right? I'm sure you can relate to that, Chris. Oh yeah.

Well, it was easy back in the day, right now you had to get on the internet and you needed a firewall. Get a firewall, right? And I'm thinking as Kurt and Catherine, your comments remind me of these transitions we go through.

Like there was the mainframes before our time, but you know, I, I'm old enough to have seen the end of that where all of your capabilities are just to keep one computer running and run terminals and printers and things off that. And then we get into where I came in, where we're starting to build networks. Fractally more complicated.

Just, you know, how do we do that with, when all of our resources were just keeping one computer running, we figured it out, you know, now we're here, we're talking about web APIs, Catherine, you, you know, the data going in and out on with being stored 30 years ago, you couldn't have that conversation. Now we're saying, alright, what do we do in this case? And it's very complicated and I think in, and Catherine you mentioned the supply chain.

This is, I think we're filling in the dots. Security has been is not, i i is not new, right? People have been saying, you should know your inventory for a long, long time.

And we gotten away with not knowing it. Now we're starting to fill it in, need to actually know where the software is, where the data is, and we're working through that. So it's exciting times, but it's not different in type than other transitional periods.

Certainly is an evolution, right. Of what we've gone through. And to think about, you know, from the Baston host days earlier on, free firewall, um, Well, firewalls used to be a million dollars a year.

I think when I got involved, you know, at least as I tell the story, there were a hundred in the world and they typically were seven computers and a team of people. And my argument at the time was my mom needs one. Yeah.

You know, and so we're at this stage where what used to take so much time in here in the API, uh, world, it has to take less time a lot. It, it, so let me, let me throw out this hypothesis here. I think it may be pretty obvious, but maybe it isn't, is I think we live in a world, you know, now we we're thinking about things as zero trust, right?

Of of, you know, anything is susceptible, being compromised and could compromise other things. How do you pro protect all parts of the network applications, the infrastructure? But we're also living in a world where if so much is determined by what our applications do, not just connecting users to AppSec, but applications really utilizing the network, being part of the network.

It's a dynamic world, right? It, it isn't a good set of firewall rules and an application firewall, and we're all good, kind of set that up. And it isn't the old days of, if I've got a pizza box in, in my rack for every function that I need, and they're all doing their thing, I'm good.

Right? We need it. It's a much more dynamic environment.

So I'm not saying we're reconfiguring our security all the time, but a security has to adapt to, you know, what's happening in the application. Because we may distribute it to a different part of the edge tomorrow with Kubernetes, or we may, you know, uh, acquire businesses. Suddenly a network is looked much different than it did, you know, three weeks ago.

I'm, I'm curious, Kurt, as a practitioner, you know, how do you think about that of, you know, you mentioned microservices and all the things that are being created, you know, in the groups that you're working with. Um, we, we hate for security to be sort of the last thing to be thought of, but you wanna be in the conversation so you can prepare as well as react when you need to react. I think what you just said is, is really important, but you wanna be in the conversation.

You don't wanna be doing this retroactively. And so when you're, when you try to tackle security retroactively, it is infinitely harder to accomplish than if you do it from the beginning. So I have, I do it both ways.

I have teams that we work with proactively where they bring us in at the very start and we're building the design with them shoulder to shoulder, drawing the picture in doing security by design, or by default as we like to say now. Or we have legacy applications, which you're doing retroactively and they're quite a bit higher in terms of risk because they've been neglected for so long. Or you find out about something after the fact and it's like, well, how did this get out there?

Well, there's shadow IP in a lot of the world. And so it's, it's hard to, to really kind of put a, a recipe together that successfully achieves it. And then with the, the rapid pace of technology today and how the cloud has just kind of blown this wide open where people can deploy new applications in a hundred different ways faster than ever.

How do you keep up? So you have to implement tooling within reason without doing, without having too much sprawl. You have to have the right personnel partnering with these teams, uh, to ensure that you have coverage and that you, you're really architecting things from the start.

Um, and not just kind of using band-aids and bubble gum per se, to, to secure your environment later on. Catherine, appreciate your thoughts on this because, you know, I remember the days of networks for speeds and feeds and points of presence and connecting A to B and kinda looked like this nice diagram that you stitched together and that was a network and you secured it. Now it's overlay on top of overlay and it's changing and mm-hmm.

You know, it's, it's multiple pieces that, uh, much more complex to, to secure. How do you, how do you have this conversation with people? Yeah, definitely.

So as you were sort of talking about this, you know, obviously there's a need for responsiveness and customizability and security, but I actually also wanna make the argument for unified policy management in application security. This is something that I've seen actually, for example, um, we have some customers who have protected their SaaS AppSec, like what is traditionally more of a network firewall or zero trust type use case with the same policy they're using for their web applications. And by doing this, they're able to do things like make sure that zero day exploits aren't able to exploit their SaaS AppSec, you know, as well as their, um, web AppSec.

And we see a lot of value out of these unified policy managements. I was talking earlier about, you know, how we have all these AppSec hosted in different places. We see a lot of customers, for example, will host, um, you know, an app across multiple clouds for like a resiliency use case.

If they're worried about outages, you'll, you'll certainly see that, um, for example. But then how do you have to, you know, actually secure an app that's stored in multiple places? Do you write different policies for, for wherever those are stored?

Um, do you write different policies for APIs versus, you know, traditional AppSec? Um, so we see a lot of benefit out of like a unified policy for all of those disparate sort of endpoints and all of those disparate, um, locations that they're stored. Uh, for CloudFlare in particular, how this sort of works out is our WAF is like the backbone, the architectural backbone of the rest of our application, um, security portfolio.

And this works out really well because you can do things like have a WAF and an API like positive security model protecting your APIs. Um, so you could do things like detect zero days and volumetric attacks, which are, you know, APIs can also be susceptible to as well as, you know, do the things like Ebola and, and all those API specific attacks all within sort of one, um, control plane, which we find a lot of people get a lot of value out of because of this really, really disparate environment. Okay.

Chris, I saw a lot of hand waving head nodding you about, jumped outta your chair on this one. And so I kind of have a feeling you might resonate with this. No, uh, I, I gotta throw out there, I was gonna, uh, before Catherine got into the, the policy thing ware, it's been on lot time, but yeah, the concept of an SBO m the software bill of material for the current release version of Adobe Acrobat as opposed to an SBO M four as we're look talking about here, some ephemeral web app that one time for five seconds exists in the cloud.

You know, think about that. How do we, how do we deal with that? And I, and, but I think policy is, is the answer all hacking?

All hacking is policy hacking. I will figure out how you do things and I will figure out where the gaps are and I'll engineer that gap. And we live in a world right now where we generally have no idea what policy applies to any of us anywhere, with few exceptions.

And in this topic, and because I'm used to the supply chain topic, imagine I needed to get the, the SBO M or custody information about a piece of software on his phone right now. I could get it in between five days and six months today I need to get it in half a second. That means I need to read the policies between me, the person who bought the phone and the first time the company I bought it from, and like their relationship, their contracts, their policies, you know, upstream all the way.

And we have to get that done in the next decade. So without unified and, and, and adaptable, you know, transparent policy frameworks, none of this technology is gonna make a difference. So I think we, we will do that.

And there's interesting things going on down that path. It's kinda interesting in a way, just connecting dots between what you said, Catherine, and you were talking about Chris, there's your own unified policy management, right, of what you're doing. So you know, you're, you, what you're applying where and how you're applying it, and then that's how that interconnects or interrelates with the people you connect with, work with, use their service product, whatever that is too.

And I, and I appreciate what you said Chris, about, think about just serverless technology like a lamb to kind of service, right? That, you know, it's there now, it's gone tomorrow may not be the same thing. It was a second ago when it, when it ran.

Um, so in some ways, Catherine, it's sort of a dynamic unified policy management, right? It can't be a static thing. Am I, am I on base here?

Yes, of course. You know, you do have to be responsive to the environment, um, you know, a threat landscape. Um, this is one, one area where I strongly advocate for actually ML driven, um, detections and policy.

Uh, this is a thing where, for example, if you have a really large data set, uh, you can train your ML models. Um, how we do this at CloudFlare, just 'cause I think it's a little easier if I give an example and it's, uh, we will score each request on a scale of like one to 99. And if something is less than 30, that means like it is very likely to be an attack.

And because we have, um, hundreds of terabytes of requests, or sorry, hundreds of millions of requests every single day, um, we have so much data we could train this on and say a little blog in Malaysia gets attacked by a new attack we've never seen before. Suddenly, because that tiny blog in Malaysia got attacked that gets feed and fed into our ML model, we don't have to rely on a security engineer to like go and find and analyze that attack and turn it into a regular expression like firewall rule. Um, the ML will basically just say, okay, like, since it matches something like this, um, we will just automatically block it.

And this is why I'd say ml um, sort of combined with that traditional, um, you know, security analyst looks at the traffic and writes a rule that matches it and then blocks traffic. Um, you gotta combine I think these types of approaches. So ML is a really, really great application, um, when it comes to being responsive to the threat landscape.

And we have some data around this as well. Um, we recently, not that recently, like half a year ago released our annual application security trends report. Um, and we found out that, uh, for example, like zero day vulnerabilities, um, we probably wouldn't have been able to find this out with just security engineers analyzing it.

But with our ml, we were able to detect, um, and exploit 22 minutes after the, uh, proof of concept was posted online. So, um, really, really great applications there. A lot of interesting stuff going on for sure.

Well, that doesn't make the case for dynamic security. What does, right. Um, I, I'm curious, Kurt, how do you, is, is someone, you know, applying these things, applying security?

Are you, are you looking at things like ml? Are you doing it via yourself? It's something you look for in the vendors, the partners that you work with.

How do you leveraging either that or the kind of technologies to help shorten that cycle between when things change and how you can account for it and secure it? Right. The, I think the ML piece of it is, is hugely important because I mean, humans, we're slow.

The, the technologies we use, the, the computers and, and each servers process all of this far faster than the human brain and I ever could. And so we need to augment ourselves with this technology. So anytime we're evaluating new solutions and bringing them in, like I'm currently in the process of implementing a big one right now that focuses on platformization and ai, ml, it's all part of it because it humans with eyes on glass, like it's great to have those guys in the sock, but they'll get overwhelmed very easily with the speed at which things happen today.

And so we need to leverage technology and machine learning enables us to do this faster than ever, and it's only getting better, right? And so augment the human with that technology and you can very quickly pare down all of that information to what matters most and focus on real attacks like Katherine was just talking about. I wonder, you know, there's so much activity around ai, of course, a lot of it because of gen generative ai, um, Chris to the security engineers have to become machine learning experts to be able to do this stuff.

What does it take to really leverage it? No, but knowing, knowing something isn't gonna, um, uh, causing any problems. But, uh, I, I just can agree more with, with both, uh, with Kurt and Catherine.

'cause you know, and, and you're point, it's all about time, time the transparency. How, how long, and again, I've seen this over and over in my career where we get to these points where what we're mostly doing is sharing the war stories. You know, I have no idea it was 72 hours, none of us slept.

There was caffeine. And, and my my question always is, okay, if there was twice as much, what would you do? Because obviously that we're at the limit.

We can't possibly work any harder, stay awake any longer. And, and this, yeah, ai, ml, Oracles, whatever we call it in this, uh, my, a big has been a big part of my, uh, my focus on supply chain before it would, you know, AI became, you know, uh, a general, um, uh, generative, what the hell do we call it? I'm sorry, I forgot.

Yeah. Generative ai. Yep.

Generative ai. Yes. Um, too many terms To throw Around.

Yeah, because again, we need to, you know, just for supply chain things, I need to read the contracts. I mean, I can literally call someone up, you know, it's not a security engineer, but it's some administrative person of the company, and I had to get them on the phone and get them to pull A-A-P-D-F and read the contract and find out if the clause allows me to get the information I need. That's not worth a human's time.

I mean, that's the kind of stuff that computers can do really well, and they're just beginning, but that's obviously the direction we're going. And if you can't see your policy environment five years from now, by various definitions, your competitors will be so much faster than you are. That don't matter anymore.

Kurt, I'm, I'm curious, without giving us too many specifics about Teradata not asking you for that, but what's your sense of, what are the, what are the new priorities that are on your Yeah, on your horizon or things you're dealing with now and that you've kind of added in the last year or so? What's changed about how you're thinking about security and that you've gotta address now? I think there's, there's always classic problems that we, we have to deal with and tackle.

Like, we can't forget things like identity and network security and the rest of it. But the, the prevalence in the emergence of generative AI and putting AI and machine learning in everyone's hands has meant that security teams have to be hyper aware more so than ever because these new technologies, people are latching onto them without considering the risks. They're like, that's awesome.

I can speed up everything I'm doing. And suddenly you see a news story about, well, what was it like Samsung engineers leaked their code through regenerative AI solution or whatever. So you're, you can quickly lose intellectual property or put it at risk.

And so we have to think about securing our environment for those solutions, or putting the guidance out for people to use AI and machine learning. Um, and I mean, getting visibility of all of this, and another big one that's been getting pretty popular and we're seeing a lot from different vendors and acquisitions and whatever, is data security, posture management. Where is my data?

Where is it moving? How secure is it? Because at the end of the day, that's what the attackers want.

They don't wanna sit in your network and use your resources to, to launch attacks as much as they used to. They wanna grab your data, steal it, monetize it. So need, we're, we're focusing on data security big time in, in the more recent years, especially, um, forward looking because we have more data than ever.

Interesting. Catherine, from your perspective, you know, communicating with so many companies, what are some of the changing priorities from your, from your viewpoint? Yeah, I mean, certainly the gen ai, um, piece is something we're seeing a lot.

Um, everybody wants to put an an LLM on their web application. Um, and of course that means that you have to think of that as like a data security concern as well. Um, because you wanna make sure your LLM is not gonna like accidentally leak somebody else's social security number, because that's certainly happened before.

Um, and so at, at CloudFlare we're thinking about this of like, basically how could you basically just put a WAF in front of an LLM, um, from that perspective, how could you prevent it from exposing sensitive data to the end user? Um, but then, you know, you gotta think about these more complex issues as well. Like, how do you prevent somebody from poisoning the model?

How do you prevent, um, you know, some of these other, like, how do you prevent it from hallucinating? Uh, these are all, you know, sort of adjacent to security concerns. But, um, but nonetheless, we see some security teams focusing on this, um, increasingly.

Um, additionally we also think about, you know, the, the LLM sort of security use case is a little bit of a just, um, increased API security use case since a lot of times, um, people are not building these LLMs themself and hosting them themselves. They're often, you know, bringing in LLMs from third parties, which, uh, necessitates, um, APIs, right, for integration. So how can you make sure that these APIs are staying secure and not leaking them back to the host and whatnot.

Um, so that's definitely something we're seeing as well. Um, I would say additionally, one thing I've been hearing a lot lately is, uh, software supply chain security. Um, I think Kurt mentioned the beginning, um, sort of securing code that lives on the client device as well.

Um, this is something that we've been hearing a lot about, especially as it comes with the PCI four, um, compliance, which is gonna be mandated at the end of March, um, in a couple months. Um, PCI four has a new compliance requirement around client side security and securing, um, the client side, like software supply chain. Um, so this is something we've been getting a lot of questions and inquiries lately.

Um, you know, how much are organizations responsible for, um, the code that loads on their end user's devices, uh, when they visit their websites? Um, this is something we're seeing a lot of people trying to actually actively get control over, um, and make sure that they're not, you know, serving, uh, code to the client devices that could do things like download a crypto mining software onto their phone, which, um, believe it or not, we have seen somebody's trying to make, you know, personal laptops part of a crypto mining network, which is pretty crazy. But, um, so yeah, I would say the client side component is, is something I've been hearing a lot lately as well.

I, I just have to say, I, I love living in a world where we can use the term, uh, you know, hallucinating artificial intelligence in a conversation like this. Seriously, just, we, we understand about that. It's Not a sci-fi movie.

It's real. Oh, It's, it's real. Yeah.

Hey, so I've, I've kind of a left field question for you, Chris. So if this, if I throw you too far off the track, I'm guessing you're thinking about this though, is, is there an SBO in our future for LLMs and s SLMs and all of these things? 'cause in a way, this is a whole nother part of the software supply chain, right?

We're handing off to something that's doing inferencing, either on a chip on our handset or in the cloud, all of the above. How does that fit into, do we need to be thinking or at least wondering how we're gonna solve this problem And not only not left field, and that's, that's right in the middle of the, the track. So in short, yes.

You know, there's ano there's another system working group, um, Dmitri Rayman, uh, my colleague CTO at at side beats is, uh, a co-chairing now on, on AI bomb, right? An AI bomb has been talked about for a long time. So what does that even mean?

You know, so AI is code. So there's this, you know, same sort of standard SBOs stuff about that, but there's also the training data and the models are produced, right? And this sort of goes back to my last comment about ephemeral, ephemeral SBOs.

You know, we start with the idea that IMA software provider and every 16 years I release new code and I carve a new sbo, you know, on purist graphite. Um, but we live in a world where code gets compiled and used all over the place. You know, how do we even look forward and say that I can commit to a policy that says I will if asked to provide the contents of this code without, um, actually going out and printing or saving or producing quadrillions of SBOs forever, you know, in, in exabyte storage.

Uh, so this AI is, you know, what we're currently calling AI is just another forcing function of the level of complexity we're at. So we need to be able to provide the answers to live up to the policies that we've agreed to, um, which is, you know, you know, in the SOM case we're talking about a software inventory that I will be able to tell you what code that was running or you know, what data set was used, and we have to get there. And, and, and it is, it is reasonable progress down that path.

It's a, it's a complicated one that is very similar patterns to how we'll do other things of similar complexity. Um, Kurt is, is that on your radar yet at all, kind of thinking about security of, from a supply chain for LLMs and AI and ML algorithms and all that kind of stuff? No, I mean, it's, it's certainly jumped up on the radar, especially since the whole SolarWinds thing happened.

Um, as Chris Blask talking, it got the wheels streaming in mind of, well, if we're gonna be kinda, we're moving towards leveraging a AI in the sense and dynamically generating SBOs and things, is this another attack vector we potentially have to watch out for? Is how do you weaponize that and, and protect against it? Because I mean, as we see attackers evolve their tactics and techniques faster than ever, they're coming up with new creative ways that defeat the traditional approach in microseconds.

And so how, how do you stay ahead of that curve now? And so I obviously, I don't have the answer right now, but it's, it's really interesting as Chris talked to start thinking about this, this new sort of problem that we're facing. And again, it all falls back to the rapid evolution of technology.

Yeah. Speaking of that evolution, uh, just in the last week or so, uh, Satya Nadal, the head of, uh, Microsoft was talking about the death of SaaS, meaning that's kinda the clickbait one-liner that what I think he was really talking about is evolving nature of software architecture that I would describe it as Today's microservices or backend code are tomorrow's AI agents, right? We'll see more and more parts of AppSec built through, you know, with or through or maybe completely with AI agents.

And it reminds me of going into the, uh, cloud native era of, oh, how do we secure microservices now that we're gonna do that kind of thing? That's kind of the, that's the next edge that we're, we have to work on and think about how, uh, there are different things we have to do for securing AI agents. How are they orchestrated?

Is it Kubernetes or it, some other thing that's managing all those things. And, uh, given that we're putting AI agent building capabilities in everybody's hands, in many cases, it, uh, could make for interesting. I use that in a nice way, uh, interesting environment to try to secure and manage.

So in some ways, the future is bright, but it may be a pretty intense at the same time, same time. Well, and I think kind of building on that too is the, the technology behind ai, it's backed by machine learning. Like you're, you're making technology autonomous, right?

So it's not as predictable anymore. So how do you secure what, when you don't exactly know what turn it's gonna take next, Non-deterministic, right. Well, I, I gotta add a note, a note of hope though, because it's easy, you know, to your point, uh, Kurt, the short answer is yes, because there's a new attack vector.

Oh, yeah. Um, but, you know, you know, throughout my career I've been arguing this one, it's like, we'll probably keep the lights on. It's like, no, no, if we don't do this and that, then you, we will, you know, the, the, we're on this, we're doing this call right now.

We've managed to figure out everything else over this point. And not only that, but I think that we're, we've been mowing the lawn. I think, you know, what we need to do, generally speaking, and cybersecurity has been known maybe forever, certainly 50 years, but we haven't gone around to doing the vast majority of it yet.

'cause we haven't had to. But as we do, and I, I will take a risk and, and put a lot of my, my faith in policy, you know, in, in real policy transparency, you know, in, again, in this decade, it gets harder to be an adversary because, you know, these are the happy World War II fans out there, you know, or no fans, you know, but the, the ubo wars, right? There was the happy days when you could just have a U-boat and sink shipping all day long.

You know, that's kind of most of the world, most of the, the history of the internet to date. It's not necessarily gonna stay that way that long forever, where there's always a new attack service, and there's always a, a, a new way when the last one is, is locked. I think we will, will keep it running.

We will all be fine. And I think over, you know, at least over a period of decades, being an attacker will become much, much more difficult. I mean, I might argue it already is becoming more difficult.

It 'cause the, while, while the, the technologies we use as practitioners are getting more advanced, that helps make it more difficult for the adversaries of the world. But that's not to say that they can't employ similar technologies, right? So now we're kind of, we're creating that chicken and egg problem all over again and playing a game of cat and mouth.

It's kind of the next arms race, if you will, as technology evolves, everybody has access to it. Well, let's do this. I appreciate all the conversation, and we brought up a number of topics, um, just as a kind of concluding thought.

Uh, we, we've been talking about what are the things we need to be thinking about? Maybe they're newer, maybe they're on the horizon, maybe we're already working on this today. Um, if you had to say, there's one thing you'd really want to emphasize this, if you were, you know, somebody who's listening to this and maybe making a few notes, the thing that sort of stands out to you as something really important to be thinking about in the next, say, six to 12 months, if not today.

Um, Kurt, do you want to give us your thoughts? And then Kathleen, if you would, and Chris, you can wrap it up for us. Sorry, did I say Kathleen?

I mean Catherine, excuse me. Kathleen. I work with a Kathleen.

Sorry if I've been doing that. All good. Okay.

Yeah, I, go ahead, Kurt. I mean, it's, we wanna avoid that situation where everything is a priority, so nothing's a priority, right? I think we, throughout this conversation, we've highlighted the importance of, we've highlighted the importance of application security and how it's, it's becoming more important than ever because our application code is, is literally going everywhere.

And that's, that's kind of the gateway for a lot of the attacks we're seeing in the world today. And so I think the, the emphasis is on application security, but it's also to say, let's not forget the rest of it, because all of the, the other parts of cybersecurity are hugely important. And we still need that visibility.

We still need the coverage, and we need to be thinking about ease of use as well, and avoiding the sprawl. So I know these aren't necessarily specific cybersecurity things, but they, they help you simplify your approach and, and focus on what matters. And that depends, that, that changes everywhere you go.

Every enterprise or company has different priorities. And so I think focusing on those things help enable us to, to focus on what matters for where we're at currently. Good.

Catherine? Yeah, so I mean, like Kurt said, you know, we wanna make sure that we're not making everything equal priorities. So I think when it comes to application security, which is of course, my area, what I would say is most important in this space is visibility.

Um, the attack surface is getting more complex, applications are getting more complex, um, you know, where they're hosted is getting more complex. So how do we actually have visibility into our entire, entire application attack service? How do we have visibility into the APIs developers are creating so we can actually secure them?

How do we have visibility into the software they're adding, um, to these AppSec? Uh, that I would say is probably the most important thing for application security and also one of the most challenging things. Excellent.

Chris, Uh, the, you know, Kurt and Catherine both want exactly where I'm going, so I'll just build on that. You know, do do things that save you time to transparency. You know, if you, you know, don't panic, nothing's on fire.

And, and when things are on fire, panic less, right? Just take your time and, uh, getting visibility, you know? Yeah.

Look at how long it takes you to figure out. And anytime you find a, a, a way, you know, in this, in this topic we're talking about here, to spend less time to figure things out, you have all that time back to do things. And it's easy to just, you know, particularly in transitional periods, to just do more and more and more of what you've been doing, you know?

But, uh, understanding the environment you're in so you can apply your resources appropriately is, is everything. And there are lots of ways to do that these days. You know, there, there's a lot of rush, panic, and there are a lot of, you know, I will say it, AI and things like that out there who will actually make your life easier, give you some of your time back.

Mm-hmm. And you'll that point feel better what's going on, make, make, and make better plans, have better strategy. Yeah.

To that point. Exactly. Chris, and, and Catherine mentioned it c around, uh, ml, you, some of the things that I'm really excited about AI is actually just the understandability of what's happening.

You know, Kurt mentioned about as things ramped up or, or you did, uh, uh, in, in the, if the tax doubled, right, how would we handle that if we're already maxed out? So some of it is just handling the volume of things that are happening. But I think one of the things that I think is most exciting about generative AI is it's also so complex.

No one person can understand the full system, right? Or maybe even understand truly what's going on in a case of an attack or where you have vulnerabilities. And generative AI is starting to make some inroads and help us understand systems and, and giving us some insights to some of the complexity.

We may not be able to fully get into our head all at once. So for example, I've been doing some work around how do you modernize mainframe applications? Well, nobody was around that built those things.

Well, maybe people that built the network aren't even around, right? So help us understand what really is happening with all this data that we've collected. And the natural language interface through that is, is a great aid, and I think it's just a real practical thing that we can start to begin to use today.

So don't think of AI as just as the next, you know, it's gonna replace all of our software and it's all gonna be different. And what do we do? There's things today that is already helping us with.

So, you know, there's some real things too, not just what's on the horizon. Well, thanks to all of you. It's been great, Catherine.

Uh, we appreciate your perspective, and Kurt, you're bringing, um, your experience and perspective. And of course, Chris, always good to be chatting with you and your connections into the security world. And some of the folks are working, collaborating together, which by the way, is another superpower we have in security.

And that's the fact that we work together and collaborate on, on these things. We're not going at it alone. So thank everybody for the good work that we're doing to help advance.

We hope this has been a helpful conversation for you in thinking about the, the last great cloud transformation, what we're doing differently and thinking about, uh, as we move forward. So as we've got our heads down, getting stuff done, getting our priorities done, getting our plans in place, and executing for 2025, but also kind of thinking a little bit about what's next and what we might be considering and learning from others that are working in our space. So thanks to all of you.

Thanks everybody for joining us today, and thank you to the Cloud four team for, uh, for sponsoring, um, our show today. And we look forward to joining us either on another recording or Sure. And check the calendar for one of our live events where folks can ask questions and engage with us in a similar kind of conversation.

We have many of those coming up. We'll talk to you again soon. Take care, everybody.

All right. Well, thank you guys so much for coming to my talk. I'm talking about, uh, when Agile doesn't work, um, and just replies to common objections to agility.

Um, so I've been doing work with software for about 12 years. Um, I see here, I said primarily in agility because I've been in Agiles, a scrum master for, uh, several of those years. But I've kind of dipped in and out of several different roles.

Um, I've been in product ownership for a while, a little bit here and there. I've been a business business analyst for a while. I've done some software development, so I've kind of been all over.

Um, and so that's kind of where I've been. Majority of my work is in agility. So with that, um, that's where a lot of my, um, experience comes in.

I have some, some stuff to say about, um, you know, how agility works. So, um, and this is a picture of my wife and my child, just so you guys kind of know who's talking to you a little bit, a little bit of connection there. My, um, my child is here, he is, uh, three or four, I think, but he's, uh, seven now.

Um, just about ready to turn eight. So he is gotten a lot bigger, but I just really love this picture, so I just wanted to share that with you all. Uh, so yeah, there you go.

So what I wanted to hear today is, um, like I said, I've been here, uh, several years, and especially in agility, and I have met a lot of people that have a lot of stuff to say about agility and why, you know, it doesn't work or why, you know, it might be good generally, but hey, it doesn't work here. And, um, and so really walking through those arguments that I've heard, and then kind of some things that I've experienced along the way that kind of serve as a counter argument. Um, so, so I've got several slides that kind of have that same format, just the, the argument and then kind of counter arguments against that, so to speak.

But then the next part is how to engage with people. Um, because when, um, when I first gave this presentation several years ago, that's one thing that I didn't really include. And I feel like now I really need to include some of that, because as I was going back to these slides, it felt a little bit like a, here's why I'm right and why you're wrong, kind of thing.

Like, Hey, agility works. You're wrong. Here's some counterarguments to prove that it works.

And like, no, that's not really the point. That's not, I'm not here to try to be right. I don't think anybody here hopefully is here to be, right?

I think we're trying to do right for our business. Everybody cares about the business. People that don't like agility, they care about doing things well.

Um, and we care about doing things well, um, as agileists. Um, and so really just wanna make sure, like as an agileists, I, I believe agility is correct. I believe it's the right way of going about doing things.

Um, I wanna just talk about how can we engage people in a way that's effective to help you under help 'em understand the benefits of agility. So, um, but yeah, first talk about the things I've heard against agility, so we can kind of walk through them. So, um, I'll go ahead and start down that path here.

So one of the things that I first heard when I was starting my career was that Ag Agile isn't really known for quality. We really need high quality. And again, some of these things you may have heard and then other things you may be like, no, I haven't even heard that, because it, it just doesn't seem true at all.

This is one thing I have heard, and just as a caveat, this is really something I heard when I first started out. Uh, and the company I started out at was a logistics company. Um, and they were smaller company, but they were hungry.

They really, really wanted to just go after growth and go after all these new clients and grow as fast as they can, as strong as they can. And, um, and so they had a backlog of a ton of things that they really wanted to get done on our, on our products. They wanted to add a lot of new features, um, a lot of really good features we were all excited about.

Um, but they wanted to move so fast, and they were like, Hey, we gotta move faster. Like, let's, let's try this agile thing. Like let's get into this whole agile thing.

Let's really step that up so we can go faster. And so, um, a lot of the dev team that I was on, they kind of equated, oh, agile is that thing where you just push so fast and so hard, you just neglect quality along the way. And so this is what I heard at that point, well, agile isn't known for quality.

We really need high quality. We can't skimp on quality as we build this thing. And, um, you know, again, the the counter, and again, all these things, as I mentioned earlier, I wouldn't just say, here's the counter, like, this is why you're wrong.

But I want us to understand from our perspective, um, kinda what the counter is in ourselves so that whenever we do engage with people, we know what's on, what's going on. So the counter that I have for this is that without exception that I've seen, like this is true when I gave this talk. And it's, it's also true now, um, when I first get this talk that is, is that without exceptions, teams that I have personally seen, do Agile have been of much higher quality than the ones that have stuck to Waterfall, um, just due to, uh, various factors.

Um, and I mentioned Scrum as well. I hear as if you using Scrum. Uh, so if you're using Scrum teams can specify some quality gates in the definition of done.

So those of you that use Agile, most of the time you are using Scrum. Um, if you don't know the difference, uh, agile is a very broad term. Uh, there's, there's a manifesto out there that has just four things that describe what Agile is, and then 12 principles, and that's all there is.

You can read that in three minutes and you'll, you'll understand to some degree what Agile is. Whereas Scrum is kind of one layer above that. And it's a framework that is a little bit more intense.

It's not, it's not fully intense, it's only 22 pages, but it's definitely more than one page. So it's a little bit more than Agile. And, uh, if you're unfamiliar with what Scrum is, although I can't imagine that because it's so widely used, but Scrum is where you kind of take a specific measure of time.

Like two weeks is pretty typical. You say, for the next two weeks we're gonna make a plan for these next two weeks we're gonna do that plan. And then at the end of that two weeks we have this thing, we've built this piece of software that gets integrated into our app and that piece of software we're gonna test, we're gonna, we're gonna review that with people, we're gonna review how we did our work, and then we're gonna start the whole process over again.

You're gonna keep doing that time after time after time, um, until you build something great, you know, incrementally, uh, one iteration on top of another. Um, so when you do that, uh, one thing that's really important is when you get to the end of that, of that time box, so to speak, and the, the end of that two weeks, you want to have that thing that you've built quality. You wanna say, okay, we cannot call this thing done.

We can't call this thing done unless it meets our criteria for what done even is. There's a bunch of things that we have to have in place if we wanna call that done. And, uh, that's called the definition of done and in Scrum.

And so if you have that definition of done, you can add anything you want into it that you and your team know to be a good quality game. So, for example, you can be like, I don't want this thing released until it's fully tested, a hundred percent code coverage. I don't want this thing out until QA has given, uh, has done a full aggression test.

I don't want this thing out until, um, our, our, um, our PO has a product owner has done their due diligence and looked, looked over the whole thing. Um, I know whatever else there is, uh, out there that's gotta go into your definition of done. So you can make sure you say that thing can't pass our, our can't pass into the, into, um, into the next stage until it meets that definition of done.

So in that way, yeah, that the quality is a very big part of Scrum at that level. And then the last part I just wanted to to touch on is that quality is actually, so it's something that's a good practice, whether it's Agile or Waterfall, it's not necessarily that like, well, you're agile, you're less quality, or 'cause you're waterfall, you are less quality. Like these quality gates are important, you know, whatever you do.

Um, I think it pairs better with agility though, as you see there at the bottom. And the reason I say that is, is really because of the iter iterative nature of agility. Like if you're spending two weeks doing something and then testing it, and then even if you release it within that two weeks and then you release these small chunks of work, um, you only find a few bugs at a time and they come out iteratively and incrementally.

Whereas if you have waterfall, you'll spend months and months and months building something. I've seen this spend months and months and months building something, release it, and then a plethora of bugs come in. And that just doesn't seem quite as tolerable to users that I've seen.

As you release something, a few bugs come in, but you fix 'em in a day or two, and that's like, okay, that's fine. Waterfall usually something after years, and then it's just buggy. And people are like, well, these guys built a buggy app.

So, um, anyway, that's why I think it pairs better with agility. But either way, uh, quality is not something that I would say is anti agile or Agile is not so anti quality. It's the opposite, um, one I can see.

So that's the first one. Uh, another thing that I've seen a lot or I've heard people say a lot is that, um, you know, we developers know best. You know, we built the app, we've done the coding.

We, we know what this app really needs, but, uh, we lose our voice. We give it to the business. The business just continually just in Agile.

There's a product owner. The product owner tells us everything that we need to do, and we have no voice in that. Um, and uh, that's absolutely not what Agile's about.

In fact, agile, one of the 12 principles of Agile said that one pager that kind of tells everything that Agile is one of those 12 principles, is that business people and developers must work together. Um, and I, I'll keep on reading some of these things here, is if the product owner is too focused on one side, so if they're too business focused or if they're just a really technical product owner and they're just really focused on the technical side and they don't really prioritize some of the stuff that the business is asking for, um, then that's a good opportunity for you Scrum Master to get them, get involved and, and, and help the product owner and say, Hey, let's come up with some kind of a balance here. How can we make this thing work for everybody?

Um, and the really key things are the bottom two, being open and building trust. So as a dev team, you don't wanna cannibalize all the features. Kinda like I was saying earlier.

There needs to be some kind of a balance there. I have seen both sides. I have seen, uh, the business come in and say, Hey, you really need to get our features done.

This is important. We really need to get these done. And devs like, no, we can't get these done.

We really gotta work on these quality things. And then the dev side actually gets the point that somehow they're bullying the other side and saying, no, we are, we are doing all these refactors and you're getting nothing. It's really been kind of toxic and there wasn't a lot of trust built, but that's what's really key.

Building trust, being open product owners need to understand the value of the technical stuff you're doing and being able to understand the value of the business stuff you're doing and figure out which one's your true priorities, um, and keep a good pulse on that. So that's, that's one very important thing. Okay.

Uh, next we have, uh, something that I hear quite often. I've heard it pretty much everywhere I've worked, which is that what do you do when you finish your sprint early? You know, so they say here, sometimes we finish a sprint early, what do we do now?

It just seems like wasteful to do nothing. We're just sitting there. We, we planned, uh, a sprint, so we're gonna do this work in two weeks and we're gonna take two weeks to do it.

Well, we've got it done in a week and a day. We have four days left over. So now we're just sitting here.

Well, the first thing I just, I just wanna get this outta the way. I didn't even have this as my first point, the first time I gave this talk, but I really, I added it because it's, it's, uh, something I feel like just needs to get outta the way, which is you don't have to do nothing and you shouldn't just do nothing like that. Just that, that is a little wasteful.

So you're free to get a headstart on the items in the backlog. So just because it's not in the sprint, you can go to the backlog and find some things to do and, and get started on those. The only thing I would say is a caveat is one of the benefits of having a, uh, a sprint is that you all talk about it, you plan together, uh, you know, everything just to know, uh, about it 'cause you've planned together.

Um, and there's, there's conversations that happen along the way, but generally you have an idea of what you're doing. Um, so if you do just wanna do something in the backlog, make sure that you talk it over with whoever needs to be talking to make sure that the, the product owner knows you're doing it. That, um, that it has all the details.

Maybe there's something that the product owner wanted to put in there, didn't get around to yet, or whatever. Just make sure that there's some clarity that like, I'm doing this, okay, I got, let me, let me go ahead and get started on this. But generally, you're free to get ahead, start the point of the, of the, of the time block of the, of the sprint, um, isn't to lock out things that you, it's just to, to plan what you want to deliver.

So if you've gotten that done, great, let you know, do other stuff. That's all good. Um, but I would go on down the next point here is what is waste?

So in my earlier career, I did a lot of work with, um, with Lean Six Sigma and in the Lean Six Sigma world, waste is an incredibly well-defined, strictly defined term. Like if someone were to say, what is waste? From a lean perspective, you can go, ah, I know, I know exactly what waste is.

In fact, there are eight different kinds of waste. And they form a, uh, an acronym. They form the acronym downtime.

You have defects over production, um, waiting, um, not engaging employees, and you can keep going down. And, and the point of that is that you can, uh, and this is from a manufacturing standpoint, if you can walk into like a manufacturing facility and you can just look around and you can say, oh, there's inventory over there just waiting, that's waste. Oh, there's this person over here kind of just leaning against, that's waste because you trained to kind of use your eyes to see waste.

And it's a little bit different for software. Um, we, we don't exactly, uh, have the ability to kind of just see inventory in a corner. Um, but we do as a, as a kind of a, a caveat, we do see like Kanban boards and stuff that is, is waiting in a Kanban board.

So you can see some of that inventory in a digital stance, excuse me. 'cause every, every ticket that you have in your board, uh, whatever column it's in, whether it's to do in progress, those tickets, um, they represent some work that went into it. So people met to talk about those tickets, right?

The business or the product owner met with some stakeholder and met with, I don't know, business analyst, whoever it is, I don't know. But, but somebody met to build out that backlog and put a lot of effort into that ticket that is just to most people, just little couple lines on a board. But that represents a lot of time and effort.

And when it's just sitting there on a board, it, it provides no value. The only way that that value is realized is when that ticket gets developed and makes it into production, and then they can start realizing the value it was created for. Until then, it's not producing any value.

So there is legitimate waste in the sense of it's sitting there producing no value, but work went into making that. Um, so that's true, but within the context of a product waste, there's a bigger problem out there. And that bigger problem is what if you spent all this time building the wrong thing, right?

Like, what if we had the product owner, the, the business analyst, the developer, all these people meet, discuss, and build out the requirements or the tickets or whatever for this thing, and they went into the backlog and it went into development and someone said developing it, then it went into code review and someone spent a lot of time code reviewing it, and then it went into, uh, QA and someone spent a lot of time QAing that. Then it went into user acceptance and, and your PO spent a lot of time, you know, you're doing that. And then it was released and all this work and all, who even knows you were to just measure the amount of touches that that went through and the amount of, uh, of billable hours that went into that.

Who knows how much that would have cost to get that through. And then you get into production and nobody uses it, or it provides no value, or it didn't do anything near what it was supposed to have done. That is the biggest kind of waste, which is building something that's just not valuable.

Um, building the wrong features the wrong way, the customer doesn't want it, whatever. And that is something that Agile is really focused on, is particularly have, scrum is heavily product focused. It can, and it, it is, it does tolerate a little bit of process waste.

You know, like, it, it, it basically says that the most important thing is making sure you're delivering well, right? That you're iterating, you're getting feedback on stuff and then feeding that back into your product and then building the right thing. Um, so that is something that Scrum is heavy, heavily focused on.

Um, and maybe there's a little bit of process waste in there and that you have to wait a little while here and there, but considering what it's trying to prevent, that's a tolerable, in my opinion, um, a tolerable, um, thing to, to take on. But if it is a major issue, if you're like, I can't tolerate any process waste at all, no process waste, total waste free. Um, there are other things besides Scrum, like we have Kanban, like if you're familiar with Kanban, it's like Scrum in some ways.

But that has got, um, uh, there is usually you have like a, like a board with like to-do and progress and different kind of columns, but you don't have a sprint. You just kind of pull whatever's next on the top of the, of the lift and you do it that way. So whatever's the top is good.

So if it truly is something that's important, um, you know, there's always the ability of using something like Kanban to manage your processes. All right, next we have in our business, things change so much we can't really commit to the work for a full sprint. I've heard this one, um, actually pretty often at several different companies, and it, it does seem strange to me because, um, the company that we've started that I started with, like I said, they, they really kind of adopted Agile.

Um, but when I got there, they started, you know, their agile journey, um, from like a more waterfall way of working. Um, and they were saying, well, we can't do Agile because it's too, things are too chaotic here. That seems strange to me because wouldn't that pair more with Agile than Waterfall?

The things are chaotic, but regardless, they were like, we just can't do the sprinting thing. We can't, we can't create a sprint. Two weeks is too, too, uh, is too short of a time box or too, too big of a time box.

We need, uh, we need to just be free flowing. And um, initially my thought, my thought before I really put a lot of this together was, okay, um, shorten your time box if you have a two week sprint that you spend two weeks building something and then you start over and build something for another two weeks. So on, just shorten it to one week.

Um, and I thought about that for a little bit and I was like, you know what though? Um, if you can't get, if you can't, two weeks seems just like a lot. Um, so how do I wanna say this?

Um, if you can't plan two weeks, it's probably a bigger problem. Like there's a fine line, there's a fine line between being agile in the sense of like, we want to respond to change. And then it just saying we're chaotic.

You know, if you're responding to change, if you have a short change window, okay, that's tolerable. You're using agile, that's okay. If you're chaotic, you're like, I can't even plan two weeks without something going crazy and blowing up in our face.

That's probably something you need a little bit more problem solving around. Like, that's probably a legitimate problem. So like, um, I'm just looking at my time to see if I have time for the story.

Um, yeah, so when I was, again, earlier in my career, one of the big things, um, somebody come up to us and said, Hey, we just, we just sold this thing, like we have to start building this today or tomorrow, like soon we gotta get on this. And I was like, well, well we gotta have a little bit of conversation around this. Like, let's plan a little bit more.

And, and, you know, uh, the guy who was selling it, uh, who happened to be kind of in the, uh, executive um, area was like, no, we need to do this now. He pulled a developer aside and said, Hey, what are you working on? Like, oh, I'm doing this.

Like, not anymore. You're doing this now. And, um, just, just, you know, totally blew it up.

And, um, well long story short, we didn't build the right thing and there's a lot of problems around that. And, um, whenever you dig into the actual root cause of that, um, sales was really struggling. Basically.

Uh, they weren't meeting the numbers they were really hoping for and they were just throwing as many hail Marys as they can, like trying to close these deals, promising things that they, they're just, they're like, I don't know if we can deliver it, but we gotta get this thing done. Like really just working, working themselves to death, trying to get these sales closed. And so we were kind of in the middle of that.

And so that was a bigger, deeper problem. It wasn't just, you know, well, we need to do better within it, or, you know, lock people out and say like, no, you can't do that. It was like, we need to fix our sales problem as a company.

So that was a big problem solving issue for us. And so, um, you know, just kind of goes to show that sometimes the problems aren't always one, they're just right in front of us. They can go a little deeper than that.

And so that does need a little bit of problem solving. Um, and again, just like I said last time, if this is truly is an unavoidable issue, if you're like, you know what, we can problem solve what we want, but this is just the nature of what we do as a business. I, I don't really know a good example, but there many businesses that are just like, we are chaotic.

That's part of what makes us us. And there's not really a way around it. Okay, scrums probably not the right framework and you can use Kanban, and Kanban is also agile.

It's just a different way of being agile. So I would just recommend looking into that if that's, uh, truly a problem. Okay.

Next we have, developers don't want to give us timelines, but we really do need them. With Agile, we're having trouble getting these. So I heard this a lot, uh, again, just, uh, really throughout my career is just, you know, well one of the big things about Agile is that it always goes longer than the timeline, which isn't necessarily true.

Uh, waterfall also goes way farther than the timeline. Well, but, um, but one of the things I, the misconception I think is that you just throw timelines out the window. With Agile, that's not really the case.

What you do is you respond to change with agile. So Agile's not anti timeline, it's just pro responding to change. So you can build a timeline and then just make sure that you're, uh, adaptable with it and you're able to change and willing to change if and when things change out there in the world.

Uh, if you're stakeholders just, you know, understand that there's something else that they need or, or whatever. So as you kind of see here, it's really a function, a lot of just the kind of team you have. Um, I've been on newer teams and um, you know, timelines are a struggle.

There might be teams that are like, oh, we're so new. We don't really know the tech stack very well. Um, I think this will take like two days, but just to be safe, let's call it a week.

'cause we're so new and it takes like two to three weeks. Um, so that, that's how bad these things can get sometimes. But once you get more mature, I do see that there's a lot more comfort with, uh, with these, with these estimates.

Once you know the tech stack, once you know what's what goes on, it gets a little better. You also have data as you get more mature, you can be like, how long is this going to take? Well, we had something similar that happened, you know, six months ago, and that one, um, took about, you know, three months.

So it'll probably take about three months to, um, you know, we may buffer a little bit, maybe four months, but generally we're probably gonna be good around three months. And that, that's a little bit more of an accurate statement. As you get more mature, you just get better at understanding what your um, uh, you know, what your timelines are gonna be.

Um, but then down there at the bottom, like I I said, it kinda goes back to what we were saying earlier, timelines are estimates and they should be reviewed and adjusted and adjusted in the sprint review. Like the reason that people want timelines, there's a lot of reasons people want timelines. One of them may strictly just be for budgeting.

It's like, you know, we, we, we think we're gonna make this much money from this feature. Um, we won't spend that much on it, so we have a good ROI, whatever, you know, it's just stuff that people, uh, people want just for that reason. And, um, and so when we get to our sprint review, so again, with the sprint review, we spent two weeks building a portion of this feature.

We got to a place that that thing has at least some part of it done, and we're going to review that with our stakeholders. When you get to the table with that, you can say, okay, here's the features we built. If they're like, oh man, I just think maybe I see it now and I, I know we said this, but I'm seeing it now and I think maybe that over here is better, can we adjust it to be that?

Well, you can pull off a timeline and go, cool, um, yeah, that's gonna take a little bit longer. So maybe this is a three month project now it's gonna be a three month and a week or three month and two weeks, you know, adding more sprint onto it. Is that okay?

And you know, maybe they'll be like, yeah, yeah, that's good, and you would just adjust the timeline based off that. Or maybe they're like, no, honestly we, we kind of have a deadline. We really gotta be done in three months.

We can't do that. Okay, okay, cool. That's perfectly fine.

There are other things in this timeline that we can probably remove to make room for this other thing you're asking for. Is that okay if we remove this thing here and maybe at some point in all that time you'll get to some kind of good agreement. But having the timeline, there is a, it's really good to have that artifact there to look at and, and kind of understand where things are landing, uh, as far as, you know, the plan and, and how and, and where, where you're gonna get done and what's gonna be in there and things like that.

So yeah, generally use the timeline's fine. Nothing wrong with that. Um, if your customer doesn't want a timeline, okay, great, no need to.

But if they do want one, you know, no big deal. You just gotta make sure that everybody kind of agrees that there's adaptability there, um, in case you want something different. So, um, this leads to kind of the last portion of this, which is what to do with meeting resistance.

This is, like I was saying earlier, like a lot of these things as I've been going through them, it might be like, well, yeah, like these are, these, these are legitimate things people believe about Agile, why it doesn't work. Um, and you know, I just, it just feels like it's very argumentative to say, well here's why it does work. You're wrong.

And that, that's not the goal. Like the goal is for us to understand, like I was saying earlier, for us to understand why agility works and the face of when people say it doesn't, or we know it does, uh, here's some ways we can kind of look at that. But as far as you're talking to other people, if you're talking to other people and they're telling you these things like, I doesn't work as X, Y, and Z, the best thing you do is just, you know, ask them things like, Hey, well would you be willing to try this?

Would you willing to do an experiment? Like not telling them here's why you're wrong. Being like, well, I hear what you're saying, um, let's just showing them that it works.

Like what kind of experiments can you run to show people that this little thing right here that we wanna try to put into place might actually work? And, and just helping them kind of get there. Um, and the next thing is this really important, and this was added since the first time I added, I did this talk as well, which is showing empathy.

So again, you're not there to be, right? Like we're all here to succeed together. Like you're trying to help people, you're trying, trying to empower people to do the best that they can.

And agility is kind of the tool we have in our tool belt to help people do that. So show empathy, like when someone's like, I just, this isn't gonna work. Like a lot of people have been hurt by agile, if you wanna call it that.

Um, Agile's been used as a weapon in a lot of ways, you know, and not agile, that, not true agile, but that, that there's some things that people call Agile. Um, like maybe someone was like, you know, agile to us was that we had to use this tool, we had to use this exact template, we had to do it this exact way. We had to meet these exact standards and we didn't, we're not meeting our agile quality, you know, controls or whatever.

And it was just dumb, you know, and that's their experience of Agile. And so they're coming to you saying, no, we don't like Agile. It's really bad.

It doesn't work. And you know, some of the, some of what you do is you say, I'm sorry, you know, I'm sorry that Agile didn't work for you. I'm sorry you experienced that, that wasn't the right thing to do.

Um, but I think you have an experience of Agile that's not right. Like I think maybe if we work together, we can figure out something better than what you've experienced in the past. And that's important too.

And um, and if you can meet them at that level, like understand where their, where their heart's at and their head's at with this agile journey and meet them at that level, that'll go a lot further than just trying to dogmatically say, here's why I'm right. So that's a good way of kind of meeting people where they are. Um, but the last thing you wanna do is when Agile really doesn't work.

So other situations where people are right, where they're like, agile won't work here because this, and, um, I put here when change is expensive, 'cause that's the whole point of agile is that you, you are able to change direction a little bit based on what you, what feedback you get from your products and from other places. So if you can't change very easily, if change is very expensive, then Agile doesn't work very well. Now one thing I'll say is that in my, the first time I gave this talk, there's a lot of places I would've said, oh yeah, like, here, here and here.

But as I've gained more experience, I've seen Agile work in places, I would've thought it wouldn't have worked very well. Like I would've said something earlier in my career, I would've said something like, well, agile won't work very well in like a highly regulated place like the government or the bank. And I'm seeing a lot of agile teams in the government.

I'm seeing a lot of agile teams in the bank. I work at a bank now we were agile and um, you know, the reason I would've said that they wouldn't work is because of all the government regulations. It's like, well you can't release this thing until you have all your paperwork in place.

And that could take forever. And believe me, I was at a bank, it did take forever, but we were able to do it iteratively too. And so it, it actually, we were able to problem solve a lot of those things and get to a better place.

And so that list of places that I would say Agile probably won't work is, is shrinking to the point that now I can't really think of a place that I would say, oh, agile won't work here. Like even if you're building a building or something, um, it's always a good pro like good thing to do to occasionally bring the client into the building and have them look around so they can tell you things like, well, I don't know, I don't, I don't really like that plug over there. I want you to move that over the other side of the room.

It's better to know that now than when the building's built and you gotta undo a bunch of stuff. So even there, I would say probably Agile's a good practice. It may not be as, as effective as other places, but it is still a good practice.

So yeah, I would say most places, if, if it won't work at the very least, you can glean a lot from it. So I would still be very cautious to say it doesn't work. Uh, more so now when I first gave this talk.

And that is all I got. So again, I wanna thank you guys for choosing to, um, to come to my talk or to listen to me or having me here. Um, yeah, thank you guys so much.

Appreciate your time. See ya.