Techstrong TV – January 13, 2025

Hey, everyone. Happy Monday. Hey, will Legent AI kill the SAS Radio star you're watching?

Text on Gang. Hey everyone. Alan Shimel here from Textron Gang.

Welcome to our Monday show. Happy Monday to you. You know what?

New Year's is over. Get back into the full swing of thing. No one's waiting around for you or going to give you any kinda, you know, easy time off.

We, we were all back to work. Um, and there's a lot going on. We've got a great gang to talk about it today.

Let me introduce you to them. First of all, it, it's kind of a, some of our regular Monday gang members joining us from home in New Mexico, deploy hub CEO Actually I see you're speaking. Are you speaking at the CDF or doing something with that Tracy?

Uh, I have submitted to speak for the C submitted. I I did get, um, reelected to the open SSF governing board. Yeah, you did.

And we did great. And we did Reinitiate. We initiated a new sig at the CDF that I'm in charge of, called CI/CD cybersecurity.

So maybe that's what you saw. Oh, That maybe what I saw. That sounds like, Gwen, I may want to get involved in Let's talk.

No, no, I, I heard that election was stolen. I don't know what's going on. Yeah, actually I was really shocked.

I was like, I can't Believe I got an elected on Mike. Count on Mike Debris. I'm not gonna take the bait.

I'm not gonna take it. Tracy, welcome. Let's move on.

Moving into Texas. I assume he's in Texas though. I see.

Well, there's leaves on the tree. You gotta be somewhere where there's still leaves on trees. He's future analyst vi co-founder Guy Currier.

Hey, guy. How are you, man? Good.

Yeah, I got back from, uh, uh, Vegas in this, uh, CES How was C in the morning Last night? Um, my first CES ever. Mm-hmm.

And, um, I will say one thing, it's big. Yes, it is. Yes.

That Is a big show. So I got a cryptic message last night from Lisa Martin. It's CES that you found My flying car?

Yeah. There was a flying car there. Uh, it was a, it was a car, um, that looked a lot like a cross between a cyber truck and a Humvee that had, where, where a, you know, very heavy, uh, it had six wheels on it, by the way.

And a heavy, um, hover, not hover, sorry. Drone like platform pulls out of the back and you attach it to the roof and then it, you know, it's all these flying cars these days have, uh, have like all these multiple propellers and all that kind of thing. Well, more, more like a a, an Osprey air aircraft.

And let's say a Harrier jump jet or an F 35 jump jet look look more like a drone than a, than a Jetsons flying. It looks like, it looks like a giant drone with a couple people inside. I sure Can't.

So it doesn't fold up into a briefcase then? No, it, it doesn't fold up into a briefcase and you don't launch it out of your garage, uh, to Noting It's probably not a one hour process to get that thing on the back, but it was slick. I mean, the, the automotive presence there is really big.

And so there, there's just so much implication to everything they're doing on that side of the, uh, the sort of consumer side of the fence to everything, you know, we're, we're doing in the enterprise. So it was Good. Cool.

Well, welcome and thanks for being here, guy. Good to be here. Let's move on there to up to, uh, high up into the mountains, to the guitar man.

Right. Mitch Ashley, future of Analyst v vp. Hey, Mitch, how are you?

Hey. Hey. Good to be here.

From the single digit lows of Colorado, no single digits. Single digits, man. We've had some that's not like Colorado.

Usually it changes. It. It's, you know, January, that's when we have our cold Spell.

Hang on. We have an international audience. We're in the single digits here in Austin.

It's just on a different scale. Celsius, single digit. Okay.

Are you, I'm using two single digits. No. Is it that cold there guy?

Oh, yeah. Oh yeah. It was the high today is is in the low forties, but, uh, yesterday the high was around 36 Fahrenheit.

Ooh, it's chilly. Well, It was, it's three degrees in New Mexico this morning. Ooh, you're colder than we are.

So, well, we were cold yesterday, but it's, it's in the mid seventies and sunny today. It's good to be, it's good to be a Florida Gator. Still in my college football thing from the orange ball.

Uh, moving on from Mitchell though, Mitch, thanks for being here today. I don't know how cold it is though. You know, you can never tell from the flannel shirts you wear every day lately.

It's our Chief Content Officer, Mike Zahar. It's 40 degrees here, baby. And you know what?

It's balmy. It's so balmy. 40.

All right. Well, welcome gang members. Thanks for being here.

We're gonna kick things off today. You know, uh, Satalia Nadella from Microsoft recently did an interview on someone's podcast, and he basically said, SaaS is dead, will be sued, you know, let, maybe a little premature to print the obituary, but, you know, he claims the killer is Agentic ai. Uh, Mike, why don't you, Yeah, and we've alluded to this in, uh, some previous conversations we've had about this, where, you know, the debate is, am I gonna use all these AI agents in the SaaS applications provided by Salesforce, or am I gonna go and just have my own and maybe stitch it all together?

Of course, I think Microsoft has a dream where you're just gonna use their agents for everything. And you're gonna turn all these other applications into headless services, essentially, where I'm never gonna see them. I'm just gonna invoke, whether it's a spreadsheet or a CRM app or whatever it is, I'm just gonna call on that, but I'm never actually gonna go enter something into it.

'cause the agent will do that. But Mitch, I know you're tracking this. What's your take?

Well, I think you, you, it was a really good interview. If you get a chance to listen to the podcast, it's an hour and something long. He's not on all of it.

But, um, essentially the, the SAS is dead. Is the clickbait coming out of it. He was really saying, I think, here's how I interpret what he was saying is, is the use of AI and use of AI agents is kind of the next evolution of our software architecture.

You can say whether it's a revolution or not. Um, but we've been through this process, right? We do.

We just went, have been through the cloud native transition with microservices and containers and breaking things up and distributing them and orchestrators that was, you know, what we've kind living with now. And the next evolution of that is those microservices, if you wanna think of 'em that way, or parts of our applications, um, our, our AI agents. They don't have to be software that we've written traditionally or maybe a software written by the agent itself.

To me, it's, it's kind of a whole lot of nothing. 'cause there's a lot said about, well, Excel is just a gooey on top of a database. Well, so is every other application, but that's exactly the point.

He said that all business, virtually all business applications are no more than a database with a gooey on top of it. So my response is, no s**t, it's been that way for decades. When we started, when we started Mitchell, separating the user interface, sorry, it's Monday.

So I had to start, got PG rating. Yeah, I, sorry, I I'll bring it back. No, I gotta put a warning on flashing lights in Mitchell.

Cursing model view controller. Have you heard, unless you watch it after controller, if you've been in software for any length of time, we've been separating the user interface from the business logic, from the database access. This is, it's the same thing.

It's a new form of that, right? It's there, there's not any real new news here other than just saying he wants everything to be agents on the backend instead of traditional software. Okay.

Alright. That's, maybe that's news. I don't think it is.

So I think it's a whole lot of, it's not a whole lot of nothing. As in he's not saying anything. He's just saying this is where software is going and where he thinks what it thinks are gonna look like.

Is it the death of SaaS? Oh, heck no. I mean, at, at all of it is gonna be SaaS in some way.

If you want to think it that way. It's part back end of AppSec, front end of AppSec, living in clouds, talking over ai. We'll have orchestrators for ai.

Maybe, maybe it's Kubernetes, maybe it's something else. Um, that'll be managing all these agents. So I, I think it's just kind of a, it, it's an interesting conversation if you sort of peel away the, the, uh, you know, the clickbait part of it.

Um, but it, it isn't spelling the death of software, period. So I'm an Associate, I'm an agreement. I'm in agreement with Mitch.

A hundred percent. Yeah. I thought the same thing.

So let me, lemme, I did, Lemme see if I could massage this so you guys can get your head around it, because obviously it struck a nerve with both of you, Right? And Me and guys. Well, all right.

So you can't be so attached it. They're cattle, not pets, guys. Okay, here, here's what I'm trying, I think here's what Sacha is getting at, and here's how I look at it.

Will it stay? Will there be stuff hosted in the cloud? Yeah, of course there will be.

The data is going to be up there in the cloud, the bi So if you look at AppSec at their component level, and Mitchell, you said it, you agree with this, there's the data, there's the gui, there's the business logic. Those are the three components here, right? Those are the three components.

The data's not going nowhere. We're, we're gonna still need the data, right? Because at the end of the day, it's all about the data.

Stupid, right? The gooey, I think gooeys are changing. I think, I think one of the things that gen AI and this whole chat bot revolution kind of thing or evolution has taught us is you don't really need a gooey, I think people would much prefer to talk to their computer and tell them what it wants.

You know, ala Scotty and Star Trek, uh, you know, hello computer. Um, yeah. You know, so I, I think a gooey is the gooeys as we know it, as the experience is I think going to give way to just a much more simpler human computer interface.

The business logic is the weak point here, and I think that's what Satya is talking about. The business logic will be controlled by agents. Now, what I find really fascinating is, for the last three years, I don't know how much wind we've spent or ex exhale talking about, well, what's after Kubernetes?

Is there anything on the horizon that can replace Kubernetes? We got something. I don't think Kubernetes becomes the orchestra orchestrator for AI agents.

I think we're gonna see something new and, and these agents are going to represent the business logic in AppSec, and it'll, this could be a Kubernetes replacement on the horizon. Whatever comes out to do that. Now, does that mean that SaaS is going away?

No, we're still gonna host stuff in the web and access it. Does it mean SaaS AppSec as we think of them with the pretty, you know, they're not so pretty, but with the interfaces that we type into and all of that is going away. Yeah, I think so.

I'm always a little suspicious about the timing of these conversations. So work, walk this through with me. Like what's the motivation for saying this now?

It seems to me, you know, this isn't a, almost a boldface effort to tank the valuations of some of those SA SaaS companies when basically trying to say that all these things are gonna wind up being features of some AI agentic framework. And maybe just maybe if those valuations tank, we'll start to see a lot of mergers and acquisitions. So maybe this is an effort to reduce the cost of buying other companies for 2025.

Call me suspicious. You're so, No, I, I, I think I wanna look at the, uh, Mike, I wanna look at the co I would love to look at the co-pilot roadmap because co-pilot right now is not to my knowledge, particularly ag agent and release, but I guess that's where it's going. I Think's another thing guy that, you know, the counter Mike's argument.

Where does, where does Microsoft make a lot of money today hosting SaaS-based AppSec in the cloud? Yeah. Your agents in the cloud, right?

So I, so my perspective is sort of a, I think a long, very long perspective. I mean, Mike, you might remember way back in the day when we were at, uh, uh, zip Davis together. Um, one of the thing, one of the little themes I had at the time was, uh, what we are calling cloud computing today.

We're just gonna call computing in 10 years. And I, I think that played out. I mean, we're not calling it computing exactly, but just everything is cloud, so to speak, unless it's not.

And the distinction has to do with use case, if anything. And there are, you know, so like, it's just, you know, it's just part of, of the infrastructure landscape right now. Similarly, uh, I don't see why SaaS dies, because ag agentic AI takes over.

It's just what we call ag agentic ai right now. We're going to just call SaaS tomorrow still or something, if you follow me. It's, it's a way to deliver software as a service.

Now the software is hosted and run by agentic ai. It's, it morphs more, it responds to you, it learns all those sorts of things that you expect it to do. But how is it exactly different from SaaS?

That's what I don't understand at all. Yeah, that's what I couldn't get my head wrapped around either. It's like, why is this not a software as a service?

Just because the technology. It's, that's exactly, it's What, it's, it's Absolute, it's not the software as a service app as we've known it. I think that's the point.

Okay, great. That's Fine. Yeah.

And, and I, I do think the, the future of copilot is to become an agent, and maybe copilot becomes that agent orchestrator, right? So be multiple agents. A I have a different view of copilot.

My, my prediction is copilot overtakes the IDE for people. Essentially a copilot is, yes, is an adjunct additive to an IDE today, the more copilot does or whatever form it becomes, essentially that becomes your new IDE, especially for non-developers, if you will. But that's the orchestrator of your development process, creating software, creating bot, managing bot, whatever it is.

I think that's kind of where it's heading, at least in my view of it. Maybe I'm wrong. We'll see what happens.

We'll see, I dunno, that's, everything gets going. Microsoft describes Azure as quote the world's computer. This is the same Microsoft that it's always been.

Every time it looks at something, it says, here's this one unified thing that we're gonna own end to end. And everything else is, you know, get out of the way or be absorbed. We are in a hype cycle, and we have to consider a lot of what we see coming out of these, the, what I like to call the giants hype.

Uh, it just, it, it happens for every single time we change technology. Look at all the hype that was around Kubernetes, and rightfully so. We had a lot to learn, and we have a lot to learn about AI and, and how we're going to apply it.

And there may be some really interesting outcomes of that. In in particular, you know, distributed databases where there's data across smaller chunks of data across organizations or even open data that we can, that these AI agents can get information from. But it's just an evolution of software.

So I guess that's where I was struggling. And yes, I would love to ditch the mouse. I'm so tired of my mouse, but I still want my keyboard.

I hear you. So here's, here's, I wanna, I wanna, I wanna say I give my interpretation of what Satya was saying. What he was saying was that right now you buy a set of, of, you know, soft a set of software, some of which are SaaS that carry out certain functions for you, for, for your job function.

What he's saying is that's gonna be replaced by an, you know, an AI agent or package of AI agents that are for you and your function, and you as a person and how you carry out your work. And those will go and get the necessarily necessary crud functions and put them together and change them as you, and learn and grow with you, so to speak. So instead of buying SaaS, you're buying AI agents.

I think that's what he meant. But that is, I agree with the panel here, a distinction without a difference. I think we need to introduce a new term, which is a agent hype building up what piece.

So, on, on that note hype. Let, let me, let me tie a bow on this one and we'll break, um, today's Monday, right? I do th Monday, January 13th.

I do believe that 2025 is going to be the year of AG agentic ai. I think we've gotta stop looking at AI as a monolith and, and recognize that there are, there are divisions and, and features within the AI monolith and agentic AI is one, and it's gonna be huge. But if you really interested in what 2025 has in store tomorrow is our Predict 2025 session.

I'm doing a great panel with a bunch of C-level folks on what DevOps and AI have in store in 2025. Mitch, I know you, I saw you, uh, yeah, doing your session the other day. I, my top 10 predictions, top top 10 predictions.

And that's good. That's both we'll replay that during Christmas next year. And either the last we'll do backwards, we do David or something.

Guy, I think you have a session too. No, I do You wanna say anything about it? Yeah, it's, uh, it's work With me guy.

My, my predictions, the 2025 will be the year the fragile app, The fragile app ex, My prediction is my AI agent will be able to beat up your AI agent. That's what I, That's part of what's gonna make my app fragile. Okay.

com right now. Register, it's free. I've got Daniel Newman talking about stuff.

We've got Steven Foskett a bunch of the FU folks, but we've got people from, I think Google, Microsoft, bunch of the great DevOps people. Good cyber stuff. Don't miss it.

Predict 2025. You're watching Textron Gang. We'll be back in a minute.

Hey everyone, it's Sunny And Cher. Ladies and gentlemen, tech enthusiasts and future Gazers gather round the biggest, boldest, most mind blowing predictions for 2025 are coming your way at the Predict 2025 virtual event on January nights. Oh, sunny, you're predicting something again, last time you tried this, you said laser dis for the future.

How'd that work out for you? Hey, hey, Cher. Not every prediction's a hit, you know, but that's why we've got the real experts this time, top analysts, visionaries and tech leaders sharing what's going to rock our world in 2025.

So, no sunny predictions this time, no flying toasters making a comeback. Very funny share. But seriously, we're talking AI breakthroughs, cybersecurity game changers, the future of DevOps, cloud Innovations, and so much More.

And what about my favorite prediction? A smart mirror that tells you how fabulous you look every morning. That's real innovation.

You're already ahead of the tech share. But if you want to hear the really big stories in tech for 2025, you've gotta tune in on January 9th, the event kicks off at 8:45 AM Eastern and runs until 2:30 PM And the best part, it's all virtual. No stuffy conference rooms, no longer commutes.

Just grab your coffee, your laptop, and join us from anywhere in the world. You know what else? It's not just predictions, it's insights, strategies, and a whole lot of fun.

0, predicting your jokes before you tell them. That's a good one, Cher. But the real joke is if you're missing out on this, so don't miss predict 2025.

That's right. Mark your calendars January 9th, 8:45 AM You start, be there. Or you'll miss the biggest scoop on the future of tech.

See you. I predict 6 20, 25. Be there.

All right, folks, we're back. And we're talking about AI and DevOps again, because there was two reports last week. One from IB M1 from harness, both highlighting that, well, maybe all the things are not quite cracked up as they were hyped.

We're back to that word again. Um, the issue seems to be that the AI tools are generating code without any context for the production environment. So the code doesn't run, and then people spend more time debugging the code than it would've they had actually written it themselves in the first place.

At least that's the theory. But Tracy, I know you've been kind of following this all last year and we're coming into this year and it, what is the fundamental issue that we need to kind of work through here to make all this AI stuff really work? Well, I think first we have to separate out DevOps, to be honest.

Um, I would say Dev has embraced ai, you know, with tools like copilot. Um, I've, even myself, I use, uh, chat GBT to generate my markdown from my websites. There's quite a bit that, that we do on the development side.

But to be quite honest, DevOps hasn't really changed that much. Um, and there may be tooling out there like, you know, AI and, and testing that can be updated in the DevOps process. But I think this, this, you know, this survey and this article and this information is spot on.

We, um, we have, I feel like throwing DevOps, uh, DevOps out the door a bit. And to say that we're doing DevOps and ai, is it throughout the life cycle is, is simply wrong. We're not, so when we see these surveys, and I think we have to, you know, I participate in surveys when they come across, but it, you know, it stays that 99% are using coding tools infused with ai, which is probably right.

But few of them are implementing any kind of really solid a, uh, AI practices in the DevOps pipeline. And I, you know, I, I'm very black and white about this. There's no gray for me when it comes to a Jenkins workflow.

And have we changed anything in that over the course of time? Not really. We have not evolved the core process of our software factory floor.

Uh, so while we talk about it, there is a lot of hype in what we talk about. You know, just being able to use AI to go out and find a, uh, somebody to approve your code isn't enough. It's really not enough.

And most of the DevOps, uh, engineers that I speak to, they don't have a lot of experience, um, in you, in what we need to do in the DevOps pipeline to start catching these kinds of problems. Um, we rely on the, you know, somebody to check in code and then approve it and push it across the, the DevOps engine takes, uh, starts working. Is it gonna find code that wasn't really written?

Well, I don't think so. It could scan it and find out that it's got, you know, it's pretty clean, but does it work Well? I don't know.

So I feel like we are in this hype cycle for ai. A lot of these companies want to get out in the front saying that they're doing AI and lifecycle management, but it's not happening in reality. I don't see it, and I'd love for the panel to prove me wrong.

I think that's exactly right. I think that, um, you know, we, we, we, we, AI is an, is an assist in so many helpful ways, um, to get you started, to get, i, I call it a reliability and a quality boost. But the quality part requires your intervention review.

Um, if what you're aiming for is speed, then I think, uh, you're just gonna get more garbage faster. And that's what we're seeing. Um, if you disconnect that speed, you know, productivity, if you like element of it, that really honestly gets all the focus.

But if you take that out and you realize that you can produce better annotated, uh, you know, better, you know, better functioning, um, and, uh, uh, more reliably produced code that still runs through the same processes that you already had, um, then you're using it well. And I think that, that, you know, the developer community as a whole has not realize this. And I, what was not as clear to me, um, although you have pointed out to me a few times, Tracy, is the stress it's putting on ops teams for whom the, you know, the code is, the code is the code, and, uh, nonetheless, they have to deal with the consequences.

So it was very illuminating in that sense. One thing I noticed also though, was, um, that this survey covered, um, both the coding side and the incorporation of services side. And that's a whole other reliability, um, uh, question, right?

Um, which is incorporation and use of, of services, which can and should change over time, as pointed out in the article, um, that creates its own set of instabilities and its own, if I may say so, fragilities, um, that, uh, uh, can affect the, the app, um, the app app performance and outcomes. I See what you did there, Greg. I think we're still at the, I'm sorry, Mitch, go ahead.

We're, we're still at the individual productivity level with AI and development. It's a copilot, it's an augmentation to the work. It's maybe there's some mag magenta functions that it performs, you know, software quality teams still have to treat all code as suspect, whether it's AI generated or not.

They kind of have to not care. Everything needs to go through the, the quality process. And, and to your point, Tracy, it hasn't made, its a, has not, has not made its way into the workflows, into the pipelines, into the CI/CD into the processes underlying it.

At least not yet. Maybe that'll hap start to happen in 2025, but I think a, and maybe it's a agent AI or AI agents that will help us start to move beyond just helping Mitch Ashley an individual developer or helper help, uh, you as a, as an individual software tester, whatever it might be. But we haven't changed the underlying framework of how DevOp works.

And it's a lot to disrupt. I mean, don't, don't get me wrong. It's not like, oh, let's go change this.

You know, to your point, Tracy, there's a lot of Jenkins and CircleCI and everything else, CI/CD pipelines that you don't wanna touch, because when those things break, it's not one of 'em, it's dozens. And how, how do you, how do you de diagnose that and fix it? So it's definitely a place for some disruption, but I think you have to look at it as productivity of the process, not productivity of the individual.

So, Mitch, to that point, I'd love to get your thoughts. You know, I was talking to Nick Durkin, who's the field CTO for harness, and he was of the opinion, and admittedly this is biased on his part, but he was saying that the AI agents used for coding need to be aware of the AI agents that will be used for the ops side of the equation. And that you need to have a cohesive framework for those things.

Otherwise, you know, you're gonna have scenarios where the AI agent used for the coding tool is gonna be working across purposes from the AI agent used for the CI/CD and the op side of the equation. So I don't know if we need to rethink this or, you know, can all these AI agents find some way to be compatible? Well, maybe that's kind of where guy's going with his fragile app app idea.

But you know, it, it's everybody's, or every AI agent doing whatever it's doing in its own context, limited world of course, could very easily work across purposes, right? My agent, AI agent can beat up your AI agent. One of the things that helps is not the silver bullet, but is that context knowledge graph, right?

So that I know well, by changing this bit of code that affects these people in operations or security or whatever functionality, it's the, it's the broader context of what's happening. And I don't know that we've got sophisticated enough knowledge graphs to solve all the problems that might come up from, you know, AI generated code. But that's at least the start of what we're using to try to add more context than just a single user.

I, I tell You, and I That's a terrific point. This is true. Where does context come from in these, you know, uh, I, is it come, does it come from the people who train the AI in, in a certain sense?

Like you are talking about two different context lobes, dev and ops. Where, where at the point of prompt does the context come from other than the user, Tracy? Well, I, you know, in this discussion, I, I just wanna create a re like a use case.

My niece, um, told me when I asked her about she's got a new job just outta college. She told me once she could generate 6,000 lines of code a day, but she couldn't figure out what was in it. And in this, uh, in this harness survey, it says in here, I'm just reading that 67% of developers space spend more time debugging AI generated code, which is kind of what she was telling me.

But even more concerning 68% spend more time resolving AI related security vulnerabilities. So I think it's sometimes it's, you know, we more code is not always better. And, and in these AI generated environments, these AI where the developers are starting to use AI to generate their code, they have to get better at learning how to use it.

What she told me was, she's starting to stop. She's not using it as often now. I just talked to her this over Christmas.

And what she's learned to do is be more specific in using AI when she gets stuck on a particular function. So she's trying to minimize how she's using AI because of the issues that she found with having to debug it. Because she said, if AI generates it, I can't debug it as quickly.

I don't understand it. So it's harder to see what's what it's doing. So that that's, that's a very real use case for somebody who's just outta college, got her first coding job and having to learn to use these tools.

So it's kind of like prop training, right? How do you get better, get better code generated? Because you're not asking it to do everything.

You're only asking it to do smaller functions. Now, if we, if teams get better at using ai, then maybe these numbers will come down. But if right now we're relying so much on AI and tools like copilot just to go out there and generate this stuff, we'd better be running some good SBOs and understanding what it's pulling in, understanding the licenses, if it's, you know, if these tools are, are not approved, um, across the organization, we have to understand if they're, if they're compliant within the organization.

So there is so much more to do that AI could be solving for us than just generating code. And it kind of goes back to some conversations we had last year about the ghost developer, right? Uh, and I, and I complained, I said, it's not just about coding.

There's so much more that's done in software development that we're not trying to use AI to solve. That could be done. And I, I think that we, we should be seeing maybe a prediction for 2025 is that, you know, AI ops we're, we'll start having more conversations about AI ops and what that is.

It is using AI To that term being oped op, that term's being corrupted for ml. I know, I know. But it needs to be clarified because we need to learn how to pull this stuff into the DevOps pipeline, um, and start fixing it.

So I've got some thoughts. I've, I've listened to you all here and I got, you know, I, I should have been taking notes 'cause my brain won't remember everything, but I got a lot to unpack here. So first of all, I mentioned before about the need for an AI agent orchestrator.

That's exactly, Mitch, what you're talking about, guide that you're talking about, right? My, my dev AI agent doesn't really jive with my ops AI agent and it doesn't really jive with my security AI agent. And so they're working at odds and loggerheads to each other.

That's a temporary phase. And that too shall pass, right? We're going to have to have some sort of agent orchestrator that makes sure these things are in concert.

Is that gonna pass like a kidney stone? Just wanna make sure I understand. No, no.

It's like your grandmother told you when you were younger. Don't worry, that too shall pass. It's just a phase.

Um, you were experimenting, but anyway. Yeah. But so that's number one.

Number two to say, you know, what challenges and what failures or, or successes we've had in AI and DevOps is like looking with a very tiny telescope at Uranus or Neptune and trying to tell me if there's oxygen there or gold there or something. Trying to figure out what's going on in that planet from a small telescope here on earth or even a distance star. You, you don't see the, you don't see the features, you're seeing a blob, right?

But yet when we dive in beneath the clouds to the individual pieces, we are seeing AI at every step of the way. We are clearly devs are using ai, are they overusing it? The, the, the experience of your niece, Tracy.

Again, that's a phase that'll pass as AI gets better, right? You're gonna have less security bugs. It, it's already measurably better than it was a year, year and a half ago.

Right? Which is why it's being used more so, and it'll continue to get better. I, I think the other thing is you haven't mentioned platform engineering, and I think that's a big story for 2025.

It was a big story in 2024. com, check it out. It's one of our sites.

I think that's where the agents for ops live. In many ways. Platform engineering is more about the ops than it is necessarily DevOps or, you know, they, there, there's not platform engineering didn't invent a lot of new things.

It reused a lot of old things. And so it's sets sets up that environment for the devs to go faster. And so is platform engineering using agents or ai, I think we're gonna see more and more of that in 2025, right?

And I think the role of platform engineering in ops is gonna be a huge story. Right now we've thought of platform engineering and how it's helped devs, right? 'cause they don't have to build their own environment, but it's really about ops best practices.

So I I I think that's another thing in here. I think it's gonna be a cultural issue as much as a technical issue. Mm-hmm.

And my prediction is that a year from now, we'll still be talking about this same brawl. Absolutely. But it'll be better.

One other thing I wanted to mention, Ja Chase, you mentioned Jenkins. Look tomorrow at Predict 25, another subtle plug for Predict 25. We're gonna announce the winners of the DevOps Dozen awards.

Now those good folks at KPMG have given me the Silver Briefcase. So I know who won already. KPMG didn't do it, but just, you know, I I I recently watched some awards, oh, the Golden Globes and they had the KPMG guys.

But I've looked at the winners for the DevOps dozen awards, I'm announcing them tomorrow. You'd be surprised what's old is new again. And some things don't change in terms of what DevOps tools and CI/CD tools still rule the roost.

And they're not what you would think of when you say, oh, what's AI empowered? But yet when you look under the covers, they're using more AI for testing both security testing, quality testing, security and quality can be the same thing. Load testing, unit testing.

There's AI being used in deciding what my test coverage is, what tests run, how tests run in, in moving things along the pipeline. So again, when you look at the, the planet through the telescope, yeah, you may not see it, but when you, if you could dive down under the clouds and get to the surface a little bit, there's all these little things going on that are using ai and at some point it, it, it just dawns on you and say, wow, it's all over the place. It's, it's ubiquitous.

And I'm gonna leave us with that word ubiquitous. Um, let's take a break on Textron gang. We're gonna come back and we're gonna talk about who's gaming the GitHub stars you're watching.

Text Gang Modernize your business to fuel innovation and elevate customer experiences with the builder community. Hub AWS and its partner network. Provide essential tools for transforming applications and infrastructure to fully leverage the cloud.

Discover free trials, in-depth demos and essential resources to empower DevOps engineers and developers to deliver value faster and more reliably. Visit the builder community hub to learn more. All right folks, we're back on the C block and there was a report that came out over the holidays talking about how the bad guys have figured out how to game the GitHub system.

They are creating, um, code sticking in a repository and figuring out how to give themselves a whole lot of stars that entice people to download that stuff. And surprise, surprise, it's loaded with malware and we've seen GitHub kind of evolve for all the good things it does. It's also become a distribution network for malware.

So I feel like something needs to be done here, but Alan, what's your take on what's going on here and how do we kind of make this work in a way as intended? Well, you know, this is not a new problem. As you said, Mike.

This is, this is the developer dilemma. A kid to people who believe Amazon reviews, right? You, you go to Amazon and it's got, oh, this product, is this product better than that protocol?

This product has 456 reviews. You know, I'd love to be able to see what IP address put up 435 of those reviews, uh, because it's probably the same IP address and it's just people gaining the system. But, you know, why is this not a new problem?

Look, I remember having this discussion five years ago about docker repositories, right? If, if, if the docker image, you know, container image to do, um, I don't know, to, to do, you know, a very common call that would be in one container, uh, a mail call or something, right? Or a send mail call.

And then let's call that docker image was called dot send mail. Someone would, would put up a docker image that was dot 10 mail, right? And then someone sloppy would not see the tend for the send and hit 10 and boom, they got, they got eaten up, right?

It was full of malware. It's a very similar kind of thing combining sort of the Amazon, you know, review gaming with another way of, of GitHub's a repo. It's what it is.

Like all the other repos. And there's a lot of bad crap in our repos. The answer to me for this issue is we need repo firewalls.

Sonatype has talked about implementing one. Mitch, as you know, I know you covered Sonatype a lot. They have one.

We need a firewall in front of every repo that before I download anything from a repo, it is checked from malware. It's either in a sandbox and opened up or something happens and it's checked from malware. It's not a GitHub issue though.

GitHub is unique in their star system. Maybe another way of gaming the system. But we need repo firewalls.

I can't, can't even begin to fat fathom how much safer and better off we would all be if we had re if we had firewalls in front of every repo that was checking every package that gets downloaded. I'll leave it there. Yeah.

When I saw this article, I just, WII, it frustrated me so much because as a, you know, we have an open source part of our product and we worked really hard to get the 300 stars that we have out there. Probably, you know, talking to people, getting their opinion, asking 'em to go out and look at our code. So it's, it's, it hurts, right?

Just have to no more than it hurts. An Amazon seller who works, works really hard getting their product highly rated know or a book an author or something. I know Know, I know exactly.

But there is a part of the human brain. I'm reading this very interesting book called SAPs. Um, I highly recommend it.

And it talks about the, our brain having been kind of evolved around two things, gossip and fiction and how we choose to believe whatever we want to believe. So if you see an open source project out there that's got a hundred thousand stars, you know, and you look at Kubernetes and it's got probably close to that, Jenkins probably has half that. You might wanna question it, but maybe we don't because we believe what we want to believe.

And so this is why this continues, because fake stars is like fake news. If we wanna believe it, we're gonna believe it. 'cause we love gossip and we love fiction.

It's a human nature. That's interesting. I like that.

Tracy. I think there's a, I think there's a safety factor too. Like I think about my behavior on Amazon of, well this has got 3000 stars and it's ratings and it's four and a half and this has got 17 in it's four and a half.

The 3000 must be better, right? More is more is better when in fact it's probably worse. Who knows what it is.

I think there's just that natural tendency that, that whoever is faking the, you know, gaming the system to get more malware out there, it's just playing upon people's natural predilection to say, I need a quick de easy decision if unless guy recommend it to me and I have to go out and figure it out on my, on my own, while I'll just look at those ratings and go, what's sort of, what looks like the best? I'm not gonna research it, right? In most cases.

Well, but, well, yeah, yeah. Two perspectives here, because we got Tracy who's like, people will believe what they wanna believe. Well, people are pre-wired to believe what they wanted to believe.

The Prewired. Yeah. And to do that.

And then you got Mitch's, which which boils down to can't believe anything. So I'm just not even going to do that. I'm gonna, I don't know if I'd recommend calling me, but I'm gonna, you know, check with someone and, and, and, you know, and, and find out.

And I, I just, you know, I I scratch my head Mike Vizard, like when I see this sort of thing and just ask myself are, are we just in an age now where the, the, the BS is just accelerated because I think this BS existed a hundred years ago, 200 years ago with pamphlets of unknown prominence. I I, you know, I think I've used this saying on this, on this, uh, show before that the truth always wins is not when it wins, but not whether it wins. It's when, and, uh, folks using these high star reviews, it may not even be a breach.

It just could be somebody who's gaming the, the system so that they get more popularity. And if it's crap, it's crap. And if you are, uh, uh, using these repos or, or any open source source that's highly rated even by guy personally, and, uh, and, and it doesn't work out for you, at some point you're gonna pay, What is it?

Half a statistics are fake and the other half are made up. Yes. Well exactly live.

That's what you come to me for God. But can you show me how you do that? Scratch your head, Mike Vizard, like, oh, okay.

There it is. She's, I think what we have to go back to, you know, um, John, I always bring this up 'cause I think about it all the time. John Schwartz used this term authoritative bias.

Mm-hmm. And stars are authoritative bias, right? They, so we have to, we have to learn to question authority, But, but that they've Forgotten how to Do that.

That's a double-edged. So two things. Number one, how does Amazon try to un game that system?

They've now have reviews that are by a verified purchaser. You've gotta be someone who actually bought this. And there's ways of gaming that obviously, but they're, They're reviewing their reviews, Alan.

And so then the authority you're trusting is Amazon, Well at least, you know, was a verified purchaser. But secondly, look, this is a society wide scourge that affects much more than our software and malware. This is why many Gen Z and I, I speak to a lot of Gen Z young people, my kids' age, they don't believe a g*****n thing they read on the internet or see on tv, especially if it's from mainstream media.

They think mainstream media is more likely to tell them wrong or lies or non-truths. Then some dude sitting in his mom's basement in Texas and, and podcasting once a week, him will believe. 'cause what does he have to lie about?

Who knows what he does when he is not in his mom's basement? But thi this is, this is a major problem in our society is, you know, the, the ability to separate veracity from, from nonsense, from lies, from, and in this case, malware from people that's gonna, it's the ability to separate Russian propaganda from, from truth or Chinese problem or whoever your, the scourge of the day is. How do we do be, and and I think it goes back to what Tracy said is to a certain extent, we're wired, we're wired for clickbait, we're wired for gossip.

Yep. I, I know what that answer is, but, You know, we're talking 40,000 years ago, we have a lion headed man who may have started a religion, right? Mm-hmm.

And we believed it as omo sapiens. It's a very fascinating read and it, it, it, it answers so big questions for me right now. And I wanna point out that, you know, the CNCF used to use stars for, uh, qualifying to get into their incubation programs for open source.

I believe they've stopped doing that though. Maybe they recognize something that this article pointed out that I wasn't aware of before. Absolutely.

Maybe it's like the old loony tune. The stars are really the, just those things circling around your head after, well whoa, whoa, whoa. Well that, that, that's all folks.

Um, What's a good way to end it? Um, hey, what a great Monday conversation. Chay guy, Mitch.

Mike, thanks for joining us here. Thank you for joining us on Text Drunk Gang. As usual, we've got Text Drunk TV following, we've got a lot of great text, drunk TV stuff if you're in the knowledge graphs.

I think one of the interviews I've done is gonna be playing Right During Today. It's about a company that's working with not with, uh, graph racks, right? Really cool stuff, Mitch.

If you want a guy, Tracy, check it out. com. Don't miss so much of what we talk about here for the last couple months.

We're gonna be really diving and it's gonna be a great show. Until then, though, there's Alan Shimel for Textron Gang and Textron tv. Thanks for joining us.

This is Techstrong tv. Hey everyone, welcome back here to Techstrong tv. I got a new guy to introduce you two here.

It's his first time on with us. His name is Sean Jacks. Sean is the SVP of Software engineering at a company called Formstack.

And let's welcome him. Hey Sean, how are you? Thanks for joining us.

I'm doing great. Having a great Thursday and ready to talk some tech, so it's all good. Alright, well you came to the right place then.

Um, Sean, before we talk tech or maybe as part of tech, you know, I always like to let our audience have a little insight into who's talking to 'em. So I mentioned your SVP of software engineering, right? Running, running, uh, I'm assuming, you know, the engineering development team there, but be, give us a sense of your journey, Sean, how did you, how'd you wind up here?

Sure thing. So, way longer ago than I wanna admit. I went to Cal Poly, San Luis Obispo, central coast of California.

Got a computer science degree. Spent about 15 years as a developer. I worked at 10 person companies, worked at a thousand person companies, um, did a lot of software development and eventually I got pulled into management.

They found out I could talk to people a little bit, started managing teams of developers. I bounced back and forth between developers, software architect managers several times, you know, but they kept pulling me back in. Uh, and so eventually I went, uh, I was running a 30 person team of engineers at a company called Level.

I decided I wanna go back to engineering. Went back to be a senior software developer at Amazon. Spent about a year doing that, and they're like, Hey, you should manage development team.

So I'm like, oh, all right, I guess I'm doing it again. Uh, and so was a development manager at Amazon for five or six years, five years, and then, uh, went over to mindbody, which was a, um, health and wellness SaaS company. And several hundred there I'm Familiar with, with it.

Actually I have the app. Ah, you have the app. Perfect.

Well, we, I spent, uh, six years at mindbody. I grew the development team there from 70 people to over 200 people. Lots of fun, interesting tech challenges there.

Um, did a replatforming, went from a monolith to microservices, went from on-premise to a WSA lot of good, good challenges there. And then for the last couple years I've been at Formstack, which is, uh, process automation, uh, data collection, so forms, documents, signatures, workflows to wrap it, uh, all around that. And then we're on the journey of making a new platform that's coming out later this year around kind of full process automation and data security and how to pull those two things together.

That's an interesting, that's an interesting journey, you know, with the mind body. What I, what I discovered in using, and I've been using it now probably a couple years, is a lot of the providers, and I don't mean software people, I mean like, whether it's a my haircut or a salon my wife goes to, or or it's something, uh, yogurt Studio or, you know, they all use Mind Body as their scheduler basically as their book. Yeah.

Right. Yeah. And that's how you engage with these folks.

And because they don't have the assets to build something like that, or a single yoga studio or a single, you know, uh, haircut, a barbershop or whatever you wanna call it. And, um, you know, having a mind body out there as a solution to them is, is a godsend. So it's good stuff.

Now as it relates to Formstack though, you know, process automation BPA and and stuff like this, not much going on there recently. Is there, Sean? I mean, yeah.

This is a, a market that's kind of being turned on its head with AI and, and some of the, you know, capabilities that they bring to the table. Yeah. And that's, you know, well we've been really fortunate.

You know, we had all of that kind of base level forms, docs, workflows that I talked about, and we made an acquisition middle of last year of a company called Open Raven that did data discovery, data security. And, uh, it was a natural path to bring those two things together and say, okay, we've got all of this data collection and workflow and process over here. We've got data security and discovery over here.

How do we marry those two into a solution that lets our customers really, you know, do their magic, right? Our, you know, our goal is to automate the mundane so that that toil and that, you know, just stuff you have to get done is taken care of so that you can go do the rest of your work, whatever that is. Whether that's, you know, in MINDBODY's case it was making people healthy with gyms or, you know, making people beautiful at spa and salons at Formstack, it's more generic.

We can do that for anybody, whether that's helping healthcare professionals service more people or helping, um, lawyers get to more clients or helping people that are applying to colleges, making that super easy and error free. So we're kind of broadening that spread and letting anybody take the tools we build to make whatever job they're trying to do. Very simple and easy.

Love it. Let's talk about your customers though. Like, you know, I mentioned the MINDBODY thing and it was clear who the customer, who your customer was there, and then I was the customer of your customer.

Yeah. Formstack, who's who, who, who, who are the customers? You know, who, who uses this?

Sure. So anyone in a business that, you know, this is, this is gonna be a generic answer and then we can use some examples. But anybody in a business that needs to move data around, particularly if they need to also work with their customers.

So think of, it's like the B two, B2C model, right? Mm-hmm. MINDBODY was that way.

Form stack is the same way, but it's not just for healthcare or, or health and wellness. It's for anyone that needs to take that data in their systems, collect it from users, and then pass it around to others. So for example, in healthcare, this would be your healthcare kind of IT administrators, professionals, support staff, and they can say, great, we've got information for our doctors, we've got information for our patients, and we need to talk about addresses and health history and medications and get that to people in billing and to the doctors and to the nurses and to the ER staff and, and all those things.

And how do we get authoritative safe information to all those people across lots of different systems? That's kind of where we come in and we help. Got it.

You know, I mean, you mentioned healthcare and immediately, you know, the alarms go from a head. Well, that's hipaa, right? That, and that's why I could see why security becomes a really big, big consideration here.

If you're gonna, I mean, there's data and there's data, right? And, and some data is, is not, I want to say not more valuable, but it has more security considerations attached to it than other data. And, um, so that has to be, I guess, part of, part of the equation for you guys.

What did you do before you bought the, what was it, Raven's? Uh, what was Raven Company you acquired over Raven? Yeah.

Yeah. I mean, that had to be a huge hole that got filled there. Absolutely.

And we had a few kind of individual specific things for HIPAA before that. But now with this new platform, we've got, uh, detailed, uh, machine learning model. So this is where some part of the piece of the AI comes in, right?

And what we'll do with this new platform is we'll connect to these various data sources within the hospital. So they might have Epic and Cerner, they might have their own SQL databases. We'll connect to all those.

We read that data in, we classify it on the fly and build a data catalog that says, here's the 500 data fields you have. Here are the ones that are PII, here are the ones that are hipaa, here are the ones in these different security categories. And then you can build data sets that you work with and say, don't put any HIPAA data in that one, or don't put any PI in that one.

Or market is secure because we've introspective and discovered your data in your data sources. Classified it using ai, brought it into our system, and then let someone who's making a form say, you no hipaa. But I can do PII because I'm gonna ask people for their address and whatever else.

Let me make a workflow and a form to collect that from the user. And then at the back end of that, I can write that, not just back to the admin database, but I can also say, send the updated patient medication to the doctor's notes. And so now it'll be in the source of truth, but it'll also be somewhere else that someone else needs to use in the system again, to make that process really easy.

That's where the process automation piece comes in. The whole data space is really, I, I would say since Covid, we kinda rediscovered data, right? For a long time we were so app focused and, and this may have been your journey as well.

Yep. Sean, as we were all focused on, does my app work well on Android and iPhone? Is it, you know, is it, is it, is there a, a security, an AppSec security issue?

Is, you know, all of these is there response rate, blah, blah, blah. And what we didn't focus on the data, the app is just the window into the data for the most part. And now we've, we've kind rediscovered data and, and how important data is, and not just data security, but using that data.

But we've also discovered that, man, it could get expensive storing data, right? If we wanna, because yeah. We wanna store everything.

Um, and then in terms of data management, is that something we do in a SaaS model? Is that something where we, it locks us into a particular platform? You know, how, how transportable is it?

Yeah. Wondering at Formstack, how do you guys deal with that kind of thing? Yeah.

And you know, that's one of the, the key benefits of this new platform is we certainly at Formstack and, and mindbody before that we had those challenges. Um, with our new platform, we give customers the capability to make that all transient. So we read that data in, you know, we put it in the Redis cache, or we'll, we'll cache it in memory, work with it, and then write it right back out to their systems.

So we're not storing in a permanent fashion any of their sensitive data. We're not duplicating it or having questions about, you know, who's the source of data authority, right? We're taking it from where it is, working with it, adjusting it, maybe writing it back to multiple places, but then cycling it back out of kind of our SaaS system so it stays in a good secure pipeline all the way through.

And we don't have that kind of data explosion that you sometimes get with modern software systems. Sure. A absolutely sounds good.

Let me, we're running low. It's a little bit on pipe. Sean, let me, let me move to the customer journey, the customer experience here.

How do they, how do you, but if it sounds good to me, I want to interact, I wanna maybe check Formstack out. What, what's that on-ramp look like, if you will? com.

There's a lot of information there. And then depending on the size of your deal, if it's pretty small, you can do self onboarding. You know, you can, it's the kind of the standard model.

There's a few different pricing packages that on the lower end you can come in, set up your account onboard, great. You're off to the races to go right? With this new platform and with our higher end pricing and the current platform, that's more of an enterprise sales motion.

So you can call up sales, talk to them and work out the details. You know, oh, you need forms and docs and sign and you want, you know, you wanna kind of mix and match into a more complex, um, uh, price book effectively. So it just kind of depends on where you are.

If you're in the, the kinda lower end of the market or the smaller size price points, self on board, it works great. If you're something that's more complicated, more enterprisey, um, then call us and our sales team will take care of you and we'll get you set up with the right, um, set of tools for the particular job you're trying to use this for. Got it.

Um, I mean, the nice thing about these kind of SaaS things is no job too small, no job too big kind of thing, or Yeah, yeah. We can do. Is there a sweet Spot for You?

You wanna make a couple forms for your daughter's bake sale? Awesome. We, we've got a place for you, right?

If you want to run a, you know, thousands of hotels or thousands of, um, you know, uh, small colleges or, you know, we've got a customers that cross from mom and pops to big enterprise names. So it's, it's really about, you know, do you have data you wanna move around and do you want your users to interact with it? You know, do you want your end customers to be able to fill out a form, put stuff in a document, sign a document, you know, that kind of thing.

Then, then we're here for you. Love it. Sean, we're outta time.

I want to thank you for coming on here today and, and telling us a little bit about Formstack. Uh, you know, as I said earlier, I think this is an area where we're gonna see some rapid changes. We didn't even start talking about tic ai and what all of that might do to this, but, um, continued success over there and, and please come back and keep us posted.

Awesome. Absolutely. Would love to talk to you again.

And, uh, there'll be some interesting stuff coming out this year and, and there's definitely an agentic AI piece in there, so there'll be some fun stuff as well. I'm sure there is. Yeah, it makes, it's a no brainer for you guys.

Sean, Jack, SVP software engineering, a form stock here on Tech Textron tv. We'll take a break. We're gonna be back in a moment.

Welcome back to Textron Unplugged. My name is Cassandra Chin, and today we have Tejas Kumar. Hi.

Can you introduce yourself? Yeah, I'm Tejas Kumar. I've been building on the web for over 20 years of places like Versal and Spotify and Zeta and more.

And today I spoke at info web shift. We talked about AI engineering, what AI engineering even is, how it, how it exists today, what problems it has and how it might evolve tomorrow. Do you wanna talk a little bit about your talk and like the AI engineering?

Yeah, I think one thing that people tend to not fully understand is what an AI engineer even is. And I've asked so many people in preparing to come here, like what, um, do they think they could be AI engineers? What is AI engineering?

And a lot of folks think they don't, they feel imposter syndrome and they go, I can't do this because I don't know, machine learning, I don't know Python, I haven't gone to university. And the truth is, AI engineering has nothing to do with that. It's, it's entirely different.

It's talking to an API and getting a response and using that to solve a problem. Um, and the AI side of that is you're interacting via an API with large language models or some other model, right? Um, and I think that's something that people don't know, but once you tell them, like if they can just like speak to something over HT http, get an answer and then use it, that's all it takes to be an AI engineer in a, in the most simplistic terms, they go like, oh, wow, I could now, and it eliminates some of that imposter syndrome.

Have there always been AI engineers? That's such a good question. Um, I think so, yeah, depending on the kind of ai, like previously we had games like Pacman where there's AI in there, right?

Like the ghosts are trying to chase you, and when you eat the little cherries, then the ghosts turn blue and like they, they avoid you. That's ai. But it's rule-based ai.

So programmers knew the rules of the game and they wrote algorithms to enact them. Some would call that ai. So there have always been AI engineers, but it's evolved over time.

The AI engineering we're talking about today is to do with generative ai, which usually includes generating something, images, video, text, and generative models like, um, GPD three, GP four, et cetera, these models. Is there really a big difference between like a software developer and an AI engineer? It's a great question.

Mm. I think there one is general and one is specific. There's so many millions and millions, even billions probably of software developers in the world.

But a fraction of that would be front-end developers, right? Because there's backend developers, there's, um, ops engineers is specializations, so software developers broad, and there's specializations inside. And so AI engineer, I think of it as like a single specialization within software developers.

So Would using copilot qualify you as an AI engineer? I don't think so. In, in this case, you're a developer assisted by ai, but you're not engineering anything that uses a large language model.

So you're, in this case, you're not an AI engineer, you're an AI user. 'cause you're using it, you're not making anything, you're not what is engineering. It's problem solving through the application of science.

So you're not actually solving a problem with the code. You know, you're not writing, in fact the code is solving your problem. So yeah, you're more user than an engineer.

That's an interesting perspective that like developers are now users. Yeah, definitely. But the cool thing is we don't have to be like, it's a, I was speaking with someone from Microsoft earlier and we were exploring the question like, does, does ai, will AI replace engineers?

You know, um, and I just, I don't think that's ever possible. 'cause like we can, we come up with the ideas and then we can build, we can build them. And then when it comes to actually shipping them, like there's always a review process.

Even now there's code review and at some point we may have large language models and AI that can review its own pull requests and stuff. But I feel like you always need human beings, you know? So like all that to say, yeah, we've become users, but we still build.

And in fact, it accelerates the way we build. Like there's still a creative aspect which humans have. Exactly.

And I think that's kind of beautiful. 'cause humans are just always irrational. And I think as AI continues to grow, it's so interesting.

We often like have a dislike of AI generated content. I, I don't know that I do, but when I talk to a lot of folks and I show them a picture and I'm like, Hey, this picture was generated with ai, they immediately like it less, they ascribe less value to it. And I think we do that also for the code written.

And so how can we replace the human element that way, you know? That is interesting. Yeah.

Like how people view AI generated pictures differently. Do you view AI generated things less, um, valuably as you do human generated thing? Personally, I do.

I Why is that? Um, I think I value someone's time they put into it. Yeah.

Not just their time, but like it's their own creative process. Yeah. Yeah.

I think we need a word for that. I don't think, I think we don't have it, but when somebody puts in time, effort and, and attention, what is that called? You know, we don't have a word for that, but that's the differentiator between AI made stuff and human made stuff.

Yeah. I don't, I don't know if I agree. Um, I once got in trouble for suggesting a future that people didn't want.

I said, imagine the year 2050 where instead of like wa waiting for the new Marvel movie to come out where it's the Avengers, whatever, you come back home from work, you sit down and you just write, I wanna watch a movie with Spider-Man and the Hulk and Batman enter. And then it's just made for you, like instantly. And it's feature length, it's two and a half hours, it tells a great story.

And then tomorrow you come back from, from work and you're right, I want a sequel. And it just like exists. And it's perfect.

You can't tell a difference. I I just talking about this with some friends, we were doing a thought experiment and they were like, I would hate that. And I was like, why?

At the end of the day, you're getting your entertainment. You know, like what's the problem? The problem is humans aren't involved.

But why does that matter? Like, there's a lot of, and this is something people know, there's a lot of stuff on the web today that looks human generated, but is not like accounts on Twitter, accounts on Reddit. If you post on Reddit and you get a reply to your comment, there's a high chance this is a bot.

You just don't, it just doesn't look like a bot on the Topic of generating movies to watch. Yeah. Would you change your perspective if you were generating a cartoon for your kids to watch?

That's a good question. Like maybe we'd be a bit more picky if our young ones, like that's their education. Yeah.

Yeah. I don't, I might change, I might like it if, if, because then I can choose what to show my kids, you know, maybe I don't want them to watch. Um, Rick and Morty, imagine like, they just like find, they're like, oh, this thing looks like a cartoon, but it's Rick and Morty.

I would not want children to watch that. So I think another topic you're passionate about is health. Yeah.

Yeah, totally. Um, I come from, I'm just not a healthy person. I mean, I, I am now more, but, um, I've got a, a disease right?

Where I, I don't know, um, if you know the term hemophilia, I never stop bleeding once I start either internally or externally or whatever. And so a lot of the reason I'm alive today literally is because of tech, because of, so, because of web engineering, what I do. Um, because as a, as a kid to do anything to like, to lift a heavy object or to walk around with a backpack, to go up some stairs to go to the bathroom, all of this would cause me to bleed internally.

And then I wouldn't stop. And at some point I would pass out. 'cause I have a lot of blood loss.

The only thing I could do was this without like, harm. And so I started by playing computer games, but I got bored and I was like, how do they make these? And then I learned that and I slowly just learned a bunch of things about code until I got really good at it.

Um, and then moved, literally got a job by learning how to code in Germany and moved to Germany. I I live in Germany now, and there's a great healthcare system that takes care of me. And I'm like living my best life.

When previously I was living my worst life, it was the treatment model was, if you're dying, they'll save your life, but if not for free, if not, you have to pay. And the bills were like $12,000 a week, a week. And so, um, it was, it was crazy 'cause like I'd wait until I'm in critical condition, go to the emergency, they save my life and go back.

It was lots of trauma for like 20 years. But then I moved to Germany by learning how to code, which is so cool. 'cause there's so many young people who just don't know what they wanna do with their life.

I, and, and now, like I'm, I'm 31 years old, I've been doing this a long time. I get the occasional young person will come and ask me like, Hey, I'm finishing high school. I'm a senior.

What should I do? And ask them, what do you like? And there's no answer.

There's like, I don't know, I management, like, it's very confused. And I have never had that, 'cause I've just been writing code since I was eight years old. It's the only thing I could do.

And in so many ways it like saved me from this. Like, I don't know what I'm gonna do. You know?

And so that was before. And then I moved to Germany and I have this healthcare. And so now I'm like, what else can I do now that my health is taken care of?

How can I optimize it and maximize it further? And so that gives rise to a whole like, different discussion on nutrition and sleep and supplementation and food and exercise. And so that's outside of tech.

I spend most of my time there because it's been a story of health from the beginning. Uh, can you describe more what you mean by focus on like sleep and nutrition? Yeah.

I think as developers there's oftentimes a pressure that we give into. Um, I've been a developer for many years, right? And there's usually a performance anxiety, like I'm not doing enough.

That comes from, I don't know, where either the company or the individual. Then there's this pressure to I need to ship something. There's performance reviews.

And when you tie all of this together, we're incentivized to ship stuff. And sometimes we won't stop. At least with me, I don't stop until something is shipped.

And in the age of remote work, like if I'm working from home, I'll start in the morning and then I'll just continue until the problem is solved. And we tend to underestimate problems. So, oh yeah, I could just build this.

No problem. Next thing you know, like it's three days have passed and you've been in the same spot. Um, we often, especially 'cause it's so easy to get those rewards by writing code, it compiles Oh wow.

And you just keep chasing those keyboard only rewards that you don't get to pursue like physical, real life reward by like running the fastest mile you ever ran. And so we're incentivized to neglect our health by our, either by our organizations or by our own, like reward pursuit behavior. Like, I'm just gonna keep building this.

Wow. It's, the feedback loop with code is so fast that you're like, oh, I'm just gonna, and you get stuck in there. And, um, I feel like this is the typical developer look around, right?

There's people who don't look the way they want and aren't as strong as they wanna be. And that's just the reality for us. And so I spend a considerable amount of time looking at how can I optimize this?

And that comes down to, um, being very mindful with the balance between work and the rest of my life. And so for me, I tend to work very well in like, highly focused bouts that have a time cap. So I'll do like an hour and a half to two hours of like maximum focus.

Like you can't imagine I'll leave my phone somewhere and I'm just like locked in. Um, and that's my deeply focused work for the day. That's it.

Just an hour and a half, two hours. And I call this offensive work 'cause I'm moving the needle, I'm doing the thing. And then the rest of the day is defensive work, meaning there's all kinds of stuff flying around.

There's messages from Slack and Discord and people are tweeting at me and people wanna do interviews. Just all this stuff that shows up in your day that is defensive work. And a good combination of this is very helpful for me.

So when I finish that intense work bout in my mind, I'm like, great, I've moved the needle. When I finish the defensive work, I'm like, great, I've served a bunch of people. And that just automatically, it's so beautiful.

That automatically makes room for the rest of life. How do you know if until put a stop to defensive work? That's a very good question.

Um, I don't, and so I try my best to, um, avoid more defensive war. And sometimes I've gotten very good at this recently, but previously I was notoriously bad at like just closing the computer and putting it away. But now I think, you know, what happened, I think making way for offensive work has taught me, because when I finish offensive work, I just literally close everything I'm done for the day.

And I think continually closing things for offensive work has reinforced the habit of make getting better at closing things for defensive work as well. One thing that puts me at ease for defensive work is Slack. The chat app has this feature called Catch up, but only on the iPhone, only on the mobile phone app, which is really awesome.

It's like Tinder. So like it shows you cards of like the latest messages and you can swipe left to keep them unread until you come back. Or you can swipe right to say, this doesn't matter.

And this, I use this religiously for protecting against defensive work. Even right now, I could open my slack and open catch up and I I will just like, I need this later, I need this later, I need this later. That doesn't matter.

That doesn't matter. And then when I, when it's time to do offensive work or when it's time to start the defensive work, I'm like, look, it's all this nicely curated list that passed me curated for right now. You know?

And so a great help for buffering defensive work, uh, and stopping is to say, great, I'm just gonna use the catch up feature and come back to this later. I think that's an interesting approach. It's like saving it for later.

It's very important because if we don't, and this is so I think there's a lot of synthetic pressure that tells us we need to respond to this thing. Now, that's never true unless you're like an airplane pilot and you need to start descending or something. For us, we put rectangles on screens, like things are very seldom as time sensitive as they may seem.

And so for me, yeah, I'll come back to this later, super helpful. Also, I do this with Slack, you can mark a message as unread and with email you could just mark it for un mark it as unread. And I come back later and I'm like, ah, it's unread.

Let's sort it out now. So, so again, to answer your question, my strategy for defensive work is I just mark things as unread and say, this is a problem for future me. Yeah, that's a good strategy.

It works. It works really well. Yeah.

Um, is this like canal with like virtual work, this kind of strategy? No. Um, even in the physical.

So I used to work in an office before, and there's a lot of things that require defensive war. The worst is probably inter like, people will just like straight up show up and be like, Hey, can I get your opinion on this? And without even knowing, maybe they've taken you out of your flow just like that.

And then you've gotta go look at their thing. Or even if you reject them, you're still out of your flow. So you've lost.

So now you might as well, okay, come, I'll look at it with you. And then you come back to your desk and you're like, where was I? And that takes time.

Um, so no, I, I think it was maybe even worse in the office because when I'm at home, I don't have a person physically interrupting me and, and I don't see their face. And I think this is huge. When we see a face, like in physically, we can't help but respond with compassion.

It just, it's in human. There's a great book about this called Hope for Cynics by Dr. Jamil Zaki, talking about scientifically proving that human beings are just naturally good.

It's an excellent piece of literature, and it's very true. Like if I, if I see a face asking me for help, the chance that I will stop what I'm doing and help them is exponentially higher than if somebody just writes me on Slack, Hey, can you help me? I'm like, no unread, you know?

Um, so the problem is better. I feel, at least for me, um, remotely. That's interesting.

Um, like for me, like I have a dog and when he looks at me, it's like, I kind of quit what I'm doing and just Because of the face. And they, they're master manipulators. Like they know how to, like, they'll do the thing where they tilt their head to the side, they master the art of like, please, you know, they, yeah, they totally, Like, there's some similarities there.

Yeah, definitely. We, because dogs are just mammals. We can look at any mammal that has a face and feel empathy for it.

For real. We can't. And if we can't, we maybe have some type of sickness.

Um, but yeah, anything with a face cats, you can see them. Some people say, I'm a dog person, not a cat person. But if they look at a cat's face, they can infer like, what does it feel like?

Yeah, That's interesting. 'cause I've never considered myself a dog or cat person. Yeah.

I Actually have both. I'm a mammal person. I'm, I'm in the same camp.

I just, if it's living and it's not parasitic, let's go. I hate mosquitoes. That's the only thing I can't handle.

Yeah. I think we've had a really good chat today, so thank you, Tejas. My pleasure.

My pleasure. It's a pleasure talking to you. Yeah.

Hey everyone, I'm Alan Shimel of Techstrong and welcome to the last great Cloud transformation. This is, uh, episode two in our ongoing series of the last great cloud transformation. And it's a joint production between us here at Textron Group and our friends and sponsors of of, of this, uh, show CloudFlare.

And so many, many thanks to CloudFlare for, um, sponsoring and co-producing with us. And, uh, we're excited to be doing this. If you miss the first episode, it is available on Techstrong tv, and I highly, highly encourage you to go back and take a look at it.

But for today's show, new show, va, as I said, episode two. Uh, the title for today's show is Running AI Inference at the Edge. We're peeling down a little bit right on this I idea of a connectivity cloud and this last great cloud transformation.

Uh, before we jump into the subject matter though, I want to introduce you to our panel for today. In addition to myself, I'm happy to be joined by Rainy Hobby and if I got it wrong, hey, B is how I pronounce Randy pronounce it for so we get it right. Uh, hi, I'm Ranny Haiby.

I'm the CTO of Networking and Edge at the Williams Foundation, where I work with our open source communities, helping them kind of, uh, shape their strategic direction around open source networking and open source edge, uh, finding new technologies that are relevant, finding a lot of new synergies between our projects and between our project and external, uh, projects and organizations. So I'm kind of trying to keep my hand on the pulse of what's the latest on networking and Edge, and obviously, uh, ai, uh, and network connectivity for ai and doing all that at the edge is of course a hot topic these days. Um, so I'm seeing a lot of activities in our community around that.

Excellent. Thank you Rain, and thank you and the Linux Foundation for participating today. Our next panel member is Rita Kozlov, R is the VP product management, CloudFlare.

Rita, welcome. And maybe give a little bit of your background. Sure.

Thank you for having me. I'm Rita. I have been a CloudFlare for the past eight years, and for the majority of that time, been building out Cloudflare's developer platform workers, which includes many services, um, some of which I, I think we're gonna talk about today.

Uh, and before that started, you know, my career in software engineering and I have been deep down a big nerd and programmer since then. Excellent. Love it.

Then our third member of our panel today is, is my, uh, partner in co-host Mitch Ashley. He's the CTO here at Techstar Group and a CTA at futur. Hey Mitchell, welcome and thanks for coming on.

Yeah, good to be here. Boy, both listening the both of the backgrounds. I was like, oh man, I can't wait to talk about this.

Hope this goes for four hours. Is that how long we're gonna go? Not quite that long.

Not quite. I don't know if, I don't know if the folks out there wanna sit in and listen to us for four hours, maybe we'll have after the show conversation, maybe we could chop it into 10 parts. But, um, anyway, I, but I agree, we've got some great panel members who are bringing some amazing focus on, on this issue, this, so we're gonna talk about running AI inference at the edge, but before I jump into that, a, a quick word about AI at the edge.

Both, both, all three of our panel members are, are talking about this and that, Hey, it's amazing, right? We two years in generative AI is cool, it's cool, we love it, but we're recognizing that we can't run, we can't do all the great things we want to do with AI going, you know, backhoeing, so to speak, back to the, the, the, you know, the main cloud center, these hyperscale centers, we need to be closer to where the action is. On the other hand, most of our endpoint devices, unless you listen to Apple, about Apple six, the Apple 16 phone right, are not super computers and they can't run everything we, you know, larger models of AI on that device.

It's, they just don't have the footprint. So we need something in between. And the edge is that place where we can get closer to where the action is, but still have the footprint to do the heavy computational work that's necessary.

And that's driving this whole, there's a whole thing going on. Let's, I don't wanna call it a next generation of AI or heaven forbid, gen AI two O or something like that, but that's what's driving it. And so we wanna get smart about running AI at the edge.

What, how do we run it? What parts of it do we run all of it? When should we go back when?

How did they all work together? Um, Randy, I get the impression this is something you are, you know, putting a lot of thought into. What do you think?

Yeah, so what, what we're seeing is indeed this desire to run, as you said, the inference mostly close to the edge. And when I say close, it means two things. One is close in terms of latency when decisions need to be made fast and, and provided back to the end users.

That's one type of closeness. But the other kind of closeness is where maybe you don't want the da your data to travel all the way to the cloud. Maybe there are security issues, sovereignty, privacy, so on and so forth.

And this is when we see that in use cases, people want, wanna keep the data close to where it's generated, but still do all those fancy things of, uh, training on this data insurance based on this data and providing insights, uh, based on this data. So that kind of puts edge computing at the sweet spot of being able to do all that and still comply with these, uh, strict requirements for latencies regulations and, and privacy requirements. Fair, fair.

Mitchell, uh, Rita, thoughts on, on what Randy said or I, I had some ideas on what did it take that said, Well, I, I love the, um, developer perspective on this, right? Because people that are creating applications, they're trying to, they're wrestling with where do we put things right? Do, what do we put on the device?

What do we, is there someplace close to the edge or at the edge of what we're connecting to what resources? It, it's like, it's all one kind of big map of resources you, you have available to your application. It's just, where is it, how do we get to it and what's the optimal way to leverage that?

And that may be different two years from now, right? We may do more heavier things on the device than, than we do today. So it's kind of a moving target.

Um, I, I know one of the things that I really, and by the way, just full disclosure, we're a, we're a CloudFlare customer also, so we're, we're very familiar with using their services. I've been really, I don't, this is not just being nice, I'm really impressed with the progress you've made on the developer portal and the APIs and really kind of programmatic programmability of the edge. And so I, I'd love to hear, Rita, your thoughts on, so how did you decide that the approach that you took, how do you help people do?

Maybe they don't know what they're doing, what a one they wanna do yet, you're kind of helping them create the future. Yeah, I, I think your point about this being interesting from the developer standpoint is really good. And that was the way that we approached it, where, from our point of view, we've been watching, you know, developers building applications for as long as CloudFlare has existed, right?

So for the past 14 or so years. And so we've seen developers struggling with all sorts of challenges from, you know, whether you're developing on-prem or in the cloud, uh, especially, uh, you, you still have to do a lot of work around provisioning infrastructure, right? Where, uh, when you're in the seat of the developer, actually your, your task for the day, uh, doesn't end with, oh, great, I provisioned a server.

Um, the actual ticket item that you're trying to check off is, you know, I ship this feature and now it's in customer's hands. And we've been helping solve that challenge of, you know, how do you unblock developers to focus on just that for, uh, as long as the developer platform has existed, by allowing developers to deploy code directly to our edge. And what's been really interesting about, uh, you know, this rise of AI over the past two or so years is we've seen a lot of these problems come back, but in actually a kind of even hairier way where when you're deploying AI applications, um, first of all, there are a lot of different things that you need.

Um, running the model itself is just one part of it, right? And here you're trying to focus on bringing the best experience possible to the user, but you, you have to, um, you, you have to do all of this provisioning upfront, right? So, um, you, you need all of these, uh, different tools.

You need the model, you need a vector database, um, and, you know, we, we can talk more about that later. Um, but you also need to make sure that your application to Rainey's point is really, really performant. Um, you need to make sure that it scales.

And I think that that's been one of the really hard challenges is when you're launching these new AI products and features, you actually have no idea is it gonna be successful on day one? Maybe. Yeah.

Or maybe not, Maybe not, right? Um, and there's actually a real cost to getting that question wrong. Uh, there's a penalty in either direction, right?

So either, um, you know, you're like, okay, I'm gonna have a bunch of users on the first day, I'm gonna go ahead and provision all of these resources, and then no one shows up. And then you get your cloud bill at the end of the month and you're like, great, I had a bunch of boxes on, you know, standby, uh, or inversely, uh, you under provision, and then all of these users show up and all of your hardware kind of goes to nothing because the traffic falls on its floor since there's no way to swallow it. So the the way that we approached it was really from that perspective of, okay, how do we like step one, let's enable people to build stuff, and we're good at infrastructure and we will handle it on our backend.

And we really wanted to take this kind of serverless first approach where you focus on building in the application, we'll take care of provisioning the infrastructure and, you know, let, let's see what direction the industry evolves in and take it from there. Agreed. Um, you know, what I find important?

So look, the whole reason about being at the edge is to be close to your endpoints, or I've waited 40 years as a yes fan to say this, to be close to the edge. Um, you gotta be my age to, to appreciate it. But anyway, the fact of the matter is, the edge is doesn't, it's not the edge in the edge alone, though.

It, this is, we gotta think about this as an interconnected web of things here, right? You can't do everything on the edge. The edge is, you know, one area, but we still need that core, if we could call it that core data center, right?

The hyperscale data center, right? We need, we need the horsepower that only those places bring to bear on, on some things that we need the edge and its unique capability of being proximity wise close, but still have, you know, moderate to, to heavy horsepower. And then we still can do some things on the endpoint, especially all kidding aside, as these next generation of endpoints incorporate faster, better processing power and more, you know, more capability.

And, and quite frankly, as we get more efficient in running AI applications and AI assistant applications, so now you got this picture right of this interconnected endpoint data center edge. Well, you know what, emphasis on the word connected. We, we've gotta connect them.

And that's part of this last great cloud migration. The connectivity between them has to be, you know, 'cause latency counts here, right? Every millisecond counts.

So we need providers, we need solutions that have that kind of connectivity. And anytime you're doing that kind kind of connectivity, you need security, right? And I mean, Rita, obviously these are two things that CloudFlare is known for, but Randy, you look at it with open source tools and there's open source edge going, a ton going on at lf.

Let, let's talk about what are the kind of unique connectivity needs to make this system work? Yeah, so I think you kind of alluded to that when you said that it's a very complex mesh of things on the edge and the cloud. And on the other hand, we have developers, I think we touched upon it, saying that developers need to focus on, and this is what they know how to do on, on developing the application.

They don't know much about cloud infrastructure, they don't know much about connectivity and, and VPCs and, and firewall rules and so on and so forth. So what we see the open source community do, and then the service providers like Cloudflare's and others actually build their solutions using is open source projects that are Dell are dealing with, uh, kind of abstracting all this complexity from the developers and doing all the nasty and, and complicated stuff as much as possible behind the scenes for, for the end users and presenting the end users or developers with very simple to consume APIs in which they can express what we call their intent. So I'm building an application and it needs to access some data set somewhere.

This is what I wanna, I wanna request from the network, and I want everything to get connected behind the scenes, uh, without me having to go and do all these, uh, point-to-point connections or setting up virtual circuits or whatnot. And if you, you, you mentioned an environment where you have multi edge and multi-cloud, and we have one of our projects, uh, newest project Paraglider is dealing exactly with that, with abstracting the consumption of the network connectivity between multiple clouds and cloud and edge, uh, and, and making it consumable by, by the developers and providing, uh, unified API. So no matter where your data or, or workers are, whether they're in Asia or, uh, um, in AWS or on-prem or on some, uh, edge cloud, you want the same API to set up the connectivity and, and you wanna let to free up the developer to develop the, the business logic, the application, and not worry about how it's interconnected.

Fair. Um, so Rita, you know that the Linux Foundation obviously champion of open source champions of open source. I wonder how does that, if you could compare and contrast that to the, uh, CloudFlare solutions for, you know, the connectivity cloud, if we could call it that.

Uh, and I'm sure there are some things that are the same that rainey's talking about in the open source, uh, you know, uh, model, but you guys have also had 20 years to play, you know, to play in this arena. And I, I assume you've learned some lessons and would love to hear from Matt. Yeah, I mean, first of all, the, the two are not at odds with each other.

And actually, um, yeah, a a lot of open source projects are, uh, built on top of CloudFlare and, uh, v vice versa, right? Um, CloudFlare itself is built on a lot of open source technology. Um, and yeah, the, the way that we kind of approach it is, you know, as you mentioned, uh, you need this connectivity between all of these different endpoints, um, across device, across the edge, then you have the cloud, um, and we, we've, what we've really done over the past 15 years is build out this network, right?

Um, and so with the network, we have different angles or different ways of looking at it in order to connect all these pieces. So a lot of people when they think of Cloud layer, they think of, um, C-D-N-D-N-S DDoS. And in that capacity, actually we've been, uh, you know, using machine learning and AI or, you know, I was called machine learning before.

Everyone got super excited about it, um, for basically the entire time to observe things like DDoS attacks, right? And understand, okay, this is what the pattern looks like. We build a model on top of that, and then we have it running on our network in a way that can, uh, allow through traffic that is legitimate and prevent traffic that's not legitimate.

Um, so we, we started thinking about it from that angle. Then there's, uh, the developer platform angle, which we touched on briefly before, but I, I think from that standpoint, it's like, okay, what are all of developers needs when it comes to developing an application? And how do we bring all of those to a single place so that you kind of have, you know, I like cooking, so I like to think of it as a museum floss, right?

Um, but you kind of have everything that you need in one place, whether it's running your compute, running your, uh, ai, running a database, running your storage. Um, and, and that's kind of our take on connecting things there. And then, um, obviously, uh, we have companies that are using us to secure their network, secure their own devices and employees.

Um, and there's actually a fair amount of AI that's involved in there. And actually with, um, more and more companies deploying ai, uh, solutions like DLP have started to become more prevalent where people are worried about, you know, I have, uh, I have all these employees and maybe I put these policies in place, um, that say, you know, you're not allowed to use these tools because we're worried about our data leaking out, and then models being trained on top of that. Um, but obviously if, you know, getting an answer is as easy as typing a question to chat GPT, you're gonna have people that go and do that.

And so having that visibility through our network becomes really, really important. Um, so all of those things are kind of what we've observed, uh, in terms of problems that our customers have. And again, the way that we've tried to, or the way that we approach solving it is, okay, we have this network and what are the different ways that we can deploy it in order to tackle that, if that makes sense.

Uh, it makes perfect sense to me. I mean, and, and, and I like that you really hit the second piece of which, which is the security aspect of it, right? Because if we're not comfortable with that data on the edge, and then we're not comfortable with that data, you know, zipping around Cloud Edge endpoint and back around again, it just doesn't work and it just doesn't work for us.

Um, Randy, interesting. I know Linux Founda, obviously the Linux Foundation is a foundation of foundations like 40 some odd daughter foundations of lf, one of which is, for instance, the open s uh, open Source Security Foundation, OSSF, there's several different security themed organizations as well as security sort of built in security projects built into all of the various order foundations. You heard what Rita mentioned, especially, especially around security.

How important is, well, I know it's important, but what, what's the LF doing on that end as well? Yeah, so, um, maybe it's the, uh, opportunity to mention that indeed, uh, telling Foundation now hosts over 1000 different open source project, and some of them are organizing to these, uh, uh, dozens of, of sub foundations. And, uh, one of them is open, uh, SSF Open Secure Software Foundation, which provides tools, best practices, and methodologies for developing, deploying, and managing software across the entire life cycle.

So, uh, the, uh, the product of the open SSF can be consumed by anyone, uh, developing or deploying software. But another thing is that, uh, many of our, uh, other foundations under the Lung Foundation, like networking, like Edge, like, uh, energy and others are actually working closely with the open SSF and integrating these tools as part of our software development pipeline. So, uh, things like software build of materials, uh, are already embedded into the pipeline of building many of our open source projects.

So if you are using some open source technology as, as a service provider like CloudFlare or as a technology vendor, uh, you get software that is secured using the most, uh, recent, uh, and broadest set of tools, making sure the software has no back doors, your, your supply chain is secure, uh, and so on and so forth. So I think that we, again, developers are always tend to kind of leave security, uh, for the end and, uh, try to, um, edit as an afterthought. But what we are trying to educate the open source communities is that it's not the best way to do it.

And you better start integrating these tools from open SSF and other places early on in, in the lifecycle of the software that can save you a lot of headache further down the road when you have to deal with some fire drills or breaches and stuff like that, where it's way too late to, to deal with that. So I think, again, I want to debunk the meat that open source software is an alternative to the service providers or vendors. We are just, our communities are actually, they consist of, of contributors from all these service providers and vendors that are building, commonly building the technology and then using it in their commercial offering.

So everybody who's, who's collaborating with the Learning Foundation on on the those collaborative project actually benefits from, uh, using the software that has these built in, uh, security mechanisms. Are we perfect on that front? I'm obviously not.

Nobody is. Uh, we try to learn from our experience and to constantly evolve and as new tools and, and best practices evolve, we try to incorporate them into our open source projects and, uh, make the projects that products and services that are built on top of those projects more secure. They're great.

I mean, we're all one community here. Um, I got two other issues I wanted to bring up. One, I, I'd like to have your thoughts on it, Mitch.

You know, look, the title of today's episode on the last Great Cloud Regression Transformation is running AI inference at the Edge, right? And, and we're giving that as the reason why we need AI on the edge, right? I don't know if everyone out here understands what we mean by AI inference and why we think it can only run on the edge.

Mitch, can you give us the, the 4 1 1 on that? Sure. Um, well, first of all, we're, we're not, probably not talking about training large language models that happens in great big giant, you know, GPU data centers where you need those things very close to each other.

We're talking more of applications or use cases where, um, AI might, AI might not be the application, but it's in built inside of your financial banking application. Maybe you, you've got local data on your computer or your, or a laptop or your, uh, cell phone or some other device that you want to be able to analyze without passing it up, right? And some of this also goes into the regulatory community, right?

The edge is kind of an amorphous thing. What the edge is today isn't what the edge is tomorrow, and it doesn't just stop at the provider level. It's at what are, what's consuming content at the edge.

And AI is one where there's a lot of concern. Um, Rita mentioned, uh, data loss prevention, uh, as an issue. Um, even in some cases, you don't want it left outside of the hardware.

You don't wanna leaving the GPU and the memory that's, uh, that's processing that data, um, because of the sensitivity of it. So depending on the application, it can be proximity to the user and the data. It can also be security that drives it, that we don't want that data pushed because maybe that data center isn't in a regulatory compliant area where we can be sharing that information or centrally housing it.

I think, I think there's a lot of innovation happening too about how we do use ai, AI at the edge. You mentioned, uh, you know, the new iPhone and Apple's inference or Apple Intelligence, and we have, uh, copilot plus and a lot of things that are being added to devices. And we're really kind of at the beginning of what some of those AppSec are gonna do.

I mean, I think in a year or two we're gonna go, wow, we never thought about that. That's very cool. We may not even know it's inside of our app that that's actually AI performing those functions.

So, you know, I used to have this debate, is AI a product or a feature? Well, I think it may be an embedded capability. Sometimes we don't even know it's there.

Uh, so there's some unique requirements we have, uh, I think in this world where we have so much happening at the edge, uh, and devices communicating across it. I think, um, if I can add something to that, the really interesting thing to me there is, you know, I, I think the thing that really wowed everyone about AI and that really gets people excited about AI is the potential for a boost in productivity, right? So once you start using these AI tools, ideally they kind of augment your work in a way that allows you to, you know, maybe code faster, right?

So I've been using, uh, copilot and cursor and all these tools and, uh, you really do become so much more efficient as a result. And, um, similarly, you know, things like chat bots, which is again, to your point, I, I don't think that we've figured out quite the end use case there or you know, what that ends up looking like. But I, I think unlocking productivity is one of the things that is so exciting and so promising about it.

And that's where I, I see productivity and performance as two things that are so tightly connected, where if you are, depending on ai, you know, is it a feature? If, if it is a capability, if you're, uh, relying on AI capability for everything that you're doing several times a day, um, every single bit of performance, every single bit of latency really starts to matter a lot. Um, and so that's where, you know, as to why AI inference on the edge, we started to see, okay, well if it's not running on device, it's running on a cloud provider that's really far away.

And right now in this experimental phase, everyone has a lot of patience for it, right? It's a new technology. So you sit around, you wait for 30 seconds for AI to generate a response, and you can go, okay, that's about what I expected.

But as you really start to incorporate that into your workflow, you're like, actually, this is slowing me down more than it's speeding me up. And so that's where, you know, I I think as it becomes more and more ubiquitous and embedded into everything, uh, we're gonna see the edge play a huge role in unlocking that next phase of applications and inference. Agreed.

Yeah. We, we also keep talking about that, whether the data is on device, on the edge, device on the cloud, what I'm starting to sense is that maybe there's a desire to blur the lines a little bit and maybe keep a mix of that and make the data accessible from anywhere. So we have one interesting, uh, open source project that we launched recently called Edge Lake, where it lets data stay on a distributed edge, but be queried from anywhere either the edge or, or a central location.

So that kind of addresses many use cases where the data might not all reside on the edge or on device or in the cloud, but again, whoever is developing the application needs to access it wherever it may be. So we kind of, at least in the open source world, we kind of see a desire to, uh, make this access, um, kind of independent from where the data is and, and open up the door for new type of applications that don't care where, where the data is. Agreed.

Guys, I'd like to bring up one other topic 'cause we're, we're running low on time. I mentioned this term connectivity cloud nailed it earlier and we really didn't dive more into it, but, you know, it really kind of describes what we're talking about, a new kind of cloud that in encompasses the edge, the core, the endpoint, the connectivity between those places and the security. There aren't a lot of players on the global stage who can, who bring all of that to the party themselves, right?

CloudFlare happens to be one of them, right? CloudFlare has the data center, the edge, connectivity, security. I'm wondering if we might not see, and, and there's probably, I could count on one hand the amount of providers who could do that besides CloudFlare worldwide, I think, right?

Quite frankly. Um, are we going to see, and maybe, and Rita, this is outside of your, you know, sweet spotted CloudFlare, but is CloudFlare partnering with other providers, maybe with some of the hyperscalers and stuff to enhance that connectivity cloud to make it available on a wider scale worldwide to, to others? And you know what I'm saying?

I do and we very much, uh, see our role here. You know, the, if you think about the word connectivity. Uh, it's about connecting two things, right?

And so the, the, the more, uh, the more organizations out there, there we can partner with, especially, uh, if we, so as a product manager, I always go back to the customer, right? And from a, the customer's perspective, it's like, okay, this is my world. These are all of the vendors that I use today.

This is the cloud that I use today. This is what my, uh, employee network looks like today. These are the devices that we have today.

And so we very much see it as a big part of our role to partner with all these different entities. Um, whether it's, um, again, more on the, uh, zero trust side where maybe we partner, you know, with, uh, companies like CrowdStrike, the DO endpoint, um, application protection, right? And we can take care of the other part of the security or with the hyperscalers.

So yes, um, if you are working on an end-to-end AI deployment chain, uh, the training part, you're not gonna run that on us, but we wanna make it easy for you to connect, um, your data that might be in R two on cloud player, right? Um, this is where, uh, even things like, uh, free egress really come in for us, where we find it really important for you to be able to, uh, then connect out to maybe different cloud providers for training if you want to broker across them and not have to spend an arm and a leg there. Um, and then be able to, again, connect to CloudFlare to do the inference part.

And so we, we really see ourselves here as, you know, if you, the, obviously, the more of CloudFlare you use, the more advantage you get to take of where, you know, the, the pieces really play nicely with each other, but at the same time, we always wanna meet the customer where they are today. And, uh, realistically that's, you know, using a few different providers in culmination. Yeah, absolutely.

Yeah. And, and I, and I think that obviously an open source future, uh, organization is the, the place to work on these unifying the way we use and consume services. There are always, I mean, it's not simple.

There's always that striking a balance between, uh, the need for member companies to stay competitive and, and differentiate and provide value and the need to make things more standard and make it maybe. So the shortsighted approach to that is no, I'll make all my interfaces unique so nobody can, can ever live. There's an, uh, uh, a well known cloud provider that was notorious for, for being kind of this Hotel California approach where you can never live.

But I, I think in the long run, it, it doesn't work. And it doesn't pay because you're losing a lot of business that way. And if you kind of align with the more standard interfaces that maybe are created by the OnSource community, there's still a lot where you can differentiate yourself on per performance, on cost, on, on outreach, and so on and so forth.

But, uh, the, the interface and API are probably not the way to differentiate. I mean, it's better to align with some standard or defacto standard of, of interface and API and, and differentiate yourself on, on the true value that You can provide. Fair, fair.

You could check out, but you can never leave. I, I get it. The World Garden.

Anyway, guys, I'm looking at my watch. We, we are at a time. I think this was a great discussion about, you know, the, as again, I don't wanna call it AI two O, but you know, as this AI generative AI works its way through our technology, recognizing that we need to be able to run these on a distributed edge network and what that kind of network needs to look at, like, or we want to call it a connectivity cloud, what that needs to look like, what kind of functionality it has.

Rainey, Rita Mitchell, I want to thank you for joining me today on this. Again, many thanks to CloudFlare for, uh, helping us produce and, and distribute this. I, I think it's a, as we go on, we're gonna dive deeper and deeper into these topics.

I'll also remind you that we have a live round table version of this one coming up, where we will have people, hopefully you watching this log on and ask the our panelists questions and contribute to the commentary and to the discussion, because it's great having experts like this on, but we really want to hear what you're doing out there. So stay tuned for that. But for now, this is Alan Shimel.

On behalf of the last great cloud transformation, CloudFlare and Techron, thanks for joining us. All right. You are watching, listening to and enjoying this move sounds of episode 65 of infrastructure Matters.

It's just me and Kimberly. This week, Diane is busy interviewing CEOs. He's released this huge CIO study on spending and trends.

com to find out more. Kimberly, how's it going? I, I've had to straighten my antlers.

They're kind of like bent all over the place here. They're Kind of so sorry. You, you can't disrespect the antlers.

You, you, you can't have bad, you can't have bad antler head. No. And on the way in this morning, so I, I, it's usually, usually what we're recording, this is eight o'clock.

I come in about 7:00 AM but coming, coming down the, the main Broadway road in, um, in Boulder, um, the, uh, one of the deer families was tripping across the four lane road there. And, uh, so gotta make sure that they don't get, it's like screeching the brakes on as a little fa They're not, the fawns aren't little anymore. They're pretty, they're pretty stout by this time of year.

So, um, anyway, so I feel like I'm kind of channeling my deer friends that live in our backyard. Yeah, yeah. You know, we, uh, we didn't have it on the agenda, but you mentioning being in the office in Boulder, we got a big air conditioning upgrade, uh, what was it yesterday?

Uh, it's been all week. So yeah, they had to rerun a bunch of electricity as well. Um, and, uh, they've got, you know, quite a bit of money that got dropped into some new air conditioning, because there's certain companies that, we have certain pieces of equipment that are blowing up the data center and just the, um, air conditioning has not been able to keep up with it.

And so, um, maybe someday we'll be a water cooled engine here, um, because it's, uh, the good news is that the guys here are super busy at the lab. The bad news is that they're super busy and they're working all over throughout Christmas. So, um, they're not happy about that right now.

Yeah, cooling is one of those data center. Coolings is one of those thankless jobs. Like no one notices it.

The, the equipment stays cool and stays up, and no one really notices the, the, those benefits other than it's more reliable and more performant. Well, when we had, we had a, fortunately we're not a high, high availability center and that kind of stuff, but we had some spikes that caused things to sh Beth spikes. I mean, we're like pushing up to a hundred here.

And so we, I mean, that, that's, that I think gets an idea about the panic that's probably in some of the other data centers that are going on. And we didn't even have the GPUs here yet. That's going into another data center that's in another location.

So that's a super excited. I can't talk about it yet. I don't think we can talk about it yet, but no, we get super excited about that.

No, yeah. But lots of, lots of big, big stuff. Big, big lab.

Everything else, they, the boys are gonna have many, many, many, many toys. I'm sorry. So Really toys, The boys and their toys.

So Let's start off the conversation. Your, uh, data infrastructure industry in 2024 m and a. Let's talk about the year and kind of review what, what was the big news of the year?

Well, you know, when people come and talked about data and data storage, they are often thinking that it's kind of one of the boring things of the world. Um, especially when you, I've hired millennials over the years and have to tell them there's quite a bit of exciting things going on here. Some people love this world, you know, they, and so, so I was just reflecting on, especially as, um, Oxia, just IPO with this last week, um, 18 billion, I think is what, you know, they are.

8 billion is kind of valuation or something along those lines that they took out. That's a hard drive in. Well, they're, there's, there's, um, solid state drives, et cetera, out of Korea.

Um, I mean, big, big IPO for a company this last year, we've had a whole bunch of m and a in efforts. Um, we've had, and also investments. Veeam took a chunk of change this year, and they, from a, um, multiple, multiple investors coming in to them, and they're still privately held.

Um, and so their valuation, I think was came up to 15 billion. They're a data protection company, guys. I mean, how boring can you get around data protection?

I mean, it's just not as sexy as ai, but here they are. And when you don't have your data, you know what's bad. Um, the other thing that happened with the big stuff was the Veritas Cohesity relationship that caused Veritas to split into two.

So some of the stuff went over to Cohesity, and they're now this big huge data, another big huge data protection firm. And they spun off, um, another portion of it, which was, um, Erra. Um, and they are, they're at 400 million a RR run rate.

Um, so, and all their products are rule of 40 kind of environment. So they've created these three divisions that are offering governance, data protection and, and, um, high availability capability. So see what they do, they may sell off those pieces over the next year.

Um, we are seeing a, what Western Digital WDC split up guard, we're gonna see Western Digital WDC become the hard dis guys. I thought hard dis were dead, but they're not. And then we have the other side, they're splitting off as the sand disk side, and that's gonna be the solid state stuff.

So we'll see some more competition with Samsung and, and those kind of folks. 5 billion, you know, uh, valuation for a global file system. I mean, that, that's, I'm, I'm amazed that they're still around and that they haven't been sub, you know, subsumed by all of the other global file systems, but Kudo.

Exactly. And then you got wca, wca, WCA as well. They raised another 140 billion, million dollars, and they're valued at one.

6 billion. So, I mean, it's kind of like, if you, you're not valued a billion dollars. I mean, we used to say those were the unicorns, but these were all, I guess all of this is unicorns.

How can they, unicorns are supposed to be something unique. It's no longer unique to be a billion dollar valuation. So I guess the really unique one would be Databricks, right?

And you were talking about what happened to Databricks. Yeah. That, that, this, you talking about a high popping, it would be amazing to value a company at $10 billion.

They're not publicly traded. They had a J round a, I didn't know J Rounds existed, $10 billion raise, which values them at $62 billion. And it, I think it shows you where, you know, AI is influencing companies like Vast and all of these companies that enable ai, the enable the, the concepts of data lakes and data warehouses, and centralizing data and making data more accessible to ai.

But that's a eye watering raise. They're, they're now, they now have the, the record for the, the largest private raise ever. This is mind boggling.

I mean, it's just, even when you talk about the OpenAI kind of stuff, you know, it Was, yeah. Yeah. I think OpenAI was the largest before that at six point something billion.

So they beat OpenAI by, you know, three and a half billion, something like that. And three a three and a half billion dollar raise would be an amazing raise. So Uni, I think unicorns, I, I think we need a new name for unicorns, maybe Super Unicorns Or nothing.

I, I have to consult my, I have to consult my five and 6-year-old nieces to find out what, what the, what's, what's more unique than a unicorn. That's true. And then thinking about 2025, we're gonna, if the market continues as strong as it is, even though we've had a little bit of a leak, um, kind of expecting a bunch of IPOs, which I already mentioned, WDC SanDisk flipping out, um, thinking va and you mentioned Vast Data, it's a good PO probability that Vast go out.

Uh, IPO, um, solid dime sometime in the next 12 months or so. There're probably, there's another solid state drive player. Vast is a big file player for ai.

And then maybe ve um, maybe, maybe, maybe not. I don't know if it'll be this year, maybe 18 months. But they're gonna want look at it.

You know, my big concern is, as personally in my investments is thinking about, are we overheating again? Um, and that's the AI's question, right? Are we overheating?

Yeah, we, uh, we, we kind of thought the days of standalone storage companies was over with Dell, EMC, NetApp, HPE Pure, and those folks kind of consolidating those markets, buying up, uh, clever startups. But the m and a from the big guys absorbing the Wicca and the vast datas of the world kind of dried up, right? And it's now, uh, these companies are showing that they're still innovation to be head being is a backup company, you know?

And Cohesity is making ways. Rubrik already went public earlier, uh, was it this year or last year? They went public.

They, it's been, I think about a year ago, about two years ago. So there's still, you know, I I, I famously wrote, uh, did a podcast maybe about five years ago that said enterprise storage was boring. It's far from boring.

It's, uh, very active and maybe overheated se uh, part of the, uh, investment in IT landscape space. Well, this gets back to Dion's CIO Insights survey. What's number one on a survey that's on top of mind Cybersecurity number two is ai, right?

And when you look at that, what's going on with the data protection is also, if we did a drill down into the other data that we have on the data protection side, and the cybersecurity side is they, are, they, the enterprises are looking very strongly at how they're protecting their data, um, because of the ransomware threats, et cetera, which is continues to ramp up. Last time last week, we talked about quantum computing and where that was going and how fast that's gonna come and come and hit us in the butt for security reasons. So why those guys are getting the valuation is, you know, they're link to those two trending areas that can have maintained strength.

Cybersecurity has been at the top of the list now for, since 2020, since we hit the fan with, uh, COVID. Um, and it's gone bonkers, and it's continued to be bonkers. Um, there's no panacea, you know, it's, Yeah.

And AI seems to be, uh, AI and cyber security are linked at the hip. There's, uh, I think we could do probably a whole episode on how AI and cybersecurity are, uh, interesting bad partners moving on to, you know, more investment news. Broadcom now a $1 trillion company.

They've joined the $1 trillion club. Maybe that's the ultra unicorn riding on, uh, uh, enterprise software, uh, assets. So Broadcom, there's a lot going on with Broadcom these days.

They're talking about their AI chips and AI strategy. Uh, there's plenty going around in their network, bu around their network chip sets that drive everything that's not named Cisco. And then some, uh, uh, instances, some lower end Cisco, uh, solutions.

They're everywhere. But the probably most interesting news is VMware, it's more profitable than Broadcom thought it would be at this point. And they are, uh, approximately, the estimates are about 50% of broad of, uh, Broadcom's $21 billion in software revenue this past quarter.

Uh, somewhere around, I think VMware's less revenue numbers was, uh, $13 billion for the year. So, uh, and I think that's the annual number. So year over year, uh, VMware is up 10%, I think in profitability.

It is a really interesting story. Uh, it's, it is been, I think, a wonder story for investors. Customers, uh, you know, well, the, the jury's still out.

I think a lot of enterprise customers are still kind of not, uh, let's say all in, in this transaction. And what it means for them, And the conversations that I've had, um, continue to have, and I'm sure you're having, is that they're not migrating necessarily off of VMware. They're developing off of VMware, which means you have a very long tail, just like you have with the other huge investments that he's made in mainframe.

Right? So, you know, I think that his, the profitability and the valuation that they have, et cetera, is, is, you know, it's, it's probably super solid. Um, Yeah, no HK to comment.

He's super excited about the growth of VCF Beware Cloud Foundation, which he hopes and promises will be that landing zone for net new development. And, you know, the devil's in the details. We don't know how much of that is because the VCF licensing is kind of the way to go.

Mm-hmm. For large enterprises, it gives you licensed portability. There's a lot of advantages.

So how much of it is buying shelfware and not actually deploying the, the advanced features of VCF and, and enjoying some of the licensing benefits for traditional vsp, and how much of it is people building new infrastructure for cloud enabled AppSec? It's a fascinating time to be a watcher. And, uh, probably uncomfortable time to be a VMware customer.

AI continues to be top of mind. Uh, I don't know if it's a surprise offering, but Nvidia announced their Jetson announced, just like the futuristic cartoon that promised us flying cars, but yet I have AI instead. I, I guess, uh, I'll, I'll get the talking robot made at any point thanks to Elon.

Hopefully Elon more you can on my, uh, SEN powered, uh, made, but the SEN is a series of devices. The, the one that got the most attention is a small, uh, I guess the best way to compare it to is a Intel nook size device that, uh, yeah. It's, uh, camera Lee's raising her fingers, representing, you know, about a, a, uh, small edge device that has quite, that packs, packs quite the bit GPU capabilities in it.

Mm-hmm. And we talked about, uh, the possibilities of, on a call yesterday, we talked about the possibilities of stuff and the 61 terabyte or 122 terabyte drive in one of these things. And what would happen as a result of what CAP and what, what could you build at the edge?

Yeah, I mean, it's pretty, it's, and what they're, what they announced was 250 bucks, right? For it's an SDK, um, software developer kit. And, uh, so I'm not sure how much that's, I'm sure it's still deployable, even if it's software developer kit.

Um, and I, I asked, uh, Ryan Shroud, who runs our Signal six five, if he was getting that for all the kids at, in the lab. So they, yeah, every, Everyone should, for Christmas, everyone should get a just an SDK kit. Yep.

Build something, build something fun. This, this holiday season, Or, or, and then the other thing I was thinking about is like, okay, so could you use this for, would this be something you could use for, um, Bitcoin? I mean, I'm thinking, okay, so yeah, I, if I'm a Bitcoin guy, I am gonna look at this and say, is this, there's something here that I can, you know, that's cheap, um, that I could stack and rack and, and create whatever I'm gonna do for Bitcoin and, and, and speed that up.

You know, there's all kinds of possibilities when you think about that power at the edge. Um, one of the companies we did a briefing with this week was Quantum, um, not as in quantum computers, but Quantum as the guys that have got, they're the data side of the house. So they well known for their tape, long-term tape and data protection, but also for StoreNext and, um, now a product called Myriad, which is a file system.

Very, very much so. They've very much so been focused on the media entertainment. What they announced was, um, something called client GPU Direct with Myriad, right?

So think about it. If you're doing, you know, what I was trying to understand about how that would work is if you're looking at, you know, you've got client systems, et cetera, that have GPUs in there, um, can you speed that up of whichever you're doing? So one of the, you know, since they're with, very much so with the Yemeni industry, um, which you're looking at is how can I do, do and improve, um, the work that the m and e industry, the media and entertainment industry is doing in terms of their rendering capabilities?

So that's what they're really focused on. You know, they've really turned their sites on to saying, often what they would have is store next would be their beside somebody else's file system, like an Isilon or Adele's Power Scale, or a NetApp on tap box. And what they're trying to do is move into that market and take that market over, um, for the prime, for the really high speed environments that they have to have.

So, kind of an interesting move for them. Yeah. And I remember talking to a company that did kind of data or SQL acceleration.

They presented at a tech field day, uh, storage Field day event, this is a couple of years ago. But the idea was to take GPU accelerators and put 'em in line with data storage devices like, uh, a Dell, uh, storage array or any one of the storage array vendors, and accelerate the processing of data for, and I think this is a little bit before the AI craze, it was for data analytics, but the math remains the same. If you can accelerate the injection of data and ve and creating vector databases and all that good stuff, if you can accelerate that, that better, uh, that better aligns to your AI objectives and performance needs.

So I can absolutely see this jets and being used for stuff other than raw ai, FRA at the Edge. Nvidia talked about plenty of services. They really want to push their software services as why as A SDK, they want to push nims.

But I think when you take a GPU shrink it down to the size and power envelope that these Jetsons are, it makes for some really interesting, uh, accelerators that we haven't really explored these off, you know, these off server accelerators that can add value in the data pipeline. The other thing that I think is really smart about this is in the pricing piece of it. Um, so what we've seen historically is technology companies will seed the market of students to learn their technology, and in hopes that when they move into working for full-time and primary companies, they're gonna say, we want this, we want this.

So if I'm seeding the comp, and one of the strengths that NVIDIA has is all their software. So if I have my guys, the, the guys coming outta college or the tech, you know, the tech schools, whatever, are trained on the tools that DIA has, um, and that runs on this thing, I'm gonna say, Hey, did this, I'm gonna move more towards that space and use that more. So, um, very, very smart on his part.

I could see, um, some requirements for colleges or the tech schools to, to acquire some of these for the classes. Um, or just be in part of your computer that you have, if, especially if you're, you know, a compsci compsci major in AI or a data AI major, something along those lines. Yeah, my youngest son just finished a program, uh, maybe about five years ago.

He's five years into his career. So this would be the Raspberry Pi requirement of, uh, computer side, uh, uh, students. The, I think it's important to highlight that this isn't a new idea.

This isn't, uh, something that Nvidia just a, a space that NVIDIA has developed. There are all alternatives. Intel a MD.

Other companies have talked about putting this much compute or similar compute at the edge, you know, may be different form factors. Uh, the A IPC is kind of an attempt at some of this capability. It's, I think Daniel, uh, uh, looked at a post from Daniel Newman, our CEO, uh, couple of days ago, and he, I think he called the A IPC supercycle a flop.

So it did not, that was kind of provocative. I'm pretty, pretty surprised that was not picked up by some of the financial media. But yes, this is not a new thing.

So whether or not it's successful or not is an interesting question. I talked to some of our peers in the industry as part of the Great Beers on Storage podcast yesterday. We're talking about the opportunity for referencing in the enterprise.

And I don't think, I personally, we haven't done enough research search on this to, to validate, but I don't think we're going to see a huge bump in acquisition of hardware for infra in the enterprise. I think most enterprises are going to use their existing VMware BPI clusters, their Intel and a MD, uh, mm-hmm c fourth generation CPUs. In the case of Intel, epic CPUs and the built in AI acceleration, uh, our former peer would tell you that IBM's mainframes have really great built in acceleration, that you'll we'll see a lot of AI on PU and smaller accelerators, and we won't see a, a, a hardware bump.

So it's going to be an interesting impact on the market. May not be a bubble in the sense that AI is disappointing, but we won't see the demand that we're seeing on the training side. com, have talked about is the coming refresh cycle for desktop PCs.

Right? Um, you're, those, those areas. So they're, they're, they have talked about, and I don't know that market, so I'm, I'm way out on my ski Here.

No, we need to have Olivier on the, on the line. And Olivia had talked about this in one of the meetings when I was sitting with him saying, really smart guy, um, and that there's, they, they're, they've, they're aging. Um, there will be a cycle of refresh, um, not driven by Windows 11.

Thank God they asked me again this morning. Do you wanna operate? No, No, I'm fine.

Thank you. I, I'm okay. My, I, I finally have my system exactly like I wanted.

I don't muck with it anyways, but talking about, you know, you have PCs that are starting age. 'cause there was a big cycle of PCs, I guess 20 20, 20, 20 21. So we are three years in, you know, into that cycle.

And so they're expecting this to cycle again this next year. Um, at least that's where some of the valuations were put or have been discussed in terms of where they're going, especially within Intel and those other companies are talking about. So maybe with those, you know, when you upgrade, you automatically upgrade to an A IPC because it, you know, why not?

Because you need it for, I don't know, copilot or something along those lines. So we'll see. Maybe we'll get a living on here to talk about that with everybody.

So I it's Not infrastructure. Yeah, it's, it's not infrastructure, it's infrastructure A adjacent. But, uh, pulling the conversation back to infrastructure and kind of a tease of our predictions, you know, we all always have to do a predictions podcast for the new year.

We're coming up on the end of a cycle for data center, though. So the, after the huge shift in work from anywhere, we completely rejiggered our data center designs to accommodate work from home. We're coming on a mass refresh of that whole investment period.

Any off the cuff insights that you think, you know, will, any markets, any vendors you think we should be watching, watching in 2025 as part of, I think this, what I'm predicting is a hyper refresh cycle for the data center. You mean for people coming back into the data center? Or it Could be for people coming back.

There's a bunch of, there's a bunch of different drivers at play. We're coming out of kind of the rush to buy all the hardware to support the pandemic to support work from home. Now we're challenged will with both, uh, the return to office and the return to data center.

I think I, I, I think I just coined a new acronym, RTDC, the return to data center, uh, RTDC, uh, yeah, from cloud providers. And there's, you know, this whole now again, rethinking of infrastructure to support these two movements of return to our own four walls. I suspect we're gonna, if that is true, and that that's curious, um, and I know the reevaluation is going on in terms of where do I place an application, um, do I leave it in the cloud, et cetera.

Um, if that is true, then I would expect an uptick with klos. Um, I would, Equinix, um, should be, uh, a plague because I may not wanna build a data center or have that kind of environment, but I may be okay with cages that would be one, one or one kind of grouping that would say I would see that would benefit from it. Um, I know that Citrix is doing extremely well, but I don't know if Citrix is doing extremely well because of the horizon un uncertainty, or if it's just because, you know, that's the vdi, those are the VDI guys, but those are still operating well.

But you, the other piece that you have to overlay with that, you know, hides that kind of, that issue is what's going on with like, what we just experienced here. You know, the, the p we power and cooling is not keeping up because we're bringing more highly powered, highly dense environments into these environments, and they're blowing our data centers up. Um, you know, and you, you have to move.

Uh, and that more than anything else is probably what's changing the data center. Um, less so moving back, um, with those applications is what I would think. Well, this, this Is, I'm guessing, you know, I am, I am talking in a very uninformed way.

So in Terms, yeah, I, I'm, I'm not, I don't think I'm, uh, I don't think I have my arms around this yet as well, but there's, I think this is a good teaser for we're going to do, uh, in rap and a predictions PO podcast, and I think that's enough to wet the audience's whistle on, uh, on what's happening. So Kaly, we can't end the podcast without asking, what's your, you know, big tech takeaway from, and, and I can't say for the year because the year's been too big. What's your le takeaway for the, the past quarter of like, I think this past quarter has equaled a year.

What's, what's been your big takeaway? I'd say the last six months, or less than six months, is the learning that we're having on what it's gonna take to deploy generative ai. And it is this super successful, oh, we, God, we failed, super successful.

Oh, we, God, we failed. And, and so, you know, the numbers I've seen is 20 or 30% of the projects that people have initiated have gone out. And those reasons are to do with everything from, of course, data classification of three NetApp processes, not maybe, you know, there and also the security issues that are there.

So that's the big issue. And you know, that classic, you know what, the organizations are learning massively how to do this differently. They're not gonna stop, they're just gonna take before we go release because we don't wanna screw up the brand.

But that to me is, that's the big lesson right now with that, um, that I, you know, I've seen. And the other thing is that what I, we know now about AI is going to change next quarter, and it's gonna change the quarter after that. It's moving so fast.

It is moving incredibly fast. What makes it, uh, exceptionally difficult to plan for. If you're sitting on the ITDM side and it, the, and I've talked to software and hardware vendors, they're in the same space.

It's moving faster than what their teams can keep up with. I just as super compute 24, they were talking about one megawatt rack. So, uh, get ready for that, that liquid cooling coming into Boulder.

This has been, I think, a really great snapshot of what's happening in the industry over the past week. It has been a amazing 2024. We're going to have Dion, and maybe we'll find one more person to join us to, to have an end of the year wrap and predictions podcast.

If you, if you smoke 'em, have a great holiday. If you don't, you know, enjoy the, the downtime that the next week or so will bring for, uh, me. I'm your host Keith Townsend and my co-host Kaly Bates.

Have a wonderful season, Have a merry, merry Christmas, happy holidays, happy Hanukkah, happy New Year, whatever, whatever you sell may Joy as my, as my friends at Blues Clues would say, I mean, not blues clues that my, as my friends at, uh, on Disney would say Happy everything. Uh, a recent report published by, uh, DevOps, uh, the, the Google's state of DevOps report for this year, uh, so they found, I think they found that, uh, there's a 50% ROI increase for the organizations, uh, who are following mature DevOp practices and followed by some very good eye-opening numbers. Uh, these macho organizations or their light performance, uh, they are having around 1 27 next faster, uh, lead time to deployments.

And they are doing like, uh, massive amount of fast, uh, deployments during a period of time. So what this all sources is speed is everything the business need is speed. And then technology has to provide or enable this speed.

And in order to do that, you have to align with some of these Dora best practices that can have a direct correlator to your revenue. So with that, uh, welcome to observability drives, modern best. There was practices skill update.

My name is Indica vi and I'm going to walk you through about how we can accelerate DevOps maturity by leveraging observability. So I will discuss about typical observability based practices and how we can leverage those best s to uplift or amplify DORA maturity. 0, which is a new concept and how that we can leverage as well.

So in high level, the agenda. So I'll go through the DORA metrics, why it's matter. I think I already shown couple of, uh, figures, which are, uh, generally eye openers, which says lot.

And why do the DORA metrics or the Dora mat is very important. And we'll go little deep into Dora metrics, and then we'll discuss about Dora and the relationship with observability. 0, which is a new trend emerging, which is about shifting left because observability people associated more with production and what is happening, uh, at the deep end or age location.

But we also want to know, can we give this benefits to the developers who actually develop code so that they are better prepared in doing their work? And we'll also follow up looking at how observable generally improved ORA metrics. We'll look at, uh, implementation roadmap, and we'll finish up with some of the best practices, what I have learned from my experience.

So moving on a little bit of myself, I'm based out of Columbus, Sri Lanka. I'm a solution architect, specialized in site reliability, engineering, DevOps, observability, AIOps, and I do some little work on generative AI as well. So I'm employed at virtu a I am kind of like involved in technical delivery and capability development at my organization.

I'm also very passionate technical trainer. I do a lot of trainings, uh, related to SRE, DevOps, observability, AIOps, and I continuously been publishing, uh, the, the technical blog post I'm writing. I can, you can find me at Dodo two.

And I'm a very proud ambassador at DevOps Institute and also AWS community builder. So with that, uh, let's dive into our topic today. So why Dora Metrics?

So why it's important and why it has been for decade. Uh, people are still getting into the these metrics. So why we track Dora metrics, because if you can see it provide a lot of, uh, data driven insight.

It provide organization with data. So we are, the organizations are making decisions based on data, uh, and it's also allowing us to benchmark ourself and set some goals like based on the, uh, the tech stack I'm coming in or the business domain, we'll be able to benchmark ourself and improve on that. And it's obviously the continuous improvement.

When we have metrics, it's easy. And then we can start coming, coming up with the short term, medium term and long term objectives and drive. And it's about prioritizing where the investments and other things has to go in.

And we can look at the business ROI and it's pretty much, uh, need and year on year. The organizations are adapting, uh, the DORA maturity and trying to get some of these benefits I have listed down in right hand side, it's about return on investment, like fast time to market. So this day and age, every business, when they have a concept, they want to get that concept out to the market as quickly as possible.

That's very important. And it's about enhancing customer experience. It's about providing quicker deliveries and keep on innovating.

Innovation is key. Innovation is now part of the blood of any organization is DNA. And this require us to do continuous, uh, changes into our environments, production environments.

And it's about increasing efficiencies in improving organizational performance. And also it's about reducing risk, right? So this day and age, a lot of organization takes some level of risk, embracing risk, but we also want to ensure the risk we are taking is controlled as well.

And this all justify why Dora metrics are still super important and what sort of ROI it's generating. And if I go a little deep, I'm pretty sure everyone is aware just to set the context clear. So when it comes to Dora, we have four metrics.

One is about, uh, chain lead time. It's about when you get a requirement, how quickly you can code, and how quickly you can test, how you can package it, and how quickly you can deploy it into production environment. It's about how frequently do deployments.

So do you have that trigger, uh, to do, uh, frequent deployments? And when you are increasing the deployment frequency, can you do that in right? First it's about, uh, reducing or minimizing change failure rate, and it's about the recovery in case there's a failed deployment, how quickly I can, uh, fix it and roll it forward, or how quickly I can roll it back so that I minimize the impact and once deployed, if there are issues in the production environment due to that, how quickly I can identify how quickly I can fix it.

So we sometimes call it meantime to resolve as well, but overall it's about how quickly we can get this business concept into the, uh, production and how quickly, how, how fast we can run successfully. And in case of any failure, how quickly we can restore things. So Dora maturity and observability go hand in hand.

So observability, the definition is, can you understand internal system state by looking at the telemetry data, which is your logs, metrics and traces With Dora in mind, it's about can we leverage observability to enable teams to optimize performance and achieve high maturity of Dora metrics? So how can we achieve, uh, the ability to, uh, do a, uh, minimize the lead time and ability to faster deployments, ability to manage or reduce deployment failures and ability to understand issues quickly, detect quickly and resolve things much faster. So those are some of the, uh, value added observability bringing in.

So nowadays, uh, every development team, if you are a developer, you need to have visibility. So you want to understand what's happening and you especially, you want to understand what's happening in your system, and you have to know what are the bottlenecks, what are the issues in my pipeline, and what can I do to understand, oh, identify some of these failures much faster. So those are kind of like a very important requirements for any development team.

And when we build this observability into our systems that pretty much facilitate these requirements, uh, uh, of our development teams. So now setting the context clear observability is key part of, uh, enabling DORA maturity. You had to observe what's happening and that will actually lead you to better outcomes and talking about it.

So when we used to say observability, uh, we had to also understand how exactly it's going to impact some of these, uh, DORA metrics as well. So if you go through the metrics, if you look at deployment frequency, so observability enable us to understand some of the bottlenecks in the deployment pipelines, understand why sometimes deployment pipelines are going in a little sluggish performance, optimize them, optimize workloads and ensure that we have better capturing management of these pipelines. So that can help us to improve some of this deployment frequency and then lead time for change.

Uh, the observability can provide more visualization into our development cycles, understand the, develop more the process of deployments and provide a lot of data which can allow us to understand, uh, how it is going. It can be about the area of the chain, it can be about area of testing we have to do. It can be about the, uh, what is required, uh, for particular code and how it is behaving.

So if you have more data, understand what's happening when you deploy that code in your development environment, you are better prepared to un and fast track that deployment to, uh, production. And also, uh, observable, help us to understand the failures, uh, especially being proactive when it comes to failures. And then, uh, super accelerate some of these fixes, which is about time to restore.

So we are able to use observability logs, metrics, and traces to develop, uh, especially metric, which is using metric. We can develop data alerts using, uh, logs. We can look at it's kind of audit frame.

We can exactly go through and understand what's happening in the system. And using traces you can find out what's your code doing, right? So with three of these, your logs, metric traces, you have the power of identifying things much faster and fixing things much faster because you can getting into that root cost much quicker.

So overall, it's a combination of everything at observability providing which help you to improve your adora metrics such as deployment frequency, lead time for change, change, failure rate, and time to restock. And we'll also look at, uh, one of the newer concept or trend happening. So when we see observability, we think observability is more of like, uh, uh, at the production then.

So it's about can I make my systems more observability observable, which is fun, end user facing. And we think observability is only related to the production systems. And because of various reason, like, because when it comes to login, we kind of like, uh, sometimes login is expensive, so we just enable a certain level of logging and production.

Sometimes, uh, the sam, the tracers for sampling, we use sampling to control, uh, in production because observability is also costly as well. The metrics, we try to kind of like look at things in a more simple manner. So because of that, because observability is also literally a cost we have to incur, we generally look at enabling observability in production systems only.

And that is a classic, uh, mistake because observability is required in our development environments as well. 0 is more of a shift left, uh, philosophy where we want to enable observability at our environments where our developers are working so that developers have more observability or able to observe what's happening, the things which are developing. So it's about acknowledging that observability is evolving and that all has to be reflected that, uh, uh, some of these, uh, low environments where the areas where developers working so that they can troubleshoot.

So recently I spoke to Thomas Johnson, who is the CTO and co-founder of multiplayer. 0 driven products to provide better capabilities to developers so that developers can also leverage these observability advantages bringing into the table. So generally what's happening is observability is more for site reliability engineers or DevOps engineers, but we also want to get this, uh, those advantages, those capabilities to our developer friends as well.

So some of the key benefits. 0 is able to provide real time context, risk rich insights, and let developers understand this unknown nodes in, uh, the codes they are reading, and especially when this code is running not only in production environment or in their test environments as well. And we want to enable fast debugging.

So as I said, the logs, metrics and traces are very important and it help us to understand and understand root causes. And sometimes even though you may go for more of, uh, flexible the log level or deep locks, uh, level, uh, but you might still not go with traces because sometimes traces can be a little expensive if you go with the observability tool or, but if you can open source it, it's much better, but it's still, there can be some cost. But if you can work around that and then enable full the capabilities of tracers that can help developers do faster debugging because they now understand how they are code runs, like it's all about the code, how fast the code running, what are the bottlenecks, and then, uh, it's about understanding, uh, your systems better.

So it's not only been about the wisdom of production, but it's about wisdom of your, uh, the test environments or developer environments as well. How clear how clarity you have related to these environments. So this is about observable.

0 is all about enhancing developer experience and naturally taking this data and insights and ability to troubleshoot. So this will all align with Dora maturity. 0 or uh, uh, enhanced developer experience, that will naturally result in high DORA maturity as well.

So with that, uh, I used to think when you are looking at observability, it's always better to have a plan with the plan. You know, where are you and you know the destinations you want to go. So observability, uh, it's around maybe around file level.

It's about keeping the lights on. It's about observing things. It's about enabling your, the tracers or real user monitoring service maps and enabling DOA metrics.

It's about measuring a customer experience, defining service level objectives and those things. And then after that, you go into more of a correlated level. You look at things in a holistic fashion, you look at metric anomalies, log anomalies, it's about using AI to understand, uh, reduce noise.

It's about, uh, correlating things much better. It's about, uh, rule-based issue resolution. It's about baseline, uh, the and correlating things so that, uh, you add a little high-end at what you do.

And when you go to predictive, it's about using AI to do self-diagnostic and then self feeling and powered by gene AI as well. So in this maturity model, uh, generally what you do is you improve your observability maturity from left to right. And there are areas where you will focus.

So we have to look at CI/CD observability and we had look at can I improve run type called performance monitoring and enabling DORA matrix plugging to your observability set up as well. So all of these things are very important. Once you do not only you get match at observability naturally, one of the ripple effect is your DORA metrics get to high end.

So you naturally becomes a a, a light DORA practitioner, right? So if you see the DORA maturity get increase while you go through this journey, so I'm requesting all of you, when you are starting your roadmap as observability roadmap, build something like this, try to understand where you are and based on your business requirement, based on your technical requirements and what is possible, develop a plan like this and, and also try to focus some of this DORA specific things as well. Naturally, re regardless you focus or not observability will provide you that benefit.

But if you can also bringing this concept like CI/CD observability, the core performance run, tide core performance, and especially specifically enable this DORA metrics and sometimes some organization have this challenge, it's good, but how can I enable it, right? How can I get IT systems to reflect it? Because you don't need manually calculating this metrics and do that manual work because it's a overhead, it'll not reflect the true picture and it's not automated and it has a lot of drawbacks.

What you want is you want Dora XB visualize and plugin in and you get this data available quickly. So if you look at it, this is the sample dashboard of Dora visualization provided by Datadog. So you can connect your, uh, Datadog instance or observability tool with the deployment pipelines and other systems Git and your code reports and other system.

With that, you will see this data in your fingertips. So now with this, you can go and start, uh, looking at where the bottlenecks are happening. You can start drilling it down to different areas.

You can look, look and set some goals, and then you can go through your roadmap. So this is a nice way for you to, uh, visualize things and always visualization is better. And this allows you, uh, to centralize this Dora capturing.

So this is one of the key thing, part of observability you can enable. And then the drill down is pretty straightforward. If you see you are struggling with your lead time or change lead time, you can drill it down, figure it out.

If you see meantime to restore struggling, you can go inside, understand the incidents, understand why it has taken time, understand how, uh, we could have done better, right? And probably it's about awareness. It's about getting everyone awareness into how to use observability, how to use this data to better, uh, provide, uh, a faster resolution or improve overall system performance.

So it's important that when you are building your observability framework, be mindful of this and also enable this kind of visualization as well. And finally, before I wrap it up, there are some best practices. You have to be mindful.

So it's about tracking the metrics. So you have to understand what are the metrics you need. It's, it's all about, uh, developing metrics which are correlated with your end user experience so that you understand what's happening, you have a clear ROI and you understand what is the benefit your end users are getting.

And then after that you have to start enabling some of these door metrics like deployment frequency, leave time for change, change failure rates and other things. So that way you can go deep into some of these metrics. And also when you are doing that, you have to understand sometimes we go with sampling or, uh, the, the ation.

So that will not give you some of these data points so you can miss out some of these things. So have a balance of, uh, limiting or sampling. So that's always important.

0, which is about improving developer experience, right? So shift left the observability so that these good things not only happen at your production, but this is happening in entire your, uh, ecosystem from your, uh, local develop environments to test environment, to E two environment, to use accept environments to, uh, staging or, uh, pre-live environments to production. So that way that you have the full stack or the full environment visibility and developers has these capabilities to enhance and improve themselves.

And while you do this, you have to look at what automated alerts you can bring in. So it's all about metrics. And once you have metrics on top of that, you can build a comprehensive, uh, alert mechanism.

So you have to get that advantage. 0 enabling this, uh, developer experience, shifting it left into invisibility so that a developer knows when he, the time they start coding. And until that code runs in production, what's happening.

So that is, uh, very rich, uh, experience and very rich data as someone can see. So those data, the visibility we had to provide end to end to everyone in our project teams organization should have that view so that we are all in one place and we are all able to plan and improve together. It's about bringing Observ built into your CI/CD pipelines.

So your pipelines are more transparent. You see the speed of deployments and you see the bottlenecks and you see the failures and you can improve as well. Sometimes those are silent killers.

There are a lot of areas where you can improve and that can, uh, provide better ROI. And it's about data. So observability is all about bringing in rich data.

Either it's metrics, logs, traces, so that data is very important that will provide you a lot of insights. Why you, why you are struggling with lead time for change, why you are developers taking time? Is it about the coming up with design and coding or it's about troubleshooting issues or it's about trying to figure out the, how that code is plugging in or it's about that one issue took entire your time that last week, right?

So it's all about looking at data and how this data can help you to do better coding, understand the issues, coming up with fixes and speeding up deployments and fast track things into production. And once you have that data, the good thing is on top of that, you can build and bringing in ai. So you can use ai, uh, ops or, uh, artificial intelligence for IT operations.

It's about running AI models on top of this data so you can plan it. So you, you can use anomaly detection or you can use forecasting or you, you can use correlation or noise reduction. You can get, I mean, when it comes to ai, almost all the benefits of AI running at other end or the production is well known, but you can use it in some of your DORA metrics as well.

You can use some things like forecast into critic, like based on the, uh, the scope of the coding you do, and uh, what do you think, like what is the realistic forecast, how quickly you can go and what are the bottlenecks and what are the anomalies, right? And anomalies of forecasting apply into door metrics, lead time for change or the deployment frequencies and other metrics that will give you, uh, a sense of being on top of this game, right? And you can definitely use AI and especially Gene as well to improve some of these areas.

And it's about genius. It's all about creating new content. You can use this and leverage to do powerful things.

And then finally, not to forget about autonomous operations. Use self-healing when you have issues. Let, let's get, uh, uh, systems to heal itself.

And that can help you in long run maturing your system, especially if you remember, or if you can go back to the time where, how much time you spend troubleshooting some of these regression issues and how much you, uh, time, uh, to spend to bring up the system because system just crash once you de had that last deployment. So if you, you want to build this self feeling and the remediation capabilities not only in production but in your low environment as well, all together will give you more benefits and achieving and going your door, uh, journey. 0, you enhance the developer experience.

This will provide you, uh, ability to do much faster, uh, requirements from your end users, develop, test and package and deploy faster into production. Ride first, reduce failures and in case if there's any failures, of course there can be you detect quickly, fix it quickly so we can run it much faster. So thank you very much for spending this time.

0, get observability in front of your developers so that you can be a, a Dora Mature organization. Thank you for taking this time to listen. If you have any question, you can put it to the chat.

Probably I'll try my best to get back to you. If there's anything, you can probably try to reach out me as well. So there are a lot of, uh, good presenters presenting various topic related to this area.

So take time to listen and it was my pleasure presenting to you. Thank you.