Jonathan LaCour on AI Adoption Following in Cloud Computing’s Footsteps

This is Textron tv. Hey guys, thanks for the throw. We're here with Jonathan Leor, who's the CTO for Mission Cloud, and we're talking about the rise of AI workloads in the hybrid cloud era.

And a lot of it's starting to feel like maybe a case of deja vu, but we'll jump into that in a minute. Jonathan, welcome to the show. Thank you so much.

I appreciate you having me. Michael, looking forward to the conversation. Well, they say history repeats itself, and we seem to be having this conversation about where to run AI workloads after somebody actually builds the model and the inference engine, what to do with that inference engine.

And there's a bait about does it go on premise in the edge in the cloud? Where, and what's your sense of what's going on here? Yeah, I think it's a, it's a really interesting time.

Uh, as you mentioned, you know, the genesis of AI reminds me quite a bit of the early days of public cloud in general. Um, I think just on a very compressed timeline. And as a result, there's just been a massive amount of activity over the last kind of 18 months.

Um, a lot of POCs from a lot of different businesses, um, but also some, you know, real production workloads as well. And that means patterns are starting to emerge, best practices, um, and people are really starting to figure it out. So they are at that conversation of like, okay, we're going to production.

Where do we put this workload? Who's in charge of deploying the AI workloads these days? And I asked the question because it used to be a data science team kind of built this thing in the corner, and then they got a small team together and deployed it.

But I feel like lately the inference engine itself is now being added to the IT operations team. What's going on there? Yeah, I think you're, you're spot on, right?

It's pulling a entire set of workloads, basically into the purview of an IT organization who manages, you know, public cloud estates. They manage the on-prem usage, you know, whatever it is that they, they have from an infrastructure perspective. And the nitty gritty details that typically a data science team would handle, um, are somewhat abstracted in many cases behind, you know, managed services like Amazon Bedrock, right?

Which gives a context for which to run, um, AI workloads. And, you know, in the cloud you have access to very expensive GPUs and, and infrastructure that you may not want to purchase on your own, uh, or maybe can't, right? Because frequently these sort sorts of situations arise where, you know, the hyperscalers are buying up all the capacity and there's nothing you can do about it, right?

So, um, yeah, I think that's sort of my perspective there. And the conversation's a little bit different in the sense that most of the data science teams that I talk to were like, wow, we got a GPU awesome. And the operations people are like, well, how much does it cost?

Where is it? How? Right.

There's a lot more questions that come from that side. I think so. And it, it's interesting because it's also moving very quickly.

As I said over the last 18 months, the sort of cost per token usage, uh, you know, come from a consumption pricing perspective has just absolutely plummeted, right? You've got all of these different vendors sort of, uh, riding the hype train and trying to be, establish themselves as the place to go, right? For a model or whatever.

It happens to be a service. Um, and as a result that, you know, price compression has really driven down the cost to a level where if you ask me 18 months ago, is there going to be a cost issue around Gen AI for, um, you know, production workloads, I would've said, yes, it's gonna be pretty significant. But, uh, that competition has really done its job.

And, uh, now you can run very sophisticated workflow workloads that are powered by gen AI and not have to break the bank at all. I think that is a particularly interesting, uh, kind of evolution of, of the market over the last 18 months. Are you worried that some of these deals might be too good to be true and people are trying to buy market share and they might not be around for the long haul?

Uh, I think it is entirely fair that, uh, when you have a circumstance like the one we're in now, you have this sort of transformative tech that arrives on the scene. It is early days, people are trying to figure it out. You're gonna have a lot of activity, a lot of funding flying around, a lot of venture capital, uh, out there saying, Hey, I wanna, you know, back a hundred companies and hope that one of them makes it as a unicorn.

Um, and so I am a hundred percent think that there's gonna be, uh, some failures in the market. People are, you know, uh, attempting to make a business work, you know, uh, speculatively around this transformative tech, and some of 'em will get gobbled up by hyperscalers, right? Uh, one of the big jokes I always have at reinvent the AWS reinvent conference is you're out on the show floor, you're looking around on all the exhibitors and saying, which of these is about to be disrupted by a change at AWS?

Right? Maybe, you know, they add something to the AWS console or they launch a service. It is directly competitive and it's just, you know, even now that happens in cloud.

Uh, and that's gonna happen quite a bit in AI kind of upfront, right? Um, I'll give you an example. Uh, I am, uh, a programmer, um, since I was very young, I still write code probably more than I should.

Um, not for production workloads, don't worry folks. Um, and, uh, I have actually been building something that, you know, basically gives me contextual AI assistance for my project from the command line. 'cause that is where I develop personally.

Um, and there weren't really any good products out there that kind of, uh, tickled by funny bone, you know, really were the right thing. Um, and Claude just announced philanthropic. Claude just announced, uh, Claude for the CLI effectively, or Claude code.

And boom, you know, there's this brand new thing. Uh, and that's just an example for me. Uh, there are going to be dozens, uh, of, of these things that happen, uh, disruptions that, you know, uh, make it such that a particular market maybe closes or the compile competition gets very, uh, crowded.

What makes it different about inference engines and AI models within that DevOps workflow that so many organizations use to build software today? Is that AI model essentially just another type of artifact? What, or, or does it behave differently?

How should I think about this thing as I go to operationalize AI within the application stack that I'm already deploying? Right. So I, I think, uh, the interesting thing about AI is that it really emerged from academia, right?

And if you actually look at a lot of the classic ml, uh, predictive models and things like that, and the tools they were being developed by, you know, I would call 'em data scientists, not, uh, cloud engineers. And what is going on now is these things are becoming operationalized, and now they're hiding behind APIs. Um, like open AI will have some, and, uh, you'll, you'll also find some from philanthropic.

And, uh, AWS is providing bedrock where, um, it's effectively like a managed service where you can drop one of these artifacts and have an execution environment for it, and you have a consistent API and all that good stuff. So, um, I feel like AWS is approach, there is much more like a typical a, uh, cloud workload, right? It, it can be controlled with infrastructure's code, uh, it is run as a managed service on your behalf, uh, integrates nicely with all the other AWS services for sort of that seamless experience versus something like an open AI or an anthropic where it is an API endpoint that you call, and then you have to architect around that.

And so, to me, it sort of depends a little bit on the approach. If you're taking that bedrock approach, I think the DevOps kind of cadences and the infrastructures code and all this, it actually a totally reasonable way to approach, you know, kind of operations. Um, if you're using third party API, it really falls more onto the engineers, right?

The developers who are consuming that API to make things happen with, uh, with, uh, you know, a NIO model somewhere. Um, and so you have that kind of spectrum that exists. I expect things to settle, um, you know, into a little bit more of a consistent kind of split between those two things.

Um, especially as, uh, the market matures and adoption kind of, uh, goes more from a solution, uh, engineering perspective than a, what's our AI strategy, uh, you know, kind of perspective, which a lot of companies are getting asked, What do you think the impact on those DevOps workflow is gonna be? 'cause I talked to some folks and they'll, you know, privately admit that they're fairly brittle as they are, and now they're stretching their heads going, well, how are we gonna handle all this code that's coming down the pike from these AI coding tools that are being added? Yeah, I do think, uh, you know, it, it is going to challenge a lot of people's, you know, roles in terms of how they they operate, but also, you know, the notion that some of this can be, uh, used to automate away certain aspects of a role.

Um, and this is something I'm actually quite passionate about, uh, kind of looking to the future here. It, you know, the reason, one of the reasons I I compare it to cloud is I disrupted a set of jobs that were either eliminated or enhanced or changed, um, just by the nature of the technology, right? And so if you were a, uh, a network engineer who was managing on-prem Cisco, uh, networking gear, you had your CCIE certification, and then your company says, Hey, we're moving to the cloud.

Well, what, what's your job now? What do you do? And, uh, you know, I saw a lot of those folks embrace, uh, public cloud and found themselves becoming cloud network engineers, cloud architects, and frankly, being quite a lot more valuable to the business.

The brittleness, as you you said around sort of current gen DevOps approaches. I actually really like the fact that this is gonna sort of disrupt there and make, uh, that category of, of sort of job rethink how they do their work, um, in the context of a new transformative technology. And you either embrace it and, and, you know, some of those folks will become, you know, uh, a highly valuable AI friendly, you know, um, resources for their business and others will become entrenched and decide, no, I don't believe in this, and I don't, you know, I'm not, uh, gonna change what I do.

And I think that it would be very wise for people to be in the former camp than the latter. What will be the relationship between managed services and what those folks are managing these days, because, um, they're everywhere, rather, AWS has them, third parties have them, A lot of them are exposed via GUIs. It's not clear how many of them are invoked via CLIs that you can do within our existing workflow.

But how's that whole relate dynamic changing between the internal IT teams, the manage service providers and the cloud service providers? Good question. So, uh, I would go back to looking at, you know, uh, AWS, um, in terms of what they offer, right?

So, uh, many of their services are effectively managed versions of open source software that they operate as a service with some secret sauce and give you APIs that you can then control and use to build out your infrastructure. And, you know, I think, uh, if you look at, and I kind of got to this earlier. If you look at an open AI or an anthropic, right?

Two of the big, um, you know, vendors in the space, their approach to this is, here's our API, right? Here's our managed service for running, you know, an execution, uh, against our set of products and services. Um, whereas AWS is looking at it more as, hey, here's another thing, right?

That has an artifact, that has some piece of, you know, open source software that can be operated as a managed service. And as these new tools kind of evolve, uh, things like a lang chain, for example, um, and, you know, plenty of other sort of, uh, best practice open source platforms that are used, um, in this kind of workload, I've very much anticipate that AWS will launch managed services around those things. Um, and Bedrock is an example of, uh, their perspective on the challenge and then of the market, Hey, we give you this, you know, common environment for execution.

It's a fully managed environment. You pick a model, you bring your own model, you get one from us. Um, and so I do think that, you know, that complicates the whole situation, uh, as you have an entire spectrum of how people are going to consume either directly or through managed services or, you know, through both through on-prem, through cloud.

Like, it's, it's gonna get really, uh, hairy really fast, but that's good for, uh, you know, consulting and managed services providers like me. So, um, uh, that complexity is power and helping customers harness that power is, is, you know, very important for us. So among the organizations that see doing well, what are they doing differently than others who are kind of stumbling through this?

There is a phenomenon in most tech companies where there is a segment of the business that is, um, several segments I should say, that are really focused on risk management and risk mitigation. Um, and when a new technology comes along, it can be scary. It can be, Hey, is this thing gonna have access to my data?

You know, what's the privacy and security look like? Will it mess with my compliance? Um, and so there is a, a large number of hurdles to jump over from people who want to say no because you know, it's new.

Um, and I'm not saying they're wrong at all. Um, but I do believe that the organizations where it's working, they are, uh, fairly permissive, uh, and smart about, um, guardrails and restrictions that allow for, you know, their business to take full advantage of what's available to them without exposing them to big distractions or issues or whatever. Um, and so the companies that are not doing so well here are the ones that are effectively waiting and seeing, right?

Um, I understand the desire to wait and see, but I also think that there is enough value out there in the market today. We are past wait and see in my opinion. Um, and, you know, individuals in their roles will get passed by if they take a wait and see for too long.

And so will businesses to me embrace, give everything a shot, understand deeply, right? Put in some guardrails that it help ensure that you're doing so in a safe way. Um, otherwise you're gonna end up with shadow it again, like happened with cloud, uh, where people will be signing up for Chacha PT on the side on their own and using it, even though you're telling them they can't.

Um, because at the end of the day, it's valuable to them. And, uh, it departments, you know, your customer is your team, your your employees, right? You wanna enable them with technology, not hold them back.

Um, and so I do think, you know, the sooner that a business can find a way to enable customer or their, their employees to take advantage of this, the better. Um, and I think we're actively seeing that right now. All right, folks, you heard, and here it's a quote, a president from long ago.

It looks like we have nothing to fear but fear itself. One more time. Hey, John, thanks for being on the show.

Thank you so much, Mike. It was a pleasure. All right, back to you guys in the studio.