Techstrong Gang – September 18, 2024

Hello everybody. We're gonna be talking today about hitless Cloud Services, and then we're going to go back in time and talk about the mainframe, which may be cool again in the age of ai. And then finally, we're gonna talk about, well, is finops gonna be embedded into software engineering practices?

I'm Mike Biard, and you're watching Tech. All right, folks, we're back and we got a full house. Today.

We're gonna start with Mitch Ashley, who joins us once again from his perch in Denver. Mitch, how you doing? Always good to join you in the gang here.

All right, then we have Paul nti, our resident expert on all things application development. Paul, how are you? Great.

Great to be here. And Paul joins us from the Futurum Group, along with Steven Dickens, who's also from the Futurum Group, where he's a chief technology advisor and an expert on all things cloud and mainframe. Steven, welcome Michelle.

Thank you. Great to be here as always. ai, and digital c xo, and of course, tech Strong, ITSM.

So, you know, she has a full plate every day, and so we appreciate her taking time out to be with us. Amanda, how you doing? Doing well, thank you.

All right, let's jump into this. There's a story over on Techron Do AI talking about how this ship to AI will further advance this notion of headless cloud services. The idea here is that, um, I'm just gonna invoke APIs to consume the resources I need, and I might not be as tied to various cloud service providers as I have been in the past.

Steven, this is not necessarily a new idea, but let's start with you a little bit on this. Is this feasible? Is this something you've seen before?

And will AI kind of drive it a little bit more aggressively in your mind? So is headless the new serverless, The light new lights out data center? Is, is that what we're calling it these days?

No, I mean, old joking aside, Mike, I, I think there's a reason why we have a podcast every week on the RUM group called Infrastructure Matters. Um, whenever I hear the phrase serverless, I ask what server it runs on just to tweak the people like Paul, who don't worry about infrastructure. I think headless for me just means that somebody in a development team doesn't wanna dirty their hands thinking about the infrastructure that it runs on.

Um, infrastructure does matter. Non-functional requirements like performance, availability, security, scalability, sovereignty, all matter. Um, and where that infrastructure resides is becoming increasingly crucial.

So whether it's headless, whether it's serverless, I think there's always gonna be a tension between the development team who genuinely don't care where this stuff runs, and the operations team that's gotta secure and make sure this stuff doesn't go bump in the middle of the night. So I think this is just for me, yet another sort of go round on that tension. I dunno, Paul, it sounded to me like Steven was giving you the back of his hand a little bit there, but, um, API it would Be the first time, would Paul?

Yeah, you Know, I mean, I'm, I I've kind of grown accustomed to doing that work and with Steven on that, but, um, but basically, uh, you know, I think the, uh, I think that the, the approach that I will kind of piggyback off what Steven's saying. I I shockingly agree with Steven this time on, on the fact that, um, that, you know, from a developer perspective and from a DevOps perspective, um, the, the business objectives have changed, okay? Over the years.

It used to be a heavy emphasis on making sure infrastructure's up, making sure SLAs are up and running, so you have, you know, five nines or, or greater. So you have planned downtime and such. But really what I'm finding from the, uh, from the business logic side, from the, uh, from the developer op and the DevOps and the, uh, side op and platform engineering set, it's really not about SLAs anymore.

It's about SLOs. It's about service level objectives. It's about making sure that your applications up and running, because you can have, uh, an infrastructure that's up and running and, and working.

Um, but if your applications are down, you're still down. So it's plain and simple, right? So it's, uh, it's, it's about the operational side of the business.

Now, I wanna come back, Mike, to your comment about, uh, about heli. Uh, you know, I was having some, uh, a little bit of deja vu. I worked in, uh, in the DXP space in the CMS space for, uh, for a, a while, uh, throughout my career.

And we moved to a headless, uh, DXPA digital experience platform approach, which is very similar to, uh, what we're talking about here, which is basically using APIs in order to connect data. I view this as, uh, from a developer perspective and from a, a DevOps perspective as a, um, bridging the gap between old and new. Uh, there, whenever we talk about application modernization, we talk about the fact that some applications just don't need to be refactored.

Some applications stay in their, in their heritage state because that's where they're, they're optimal. Um, and that might be running on a monolithic infrastructure or, or an environment or a cloud environment and such. Um, but new systems of engagement will use APIs in order to access those systems of record.

And those exec, those heritage applications become systems of record where modernized applications in the front end will use a kind of a quote heli approach to attach to that, those data sets to give a new, fresh experience to those applications in that dataset. We're seeing that a lot with LLMs and ai and ai. We're seeing that a lot with production applications.

Uh, but we're seeing that across the board and, and, and the SDLC overall. So, uh, that's where my head goes. Uh, I think that the one caution of would caution the audience here, it gets incredibly expensive when you start making all these API calls, but, well, that's a whole different conversation.

I'm gonna, I'm gonna, I'm gonna thread right down the middle of the, the two gentlemen here. So I do think it's all about APIs because it, it's about dynamic infrastructure, uh, which can be headless or not whatever you want it to be. You know, serverless is actually kind of its own, at least in terms of the land lambda kind of services.

But I think we're, we're moving past the, I have one tool to do this kind of automation one tool to do this kind of monitoring one to, to this tool to do this kind of provisioning. When I, when I wanna set things up from a platform engineering console, if you will, or portal, I want a provision in my VMware environment. I want it to set it up in my cloud environment.

I want it to load whatever appropriate images of operating system, Kubernetes containers, all this, all of the configurations that are part of what, what I need to use as a platform engineer, operations person or developer. So it, it's all about the pro programmability of the infrastructure and being able to automate it across domains as well as across tools. I, that to me is what we're driving towards.

Sometimes it doesn't always feel like, but I mean, we're getting closer and closer to that world. I, go ahead, I'm sorry. Well, I just think it takes a, a hybrid approach.

It, uh, and because the more APIs also you have to take into consideration keeping track of all the APIs and, and managing all those as well. So I, I think it takes a hybrid approach. I don't think it's gonna be easy, and I don't think it's gonna happen tomorrow, but I agree with Mitch, we are moving down the path towards, if I put APIs in front of infrastructure, and I'm programmable calling that, well, I got news for you, that makes the infrastructure behind it disposable.

And for that matter, the infrastructure that AWS already provides is kind of disposable. They don't tell you when they're upgrading their infrastructure underneath those VMs that they're providing. They just tell you that they got a, you know, a faster one that's available and it's the underneath the API you have.

So I think it's fungible Mike, not disposable matters. I can do what I want with those resources. I'm Gonna jump in and say I 100% disagree.

So Depending on the application and the non-functional requirements of that application, you do care about the underlying infrastructure. If my Netflix is down, I'll just go and watch HBO max. If my core banking is down and I can't get access to my cash, then there'll be a run on the bank.

A nationally defined critical piece of infrastructure will start to get attention, and the economy will start to unravel. If you can't use your visa card because the core infrastructure is down, then you're gonna have trouble. So there is a, there's a degree here of what infrastructure is and how fungibility is.

Some stuff has to run eight nines, nine nines, and it has to be basically always available other stuff if it's not there. Okay. That might, Yeah.

Steven, look, I, I agree with you. I think, uh, without being overly dramatic about the infrastructure, I know that those use cases certainly matter. I think that the, kind of the spirit of the comment that Mike was making is more about whatever, however you architect your underlying infrastructure would accommodate your application's needs.

You're, you're spot on. You have to have performance, uh, you know, connectivity, latency issues, all sorts of things that kind of make things work. But that is implied that it works.

And if it doesn't work on one, uh, architecture, it needs to be transparently moving over to another right. And that, that, to, to your point, the application needs to be, um, properly, uh, you know, have the, uh, the, the resource available to it. But I also think when we look at, you know, releasing and code, again, my, I'm, I'm, I'm a strong advocate of this because of the data that we have when organizations are pivoting to releasing code on an hourly basis.

Um, and, you know, the, the days of putting a help desk ticket in and waiting for things to change in the backend are long since gone, right? Uh, that has to be a self-service model for a a, a a, an architecture, an infrastructure that's, that's available to the, the business logic above. And it has to be dynamically adjustable.

So I don't think, it doesn't quote matter. I think it needs to be architected appropriately to fit the needs of the business. Mm-Hmm.

I'll disagree with my learned colleague, Steven. One more time. Um, servers crash all the time and nobody knows anything about it.

And Netflix has servers that burn out and nobody knows anything about it because the software calls are rerouted to other APIs. And the whole system is designed to accommodate that. And as that becomes more and more common, more enterprises are gonna have a similar mindset.

So, so Mike Azure Central had a 14 hour outage a few weeks back, Google had an outage. When the public cloud operates at three nines, SLA, if you have an outage in a quarter, typically your contractual re um, situation is they'll give you 10% of your bill back. That, do you want to know what three nines means from an availability point of view?

77 hours a year. 77 hours are, they just randomly happen. If that randomly happens on Black Friday and your Walmart or Best Buy, that might be a problem.

I would say that we're gonna reroute those calls over to Azure, which is not likely to be down the same time as Google. It's in the same, same data center, Mike. It doesn't have to be in the same data center.

My app can be redundant and structured to make API calls across different services. Why does the European Union put in place legislation called dora? Why did the UK this week declare this past week?

Declare data centers is national critical infrastructure stuff happens at the infrastructure level that creates outages that businesses can't survive from. Not everything needs to be that way, but in this nice fluffy world of the cloud where magic pixie dust just ensures servers are up, you've gotta be more pragmatic than that. And just saying, oh, it's in another availability zone isn't a good enough answer.

I think it's about also architecting it where you can. 'cause now some applications are not this way. It, it's, can you architect it so that you've got some workload portability, wherever that is.

Same data center, different data center from a, the same provider. To your point, Steven, you might be fixed, and this is where you have to run because of whatever regulation. And so you have, you're gonna architect based on that, right?

That's your constraints. Whereas another place, okay, I got, I've got three locations I can run in, in the us, east, west and Midwest, or whatever it might be. But can I make my workloads portable?

So if I do have an outage somewhere significant enough that I wanna re-spin up, uh, more resources in a, in another data center, that that's, I think what we're saying is it depends on the situation, right? You've got constraints that you're working with, and in some cases, this is my data center, this is all I'm working in, is that, that room over there, down the hall, and that's where all my stuff is, and that's what I've got to live with, versus I've gotta work in this data center in the cloud, or these providers in these locations. Yeah.

Mitch, to add, add data to what you just said, I agree with you. Our data sets from our research shows that 20% of respondents view application portability as critical to their environment. 67% say it's very important.

Every look, we're all in. I mean, I think violent agreement here, I think the thing that thing that's, uh, interesting is the biggest factor for HA and BC configurations for applications, whether it's portability or availability or across availability zones and such, it's not cost effective to do that for every application. So you can absolutely architect a completely redundant environment, but nobody has the budget to do that.

So, uh, that's where it runs into a well, okay, the critical applications may be, but I would say that the entire infrastructure is not completely redundant. Yeah. Business critical applications.

Sure. Or, but yeah, I Mean, I just, my sort of takeaway is not enough. People think about recovery point and recovery time objectives.

If those need to be architected as near to zero as possible, you're gonna have to think about different architectures. There's a scale and a continuum. There's lots of things that can survive with long recovery point and long recovery time.

And there's things that can can't survive with any of either. That's a long, long, long continuum, and it's very, very expensive at zero. And you can throw applications up on the public cloud really cheaply and not architects for availability.

You've just gotta start with what your recovery point and recovery time objective is, and then just plan it an architect according. So I'll agree with Steven on the issue as described for availability in monolithic applications is true. We're not there.

Um, I think as we look at more modern applications with microservices, we can architect some redundancy into the system and we can get to a headless model. The question I would ask Paul, and, um, you know, is this an argument for re-architecting monolithic applications and turning them more into microservices? Or is that a bridge too far?

All right, so it would be wrong if I didn't say it depends, because everything, it depends, right? Uh, you know, so I think that, um, you know, obviously we are refactoring, um, heritage architectures or heritage applications into mon uh, into microservices. And, and containerization is a desired state for many businesses for a number of reasons.

Um, you know, whether it's, uh, portability as we just talked about, or if it's, uh, the flexibility of cloud elasticity, of the burst capacity, uh, management of the application is a slew of things that you can kind of talk about. But the, one of the big factors that or organizations typically overlook is, um, sometimes refactoring doesn't give you any advantage except one with, with the asterisk. Sometimes refactoring doesn't give you any advantage.

So they can encapsulate that heritage application to a vm, make it cloud ready, stick it in the cloud, and it's available, right? The problem is, is a lot of times when those applications are not refactored, it does open up a security hole. Um, a lot of, a lot of those, uh, applications that are not refactored, even though you may get the same performance after refactoring.

Um, and so what's the point of doing it? The point is, is you've now have, um, you, uh, reduced your security vulnerabilities. And that's a challenge that a lot of heritage applications will have, as we all know.

Um, so, you know, I I think that if you look at it, if, if you're looking for availability, if you're looking for that portability, if you're looking for that architecture that allows for that flexibility refactoring into a cloud native state, so you can go from one of the hyperscalers to an on-prem cloud to a, to an edge location. Sure, that's fine. Um, but the dl the other factor comes into play is what do you gain, uh, if, if that's your number one thing, is the portability piece fine, but what do you gain on the other side?

Do you gain performance or do you lose performance by going to microservices? And that's where a lot of companies need to re really look at and understand what's going on with their heritage environments before they do they take that plunge into it. Mike, I suppose the, I don't accept the premise of what you're saying, that monolithic applications are about Netflix when monolithic microservices and back to monolithic, certain applications are better as monoliths depending on what your performance criteria is, your availability, your scalability, looking at those non-functional requirements, there is scenarios where monolithic applications are better than microservices.

Those might not be the mainstream. They may not be the most by volume, they may not be the sexiest. But there is scenarios where monolithic applications are better than microservices.

I think if you put an API in front of that monolithic application, you got a big ass microservice. So there you go. Or lots of them.

You say tomato, I say tomato. I mean, I mean, the interesting case of Netflix has been through this journey over the last five years, they blogged about it. They took their call streaming application, got on the bandwidth and where, and have microservices and have now taken it back to a monolith for performance reasons.

I'm not saying that every application should be a monolith, but I'm saying they have their, they have their usages. I will, I will concede the point that microservices are harder to manage and build and maintain, but the benefits of availability may outweigh those issues if you have the right team in place. True.

So Mike, if I told you that your Visa credit card application, that your Visa transactions are processed on a monolithic application and there's not been a second of planned or unplanned outage in the last 30 years, would you take that, that disproves what you just said? I would say that visa might often be the exception, given the amount of outages that we see all the time across all kinds of application environments. So When was the last time you bank?

Who do you bank with Mike? I bank with Citi. What's your, what's your bank account number Last time?

When was the last time Citi went down? I don't know. I don't monitor it.

24 7. So I don't know, I can't remember a Wall Street Journal headline saying that, And this may be because they have redundant infrastructure that they built an application that takes advantage of that. So, Or it could be that their core banking service runs on a mainframe.

Could be, well, let me, let me interject in, in the Steven, there are applications where the scale, the volume of what's happening with those, those applications, credit card transactions, video streaming, right? It isn't, you know, we're doing 10 of them. We're doing thousands of them a minute, tens of thousands, maybe hundreds of thousands.

And a constant rate that kind architecting for that kind of an application. 'cause if, if it goes down, you're talking about millions of dollars per minute that potentially is lost, right? So you're over at that scale.

You were talking about zero downtime, to your point, Systems like sre, swift payment Systems in the banking sector, core banking applications, you know, I'm talking about probably the 1% edge cases. And Mike, you are probably talking about the 99% of the rest. I'm just saying that it isn't zero Apple, that it's, it's, there is a scale.

Nobody talks about core banking applications. When Barclays went down in 2009, it was from page news. The chairman of the Bank of England caused the chairman of Barclay's Bank within 45 minutes and go, are we got a problem?

Are you guys okay? There's different applications and they need to be architected differently on different infrastructures. And I can tell you, growing up in the banking world, when we were designing new banking systems, it was all about transaction per second.

It still is, right? And now it's all also availability and everything else. It is, it's kinda like the air air traffic control system and the air airline control program priming, it's still around a CP is still around because it's high volume and it's necessary for, you know, reps at those stations in the airport to be able to transaction.

There are some things that are kind of over the top in terms of they're just at that maximum point and they're gonna run, you know, 20, 40% above full throttle all the time in red lining. That's just the way it works. And that's, See, we're not, we're not done arguing just yet, but we're kind out of time on this particular topic.

But, um, we'll be back to have a more spirited conversation about mainframes. We'll be back In a world where every line of code powers the future, every keystroke can introduce new threats as software evolves. So must security, it's time to rethink how we protect our digital world.

Join the leaders in DevSecOps and AI at the OpenText DevSecOps Virtual Summit on September 24th. Discover how innovation is transforming software delivery faster, more secure and smarter from AI driven security to the truth behind cloud security. Get the insights that will keep you ahead of the curve.

Don't just watch the future unfold. Be part of it. Register now and secure your place in tomorrow's world.

All right, folks, we're back and we're talking about mainframes as promised. There's a new survey from ndl. ai, um, talking about how well people seem optimistic about deploying AI workloads, especially inference engines on mainframe platforms.

Steve and I know you have a lot of, uh, experience with mainframes and maybe even a little bias here and there. That was pretty real segment kind of showed, but, um, Me biased about mainframe. So I dunno what you mean, Mike.

Let me write that down. What is it biased about? Mainframe.

Okay. I I doubt it. There you go.

But I, you know, all joking aside, is the mainframe poised for some sort of renaissance in the AI era? Because, well, that is where all the data seems to be. Well, it, It's an interesting one.

I mean, we did some analysis and did some commissioned research on, on this year commissioned by Broadcom 21 Cs and IBM. Um, I just published a piece that compares that about against the work that NDL did. A lot of similar findings, I think from ba The key takeaway from our research was that it was no harder to find an AI specialist or a cyber security specialist than it was to find a mainframe specialist.

There's various skills constraints in the industry that are just well known. The mainframe's one of those, it's not worse or better than any of the others. It's just one of those areas where there's not thousands and thousands of, uh, people like that would be Java programmers or web designers.

It's one of those more edge case where you've gotta think about skills programs. Lots of the vendors have been doing this for a long time. I think going back to your original point about ai, IBM made some announcements recently with its Tellum two processor at Hot Chip.

5 gigahertz. Pretty cutting edge stuff. If Intel was to launch that, people would be interested.

Um, they also know launch their new AI accelerator called Spire. Again, cutting edge technology. We're gonna see that come through in the, the new mainframe in 2025.

So hardcore chip innovation, IBM's got over 800 chip designers based in the Hudson Valley area doing cutting edge design. IBM research and labs have done two nan two nanometer technology. They've got on chip water cooling.

You know, IBM's pushing the boundaries of this stuff. This is not legacy tech. If I told you that the first commercially available seven nanometer based server was the IBM mainframe when they launched, said 16, and that IBM beat intel to market with seven nanometer, I'd just completely blows up the narratives that they're legacy tick.

So we have to put a pause on that. The piece, if you are gonna do a fraud transaction in the credit card tra actual credit card transaction loop, you can't be going out to the public cloud doing a core on Gemini and then inserting that back. There's just too much ran triple latency.

So you've gotta do those AI scoring that is fraught management, those K-Y-C-A-M-L, whatever the use case is, as close to the transaction as possible, you're gonna be doing that on ship or, um, reaching out to an accelerator that's actually in the machine. That's where my beam's go in with this. They've got over 200 use cases for AI based on these transactional systems from the conversations I've had with them.

So I think it, I think you've gotta hold, going back to our previous conversation, you've gotta hold two truths to be true. At the same time, public cloud can be growing and hybrid cloud can be growing and these infrastructures can be growing and people can be putting AI again into those public clouds. And it can also be true that there's a scenario where you want to do AI in, in transaction, at speed, at scale, on mainframe.

It's not either or. The way I describe it is boats, trains, planes, cars all exist because journeys are different. Workloads are different.

Therefore, they all need different chip architectures and underlying infrastructure to support them. A long rambling answer, Mike, but yes, mainframe's a call for the transactional AI use cases. Yeah.

If I recall, didn't that survey say like 80% of the business leaders, it was a large percent of the business leaders were looking to incorporate AI into the mainframe? Yeah, I mean, BM C'S about to drop its annual survey results in a couple of weeks time. They'll have asked questions.

They've been doing that survey for I think 19 years maybe. Could be, could be wrong in a, I think as you say, two things can be true at the same time. Public cloud can be growing.

You know, Oracle announced earnings last week and told us that OCI is growing at 45%. And on-premise, mission critical infrastructure can be growing at the same time. You know, IBM and its last earnings, uh, said that the mainframe sales group by 8%, that's in the very end of an extended product cycle.

So normally they're talking about decline by now, they've got a new system coming out next year. Normally we're in the back end of that cycle. And we're talking, you know, the CFO is saying it's a cyclical cycle.

This is eight, you know, eight to 12 quarters. We're in sort of quarter 10 of that cycle and they're still posting growth. So you two truths can be true at the same time.

It's not an either or. I think there's a lot of reasons why you want AI in the mainframe. And, but you have to look at the workload.

5 on the mainframe. You want, you want massive farms of GPUs to do model training for things like that. But if you're doing machine learning inference with live data, that's happening right there, provided it's not, let's scan all the available transactions and look for patterns of fraud.

No, you want to know, is this transaction a potential fraud? You've got a machine learning model and data that's already been analyzed, you can match that against, you want that to stay very, very high volume and stay close to where the rest of the transaction is happening. Maybe even for security reasons.

You don't want it off, you know, off memory, off chip from that machine. You don't want it to go anywhere else. 'cause it's very sensitive.

Matter of fact, machines are being architected now so that the AI inferencing can happen in a certain part of the hardware that other, uh, memory areas can't get access to, et cetera, because of the security. Um, look at AWS's chip, they, they all, they all are doing this. So it, there's a lot of good reasons why you want to do AI on the mainframe.

Is a mainframe a great general purpose for model training? Some, but probably not for most. But tying it to transactions totally makes sense.

Highly tuned. Small language models, tight use case, not really generative. Absolutely.

Sorry, Paul, you were gonna say something and I jumped in front too. Uh, no, I was going to kind of echo in from the, uh, a couple of the comments that you've that've been made on this so far. I, I would say, you know, as we kind of tie the last, uh, uh, segment into this segment as well.

There's, there's a number of things that are going on here in the, in the, uh, you know, that organization's taken into consideration. One, um, is, you know, historically, you know, when mainframes came up, there was a central set of applications that were running on a mainframe. And the, and the environments were far less complex than they are today.

I mean, we just did evidence of what Mitch was just saying. There are certain use cases that mainframe's good for and there's other use cases it's not. Um, but you know, that means that they, that organizations will have to add in additional resources to accommodate those use cases that don't have, uh, that, that that's not good for the mainframe.

But that adds complexity to the environment, right? That also adds a skill gap issue to the environment. You have, you have this kind of bifurcated skill gap of, of organizations that are using their, their dare I say, heritage environments, um, to run.

They're working and operating seamlessly, but now we have to bring on new resources, new skill to kind of bridge that gap between the two. That's one thing, but the big factor that I, I think from the, uh, application side, especially as we start going to, you know, uh, releasing applications far more than what, uh, what what's been done historically. I mean, what we're seeing in our research, um, today, applications are being released, uh, at two to three times faster than just three years ago with the same or less resources to do so.

So if that application growth is happening, that at that, at that lift, um, one thing that mainframe does bring to the picture is, uh, consolidation and sustainability, uh, message as well. So you can consolidate down those applications to a, a more environmentally friendly, uh, environment for that infrastructure, but still accommodate your business needs of re you know, producing those applications as needed. So the challenge, complexity, skill gap, the use cases are there, but again, and I'm gonna say this and Steven's gonna, you know, yell at me, but the underlying infrastructure really doesn't matter because whatever it's running on at the top, as long as it's a a, a meeting, the business need, um, the, and it, and it meets the, you know, all the different pieces, then it really doesn't matter as long as the, as long as the, uh, the, you know, the application teams are getting what they need, Know how I'm my podcast Paul infrastructure doesn't matter.

No, it sort of matters. It matters. Sometimes it Matters.

It's catchy. So I, I do wanna ask Steven something here though that I think that I'm starting to see. And initially all these AI projects were led by what you might call a TIGER team.

They had developers and data science and all these IT ops people, and they kind of operated in that fashion. I think to Mitch's point, as we run the inference engines more and more on these production environments, mainstream IT folks are gonna start managing all of that as part of their function. And we'll see the data scientists focus on training and there'll be separation of duties.

I think. So, I mean, we're, we're, we're still really early, but what are we like two years into chat GPT having changed our world? You know, so the teams are starting to come out of POCs.

They're starting to come into production use cases. You know, the first of those applications are starting to come through into that cycle. You know, if you've got a really tight, really sort of defined use case that's been through a testing cycle, and you're probably for the first time starting to deploy that in that into production in a place like a Citibank, a Wells Fargo or a Walmart, you know, maybe you've gone through that cycle faster if you are in a kind of fast moving startup.

But if you are moving that into production, you know, from po like whiteboard people staring at the universe, thinking about the first use case for AI through into putting that in, we're probably only at the start of this journey that application's gonna look like any other application for, for a team to start to think about, you know, how do we secure it? How do we back it up? How do we make sure it runs and operates?

It's gonna start getting passed over to the platform engineering team or the ops team at scale. So whether that's on the mainframe, whether that's on, you know, Kubernetes, running in Google Cloud, you know, I would imagine there's, we're only in the early days of those AI applications starting to be fully in production in big enterprises. Mitch, you agree with that?

Do you think that this is just gonna wind up being, you know, an extension of our DevOps workflows and ML ops and DevOps and feature stores and GitHub repositories and all that other stuff will just kind of meld together into one big happy workflow? It it has to be, but it won't necessarily one big happy journey. Um, because, and, and the way you're talking about Steven, by far, I don't have data to back this up, but I'm confident in saying the vast majority of AI being used today are machine learning algorithms and, and, uh, and models, expert system models.

Very little of it is, is, uh, is A-H-G-B-T kind of LM type applications where we're, that's where we're in our infancy infancy when you're talking about feature stores. We, we've been in this era where the way the model, the way the algorithms and the way the data is, is prepared for machine learning is on this own track where it goes through these cycles to get, develop the features, refine them to get the answers that we're looking for DLM based on the data and make sure the data's in representation that match, the algorithms are optimized for that. And then we release the model.

I think now we're in the era of how do we release that as part of our workflow and call it a DevOps workflow, call it whatever you want, release cycle, whatever you want it, it has its own, in some cases its own, uh, meter and tempo that takes its own different kind of work, but it's also integrated into the larger application, the front application we were talking about before. This is fraudulent transaction. Well, I wanna test by software, the app, the app against the, uh, new model, the new algorithms for machine learning when that's ready to release.

So make sure we're still in good shape. So I, I think the more, and, and there's activities that this is happening now where we can integrate machine learning and AI workflows into the overall DevOps workflow and understand where there are kind of in different fidelity of how processes work between software and, and AI models and be able to adapt to that. All right folks, so we have determined that mainframes will matter after two sessions.

We'll be back in a minute. Discover Textron Group, the epicenter of tech innovation. We are your go-to for reaching IT leaders and practitioners worldwide.

Our secret impactful content that sparks awareness, engagement, and top quality leads with us. You'll access editorial websites, streaming videos, virtual events, custom content analyst research and more. Join our satisfied clients, let's revolutionize your tech journey.

Contact us today and tell your story to the world in the most powerful way with Techron Group. All right, folks, we're back in over on Textron TV a couple of days ago. We had an interview with Bill Oink who's over at IBM talking about how finops needed to be embedded into the software engineering workflows.

And it makes a lot of sense, but it seems to be one of those things that might be easier said than done. Um, Mitchell, we've been letting developers provision stuff for years and never quite actually put any financial controls around it. It was kind of run after the fact, and in the last few years we've seen the rise of been up.

So is this a response to, I don't know, our developers kind of like drunken sailors, just consuming everything there is out there and no adult supervision or what It's, uh, like in music, you know, three chords in a guitar, you got a song, got a country song, credit card, and a browser. You got an application in the cloud. That's where, where we came from in the cloud.

And of course, once we realized how those costs roll up differently and aggregate differently, Hmm, maybe I don't want that thing running all the time when I'm not using, how do I bring it down? How do I automate that so that finops has been catching up to what we're spending and controlling spend on the backend and moving it forward. I think where, where finops is best thought about is in the architecture of the application.

The architecture of the infrastructure, um, and, and the entire kind of systemic view of how this is going to work, whether it's distributed or centralized or where, wherever that is. 'cause then you can say, well maybe I, I want to use, um, dynamo for my, my database, or maybe no, in this case I wanted to use Mongo because that's gonna be more cost effective. 'cause it handles this differently.

So that a lot of those cost decisions aren't just operational. They're also built into design, the architecture, the applications and assumptions that you make. And a lot of times as an architect, what you have to do is go create what I call architect prototypes.

Let's go try this and see what kind of volume or what kind of a a your resource usage if we take this approach or versus this approach. Okay, great, we figured out that out. Now let's make sure when we deploy bigger parts of the application that that still holds true.

So I I do agree. It's part, it's gotta be part of it. If you don't, you got the wrong architect.

I think. So for, for finops to work well, it's gonna take better communication. I think there's gonna have to be a shift in companies collaboration between all parties involved and a different mindset from the engineers about cost.

Exactly. Exactly. Paul and I are working on a total economic value sort of study as we speak around one of the big finops tools.

So I think people process tools is the kind of simple rubric that's where the majority of the cost savings are gonna be. I think, Amanda, to pick up your point, you, you need to be thinking about the people side of this and the, and how teams collaborate. How the dev team to collaborates with the ops team with infrastructure.

You know, the, and Paul, I'm going to pause in a minute 'cause I, I see you coming one it to come in, but I think it's across the whole piece. Samantha, you, you'd now to Yeah, I I think that, you know, we're all touching on, uh, valid points here. I, one of the things when, you know that I often, uh, get into conversations with, uh, whether it's advisory with, with clients or, or just end users CIOs and such, the, the biggest challenge with finops, um, is at least from the, from the application side, is usually it's an afterthought.

And, and it's an afterthought because the primary concern is the refactoring or need to modernize or I need to do something, right? And doing something is like, Hey, yeah, hey, we all high five, we did something, we, we refactored. And then to go, whoa, the costs are way outta control.

Like, so now we have to figure out like, what's the cost factor here? So when we talk to, you know, companies out there that are looking at finops solutions, right? org, right?

Um, there's a, there was IT vendors at, uh, finops six, Right? So, so, so those vendors that are out there, there's obviously a need in the market, but I believe, and I've been, this is the thing that I would advise anybody watching this video or anybody that's looking at new initiatives is to obviously not just say what's it gonna cost, but put cost measures in place as you're going through the project. Because as you're doing the project, um, if you leave it up to, you know, say DevOps teams notoriously DevOps teams are gonna overprovision because they just need the resources that are, that are needed to do the job.

So as long as they have 'em, they're gonna do it right? But that's over provisioning. It's, it's gonna cost extra money.

So understanding where the, uh, spend is going as you're implementing is probably the better approach than waiting until after the fact. And then trying to scale it back from there. I think, you know, Paul, as if you're building a house, you know, you have a certain budget you're working with and you can design a, a kitchen that's an x what your budget is by the materials you pick the providers, et cetera.

The same is, is true in software. It may not be for an individual developer. That's why I point to the architect.

Someone is, has to look at the overall decisions you're making about how you are designing, constructing, and operating those applications. 'cause those decisions drive up that cost or down. Steven, how did we get here?

Because back in the day when we had those mainframes that you sold lovingly tend to, um, we had capacity planning, we knew a lot more about the spending and we paid a lot more attention to that. And then suddenly it seemed like the cloud came along and we just forgot all those core skill sets I've got. I've got four teenage daughters and they're wearing flared jeans Fraction Comes, right?

I mean, I think We're on a journey. What are we, 18, 19 years into the public cloud? Everybody went ran and did what, um, uh, Mitch described with a credit card, you know, shadow it, span something up and then realized it was expensive.

And when they're just coming around the backside of that cycle with people going, you know, it's now platform engineering is the phrase I talk about, sort of the grownup ratio that I'm tracking, sort of the sports coat to hoodie ratio at some of these conferences. We're just getting to the point where this stuff started to mature. I think people are looking at their AWS Google and Microsoft does your bills and going is the largest line item in my IT budget, therefore I need to optimize it.

That's where the mainframe was 20, 30 years ago. It was the line largest line item in people's IT budget. So they tried to optimize it.

It's just fact. It's come what's old, what's old is new again. And we're just going through this cycle.

Paul, I think Steven is implying at least that, uh, developers or teenagers that can't be trusted with credit cards. So, um, how did we get to that Main shipment? You know, I, I think it's interesting.

I mean it's a, it's a, it's a dilemma that, uh, uh, organizations run into quite frequently. It's, uh, you know, if you're trying to meet your business KPIs of, of, uh, you know, doing your agile sprint releases every two weeks and you're trying to push code out the door, uh, and there's barriers in the way, you can either do one or two things. You can put those checks and balances in place to slow the process down, which is gonna slow down your, uh, your business KPIs or you can, you can release the hounds and let them do what they wanna do, which is gonna overspend typically.

But I think that you have to have those business KPIs aligned to the, to the, uh, the developer's expectations and vice versa. Because I think that if you don't, um, then you end up having the, the challenge of you're not gonna release your code on time or, or to your business KPIs. You're not going to have a, a successful team.

I mean, if you're dev DevOps teams and developers need to be empowered to do their jobs, right? And if they don't have the tools to do their jobs, then they are, then they're stuck. They have, there's so much on the developers now where they're sitting there with the, you know, sitting with the catchers, everything's shifting left and being thrown to them, um, that, you know, they need to be able to have a little bit of being fluid and dynamic and able to achieve their goals.

If they have all the resources and tools they need, including the spending it need to, to put in place, then that makes sense. But I definitely believe checks and balances need to be put in place to, to Mitch's earlier point, you can design a kitchen however you want, right? You, you can design an application infrastructure however you want.

It has to fit within the budget. And that budget, um, needs to have those checks and balances. All right, so Mitch Paul is insinuating that if I give teenagers credit cards with no limit, who's the pool?

So I think I'm still stuck on release the hounds. I'm, that's, that's an image I like, I think, Um, Paul makes a really good point. It it's about magnetism.

You want to just, enough friction with developers that they're being thoughtful about where they're deploying stuff and the cost. We don't want too much friction. 'cause you don't wanna slow them down and lose the benefits of turning applications faster and bringing things into, into production faster.

The best organizations are making that trade off and looking at that trade off and going, infrastructure's gotta be secure, it's gotta be available, it's gotta be the right price point depending on the type of application it is. So they're doing those things on one hand, but they're also thinking, how do I increase feature velocity, speed of application drops, get code into production faster. And those things pull people in the, in two different ways.

The best organizations are able to find that happy balance. I think it's all on how you position it. If you, if you hold the developers together and say, okay, we have to find a more cost effective way.

How could we reduce our costs here by 25%? Let's shoot for 50 and see if we can do that and go into a virtual room or a conference room and give 'em a problem to solve. They'll focus on that if you go in and slap them down.

'cause you know, you bozos, uh, that, uh, you know, they, you hounds that are running rampant around here with our credit card, we're clamping down on you and we're taking the car keys away. And you know, you'll be lucky if, we'll, we'll, we'll turn the lights on when you come in the building. I, I think, I think the term you're looking for is bad dog.

Bad dog. Exactly. Amanda, Amanda hits it perfectly I think a few moments ago.

It, it's thinking about it holistically and it's thinking about it from a people perspective. So Steven, can I rely on the cloud service providers to give me accurate info that's useful for controlling costs? Do they have a vested interest in, you know, helping those drunken sellers have another beer?

I think it's incentives. I think it's incentives. So the person is the person trying to sell you something, the best person to trust with how much it costs.

I think you want an independent third party in that loop, whether that's a finops provider, whether that's a consultant or sort of optimization team you're working with. I don't think the incentive structure for you that pick, I'm gonna pick on AWS go turning around to AWS and going, I'd like to reduce my bill by 30%. Can we spin up a project where your sales person gets less commission?

Don't give the incentive infrastructure for that works going to somebody like an IBM and app to and Cloudability and going, Hey, IBM I'd like to work with you to reduce my AWS bill. And if you and a shared risk shared reward project where if you say me 30%, you know, I'll buy some of your software, the incentive structure makes more sense. So I think if I was running a large enterprise, I probably have an A WSA Google and a Microsoft kind of engagement.

Going back to some of the stats that Paul mentioned earlier around hybrid and multi-cloud. I think I'd be looking at a separate cross platform finops PLA platform to reduce my costs on all three clouds. I don't think I'd be going to each particular cloud provider saying, Hey, can you reduce your bill by 20%?

Amanda, do you think we might be able to use AI someday to just control these spit ops costs? And you know, I'll have all the data input and in from these things through those lovely APIs we talked about earlier. Um, and I will just throw that in a model and optimize it for controlling my calls.

Oh yeah, absolutely. I mean, the use cases for AI as fast as it's evolving are growing by the day, so I'm sure that will be utilized. All right, Mitch, that sound feasible to you?

It's, it's a great machine learning opportunity because if you're doing Fed Ops as a service right through the cloud, uh, like in many other, uh, types of applications or services, you have access to all that data that's being processed by each one of your customers. You can anonymize it and then do, you know, look for patterns and new machine algorithms, machine learning algorithms against it that you might not find in one customer looking at it yourself and our own usage of this particular cloud. But if I look over a hundred or a thousand different customers, you know what?

We see some trends here and we can make some recommendations that'll help you save money. All right, folks, you heard it here. Hey, someday those AI models are gonna be voice enabled and when you overspend, they're gonna say bad hound.

Hey, thanks for spending time with us and we enjoyed the show today, and we invite you to check out the rest of the lineup that's coming up right behind us. Once again, thank you to our guests. We'll see some other folks tomorrow.

Until then, we'll see you next time. I'm Bonnie Schneider, sustainability contributor to the Techstrong Group. I'm excited to introduce you to a groundbreaking new initiative from Techstrong Research, the sustainability pulse meter.

The pulse meter offers valuable insights into how environmental responsibility factors into tech purchasing decisions for key players in the industry. Position your company as a leader in the industry and differentiate from your competitors with the sustainability Pulse meter offered exclusively from Techstrong research.