Automating IT Operations with Generative AI – Muddu Sudhakar, Aisera

This is Textron tv. Hey guys, thanks for the throw. We're here with Dr.

Muus Sudhakar, who's the c e O for i e a, which is a company specializing in all things AI ops, and we're talking about how generative AI might get applied to this whole mix. Mudu, welcome to the show. Thank you for Michael for having me here.

We've been throwing machine learning algorithms at IT operations for a while now. I don't think that's anything new, but we are seeing, uh, a new AI animal in the zoo, as they say with this generative AI stuff. So are we gonna see something that feels like multimodal ai?

Will we apply both? And if so, how and when are these things complimentary? Or does one supplant the other?

Um, what we call that whole space is AI ops and AIOps observability, right? Uh, it started with anomaly detection and rule-based and everything. Now we are at a point, we are using AOPs for different algorithms using the N L P in the AOPs world.

See, what's unique that ICE R is doing in the AOPs is we are doing not only time series analysis, we are not doing longitudinal analysis and graph analysis. We are applying N L P techniques, n u techniques, the what you call, think of, uh, the Chad G p D techniques on the textual data as well to derive where the outages will happen, where the performance issues are, where are the experience issues are, right? So the whole space of the, uh, a P M application, performance management, IT operation, cloud ops, DevOps, that whole thing is going to get a uplift with this whole new age of AIOps and observability applying a different way with the new stack.

So what will be the impact of that uplift as you describe it? I mean, how will the functions and the roles change? How do you see this all playing out?

Yeah, a lot of it, look, first is, uh, at the end of the reason you're doing this in is to what? To improve your uptime, reduce your downtime of your services, right? And making sure that you're not throwing that many resources.

You don't need thousands of DevOps tune, for example. You don't need thousands of IT operations people. People can't even afford it today.

Maybe the large public hyperscaler companies can have thousands of people monitoring their service to making sure everything. But once you are inside a corporation, an average enterprise, average Fortune 5,000 companies still want the same level of service uptime and downtimes, right? But for that, they can't afford to have thousands of people.

So by having this, it actually provides a cost effective way to manage your service and provide a good customer service, et cetera. So that's where you'll see the benefits coming in. What happens to all those people who are managing those systems?

Today, we hear a lot about keeping the human in the loop, but no one seems to know exactly where that is. So what's your sense, uh, how that's gonna work? I think all of them will become software developers.

The world will be like, nobody's gonna lose a job. AI is going to create more jobs than losing jobs, right? I don't want to be as a DevOps student, trying to tune some parameters and watch some dashboards.

So all those guys will become the software develop person to do AIOps workflows. If there is a problem, what I'm gonna do next, can I use that notification to resolve the problem? Can I write code to do, uh, what you call remediation?

Can I do that to fix the problem, right? So there's a lot of other ways that new roles will evolve, uh, in the DevOps side, in the IT operation side as well. But I don't think the view should be that by doing more automation here, some of these jobs are eliminated.

Michael, I think that the, I like to think this is world is a lot more half full than negative. I'm not even sure that people wanna do those jobs. I mean, there's a lot of toil involved and a lot of stress.

And, um, if given a choice between working for an organization that has AI and one that doesn't, I think most people are gonna go with the AI 'cause it's just gonna be more fun and more interesting, Right? And also the way I'm seeing recently, and if you saw, we announced with Microsoft AI co-pilot in the world of AIOps, the AI co-pilot functions will be very interesting that we are seeing with our customers. So we will get a alert, let's say from your application, your service, or even your laptop or your Mac device.

We using that to proactively inform you, Michael, hey, your drive is going on. Do you want me to order a new, uh, drive? Uh, I see your C P U is always failing here, or you have a memory problem yet.

So able to do proactively detect issues in your application, right? It could be a database server, um, Oracle servers acting funny on this server, this server may replacement. So able to do a, a co-pilot approach for AIOps makes it very interesting in the world of operations, right?

Uh, where I'm not gonna be right always, but the concept that somebody has, your back, Michael comes and tell you what are your top 20 things, and you as an expert come and say, you know what? Let's focus on these five and not these four, and that will change the game. So I want to proactive service, which monitoring my application, my business services, my infrastructure, and telling me what may happen, even though my probability of success there is even like say 60, 70%, right?

Today, we are already seeing in our environment, it's 80, 90% certain. We are able to predict events before they happen. I still have false positives and true negatives, but the concept of true positives that I'm generating are so valuable.

It's better than being react to organization. You kind of alluded to this, but I'd like to dive into it a little bit deeper. But we talked about the notion that, uh, a lot of organizations don't have enough people to manage it at their current level of scale, but will they be all managing it at several higher exponential levels of it, in effect, everybody's gonna become a hyperscaler?

Look, I think, uh, you can, right? Like hyperscaler companies have different, uh, business models. They can afford it and average organizations cannot be managing like hyperscaler, but they would need tools like us.

So can you provide a a thousand x cost efficient solution and thousand x less resources to manage my infrastructure? That's what the value prop companies like us are doing as an entrepreneur. My goal is how can I provide AIOps solutions at a fraction of what it takes for hyperscaler to do, but provide almost as close to that, uh, quality of service, right?

And it is doable, right? Uh, as I said, if you ask me five years back, is that possible? It was a, a pipe vision back then.

Now I'm already seeing this deployed and people are testing it. That's the thing that we did with Forrester. Uh, came out a nice report on AOPs and observed where they compared us with the various industry use cases.

That's a very wonderful what Forrester has done in that report. What will be the impact of all this on the cost of it? I mean, are we approaching a point where the cost of running it and me software development might be approaching zero?

I mean, just how, how are we gonna look at the cost of this? Nothing will become zero, but it'll come down, right? So it's called the, like if you ask me what was the cost of storage was 20 years back, cost of, sorry, of terabyte came exponentially down.

And look at with, uh, VMware and, uh, Kubernetes. What, uh, Amazon, Microsoft, and Google have done. Same things won't happen to your total.

IT costs. If I'm a C I O, my total IT cost c i o spend. If it's not coming down, I'm doing not a good service.

As a C I o I may do other things as a development. I might become a C I O to chief development officer. I may invest in technologies to build with the developers, but purely managing it.

Infras costs have to come down. Loss of scaling efficiency, that's where the AI will play a role, right? Both in terms of number of resources.

Like I don't need level one, level two players to look at, um, monitoring the solutions, incidents and all of those. So I think the scale has to come down. Company's efficiency.

That dollars will be reinvested somewhere else. It, you may not want to call it IT budget, you may call it AI budget somewhere, right? So, but the company's spend will come down.

So companies will get more scaler. I think AI will generate such a huge productivity. That's why I call this a revolution, right?

Uh, AI will change. It's like industrial revolution. It'll give you productivity across globe, across America, across societies.

It's not a zero sum game. It's not a 82 game. It'll give you net scale down on total cost.

Now that savings, where you going to invest is up to you as a, a leader of your organization. Michael, Do you think down the road we'll also see some major spike in innovation? Because I think a lot of organizations limit what they're gonna invest in from a development perspective because the costs are too high.

So they have to, uh, make fewer betts. So will organizations just make more betts on more applications and see what happens? No, I don't know.

Look, to me that's individual preference. If I'm any, every company in the world is going to be AI company or a software company. Doesn't matter, your man, if you are not investing in your IP technology, leveraging ai, you have to think about for your business model and growth.

At the end of the day, you are, are trying to be a shareholder value. Are you trying to grow your company? How you want to grow efficiently?

What is your ip? If you think that way, you have to invest in technology. And IP question is where you invest, what areas is it for reducing your costs?

Is it to improve your uptime, reducing your growing your sales productivity? You gotta think through some areas, but I think leaders will invest in technology and you ought to make your betts and this quarter you may put more on sales productive versus business versus it. But I, I'm a much more positive.

I think people are waking up to the ai, there'll be huge bets will be made across the world and every company, it's a C-level discussion. It's no longer a a it C I O discussion. The board and c o and c o are asking where, what are you doing with ai?

Where is my chat? G P T, right? How come I'm not leveraging that?

Right? So people are gonna ask that. If, if you are not waking up and providing that, you are just kicking the can down the street.

What ultimately will differentiate one AIOps platform from another? 'cause there seems like there's a lot of them floating around out there. So what should people be looking for?

A lot. It's not one thing to look for. Like I think first it's accuracy, right?

How many, how much are, it's like earthquake prediction. If my predictions are going to be 10% accuracy, micro or not use it. So number one, what is in AIOps in the future?

And is it a predictive solution? Is it a proactive solution? What is my accuracy?

What is my batting average? That's one. Second thing is when it's false negative, can I train the system to improve those?

'cause it will be wrong sometime. Think of like how when you go to charge, but you have hallucinations, they make mistakes. It may not answer it Questions.

Can I now tune it to my, now can I use, in the AIOps world, can I leverage the LLMs? How much training requires a reinforcement learning? How long does it take?

How much human in the loop is happening there, right? Does that human requires an expert person or can I use a simple annotators to train the system? Right?

Do I, if I have to invest under 10 people paying 200 thousands of $2 million on top of your aiop solution, that's not a viable, right? That's other metric that I look for, right? Um, and the last thing is, is this, once you do it, can you reach a state of what I call the self feeling, right?

Which is once you detect it, can you take actions on it? The actions. So the whole AIOps world.

Next would be is can I take remediation actions and the remedi action for Michael, it'll be different for you versus me for your SaaS service, you may want to do different action than my SaaS service. And you want to customize that. So do have a workflow engine that can be triggered from the AOPs to drive the actions.

'cause you don't want to wake up and say, I want to page you. Nobody does paging anymore. That's 20 years back.

Nobody wants to be notified through SS m s, but still the industry is still operating with paging and notifying alerts and running run books manually. There's so much inefficiency in that area of connecting AOPs to remediation actions. That integration will be also important in the tool selection As we go along.

How, how much time does it take to get the benefits of an AIOps platform? And I'm asking the question 'cause hmm, know, a year ago, two years ago, it would take several months before the system learned enough about the environment to do something. And the environments are all different.

So how, how, how fast is smart? Very, very rude question. So typically, so two things that I look for when we deploy our AIOps or anybody else, right?

How much data are you giving us to start it? So think of like when, when you see Tesla and Google driving the cars to map the streets, right? They want to know what the roads are and they're doing the mapping algorithm like that.

If you gimme your data and your data has a ground truth, the learning becomes faster. So question is the amount of data that you can provide, and the data has real good signal for the algorithms to learn these days. Typical learning is happening.

What is to take years is already come down to weeks and months, right? So we are validated, reduced that look, given that it has reduced a few weeks and months, you'll start seeing results. I think in, in our, my view, people once you deploy apps in like in day three, mar whatever, month three or month four itself.

But whenever you deploy these apps, my uh, request to customers is deploy. Don't deploy for like a POCs. Don't do it like a, you can't expect anything out of one week, two weeks, uh, proof of value or proof of concept.

Take a bet to do a six months or one year project. If you don't have that much will to do an AIOps for six months, one year, and you are a c o, maybe AOPs is not important for you. Maybe your mind is somewhere else, but this, you have to go in all in as a bank, right?

Michael, if I bet with you, I wanna bet with you for a year and in, but the results would start coming in like month, three month for itself in terms of results or even like eight, six weeks. Eight weeks. Also, the IT environments themselves are complex and they're constantly changing these days and there's a lot of, uh, microservices and containers and things are being ripped and replaced.

How does the AI system keep up with all that? Very another very good question. So this is where I call it AI discovery.

See, before you do AIOps and observative, something that we have done at i r and I I always tell all the customers to do is we are living in the world of increasing complexity. As you said, you have Kubernetes containers, you have microservices, right? And they're ephemeral.

That means they don't stay constant. They'll go up and down. You don't even know what services coming there.

So how do you keep track of that? You're not going to keep track of that next spreadsheet manually writing down or do one time asset discovery or one time C M D V. You remember the old C M D V, right?

So what I call that is do AI discovery real time all the time throughout. So that AI discovery, again, should be done in a very non, uh, noncompromised manner. What I mean by that is if my discovery requires you to put agents in every application, uh, in, in every system, and I, if you have to crawl your networks, that's a very compromised approach.

Why? Because from a cybersecurity perspective, if a hacker or a bad actor gets in into these networks and agents, the game is over. You'll be like, what just happened with, um, in recent attacks that you have seen with others, some other vendors that they got into other networks.

So the AI discovery in the future state should be based on the data. It should not be based on the agents, it should not be network crawling. So we have created something at is called AI discovery, which is based on the data.

What are data you gimme? If you gimme small data, I'll discover only those, uh, those cis and those Kubernetes and those items. And that should be such a non-intrusive noncompromised approach and use that as a base layer to do all your AIOps and observability.

So let me ask you this straight out. Sure. If you had a, if you had a child today, would you tell them to get into IT management or would you say maybe you should just become a software developer and not worry about the machines because there's no career path there?

Or what's your, you know, what, what advice are you offering the next generation? I think all of them go right. Look, if you're a software engineer coming out of engineering school, I tell the people it's first learn how to write code.

Make sure you know how to write Java code, Python code, JavaScript. Be a developer. Whether you chose to be a software developer or you want to be a career in IT industry, whether you wanna be industry in like anywhere the code, if you are not a good math class, math major, science major and programming program.

Software and programming is where you're going to create wealth for yourself or your family and for your organization that you're going to work. So the basis not be a good programmer, go get into hackathons Now, which area you want to go? Even if you want to be a good podcaster, you know, no programming, right?

And then use, be an Apache developer. Go into GitHub, right? Go into these hackathons, uh, or be an open source committer, right?

I even tell people like, become an open source committer. Write your code. Let other people value review your code.

They let them critique you. They may not get always good critic. They may tell you your code is not good.

It code sucks, but you want that, uh, honest feedback from your peers. All right, folks, you heard it here. You may not write the code, but you sure need to understand how it works and how to put it all together to create something interesting and compelling.

Mudu, thanks for being on the show, Michael. Thanks for having me. All right.

And back to you guys in the studio. I.