The Costs of IT Outages – Mohan Kompella, BigPanda
“The Modern IT Outage: Costs, Causes and Cures,” found that the average cost of an IT outage is $12,913 per minute. Produced in conjunction with Enterprise Management Associates (EMA), the report found a correlation between IT outage costs and the size of an organization, as businesses with more than 20,000 employees lose an average of $25,402 per minute due to outages, translating to more than $1.5 million per hour. Larger companies should consider things such as potential revenue at risk, number of people who could be impacted, and the possibility of associated fees, penalties or litigation.
Transcript
This is texturing TV. Hi everyone. Welcome back to techstruck TV.
Our next guest today is Mohan compare Mohan compella. Mohan is the VP of product marketing a big Banda. I know many of you probably know big Panda but we're gonna get if you don't not to worry, we're gonna wear explain that to you.
But first, let's welcome Mom and welcome to Tech strong. Welcome back to Tech strong TV. How are you?
Good good. Thank you Allen. Thanks for having me this morning a pleasure to have you on so Moen before we jump into big panda is let's jump into who is Mohan give people a sense of kind of your career your story.
Shut up, certainly. Yeah, so I kind of got my start in it Enterprise it almost 20 years ago as a consultant, I guess today you would call that customer success. So over the years I've had a variety of Consulting presets engineering technical marketing marketing roles and somewhat stayed in the it operations domain, which is also very big Panda plays in so yeah, I've really seen the landscape in the in the space evolved over the last 20 years.
You know, you know what they say the days are long with the years are short and when you look back over that it doesn't maybe doesn't seem as long but when you look day-to-day the changes they they really are I mean think about what what it was 20 years ago, right? You didn't have Cloud you didn't have so much of what we called native so much what we take for granted today anyway. You mentioned big Panda plays in the it operations it Ops space, but why don't we dig in a little deeper than that?
Tell us about big Panda, Mohan. Yeah a happy too. So like I said, I've been in the space for a long time and it's one of those funny things where you know, some of the tools that were used 20 years ago the promise of it operations never really came to fruition and that's what big Panda fixes I think for the first time companies that use big Panda big band as AI Ops platform are able to finally reduce the the frequency the duration and the the impact of it incidents and outages could also say in some ways that we help prevent outages and then if there is an outage we help you resolve it very quickly using the power of AI in machine learning.
excellent, you know and and You know we hear so much today this about Ai and ml. And you know, what's the difference and is it real is there not real? My feeling is that in some in some areas.
There is a lot of hype and there's more hype than reality but in certain instances in certain use cases. Whether you want to call an AI or ml right, I'm not gonna argue. That's like a religious War not getting into politics and religion, but in certain use cases, it really runs really works and really helps the AI Ops use cases one of those.
Why do you think that is? Yeah, another great question. I think like everything else in technology.
You generally have the hype cycle. You have the value of disappointment all of that stuff. So I'm sure but I do think that shroud of disillusionment is the yeah, that's regard or calls it, right?
Yeah, exactly. So I feel like especially an IT operations in the last seven years specifically, you know, the promise of AI is actually started to be born out. And you said why is that I think the technology, well, you know for AI to work you need large data sets, you need real-time processing power and you need pragmatic results right not smoke in mirrors, but pragmatic results.
I feel like all three things that Confluence of these three things has happened in the last seven years, and I don't think there's any Looking Back Now. actually, I don't believe so either not only that I you know, I was we had a previous guess was some service now on and we were talking and the whole ml thing came up there as well and and you know, given current macroeconomic conditions. What's going on?
Even in the tech World which has been immune from? Macroeconomic conditions for probably you know 10 12 years. People are going to have to do more with less in terms of people.
Right and and budgets and stuff. So in order to do that and the jobs not there is no going back. The job's not going to get smaller.
We're not going to shrink our infrastructures and stuff. We're going to need to rely more on. Technologies like ML and Ai and automation To allow us to to get the jobs done here.
Yeah hundred percent. I think yeah the Dior earlier Point The Tech Industries. I mean some of a Teflon industry, right the economic conditions have been completely divorced from the tech growth curve, but I do think that you know, finally in 2022, there's definitely been a reckoning and I'm sure you're seeing that going on and I do think that especially when we talk to our prospects and customers.
I think what gets some really excited is how AI or AI Ops and automation can help them whether the storm right it really comes down to how can AI help them help their teams do more with the existing head count. How do they help them get more or squeeze more Jews out of their existing tools and how do they optimize efficiencies and cut costs not just head count, right but in all the other areas, there's inefficiencies and things like that. How can Ai and ml help them optimize and automate and all of that stuff?
So we see a lot of interest in that and I do think that we have this unique ability to help move the needle there. Absolutely, absolutely. Before we even jump into to this report that we want to talk about for people want to get more information about bigpander Mohan.
io. io Allen. Yeah, you're right.
Yes. Okay. So just want to send people that out there right right off the bat.
All right, let's jump into really what we were gonna talk about today before we went down that fork there, which is a big panthers recently released a new report. And around outages and so forth. Well if you can't share with more, I'm sure with our audience.
Yeah happy to it's a topic that's a near and dear to my heart. So I talked earlier Alan maybe five minutes ago. I guess how you know big Panda's really in the business of preventing and resolving outages it outages.
Right and I think that you know various other vendors that also do something similar but you know, every every vendor has a different take on it. But you know, one of the the fundamental inputs to that process the fundamental ways in which you can measure track and communicate the value of preventing and resolving outages. That metric is really what is the cost of an outage.
Now, you're gonna be shocked to hear that while that is such a fundamental input the last known statistic that's been used in the industry is all the way back from 2014, right? Which in Tech is pretty much a lifetime. It's like doggy right here like dog years, so So imagine this is once random statistic that's floating around from a blog post from all the way from 2014 and that's been used and abused to justify investments in tools teams headcount growth budgets and so on and so forth.
So anyway, long story short, we you know, given our ringside view into the it up say of space third was high time that we fixed that so we commissioned research firm called EMA research to go survey a couple of hundred it and it Ops execs to find out what is the state of the union when it comes to the cost of an outage and we found a few things. The first thing we found was the old updated outdated stat was that the cost of downtime was 5600 a minute and we found that in 2022. No surprise I guess is that number is more than double this old data stand.
So the cost of an outage concept of downtime is on average and we can get into more detail if you have time but is almost 13,000 a minute. Wow. So what the crazy a minute?
Yeah a minute, right? So now the next thing is then you probably wondering alright, that's the wrong per minute basis, but it doesn't outage really last a minute. Does it last five minutes?
Does it last an hour? So that's the next thing we found was on average and we surveyed companies with employees between, you know, a thousand employees and all the way to like over 20,000, right? So they're multiple groups of companies.
We surveyed on average for these companies and outage lasts anywhere between an hour and two hours. 5 million and pretty much every company said they have one major outage a month to keep the math simple. Let's assum.
5 is 15 million. So every company out there with between that has like between a thousand employees and twenty thousand employees or more. He's incurring it, please 15 million dollars in wholly avoidable outage costs.
Crazy, I'll be honest with you. I don't remember what the numbers were in 2014 off the top of my head. I I do remember the the pony my Institute Larry Pony my Dr.
Larry Pony my he used to do these kinds of things but it was more for security incidents. Right? What did the average security incident?
Cost but that that's pretty much in line. I don't even know if it's gotten more expensive Mo and I think it's always been that expensive. You know, that's a great point.
I think it has been but there's been no, you know scientific, you know, you obviously you mentioned the pony one Institute, right? Obviously, they've done some work but in general it's kind of weird to me that does not be more research done by others. Hopefully now, you know, we have something again, you mentioned the big planet or IO website.
So anyone that's listening or watching can go download the full report. But yeah, we've done we've broken it down by size of the companies either team frequency of outages again, I gave you the average the headline number but there's a lot of variation there too based on who you are. So we're hoping that this is really useful for a lot of companies to help with budget planning and you know, one of the things I think I'm fond of my own voice.
So if they want more thing and pause but one of the things that it operations teams, Other is to communicate their value to management. And I think this report has a lot of good data that they can actually use to do that. And finally, you know, stop being the unsung heroes of ideas and be the recognized Heroes of it operations.
I think there's a marketing campaign in there somewhere right start being the unsung heroes. Yeah, and the real here, you know, I I Can't help but think you know our million and a half dollars an hour, right? That's you know, it's a lot of money by anybody's.
mark Again, analogizing to Mike. I was in security for 20 25 years. So everything, you know, when your Hammer everything looks like a nail everything I take back to security well in security when faced with those kinds of numbers face with those kinds of outcomes, you know, it's about risk management.
Right and in managing that risk you could say. Well look I can try to prevent. those outages from happening and that I don't know my gut tells me we could do a little better and preventing outages, but they're going to happen.
Nope, you know, I I think in terms of managing risk the amount of money you would have to put in to just totally prevent outages from happening May reach a point of diminishing returns to outages happening and then you know, the flip side of the equation is the response right? How do I minimize? Once I have an outage, right there's a big difference between a five minute outage and of one hour outage.
So how do I bring that average outage? number down and you know an easy answer you can give me is use big Panda but let's dive in a little bit beyond that. How what do you do?
Do we make a bet to try to do less outages or do we put all our money into minimizing the outages when they happen or little of both? Yeah, I think you're gonna do both right? I mean there's one of the ways we think of it as a lot of times outages have these early warning signs that are given out and if you do a good job and catching those early warning signs, then you can prevent those out of this.
We say you we kind of helps you detect an incident in real time before it escalates into a crippling outage. Well, that's one but but your spot on right you can't really do it for all the outages out there. That's just you know, unrealistic impossible.
I would laugh at anyone who claims they can 100% prevent outages the second part is all right. Now, you have an outage what do you do? And I think that's where a couple of things come in.
The first thing is, you know, it's like when you go to the ER right your triage nurse quickly looks at you for five minutes, you know takes to your temperature, maybe your blood pressure or whatever and then decides what to do and that's like one of the fundament Of things that we can do to help accelerate the incident or the outage triage process. So we route it to the right specialist for immediate Deep dive investigation Etc. That's one way you can reduce the duration.
The other thing you can do is a lot of our customers tell us that they get stuck in these endless Bridge calls. I think they call them these Bridge calls from hell. So there's a problem.
There's an outage you have 20 people join a slack room or actually a phone call and they're all trying to say, you know, who caused it whose team cost it and that's something else that we can eliminate by surfacing something called the probable root cause these teams don't need to be stuck on a three-hour call and by giving them what likely broke or caused it by surfacing the problem root cause we can drastically condense the duration of the outage. The last thing we do is once you know, what's broken, you know teams often struggle to manually go fix something right. The cliche is rebooting the server but there's other things that they do manually that take a long time that prolong the outage if we can automate that and big Panda helps you do that then we can also again cut precious minutes of the duration of the outage.
Eric absolutely and and look minutes or money, right? It's it's a very easy that's easy as part of the equation here our present. Yeah.
I mean that whole cliche the the time is money applies in Spades to Outer this for sure. Absolutely. io.
You could download the report from there and and you know dive in because they're you gave us a really great overview, but they're a layers here. They're marketing. We did report, you know, a lot of really good data highly encourage your listeners and viewers to you know, check it out.
Well, thank you so much for coming on and and telling us hey, don't be a stranger come back. He was supposed to know what's happening. Yeah, that's right.
It's fun you make it look so easy. So thank you. I know it's not I just make it look Easy by letting you talk.
I do less work. Anyway be well good luck to you happy holidays will be in touch. Thanks.
You too. Bye. Thank you Morehead.
Capella big pad to hear our text on TV. We're gonna take a break here on textual. We'll be back in a moment with another guest.