Mezmo Data Management, Observability, and Getting Value Out of Telemetry Data | DevOps Dialogues: Insights & Innovations – EP10

Hello and welcome to this edition of DevOps Dialogues. My name is Paul Nashwa. I'm the practice lead for the app dev practice at the Futureum Group, and I'm joined today by Tucker Caraway.

Tucker, would you like to introduce yourself? Yeah. Hi, my name's Tucker Callaway.

I'm the CEO of Mes o and thanks for having me Here. Excellent. I'm glad that you came and joined us today on this session.

You know, there's a lot happening in the data management space, observability space, and a lot of traffic. What's, what's, uh, MES o's claim to fame? Our claim to fame is we, we really focus on, um, helping people understand and, uh, get more value out of their telemetry data.

Okay. I think that is, is exactly where we're focused. Well, getting value out of data is key.

'cause data's the lifeline of the business. Right? Yeah.

And when we talk about it from a DevOps perspective as we're on this DevOps dialogue session, um, you know, one of the things I found in our most recent, um, observability research is we found that 52% of the, of, uh, respondents indicated that they're using six to 15 different tools in observability. Yeah. What, what does that mean?

I mean, there's a lot of stuff out there. You know, I, it's, it's an, it's, I I saw the same or similar report, and I've seen that a number of times. And six seems to be that marker as of the minimum.

And the thing we always think about as it relates to that is that means that there's most likely six different agents, six different collection mechanisms, six diff different data stores, six, six different analysis capabilities that are out there. And the bifurcation and the segmentation and data is just like incredible. And, and that happens with that.

And there might not be all, like, it might not happen, you know, six different times, but there's a tremendous amount of overlap. And the enterprise complexity that is, um, that's created outta that is, is massive. The management overhead, the lack of the ability to get business insights is, uh, is tough.

Yeah. Yeah. No, it, it makes sense.

I used to, uh, kind of joke around when we talk about observability. I said, you know, the observability people really, that their role was basically storage managers, right. They'd store, collect a bunch of information, collect a logs.

When we look at understand this, this telemetry data, you know, how do you understand these logs and break through all this noise? Well, it, it's funny, it's real funny. Real quick that you said that though, because I've, one of the things I say internally a lot is that a lot of times log management is just like French for storage.

Makes sense, right? It's like, it's really, yeah. It's really fancy, elegant in storage, uh, at the end of the day.

Um, but, uh, yeah. You know, so the, as we've embarked on this journey with the attempt to help people optimize and get more value out of their telemetry data, we found, um, this, this was surprising at first and early in our journey, and so obvious now that the fundamental problem is that people don't understand their data. They don't, they don't understand where it's coming from, where it's going to.

And there's some really great standards evolving out there today, like open telemetry mm-Hmm. Or hotel and, uh, super supportive of that in all things. Um, but that doesn't fundamentally under, like, answer the question of do you understand it?

Do you, can you, can you, um, profile it and can you categorize it and can you make decisions on what effectively to do and how to manage that across an entire enterprise? I is it a skill gap issue? Is it a, is it just a persona?

Like is it the wrong people doing the wrong job type of thing? What's, what's why? Yeah.

Good question, actually. So I've, I kind of looked at it from a different angle. I think that, that, um, there, well, I might be a little biased, but I think to some extent it's a, it's a capability problem in the industry.

Um, I think there's plenty of skills, especially in, in this particular space or, or, and, and this group of people, SREs or platform engineers are known for their ability Sure. To pick up whatever you throw out of'em, right? Give 'em, give them, you know, application monitoring, give 'em system monitoring, give 'em traces, give 'em all of it.

They'll, they take everything that gets thrown their way. So I look at it as a capability gap. And I think specifically it's a data engineering problem, but it's a data engineering problem applied to a new type of data that data engineering, your typical data engineering tools don't address today.

Right. Okay. So that makes sense.

So is it, is it the applications that are trying to understand the data? I'm, I'm really trying to understand it, like from the perspective of who from our audience here should be concerned? Who should, who should be thinking about this going?

Well, okay, I'm a DevOps person. I'm a SRE, I'm a platform engineering team. I'm a developer.

I need to create this stuff. I know I have a bunch of insights coming in or some logs coming in Mm-Hmm. I don't know what to do with it.

It's a bunch of information that's sitting there. Yeah. Where do they go from here?

So the question of, uh, you listed a bunch of roles and asked, you know, who should be concerned? And my answer would be Yes. Yes.

Uh, I think there's different sides of the different sides of the problem. On the one hand, um, you've got a number of, uh, DevOps engineers or application developers that are creating blogging capabilities because they need the visibility at a point in time. And that point in time isn't all the time, but it's, when that point in time comes, it's super important.

Right? The other side, you've got a platform engineer or like an SRE team who doesn't understand the application as well, or maybe they don't understand those logs and why they're being created as well, but they have to, they have to manage the operational overhead. Hmm.

So there's a, there's a push and pull in between those two things, right? There's a natural push and pull, and at the, on the other side it's like, what value am I getting from the data and how much is it costing me? But that decision is often made with the platform engineering team that it get at times get made on behalf of the developer or on behalf of the DevOps team.

And so when that incident happens and you want that information, it's not available. And when you have that problem, you know, the systems tend to be down longer than you would want them to be. Right.

Right. Yeah. No, that makes, that makes sense.

The thing that, um, often comes up is there's, you know, in, in this maturity Mm-Hmm. Of call it observability and such. Yeah.

But in the, in the, in the maturity in organizations, um, there are organizations that are not mature at all. Yeah. And there are organizations that are very immature, but the bell curve is somewhere in the middle, right?

Mm-Hmm. And so when you do that alerting and monitoring, and when you have the login, the tracing and that, that occurs, and then there's actionable insights, right? Mm-Hmm.

Um, what does actionable insights mean to smo? Yeah, that's a good question. I then I'm gonna give you a partially a personal answer too.

Like, so I've actually been in monitoring and observability for like 25 years. Okay. Which is kind of fascinating.

I feel like this is like my third or fourth or something. So You're an authority in the space for Sure. Yeah.

Yeah. I, I've know if authority is right, but I definitely have, have been in and around it working with customers, implementing it, and, um, I think the challenge has always, it, the challenge has fundamentally always been a data problem. Those things are mm-Hmm.

Um, but the challenge has been like, what do I instrument and what do I not instrument and do I know the right things to instrument or not instrument? And those are very, um, making those decisions can be, um, can like, be really challenging. You know, there's a lot of, there's a lot of like deep trade offs you have to make.

Am I gonna instrument this system or not? Am I gonna follow this metric or not follow this metric? If I'm gonna look at it from an end user perspective, if I'm doing end user type monitoring, do I actually have the fidelity I need to trace back and solve the problem?

Um, is it a valid user? So we have all these, all these trade-offs in terms of what do you instrument? What's the cost of that instrumentation?

How many tools do I need? And all those things. And so, so back to the question of what is like an actionable insight and what does that mean to me?

Smo, um, I, you know, I, the, we just had a conversation earlier about the response of pipelines. I think that's going to depend on the context. And so you have to have the ability to dynamically change the flow of data and information.

Either you gotta collect everything, which isn't an option, or you have to be able to make decisions of points in time. So there's clearly things that absorbability tools do well today and use some monitoring. All that stuff needs to continue to flow through and give that top level.

But once you start to dive into an incident, I think I believe that you need to be able to dynamically change the, the data that's flowing so you get the actual insight based on the context that you understand about the system and its performance today. Okay. That, that's one ver that's one answer.

And then the, the other side of that too is I actually think that there's a tremendous amount of rich business data that sits in, in, um, these, this, these telemetry data streams. And, um, I think there's a real opportunity in the industry to, to, um, harness that more, to, to harvest that and actually turn that into business context, not just technical context, but turn it into business context that allows us to answer maybe some more critical business questions out of the observability type data. Yeah, that makes sense.

So, but when we think about the, the data sets, right? Mm-Hmm. And we think about, um, anomaly detection.

You, you mentioned like what, what pieces to kind of think about to collect versus if you collect everything versus collecting certain pieces. How do you know what's important? I mean, it's a good question.

Um, like what we've spent a lot of time on is maybe the reverse of that question, which is, how do I know what's not important? That's fair. You know what I mean?

Like, like, so there's certain things that, uh, it's always because with changing context, it's always hard to know what's gonna be important at some future state. And so, um, and so if we look at it from the standpoint of like, what do I know is like factually noise? Like what is something that's just repeating, you know, tens and thousands of millions of times that I can turn that thing into a metric.

So I take something that maybe this really long log line, I can turn it into a single metric, or what can I aggregate where I take many of those and turn it into one? Or what can I, what just repeats that I can truncate every time? And, uh, what are sources that just don't, that maybe just don't matter that don't have any interesting information in 'em?

So, um, I'd have to think about that one a little bit more. Sure. But I think, I think a lot of where our customers are focused is actually looking for things that don't matter, so that you can keep those richer things that may matter.

Yeah. And I, I think from what I heard you say, is it, if we wouldn't be in it if we didn't say it depends. Yeah, Fair enough.

I mean, you know, it really does depend on like, what is your business, what do you care about, what you, what's interesting to your business, right? And, um, you know, when you look at maturity, it always kind of started this conversation with maturity. Mm-Hmm mm-Hmm.

Looking at maturity, that also means application maturity. It does. Right.

And where that lives, where the application was within its own life cycle, right? Mm-Hmm. Um, the thought around the piece that kind of comes to mind as well is the convergence of Mm-Hmm.

Taking this data and taking these, you know, what you do realize is actionable insights, right? Mm-Hmm. And then applying it across other disciplines within the, in the organization such as security.

Yeah. How does that kind of play into mes o's vision? Yeah, I think, I think that, um, well, I, I, I see, as you alluded to, I see mes o as a data company, not more than observability company or a DevOps company, or a security company.

Uh, or even ultimately people ask questions about ai. Of course. Yep.

Um, so I think it's really about applying, um, some of the techniques that we've seen in general business data space to telemetry data. And I think when you do that, which is really focusing on how do I, how do I gain a better understanding of my data? How do I optimize my data and how do I respond to the changing context that, in that you were, we're equally solving observability challenges as well as security challenges, uh, and ultimately AI challenges too, I Believe.

So where do we see me SMO going? I think we'll stay focused on the data. I think it's really important we stay focused on the data, um, and actionable insights outta that data.

So I do believe that there are many things that we can do based on our preferred access to the data as the streams come through. Mm-Hmm. And, you know, within some, you know, reasonable buffers and windows, um, that give us the ability to gain, to provide insights into very unique things across observability and cybersecurity.

Um, and then I also think that there's some great things that will always happen at the destination, and we don't, we don't need to do that. Um, but what we want to do is we want to give you those insights sooner, cheaper, um, uh, better, of course. And, um, and help people really own their telemetry data.

Mm. So at the end of the day, they feel this responsibility, that they understand it, they're in control of it, they can send it to the right destinations, and they can provide it to the right consumers in their environment. That, that makes a lot of sense.

I like, I like where you were going with that. Like, it's customer choice, right? Yeah.

You know, where they want to go. Um, you know, there's this element of, uh, you know, the tool sets the tech stack and, and where it's going. Um, I like the convergence of where you're going, the messaging.

Um, but when, you know, like, one of the things I, I was also thinking about, uh, is in our briefing that we had Mm-Hmm. We, we talk, we were talking about your use cases Mm-Hmm. Can you, can you double click down a little bit on the use cases and where, like somebody wants to get started right away?

What would you recommend they do? Yeah. I, I think the right, the, the, the two like most tangible things to do right away is to profile your data so that you understand what it is.

Mm-Hmm. And then that's a very easy means to drive, um, cost reduction, which is basically increasing the fidelity of the data. So there's more, a better single to noise ratio at the end of the day.

And the second is, while you're doing that profiling, make sure that you've, you handle all the compliance. That's like the first step. That's the first step in the maturity curve of like, I want to get control.

Once you do that, then the next step we see people take is like, start to think about what is, what is the latent value? What are those actual insights I want to get out of this data? Which in many cases is a very specific question to that enterprise.

But then, then you can start to take some of the aggregation, um, and business insight type type, um, capabilities that we have to present that data, whether it be in an observability system or any downstream system. So we see people doing things like grabbing order information, grabbing, like ticket check-ins, grabbing failure, um, presence or absence alerts, and all those things can inform a tremendous amount of business context that all sits inside of this, um, this telemetry data. No, it's very, very true.

And I think, um, it's, it's an evolution. Yeah, right? It's going to evolve.

Businesses and organizations are gonna be dynamic and fluid and try to figure out how they're going to use this. Um, you know, I know we're kind of coming up on our time here. What would you recommend what, uh, to the audience here, if they wanted to get started with mes?

How do, how do they work with you? What's the best way to kind of move forward and get Started? Yeah, great.

Thanks for asking. Yeah, of course. Yeah.

com, um, start a trial, reach out, contact us. One of the things, one of the most valuable things we can do for you is, you know, no charge or anything like that, but just like, help you profile your data, which allows you to easily tap into a source, get a full, um, kind of inventory of what's coming through and, and Mm-Hmm. And it will make recommendations to you in terms of what reductions you can expect and what, what insights you can gain from that data.

Well, I mean, it sounds like there's this really just the tip of the iceberg, right? There's just a lot to consider, A lot to think about here. Yeah.

Um, Tucker, I wanna thank you for your time today. I wanna thank you for your insights and your perspective. Yeah.

Uh, I also wanna thank the audience for attending our session today. com. Yeah.

Right. com to learn more about our research and our information. Thank you, and have a great day.