ROI of Platform Engineering – Lambros Charissis, Wise

This is Techstrong TV With a great pleasure today being joined by Lambrose Caris. Lambrose is Principal product Manager with Wise welcome, Lambrose. Thank you.

Thank you, Mitch. Good to have you here. We're gonna talk about, uh, a conversation around, uh, platform engineering and recent talk that you gave.

But first, introduce yourself. Tell us a little bit about you and, uh, what you do at Weis. Yeah.

Um, sure. So, yeah, my name is Lambrose, uh, working at Weis. Um, I've been in the platform as a product domain, let's say as a PM for the last five, six years now.

I think I'm working, uh, actually in Seattle, in the us um, for a few years at Mercedes, um, in the team that builds infrastructure, uh, the cloud and cloud migration. And now IT wise, um, in the, um, yeah, develop experience space. Basically I'm a PM in the platform engineering space.

Look over a few teams. Um, things like SRE reliability, um, observability, right? So developer tooling, but also holistically over the develop experience, basically advice, how to measure it, um, how to improve it.

And the VPM crew is also pretty involved in, you know, bringing a product mindset, uh, into platform engineering, basically advice. Cause it's still, it's a journey for every company, right? Mm-hmm.

Also part our job. Well, it's interesting because, um, you know, talking about platform engineering, but other disciplines as well. It's bringing a product mindset to it.

It's very much kind of an attitude, right? Of people are trying to take. And if you've got a product experience, that's one thing.

If you're not so versed in it, that's also something you've gotta learn as well. Um, I'm curious about, so your product experience, how is that helping you with now doing, creating platform engineering as a product? Um, so the funny thing is that actually my first, my, my first job in product wasn't platform, right?

So I've never been a PM somewhere else, basically, right? Mm-hmm. Um, so I don't think you need a PM platform.

They have a lot of companies to do a great job without PMs as well, right? But that really helps creates this healthy balance between, like the PM generally is more like, um, customer focused, right? Supports in direction of the customer.

The engineer lead is more like team focused. Once you have the team happy, right? And you have engineers like leads or staff engineers, that more technical focus.

So this creates a healthy balance between all of, all of the involved parties, basically. Um, but yeah, I mean, being a product manager and platform basically is the same as anywhere else, right? It's not, not a big difference.

Use the same methodology is the same thinking. Um, obviously you have a smaller customer base, right? You can also approach so much easier, easier, right?

They're actually your colleagues. That's, that's a big difference as well. And yeah, I mean, you, you can't, let's say, do things like AB testing because there's not enough people, right?

Mm-hmm. So that as well. But overall, I think, um, I mean the same thing, supply for platform as a product, really very interesting.

Yeah. But I know you gave a talk at, um, at that platform con talking about through the roi, right? We, it's, it's, I'm sure you've experienced this too, right?

There's always these trends happening in our industries. Like, we need to do DevOps, we need to do this, we need to do sre, we need to do platform engineering. And, and a lot of 'em are very legit.

Not saying they aren't, but it's also easy to kind of get caught into the we need to, as opposed to, well, let's look at this and then figure out why we want to do this and what we want to get out of it, and then really go for it and get some great things accomplished, showing some results, R o i, which I think is what your talk was about. So I'm real curious, tell us about your talk and, uh, maybe how, how it was received is, are people looking for, like, how do I justify this? Or how do I make sure I'm on the right track to deliver value?

Definitely, yeah. And I mean, yeah, being a PM right? Means that you ask those questions, right?

Is this actually important why we're doing this? Right? This is kind of our job.

Um, so it all started with a blog post actually on Medium, on the wise blog around KPIs in block engineering a space. And this is how I got, let's say, I know this covered in, so when I got invited to talk about this problem, right? Or this, this domain.

So, um, the blog post got incredible feedback, like still today, I get people asking me questions about it and so on. And, um, there've been a few talks from around that. And the one platform con was specifically about how you can translate really like work, uh, platform work from, um, outcome, like, almost like platform outcome into business outcome, right?

Mm-hmm. That's not that trivial. Let's say for instance, oh, no, platform, platform.

Maybe things in things in, um, in incidents or in, in lead time, right? Those are outcome metrics. You even have input such as maybe, um, build time or uh, time to gross incidents, right?

Those are input metrics, how you can get from there to, to business outcome. That's kind of the talk, right? This is the, the gist of it.

Like how can you talk the business language? How can you kind of quantify it and compare initiatives, right? They don't really, really comparable.

How can you get to one currency and compare them, right? And then also, of course, uh, how can you, um, decide on investments based on this? Because if, you know, um, how much, like how high the rise is platform, you can then decide, well, should I invest here?

Invest there, right? It's kind of really an enabler for a lot of other things as well. Yeah.

It seems like one of the complexities often is with, whether it's platform engineering or anything, trying to tie it to business outcomes, is sometimes there's a direct connection I can see by doing this thing and made the releases faster, and that lets us get this product out the door sooner. Really nice. When it's that clear, oftentimes it's that contributed along with eight other things that helped us get there.

And that's, that's usually where that sort of having some methodology at least lends credibility to your analysis of why this is valid as opposed to, well, it just helped To be, to be honest, right? I mean, um, it's hard. You can't apply this to every, everything, right?

It's, it's a, a tool to help you make decisions better, right? Um, it doesn't work always, right? And I'm not using it, let's say, as a, as a, uh, kind of mandatory thing you have to do for every work, uh, item.

No, right? It's more like a, a, a guide that helps you to make decisions, uh, longer term, like more strategic, but, uh, most tactical work, right? Tactical, um, um, you know, questions as well.

So this applies for both, right? For instance, you could say, should we, um, or what is the RI of investing in a new, um, platform for cd, right? That they continuously can make estimates, again, try to estimate the impact it has on lead time, on failure rate and so on, right?

And could quantify that, but also more short term, should we invest in, in, um, reducing build time should re invest in mt d mttr reduction, right? Um, next quarter. So it helps you really, uh, in both, in both respects, right?

Strategically and technically as well. Yeah. Interesting.

Well, to to say a bit more about your talk, um, what maybe what some of these things are, other things you might have covered in the talk there. Yeah, yeah, for sure. So basically the way we, um, we work at, at wise platforms, so we have platform mission, and we have, we have, um, four we call North Star KPIs, which are, uh, productivity, stability, efficiency, and risk, right?

Mm-hmm. And this, those are the four levers that we think we can influence to then basically have an impact on the wise mission. That's, that's what we believe basically, right?

And, um, yeah, those four are the, are the outcome KPIs. Now, there are different models on how to break those down, right? Let's say productivity, they're well defined frameworks out there, right?

What this means, um, there's Dora, there's space, right? Space framework, there's, um, in the new DevX paper research few weeks ago, right? There are a lot of frameworks that help you YouTube to understand, to create a K P I tree almost, right?

Of this productivity, um, domain. Same thing for stability, same thing, efficiency and for risk, right? And the talk really goes through each of them and explains how you can break those down and make them quantifiable, right?

But the real question is now how do we get from, uh, let's say I improve lead time to 5% to money, right? That is kind of the, this is the key question, and this is, I guess where my talk is still different to others, I think, is this the only one that really gets like into this aspect, right? Um, so as an example, right?

Um, for lead time. So if we say we have an initiative that's going to reach reduce lead time by, um, 5% plus engineering, right? On average, um, how can we quantify that in, in, let's say in pounds?

Well, what we do is we basically have to understand now what is the impact of engineering advice per quarter, right? So what is the additional revenue or engineers generate with their work in a quarter, right? Luckily, um, why is this very data driven with this generally, um, wise across the company uses a, a method to size opportunities by predicted revenue and also cross port volume, right?

So we have those numbers, which is very, very good, right? In that form. So we do know, um, how much revenue, if you want to write an engineered wise, generates a quarter on average, right?

Of course, you gotta look, you look back, it's gonna be a little, you know, it's won't be, it won't be, uh, perfect. Cause you kind look back, It's never perfect. Yeah, I had to exactly Right.

Had the same thought, but having a framework, having a process that you can fit into yeah. Is a huge plus, right? You don't have to reinvent the wheel, like fit in what you're doing, your KPIs or OKRs Yeah.

Or whatever methods tie it into the frameworks you're using if you can. Exactly. Right.

And it's not what can be used to then do the math, right? So we know the value, um, or the revenue that we generate the quarter and we can say, okay, now we have a improvement of 10% lead time, so we can now do the math and say, this is what we generated with this, right? Additional, um, we kind of push to open it forward.

But I always see, I think this is wrong in my mind. Uh, people say, well, um, we say develop powers, right? Whatever, $10 powers per quarter, that's a cost saving.

It's not really, because you don't fire those people, right? Mm-hmm. You use them to push it open forward.

That's A big difference. Reduce cost, right? You may have freed up resources maybe, but you're not measuring it truly.

Right? Exactly. Right.

This is a difference I think that we, uh, we have, and this is why we use this, we try to quantify the impact of moving the roadmap forward, right? This is exactly the big difference. And the same thing, um, again, I mean this is not going into detail, but stability, efficiency, risk, they're just models that we created to kind of translate, um, translate those k p i changes into pond, right?

Uh, really. Mm-hmm. Excellent.

What were, what were some of the questions or maybe even conversations you might have had after your, after you're around this topic with other folks, you're saying it's a virtual conference, you're not all in one place. Yeah. Yeah, yeah.

Um, so there are a few other talks, um, around ri um, and how quantify impact. Um, so yeah, I am the only one that that kind of goes to this quantify in pounds. So if you people ask about that, and then I got referred, you got referred to me, right?

Watch this guys talk basically, right? Mm-hmm. Mm-hmm.

Uh, one big question that I, it's hard to answer, right? Um, is the lead time example, right? How do you get this number of, um, impact engineer, right?

That's really hard to get if you don't have the support from analysts in products, right? So again, why is, is, uh, very data driven with their planning process. So they really, um, use those numbers to prioritize their work and we can, we use them for our purpose, right?

But not everybody does that, right? This is kind of one question I got and I told them, well, um, you gotta, they were like estimated somehow, right? Similar for stability.

How do you know, let's say if we, um, reduce incidents or if we improve our availability by one minute per quarter, like how to quantify that? Well, what we do, for instance, what we do is we think of immediate impact. So let's say a minute of downtime means, um, volume loss people can use our products, right?

They may even go somewhere else. Who knows, right? And there's alternative.

So this immediate loss of volume, um, but there's also long-term loss of, um, customer reputation, right? So we usually see a spike in customer contacts with every incident. Mm-hmm.

Again, our analysts are, are great. They quantify that. So we have a number, we can tell you how much money it costs, how much money one customer contact costs wise, right?

So speaking again, we can say if we prevent one incident per quarter, say SF one incident, this means there's that much immediate revenue loss prevented. Plus we also, um, reduce customer contacts by 10,000, right? Can be ified in that regard.

And this gives us total impact of that work, right? So it is a lot of analyst work. By the way.

We probably one of the only companies that has an analyst in platform. So we have analysts that works only for platform, which is also, I think, pretty unique. I haven't seen this anywhere else.

Um, yeah, That seems like a nice thing, a nice luxury to have. Yeah. It seems like a good, uh, at least one of the other options is sit down with a financial person, right?

And say, let's do a sanity check on my logic on my Yeah, yeah. Stretching things too far to make a connection here. Um, there's also some other things that I was thinking about I wanted to ask you about.

You know, in the s r E world, there's error budgets, right? How much I know it was around downtime and things like that is maybe a concept like around an air budget or, uh, when you're talking about improving developer productivity, time wasters, things like that, you can identify that here's waste on our system by doing these things, that by doing platform engineering project A reduced the waste that we have in the process this much, and that's performed in times and that's blah, blah, blah. So I can do some calculation of, you know, there is a savings.

It may not be a revenue improvement, but, um, to try to help quantify some of the non-revenue things. Yeah. Makes sense.

That makes sense. Um, and I guess the error budget, like how do you pick this, right? Um, that's the question as well.

This also technically comes to how much impact or loss is acceptable, right? Mm-hmm. This then connects to the risk management framework of the company as well.

So we also work a lot with risk management and try to, um, decide what is acceptable risk, right? And this is again, something that we also try to quantify this a fourth pillar of, if you remember the platform engineering pillars mm-hmm. As a risk.

Um, and we do, uh, quality risk management, check in on all risks that we own and all the teams own, and then we try to understand if it reduces risk from, from here to here, right? What's the impact of this in pounds? How is this compared, let's say, to an initiative that's, um, post productivity, right?

Again, everything has to be quantified and everything. I mean, it's, it's, um, again, approximation, but it really helps you to compare initiatives such as error, budget, risk reduction, right? Or, or other other work.

Well, fantastic. Um, sounds like a fan. Great talk and received.

Well, if other people referring people to you to say, we'll go talk to this person about talk, that's a good sign. That's, you know, that's, that's, you know, a very good compliment to, uh, the talk that you did. So congrats on, on some great work.

Thank you. Thank you. And, Uh, love to keep up with this, Sonya.

So I would love to have you back on Textron tv. We also do a lot of panels and things talking about platform engineering and ROI and transformation and all of those things. So we're all, we're all on this journey together, right?

So we have to figure out how to learn and help each other because everybody's got some good ideas that can help us. So hope to have you back. I wanna thank as well the platform organizers.

I mean, they did a great job. I think it's the first platform engineering community that's, that has been generated is I think really amazing, the one Slack channel, right? To just talk about stuff and ask questions and yeah, It's really taken off.

It's very, very well received, hasn't it? That's, that says, says a lot about it right there. Yeah, yeah, definitely.

And yeah, thank you for overall I definitely gonna gonna come back if you invite me again. I'm, I'm, I'm gonna be there. Yeah.

Fantastic. Well, lamb, Rosas, uh, Caris, who is, uh, principal Product Management with Wise, thank you very much for sharing with us both about your talk and kind of beyond and look forward to talking with you some more. So come back soon.

Thanks so much. Thank you.