Valiantys’ Philip Heijkoop on Managing Hidden Costs in Multicloud Strategies

Hey guys, thanks from the throw, we're here with Phil Haiku, who's head of developer experience for valis, and we're talking about multi-cloud and well, there's some hidden costs in there and some might even call them attacks. Bill, welcome the show. Thanks for having me.

Alright, Walk us through what goes on here, because everybody thinks that maybe they want multiple clouds and they wanna be able to put workloads everywhere, and they think that there's a lot of flexibility and all of that, but I think every time I add a new platform, I gotta add new folks with different levels of expertise. So maybe the costs outweigh the benefits. Phil, what's your take on what's going on here?

I think it's, uh, almost always gonna be a trade off question, right? And that's gonna differ contextually with everybody. But you're right.

I think the, uh, while conceptually a lot of the cloud, multi-cloud, especially the larger ones, will offer the same things. Language will be a little different. The specifics will be a little different around each one.

So you're gonna have local changes in tradition or trade offs that you wanna make. Uh, and then you'll have people who obviously have built their careers around one or the other. And then at the end of the day, I think really it's gonna be around what it is, right for you from business context.

Do you want to have very strong separation of concerns? Do you want to have very strong consolidation of your finances so you can make sure you're not paying too much for what you're doing? Um, but it all becomes just a question of like, here's the trade off.

Let's make sure we are making them deliberately. Because if you don't, you're still gonna end up paying it, but you just don't know what the trade off is. And then that might bite you later.

Is it really possible to have kinda one IT team that is master of multiple platforms or do they wind up just being kind of jacks of all trades as it were? And ultimately I wind up having to get a bunch of different specialists anyway. Uh, at a certain size it's gonna be impossible to really have one central IT team that manages everything and does all that, but that at that size, you're gonna have consultants coming every now and then for specialized things anyway.

Um, so I don't think that's also the goal, to be honest with you. I think the the thing that you do want though is making sure you have insight and some level of governance on where you're, you're having your assets deployed and where you're spending your money. Um, and that is very much possible, but like I said, it, as long as you're deliberate about what you're choosing, you don't have to be contained to one service or one company, but you just wanna make sure you're doing it with understanding the reasons and the constraints, the tradeoffs.

Mm-hmm. How do I get that visibility? Am I kind of deploying some sort of platform to drive that or am I pulling together all these different dashboards from all these different cloud providers into some single portal?

What's the magic there? Well, I don't know if there's gonna be a lot of magic involved, but you're probably doing a little bit of column, little bit of column B. So each one will their own billing.

I, um, and they'll never be quite perfect for what you want out of it. So you're always gonna have some form of observ, uh, observ observability platform or tool on top of that. Um, and um, attached to that, you also wanna make sure that you can segment it the right way.

So obviously your total dashboard, which just tells you whatever your dollar figure is and how many, if you're in AWS, right? EC2 instances you have and everything else isn't gonna give you the right contextual information that you have, you need to be able to pair that with which teams are using it, how often are they revenue generating, are they experimental, all that stuff. So if you're always gonna end up needing to pipe that to a BI tool or maybe even a, a data science team who kind of slices that up in their data lake.

Um, but again, a lot of it will, will end up being a, how much that visibility do you need? How much of that are you pushing down? Um, we see a lot of companies have a lot of success with leaving the details to the either regional business owners or, or whatever terminology they have, right?

You use GMs, um, and they just own the budget for a lot of that and they'll just come in overhead and say like, all right, we've made a deal with this company for this, but we'll take on internal IT ops centrally. So it's possible, but again, a lot of it will depend in context and it's always gonna be a combination of tools because depending on the question you have, because you're going there to answer a question, you're gonna have to, you know, heaven forbid export to Excel and write your own pivot table. But most of the time it'll be a combination of like, all right, I've got this list here.

This is what's happening. Put 'em both in your own working memory and then figure out kind of what's going on Is in your sense that some organizations are truly multi-cloud or is it more likely they have one cloud that's kind of dominant and then they have a couple of smaller clouds running because they picked them up through some sort of merger and acquisition or somebody decided that there was a particular workload that really had a run there. But, uh, for the most part they are standardized on one larger cloud and then have a bunch of smaller clouds.

There is almost always gonna be one dominant one and um, there's always kind of a, there's A and B and then everybody else is kind of much smaller in my experience, right? So you've got for companies that are software companies, right, in the traditional andreesen software leading the world terminology, like all their production work is going on a cloud somewhere or hybrid cloud, right? They'll have some data center somewhere.

That's the biggest one and that's also your most important one. 'cause frankly, like that's how you make money. Um, and so that will always be consolidated as a general rule.

You might have some separate products, like you said from m and a and stuff like that that live somewhere else, but the effort post m and a is always gonna be some form of consolidation, if that's possible. Um, and then you'll have your internal ops stuff sometimes on the same cloud, sometimes separated by design. Uh, it really kind of different, like it's kind of 50 50 in my experience, but I, I don't have a broaden up view to really give that authoritative number.

Um, and then everything else just kind of lives where it happens to live, right? The, the usual tool sprawl that just over time or somebody got a corporate credit card, they need to spin something up real quick and then well shoot, we can't fill that process yet and it's not worth migrating. So we'll just keep that over there.

Like it, it ends up in this kind of like there was an element of deliberateness here, but there's also some organic growth around the edges that you definitely will always have a little bit of like it's unavoidable. Mm-hmm. Of course you can't walk down the street these days without somebody talking about their great new AI thing.

Um, and I wonder, you know, mighty become easier to centralize the management of clouds in the age of AI when I have my small army of AI agents that have been trained to do things. I mean, is that a reasonable expectation or is that still fanciful thinking? There is probably some truth to that.

I think there's, but there's two ways to look at that really. There's where the AI agents are running, right? That's not a trivial cost in most organizations, right?

So you're gonna end up having the stuff that you're hosting yourself will live in some cloud instance and that'll be a non-trivial bill. Um, this is also what you wanna do. Um, migrations traditionally have been non-trivial in my experience, even though conceptually very similar things have been offered.

Uh, AI agents will probably make it a little easier, but you're still end up gonna have, like, you're still gonna end up having that constant conversation around is the opportunity cost of focusing on this and the potential disruption worth doing? And AI agents may change the ratio of some of that, but it's always gonna be, again, a trade off of like, is this the best production or deployment of our resources? And oftentimes in my experience, like not really until it gets to a certain size and then it becomes too complex to really want to leave it outta hand because you need an accountability aspect of it.

And I've, in my experience, while a lot of the agents show a lot of promise in what they can do, you can't hold 'em accountable for what they're supposed to do yet you still need humans for that. Mm-hmm. Now we also talked briefly about finops and I'd love to get your insight about this, but, um, are people setting up like a finops office where there's the finance team and a, and a couple of IT folks and they're kind of doing that as a center of excellence and or maybe the better part of valor is just to put some sort of cost metric in front of the SREs and DevOps team so they know how much things actually cost and they'll just do the right thing accordingly?

There's a bit of truth in the first one. I think people start with that. Like, uh, we know like over the last say two years with the macroeconomics of what it is, cost has become very much top of mind and people have looked at it that way.

Finops has become the populated to manage that. Um, historically finops sits, I think right now as the third party in what was also happening with quality and security in terms of like, it sat all the way at the end of the decision making process in terms of the chain of what happened. But the goal is to move it as left as possible, right?

Like you want security and QA to be things that are in the planning stage and the development stage, not at the deployment stage. And similarly with finops, like you actually want that to become as early a like principled concept or thing that you discuss with your product teams. And so I think you're right, like the, you want that to be, I dunno if I put it in the SREs hands to be honest with you, that's just probably gonna lead to a lot of containers getting nuked.

But I think that if, if you move that to a product piece that yeah, the end goal would be definitely for them to make that decision, right? They'll own a local budget, they can advocate from a business case perspective like, we need this money for these services and this, they know what it costs because they'll have the local bill and it's a lot easier and, and tighter. But I think that that's more the end state they're working towards.

Most organizations are really not there yet. They're really in that first area of like, we're doing some initial IT financial management. We know what our bill is, we need to bring this down.

How do we do this? Well, it starts with understanding what's running that has to be done centrally. 'cause again, a lot of these budgets have historically been managed centrally, and then you can start to push responsibility out kind of incrementally.

But I think we're on an evolution or a path there and we know where it's gonna be, but it's gonna take a while to get that culture change going. Of course, you are in charge of the developer experience. Um, and I have to ask the question, do we give the developers too much control over the cloud?

It seems like historically we have decided that speed was everything and we let these folks provision the infrastructure and then we're surprised when there are misconfigurations and mistakes get made and uh, data gets exfiltrated. But do we need to find some way to maybe more centrally manage that? Does that look like some sort of newfangled platform engineering team, or how do should we be thinking about all this?

That's a really good question. I think there's an element of risk that you have to accept on that front though. Um, as someone who's blown up our Google maps, API bill once before, like, I know what that's like, you feel bad never happens again.

Um, so there's a like learning experience cost to it and you learn from a retro perspective how to manage these things kind of proactively as well. Um, I think that because you want developers who are truly informed, like if you were having that actual agile team, they should be responsible and accountable for that. Um, historically in the zero interest rate environment, that was a lot easier 'cause cash was cheap.

Um, but now like, yeah, I think that becomes kind of a, a new muscle they have to learn. Um, there probably needs to be some central oversight. Um, but it becomes a, the typical balance of enablement and governance.

I think the enablement sits at the developer side. Governance needs to be something centrally done. And if you kind of balance that, right, this is not something that becomes overly complicated to solve.

I think the developers can still have enough freedom to do their job while not blowing up anybody's budget centrally most of the time. So what's that one thing you see folks doing that just makes you shake your head these days and go, folks? I think we need to be a little bit smarter than that.

Um, one central rule is often the case that that never works. Um, so they'll have like a central rule for example, that, um, just as an example, right? Containers need to be spun down at 5:00 PM and they'll spun back up at 8:00 PM or, you know, whatever the number ends up being right at 8:00 AM Sorry.

Um, that in theory is a great idea, but if you make that a policy across different time zones never works, uh, if you make that a policy, but you have production requirements or that are running, or you have jobs that have been all allocated over and that you need to run over the weekend, but you took down one of the microservice that they rely on, like I see that oversimplification type of thing is, is definitely something that people are, are struggling with. So they'll have a central policy, we can't do more than this, or here's our limit, and they'll cap things on a budget perspective. Um, I saw a customer, for example, they had a, uh, a hard cap on the budget for their serverless processes, but that's great, but if that's in the middle of your customer environment and they were gonna blow up their bill, but they're gonna pay you anyway, but you cut them off, now you have an unhappy customer and you take them down your services.

So I, I think the overly simplistic approach that we've seen that a lot of customers take sometimes like that has a tendency to, to not work. And it's easier to have guidance rules and then make it something that, all right, somebody exceeded this guidance, let's figure out if there's a good reason for it. Amend the rule then to have these hard and fast cutoffs.

'cause I've seen those come in and and they never work. Like they'll keep the bill down, but you're paying for it later. Mm-hmm.

Of course there's an old joke that says, you know, in the future of the data center is one person and a dog, and the dog is there to keep them from touching anything. Um, what is the level of automation that we should actually be maybe working towards? 'cause I feel like we'll never get to total automation yet.

I also feel like we seem to be erring too far these days on manual processes. I think the goal will always be a little bit more than what we have now from an automation perspective. Um, there's always, but it's, it's interesting how the same rule applies.

There's always an exception. Therefore, whatever rule we have codified doesn't work because the context, whether it's the business context or the goal for the, um, the product or like the circumstances around our, you know, infrastructure has changed a little bit and you have to change that, tweak things a little bit. Um, there there's that flexibility that you're gonna trade off.

But I think it's also important that, uh, maybe I'm old fashioned in that sense, but the fact that you can and have to understand the manual process a little bit to get it up and running is an important thing to have. Because at some point, if you abstract everything away and no one ever touches it, when something breaks, it takes a lot longer to debug and figure out. Um, so I, I I, I, I, that's not a reason to do it.

I think that this is kind of a side benefit for it, but yeah, you're always gonna try and automate a little bit more. But yeah, it's, it's kind of unavoidable that you're always gonna have some manual processes and that's okay. I don't think that the, the tax on that is too steep.

All right, folks, will, you're heard it here. Hey, no matter how much it advances, you still need to get your hands dirty from time to time. Hey Phil, thanks for being on the show.

Thanks, Mike. All right. And thank you all for watching the latest episode.

Back to you guys in the studio.