DevOps and Developer Divides Causing High Cloud Costs – Jon Osborn, Ascend.io

This is Techstrong tv. Hey guys, thanks for the throw. io, and we're talking about the cost of cloud computing and how we're all a lot more sensitive to this than this unstable economy of ours at the moment.

John, welcome to show. Hey, Mike. Uh, thanks for having me on.

I really appreciate it. And, uh, I love talking about cost of compute. It's like one of my favorite topics.

Awesome. Arguably we, during the good times, were basically consuming cloud resources, much like drunken sailors at a open bar at a wedding, and nobody was really Oh yeah. The bill.

And now the question I have for you is suddenly everybody's kind of looking at these bills and they're kind of surprised how spiky they are and they wanna reduce the cost of cloud computing. Are there smart ways of doing that and dumb ways of doing that? What do you think?

Oh, yeah, no, I think, uh, I think to some extent the, the issue is like a hell award world problem. You start and you have a small amount of data and it all works fine, but you didn't have to worry about partitioning or like egress costs or any sort of that kind of stuff. And then your data all of a sudden is really big and it's growing and now your infrastructure and your software can't support, um, you know, maintaining a, a stable cost in your cloud and your cloud costs just continue to go up and escalate exponentially in a lot of cases for sure.

Um, so I think there's definitely some strategies we can talk about to like save costs, even if you, uh, build it yourself and then there's a bunch of options out there that, um, sort of outta the box. You get a whole bunch of things that you would love to engineer on your own, but you don't have the time or the skill to do so. Um, so happy to talk through a lot of those options.

Do people have a good understanding of what types of workloads to run where and what types of cloud instances and for that matter, what type of workloads? Maybe I shouldn't be running the cloud at all. You know, one of the nice things about an on-prem environment is that it's naturally capped.

Like you only have so much compute and you only have so much memory, so you can only do so much work. Um, the cloud isn't that way. It'll obviously expand.

If you give it no constraints, it's gonna get really big and it's gonna be very costly. So, so I think there's definitely strategies that, uh, teams should use in the cloud, um, to do, uh, cost controls, right? And so one of those strategies is you gotta measure your performance and costs on a regular basis, and not like once a quarter.

It, it should be probably weekly or daily or even hourly. And really understand the patterns that your business, um, put, uh, pressures into your, um, data pipelines so that you're matching that performance and cost performance up with, with your business performance. So you can realistically say like, Hey, if uh, if my data cost is rising and my business revenue is not, that's disjointed and we should, we should think about lowering our costs because it's not scaling the way our business is scaling, it's scaling faster, which is bad.

Um, and, and I think the other, um, sort of tenant that, uh, gets missed a lot is that something we think about doing compute in a single place like aws, but lots of companies have clickstream data over maybe in gcp. And so the tendency as well, let's replicate it over into a different cloud so that we can process it there. I think there's a lot of debate, uh, or frankly concrete savings you can have about doing compute in its natural location and then doing, uh, making your pipeline span multiple clouds, uh, in really efficient ways.

That's a, uh, amazing way to save, uh, egress costs, but also centralize the compute closest to your data. So you, so it's faster and less expensive. Yeah, for sure.

Uh, one of the things at Ascend that we, uh, recently just released like a couple weeks ago is multi-cloud data flows. So you can have your cake in it too. You can do processing in, for example, in GCP against BigQuery, and you can take the aggregate result, which is oftentimes much smaller than your transactional data and move that over maybe to Snowflake and do some machine learning in, um, uh, AWS or something like that.

So, uh, there's strategies, uh, that are now available for you out of the box that, uh, are very easy to adopt and, um, increase your efficiencies to the point where maybe it pays for tools that you couldn't previously afford. You think some folks are a little intimidated by the whole thing and they see that their workloads are fairly dynamic and then as a result they're trying to close the proverbial barn door at the end of the month when the bills already arrived. So how do I get in front of all this stuff, right?

Yeah, I think having an excellent observability platform is key, right? Even down to the point where, um, I know I'm biased sort of sent of course, but we can do pipeline per pipeline cost analysis so you understand exactly what each pipeline is costing you, and then you can build a profile of that cost over time. And if you have a chargeback model or you have some model where you need to control costs in certain pipelines differently than others, now you have this level of granularity available to you that wasn't previously available.

And you have the data, like the visualization of how much I'm spending and how it correlates to my data that becomes available to you and now you can control it. So anything, if you're not measuring it, you can't control it. So a lot of times costs control because nobody's looking at it, nobody's measuring it.

Um, uh, with the level of granularity now that you can get on, on workflow, workload management, uh, it's possible to control it at an individual pipeline basis. Alright, back in the day day we had a thing called capacity planning. Is that what we're really talking about in the cloud?

Or we hear the term finops? What's the difference? Um, yeah, it's so capa I'm gonna age myself, but you know, capacity planning's important on-prem cause you gotta order those servers six months ahead of time before you need 'em, right?

So that's important. The cloud obviously takes that away, but, um, the new term is finops, right? This financial operation model.

And I think, um, it's true that you, you should understand how much it it's costing to process data. Um, you know, run machine learning models like these costs are important because they need to correlate to your business. I mean, if you're making certain amounts of revenue because you're producing these data products, you should understand what the cost of to produce that product is, and you should control that cost so that it stays within whatever range your business partners need it to be to, to maintain profitability.

So, um, spin ops is really around, I think two key topics. One is understanding how much it's actually costing me to produce this data product. And the second one is, how do I distribute the workloads into the platforms that provide me the best outcome?

Whether that outcome is cheap, like less, spend less money, or the outcome is performance cause they need more, more speed, or the outcome is maybe, um, higher quality or distribution of data or something, right? So pick your metric. Spin ops is really focused on costs, obviously, but uh, how you actually control costs, uh, might be different depending on your scenario.

Do we need to maybe put some guardrails and controls in place for developers? Cause we have been allowing them to use infrastructure as code tools to just provision stuff whenever they want and maybe mm-hmm. That requires a little more adult supervision.

Yeah, you know, that's a really great point because I think, uh, to some extent we have these amazing compute platforms now, particularly in data. So you look at, like Databricks for example, it's a, it's an amazing, um, spark platform. Uh, but maybe not every single workload should hit Databricks.

Maybe some of the playground sort of, uh, messing around workloads should be done in Spark. And so the question is, is how do you make those transitions and how do you keep those workloads in the most cost effective platform without having to do a bunch of engineering to move them around? Right?

And that's where, uh, data platform, a data platform like Ascend can really help. You can focus on code and forget about, uh, the details of where it's running and leave that to the thin ops team or the, or the, um, operations, uh, management team to decide, uh, where this workload should run versus, uh, developers making their own choices and spinning their own stuff up. Let's take those controls and uh, move them more toward our management responsibility.

It's very similar to what Kubernetes did for applications, uh, in general. Um, you know, they don't get to say how many computers it uses or whatever. The administrators figure all that out.

You just deploy your code and it runs Data's data, uh, is moving in that direction also. Who's driving this conversation? Is it the finance team or is it the C-level execs in the IT department who stands up basically and says, you know, this must change.

So I have lots of conversations with C-Suite and that's obviously very financially biased. Like they wanna know like, how much does this stuff cost? Uh, I don't think developers really are sensitive to that cuz they're not rewarded positively or negatively for, uh, spending money.

They're, they're just trying to execute the business logic that people are giving them. I think where, uh, where the, the pressure is to do these types of works as sort of like the VP level, uh, management where, uh, if the budget is too much compute, then they're, they're starting to like shift some of that money away from, uh, potentially consultants or internal FTEs and stuff. So I think there's a, this middle management pressure to make sure that that, uh, ratio of how much we're spending in cloud versus how much work we're getting done with humans is, is, uh, set up for success within their business scenario.

So I see that pressure like middle out in, in organizations frankly. Do you think that someday maybe all this artificial intelligence stuff that we're reading about will come and save us from ourselves? I don't, I dunno about that.

I do know that, um, the security model for that stuff is a little up in the air at the moment. So, so, uh, using GPT for sensitive data is probably not a good idea at the moment. However, I think there's opportunities for these types of tools to then help us, uh, provide recommendations or even automation for how to move workloads around.

So for example, if you have a workload that's running maybe in a traditional, uh, environment like, uh, snowflake relational database, but it should be run more in a spark environment, we should be able to use these tools to help us as humans figure out like, what's the right environment for this workload and for this, uh, sort of pattern of execution. And even come to the point of like estimating, like, how much would you save if you moved it from this vendor to that vendor? When you get that level of abstraction, um, you know, you're able to then move workloads in a real way across these planes and actually affect your, your bill in a, in a positive direction.

And instead of guessing, saying, oh, this platform's more expensive, so let's move to this other one. And it turns out it didn't help, well, it would've been nice to know that ahead of time. I think AI can, um, really, really help with that in the future.

So, So what is your best advice to folks? What's that one thing you see folks doing that just still makes you shake your head and go, folks? I think we're better than that.

I think, um, I saw this happen on the front end software development. Um, you know, when Kubernetes became a thing, and I came from a Cloud Foundry background too. So taking away the, the control over like what operating system and what version of my JVM or whatever am I running, taking that away actually had a, a positive effect in it.

Developers had a focus more on what's my logic and how am I testing and how am I doing all of that. I think in, on the data space, um, that transition is, is getting ready to happen. So my advice to, um, architects, engineers, like people who care about this stuff is, is it's coming to the point where you're not gonna be able to dictate what platform your data is gonna operate in.

Your pipelines are are gonna run and you're gonna be in charge of the amazing logic and using AI engines and stuff, but where it actually runs is gonna be more of a, uh, back office administrative function. And your, your logic is then gonna push the data to that plane, but you're maybe not gonna have a choice over that. So I think getting comfortable with that, like the front end people do with Kubernetes is really important.

I think that transition is, frankly, it's starting, um, maybe last year or this year, uh, but in the future that'll just be a thing. Clouds are normalizing and if I'm running a pipeline in Spark, I don't really know where it's running and should I actually care. So you're kinda saying the attributes of the workload will be analyzed and then we'll be able to automatically figure out where it should run based on, uh, what's most cost optimal.

I, um, kind of, sort of can't confirm or deny that that may or may not be possible in Ascend here in a little bit. So, All right, coach, you heard it here first. Stay tuned.

Good things are about to happen in terms of controlling cloud costs. John, thanks for being on the show, Mike. I really appreciate your time.

Thanks. All right. Back to you guys in the studio.