Momento’s Daniela Miao on Rewriting in Rust

Hey guys, thanks to the throw. We're here with Memento, CTO, Daniella Meow, and we're talking about Rust. And one of the things that's interesting here is that Memento rewrote its platform at Rust, and a lot of people are talking about making the same transition.

So we're gonna have a little chat about what went into that. Daniella, welcome the show. Hi, Mike.

Uh, thank you for having me on the show. Super excited about our chat today. Our pleasure.

Um, what was the motivation in the first place for making this transition? I mean, everybody kind of is loosely familiar with memory safe as a concept, but, uh, being aware of something and actually doing it, or of some two different things. So, um, what was the driver?

Driver for us was primarily performance. Uh, we are a very performance focused company and team. Uh, we're a real time infrastructure platform, so all of our customers are building and scaling real-time applications.

So it has to be snappy, has to be really fast. Um, and we felt like we had kind of exhausted our options on our, uh, old JVM based, uh, stack. Um, and I think engineers sort of unanimously agree that in order to get significantly more, um, optimizations outta our system, uh, we would, uh, need to pursue a different language, um, that didn't involve garbage collection, um, and more predictable memory, um, management, like you mentioned.

Um, and we landed on Rust and it's, uh, always better to do it sooner rather than later and we'll, when our stack gets even bigger. So we decided to repop the bandaid, um, and rewrote everything. And as you look back on it now, what do you know now that you kind of wish you knew, but you got earlier?

Because, you know, everybody has assumptions about things that they're gonna see or experience, but what was the reality? Um, I think it was overall a really good decision. Um, we de we're definitely happy with the results, um, that we've been getting, especially on the, on the performance side.

Um, I think that there were challenges to ramping up to a new language for sure. Um, you know, people say, oh, you know, it's, it's like a functional program, so maybe you can liken it to hasco or, uh, it's also like more like this and that, uh, it's, it's pretty unique on its own. So, um, I think that, uh, I actually wish that, uh, we didn't see it as as much of a, a, a challenge.

Um, we, you know, we didn't do it in one fell swoop. We decided to start new services in Rust first. It was rather contained.

Eventually we decided to migrate an existing service, which meant the rewrite. Um, but, you know, I, I don't think people should really see it as this like insurmountable, like huge amount of work. I think starting by tackling it, using something small like a, a tool or like a CLI something that's kind of off the critical path just to try it.

I think you will quickly see that the, the barrier to entry is not as high as, as people might think Outta curiosity. Did you get any help from Gen AI to help with the rewrite or to summarize some of the older code and just to see how it works? Um, I, I mean, I personally didn't, although I wish, you know, I, I'm not hands-on coding every day anymore, but we're definitely big, um, you know, we're big fans of, of co-pilot.

Um, I know that a lot of my engineers use it. Um, I think that at this point in terms of the gen ai, um, capabilities, um, it's helpful when you're trying to just like move boilerplate over, right? Like, um, they, they can figure out like what you're trying to do in the old language, which for us was Kotlin and they can suggest very quickly like what the boiler plate RUS code looks like.

Um, oftentimes we do have to apply a deeper review of, you know, 'cause 'cause in Russ there's a, it, it has its own ownership model. So we have to kind of rethink okay, does that apply here? And, and you know, um, I think that when it comes to writing it, uh, I mentioned the CLI tooling when it's simple and you know, what you're trying to do.

I think, uh, Jen AI was super helpful and definitely helped speed up on the velocity of our team. Uh, some of the internals of our platform gets a little more complicated with business logic, and I think it was, it was, it was less helpful there. Um, and I, and I think some of our engineers actually said they had to turn it off because it was kind of, it was interfering with the, the good old traditional auto complete and they wanted that at that point.

Did security factor into your thinking at all here? Because sometimes we talk to folks about Rust and one of the reasons they're moving is because the older languages are written at a time when we weren't aware of so many security issues in our code. So, uh, is this part of the thought process in terms of we just need a platform that's written in a language that's more secure?

Yeah, absolutely. Um, you know, we work with a lot of enterprise companies, so security is definitely top of mind. Um, now fortunately, you know, we're, we haven't actually had any, like any real security threats.

Um, but if you just think about like, well, you know, c plus plus has come a long way, C has come a long way, and those are kind of, you know, like proven, you know, that you can do very fine, um, reigned memory management. Um, but we also know like that, you know, not everybody rides perfect code, unfortunately. Um, and bugs, bugs will always be there and things like, you know, buffer overflow, um, trying to access something after it's been freed.

Um, those are all, I think, vectors for attack potentially. And if you, if you like, let that accumulate over time, it's hard for anybody on our team to guarantee that. Um, nobody's gonna come in and accidentally, you know, uh, write a bug in our system.

And we, we store data like specifically for our caching platform. Like we're constantly serving these requests to different customers. And you know, that that would be a kind of a company ending move if, you know, we accidentally serve one customer's data to another.

Um, so I think, again, coming back to kind of the primitives that Rust offers, where it forces the developer at development time to think about ownership model, who owns this variable, who can access it, um, you know, immu ability to change it later on, I think it kind of builds those guardrails, forces you to think about, uh, an, uh, upfront and then hopefully helps you prevent these kinds of bugs later on. And, and it has actually like, been very effective in doing that thus far. Did you have to invest in any training to bring your team over?

Because, uh, you know, at least in my understanding, not everybody is, knows Russ just yet, but, uh, you know, progress is being made, but what was that kind of process like? Yeah, I think the part of the decision here is that as a business we have to be ready to invest in educating and letting our engineers, um, kind of ramp up on that. Um, that's the reason that we didn't start in rust, right?

We, we hired an initial team, we had no product. We wanted to get the product out ASAP to start testing it against product market fit. So the first version was written in Kotlin, a JVM based language because everybody already knew Java and actually knew JVM debugging tooling really well.

And that was a good decision to get the product out once it was out. We had product market fit, we had confidence that we wanted to keep a service a certain way. And, you know, the architecture was kind of set in stone, well, not, not in stone, but you know what I mean, it's matured.

Um, then we, we knew we had to take, take the time if we do decide to rewrite it and rust. So of course I think lots of really good material, free content, uh, really good books, really good. Um, you know, uh, tutorials as well that like really ramp, ramp you up and a lot of our engineers went through that, but nothing beats actually making a production ready system, um, in rust.

So we, we did a lot of that as well. Um, additionally we actually work with, um, uh, another sort of, I consider them to be like a, a a design partner of, of ours, a team of X um, X Twitter XX, um, engineers who, uh, launched Rust Services in production, uh, uh, on, on their infrastructure team. Um, so we work with that team.

Um, they are consultants to us, um, on the performance and the rust debugging side. Um, so we do, you know, we do have external help as well from experts. Mm-Hmm.

And we reached some sort of point now where the legacy languages don't really enable us to get to the next level of performance that's gonna be required. And between the, you know, everybody talks about the security side of the issue, but with Russ, I mean, have we just reached a requirement where we need a different approach? Um, that's an interesting question.

Um, I still have a lot of, um, uh, I still have a lot of fondness for, you know, CNC plus plus that's kind of, you know, maybe I'm a traditionalist in that, that's where I started by learning programming. And I think understanding at the low level, the, the risks associated with managing your memory, um, you know, literally a block at a time is valuable for a developer. Um, and I think that performance at the end of the day, it's just a matter of time, right?

Like whether you can, you can get to the point where you're so sophisticatedly managing what's happening, and, and it's very specific to the product. Um, you know, what is the application lifecycle? What do you have to do with a memory?

How much do you have to churn through garbage collection? Um, so I I still think that, you know, c plus plus is super powerful, constantly evolving, and it's able to be very, very performant. The problem is the amount of time, the timeline on which you're to build a viable business, right?

So if you're, if you're developing in a rush, I just think that the language itself is, you know, it, it causes developers to make mistakes along the way because of how carefully you have to manage memory. Um, and, and also I think Russ has some really nice higher level abstractions, um, to my functional, uh, sort of language reference. Um, it kind of has functions as a first class citizen, which allows developers to relatively quickly write complex logic while, you know, still reaping the benefits of it compiling into, um, into machine code that's very performant.

So I think it's more about the balance, the, the input of effort versus the output performance, um, which I think it's like, oh, you, you get like relatively good, um, low effort input and um, just as good performance as a low level language like CNC plus plus. And what's the state of the tooling like for Rust? I mean, 'cause a lot of times you get developers who are, you know, well, I'm not excited about learning a new language.

And then they tell themselves, well, you know, the tooling will never be as sophisticated as I have for the stuff I'm in that I'm currently using. Um, what's the maturity level of the tooling? We really like it.

Um, I mean, I think it's subjective depending on what, uh, ecosystem you're coming from. Um, one thing that I will say about the popularity of Rust compared to I think also very awesome languages like er, Lang and Elixir, is that it has gone mainstream. You know, big corporations have taken on Microsoft, Mozilla obviously, um, Amazon.

Um, so I think like the community is definitely here to stay. And from our own experiences, the ecosystem is very, very vibrant and very active, constantly being developed on, um, it has pretty mature package management. So, you know, we use cargo and, and, you know, build crates, um, formatting linting, um, a a lot of this from the moment that you're writing RUS code to the moment where you have to deploy an into production, we found, um, like very little issues moving from a j VM based world into a RUS based world.

Um, and if anything, I think a lot of our developers have now become, uh, pretty passionate advocates, uh, for the language. Um, so I think one, uh, I think is already pretty mature given our experience, but two, I think it will continue, continue to get better, um, just given the amount of, uh, attention and resources that, uh, even like big companies are pouring into it. All right.

Um, assuming that we had a magic wand and we can make you queen of all things rust today, um, what would be that one thing you would want the community to kind of focus on or fix or enhance or just make better for everybody? Oh wow. Um, that's a really good question.

Um, I think for me it's interopability with other languages, um, because of this gap between, I mean, yes, we had the privilege and the fortune of being able to make the time to do the rewrite, but that is a very daunting task. And I think that's the biggest barrier to adopting it as a language is that, uh, like when are you gonna really find the time to do it? So I think, um, just like more libraries, more, um, uh, sort of, uh, ramps into like, Hey, you, you have a Python application, here are the parts where you can kind of plug in a, a rust plugin or like a rust, um, binary, uh, so that you have like a mix and you have an incremental way without having to go and rewrite everything to, to migrate to Rust.

How did you make the case to the business, which probably stood there and said, well, if it's running, why do we need to go do this and redo this in Rust? 'cause I'm sure somebody's asking a question that has, you know, the acronym ROI in it somewhere. Yeah.

Um, you know, it was actually a, a, a team decision. Um, we asked ourselves a lot of questions about what's important for the business. You know, I mentioned performance being in our DNA.

Uh, I think that it's gonna continuously be, uh, sort of the, the winning recipe, um, for us with customers as well. Um, especially like I said, in a real time platform, customers are continuously, uh, gonna be, uh, wanting better and faster, um, and their bar for that. Uh, especially with Gen AI in the picture, right?

People are constantly getting new information and want that information to be quickly processed for their use in their application. So I think latency is gonna become more and more important. And because we hit sort of like has said kind of a bottleneck with how low we could get our latency, um, and just kind of looking at the, the, uh, alternative, um, which like, we know we can get better, um, I think it was just like, well, do we wanna do this now or do we wanna do it when we absolutely must because we're losing our competitive edge in a real time, uh, you know, in the real time market.

Um, and doing it now is definitely less expensive. So I think it, it kind of, it wasn't like a solo decision by me or by my co-founder Quadra. It was kind of a group realization once we started asking more questions about like, what, what is one of our, um, tenets, which is, you know, performance is, is one of the most important.

And everybody was probably having that conversation with themselves and was happy to have it out in the open where everybody's going. Yeah. You know, I was thinking about that.

'cause sometimes, you know, it's, it's reaffirming as it were. What is your best advice then to folks who are thinking about making this transition and, and what went into that? I mean, um, is, is it, what, what are the bigger challenges?

Is it the technology or is it the cultural side of the thing? The business side of the thing? I mean, how do you get there?

Yeah, I think to the question that you already asked me, it is making that business case. Um, I definitely don't recommend doing it for the sake of, you know, rust is popular now and it's a trendy thing to do. Um, I was talking to an engineer, so I'm, you know, I'm actually, uh, a, a track host for the, for the, for the Rust in Production talk track at coupon in San Francisco later this year.

Um, so I've been talking to lots of people who's, who's been developing in Rust and one of the engineers, um, actually on the Prime video team, uh, at Amazon, um, they're switching a lot of their front end and, and you usually think of Rust as kind of a backend, you know, low, low level language. Um, they're actually using Rust to build their front end, like literally their ui. Uh, and why, because Rust compiles really nicely into WebAssembly, um, which is super efficient, super fast, um, uh, as a, as a ui uh, language.

Um, and they're switching, uh, most of the, the prime video, you know, UI into, um, into completely rewritten in rust into web assembly because of how, how much snappier it gets, it gets you in the application. So I think, um, the, the advice for people is that, you know, it, it has to be tied with a business case. It doesn't have to be, you know, as big as sort of what Memento did, which is like, you know, it's tied to our mission.

Um, it can be incremental, make a business case for it, whether it's performance or risk mitigation, or, hey, you're about to undergo this like, big project that needs a lot of concurrency, and that's always a little iffy in, in some other languages. Um, you know, so I think find ways to tie it to the business and then, like I said, don't see it as psychologically like a daunting task. It, it really ended up not being very bad.

I think good engineers, um, pick up new things very fast, um, and they expect to, um, and that, that there are incremental ways to get there, you know, one component at a time. All right, folks, you heard it here. Hey, when it comes to Russ, it's like many things.

There's nothing to fear but fear itself, right? So there you go. Yeah, exactly.

All right, Danielle, thanks for being on the show. Thank you for having me. This was fun.

Thanks, Mike. All right. And back to you guys in the.