Matt Butcher, Fermyon | AWS re:Invent 2022

This is texturong TV. All right. We're here.

We're here at AWS re:invent 2022 in Las Vegas at the tech strung studiosep the win and I have the great privilege of being joined by another great guest with blog brought you a lot of great guests today and true in this case exciting topic Matt butcher from fermia. Thanks for having me. Good to have you here.

Tell us a little bit about yourself and we're gonna get into what fermions ferr me on sorry. It's doing yeah, you're gonna try and get me to sing that far. Yeah.

Yeah. You're right. You're right.

I've made it like seven seconds in and I'm saying it already. Yeah the folks at the company are now cringing. That's right.

They are sorry. Yeah, so Matt tell us about yourself. Yeah.

So I've been in the cloud space for quite a while. I was I came into the cloud space for him like to Drupal content management world and I got a job at HP, right when they were starting to investigate open. Stack and so I came in as a CMS person.

And as soon as I saw what they were doing, I'm like, oh, I don't see him at no. I want to do that stuff. Now, you know this kind of breakdown of the we'd always up to that point, you know perceived the relationship between the operating system and the hardware is being kind of one to one and it was like one of those experiences where it sort of breaks your brain in an exciting way and so started doing that there at HP later on, you know moved into the Container ecosystem very early at was at a place called revolve that was an iot startup and got into the cloud side of iot and ended up at Deus which was building a Heroku like an open source alternative to Heroku that got me into kubernetes and you know, really kind of Dove deeply into the kubernetes World created the package manager for kubernetes called Helm and then ended up moving on from Deus into Microsoft and spent about five years there until making that big leap into the Startup world and starting firmly on about November of last year strap on that parachute.

Yeah. Oh, yeah. Yeah, nothing like jumping out to learn how to fly right Drupal you had me flashbacks to Joomla and no.

Oh your mother CMS. Oh, yeah. I was a big CMS guy in June line WordPress.

And even even some of the more esotericans like open CMS. And that was that was my world for a long time that too. Yeah.

Well we're so let's talk about web sample. A lot of Buzz starting to yeah to talk about this is maybe an alternative or an extension or replacement to Cloud native. Maybe we should start out with what webassembly is.

Yeah, and I think the interesting thing about webassembly and this happens a lot in technology, right you build a technology for a purpose and then discover after the fact that the technology is generalizable and can be applied in different places. So the original intention of webassembly was as kind of the name implies just to build a compiled Target for for different languages to be compiled and executed inside of a browser in a secure way. And I mean this came out of Mozilla and then Mozilla and Google and and Microsoft all work together and build this is great specification for executing things safely in the browser and I think figma is a great example of a user of webassembly in the browser space.

A lot of their code was written in c++, compiled a webassembly and executed inside of the Browser, but the interesting thing about it. Is that security model that webassembly provides. It's it's got an application and a number of different places and that's what really kind of excited us about it.

But really, you know, at the end of the day webassembly is just sort of a specification for how to compile things to a platform agnostic binary format where there's where the default security posture and this is the really important part right? Is that the thing you're executing you don't trust it. network the default security posture is well.

Yeah, you know, I trust the developers so I'm gonna let them use the network resources. I'm gonna let them access files on the file system. That's not that's sort of a non-starter in the browser world.

Right? And when we saw features like that where it by default the runtime doesn't trust the binary we thought okay, that's the kind of thing that we can use we could you know, pick up out of the browser and drop under the cloud side and solve a bunch of problems that we had identified there. If we haven't learned anything from plugins and add-ons and browsers and yeah territory, yeah.

Yeah, we haven't learned much. Yeah. No, we certainly have learned a lot.

So Understand I can't see some basic questions first. Just yeah folks that aren't familiar with it is web webassembly considered like a runtime or is it strictly here's how it gets compiled into a executable that already has in that it come compilation has built in the security mechanisms to prevent it from yeah wild inside your brows. Yeah.

So webassembly was standardized by W3 the same group that does CSS and HTML and there's sort of two parts of the standard one of them defines how you'd compile something into the format and the other one defines how you would execute it. And then now there are a bunch of as as happens with W3 right? There's a core standard and then there are a bunch of auxiliary standards to say.

Okay. Well in this case, this is how you would expose this particular API in this case. This is how you do threading and that's kind of where we're seeing it now, but the specification really defines sort of how to compile it to this, you know, assembly-ish kind of language and then how to execute it similar to The Bike code in virtual machine specification for Java.

Okay? Yeah. Familiar yeah much.

So yeah. Browsers today support webassembly is this something all to roll out yet all major browsers currently support at Safari Chrome all the all the Chrome family of all that you yeah V8 and then then the Mozilla family as well. So okay.

Yeah imagine that getting all of the browser vendors to agree on a standard and then implement it, you know, really? Well, it's actually yeah. Yeah exactly.

Yes. So why webassembly? Why do we need it?

I think there are kind of four different areas where it can be applied and fermione. We've really kind of zoomed in on one but one of them is of course the web browser the original case the second one I think is you already touched on this idea that we could create sort of as one of my friends calls it the last plugin system will ever need the envoy proxy for example can run webassembly plugins. So if you're writing proxy filter rules, you can compile them to webassembly and drop them in there and you get the security model and you get all the benefits of that and the developer can choose which language they want to use and if they if they like rust they can use rest they like see they can use see and and it that kind of that set of capabilities is really compelling in the plug-in model.

I think we're also seeing that in the database world as well. Third case is really kind of edge Computing and iot so far Edge and iot webassembly one of the cool things about the standard is that you can run it in an interpreted mode, which is great for a low power and small device or you can run into kind of compiled jitted mode like you With in maybe a server environment. That's how we run it for example, so, I know BBC Amazon there are a couple of different vendors who use webassembly to build the app where you watch your TV shows.

So if you're using Amazon Prime, you're running webassembly most likely because then they could write a single implementation of this and then move it to your TV or your Roku or whatever device you happen to be running it on and they don't have to maintain, you know, 1800 different code bases to execute on all these different platforms. It's kind of a list of Holy Grail of right once that's yeah. Yeah, and I think the the promise of the right ones for on anywhere with Java is really coming around again in a different iteration here with a different security profile and it's kind of exciting then the case we're really excited about is the cloud side, right?

And you know, what what we saw happening was, you know again going back to the autobiography, right? I was there solve a VMS. This was really cool and the VM is kind of like this Workhorse of the cloud, right?

We can't even imagine cloud computing. Without virtual machine technology and then Along Comes sort of like so that's really the heavyweight class of computing. The middleweight comes along a couple years later with Docker containers, which provide a nice easy way to kind of package up a segment of the file system, but not the entire image that the virtual machine has has in it, right?

So you get kind of this middleweight class and it's really good at a particular set of problems. One of the things we kind of discovered that got us interested in starting for me on where there were certain problems. We couldn't solve with the heavy weight in the middleweight classes and we started to say well maybe there's a next wave of cloud computing.

Maybe there's a third third category of cloud computing what kind of profile would we want while we want this kind of security model. We want this right one's friend anywhere kind of model. We want super fast startup times and as we kind of hunted around the landscape webassembly sort of Bubbled up to the top and we looked at that and said Okay, so this could be the kind of next Cloud side technology that's going to sit alongside Docker and going to sit alongside virtual machines, but be able to accomplish some things and solve some problems that we either haven't been able to solve or have solved in expensive and inefficient ways up to this point.

So I think someone to ask you is Would we ever containerize a webassembly application or is that sort of like, you know, you're moving a step backwards by doing that? I know I think they're actually some good use cases for container. Right just like we've run containers in Virtual machines.

I think there are cases where the sort of the nested model can can solve a problem and other cases where you don't need the nesting model. The webassembly security model is is great. So you don't necessarily need the security isolation that it container gets but someone might have a reason to say I'm going to containerize this and run it somewhere for us.

We've chosen not to go that route and and a big part of the reason is because each layer on that layer cake, you know adds a little bit of Well, it consumes more resources adds a little bit more friction. Yeah. Yeah adds a little bit more weight to your application and consequently in some cases.

It drives the cost up to operate it. And one of the things we're really excited about with webassembly is that the resource consumption is very very thin. So when you think about a virtual machine you're talking about a gigantic image that then requires a large amount of CPU and a big amount of memory to execute and so again not kind of extra large heavyweight model containers, you know little more spelled you can get you know, 25 Meg container image, but webassembly we can get those sizes way down to just the size of the code or the code in and supporting files and likewise startup time a virtual machine takes a few minutes to start up a container takes, you know, a couple dozen seconds to start up the webassembly.

This is where we're really excited about it webassembly we targeted saying, okay. We want to start up time around 100 milliseconds. While turns out we can get it way below a hundred milliseconds.

We can actually get to the first execution of an instruction in under one millisecond. And that's a that's one of the really key things that's exciting to us because as serverless Technologies are as we start to understand what we can get out of them every little bit of speed that we can squeeze out of it means we can be a little more efficient in the way. We use cloud computing resources.

And ultimately, I mean you can you can kind of parse that out different ways, right? We want to save power. We want to be a little more eco-conscious about the the amount of computing that we do in the resources that consumes.

We also want to save money and that means if you can run more stuff on fewer Cloud VMS and things like that, then you're going to save substantial amounts of money. You can keep startup time under the millisecond latency of my Wi-Fi network and the hotel. Yeah.

Yeah, you know, it has been it has been really fun, you know, you get going on a startup and you're like Theoretically I can do this. So I'm going to tell people you know, we're gonna do this and then the rubber hits the road and you have to actually prove it and so one of our first projects at fermione was to say can we build and run our very own website on this technology and can we get it, you know at the time it was into that hundred sub 100 millisecond pages, right? Yeah, and when we you know, as we got things built and February of last year was creeping around and we started running the Google performance test Suite against it and we're targeting like a 90 out of 100 on the Google score and we hit a 99 out of 100 and we went this is faster than we thought we were actually under selling the technology and and consequently, you know from from then on we started to really learn how to design this system to maximize efficiency.

So talk to us about what fermione then offers and so we set out to build three things. And now as of October, we you know, we've released our third one. But the first one was we wanted to build a really good developer tool to make it easy for developers to get going and and easy for them to pick the language.

They want it and build a quick HTTP application that would Ideally be you know, a couple dozen lines of code to get started and then you know, you dive straight into the business logic. Second thing and that was a spin but your great microservice. Yeah, you know, yeah and it is we wanted microservices are the kind of applications.

We think webassembly is perfect for and so if we can build a developer tool that says, hey, we'll make it super easy for you to get going on a microservice and and so easy that you can get from a blinking cursor to a deployed application in two minutes or less. That's our guiding user story. Then you know that's a really compelling thing to offer to developers.

And if you look at why Docker was successful early on it was because the docker tool made it not just easy to package up things and then push it out to registry but actually kind of fun right faster. Yeah. Yeah.

We got a buzz because building Amis was hard at that time. Right and we thought okay. Well web assembly we should make it as easy as possible to build these so spin was our first tool.

We released it on at the very end of March last year because we didn't want to release it on April Fool's Day. Like, let's get it out the door without a bad marketing. Yeah.

And then on April Fool's Day, it trended on Hacker News. So we're doing well. I don't know I guess that's better than good intention.

Yeah. Yeah, and then the second tool we wanted to do is an open source platform that people could Deploy on their own clouds and be able to try this out and it's not, you know, it's not the kind of thing that'll scale out to hundreds of thousands of applications, but it's the kind of thing that a team can use to deploy their first, you know, 10 20 50 applications kind of a reference model. Implementing exactly.

Yeah, and we wanted to show people how we had stopped using kubernetes and started using hashicorps Nomad scheduler, which is just a remarkably flexible tool and allowed us to not have to build an orchestrator and that's like personal goal of mine orchestration is hard. So we released that open source platform at open source Summit in June of this year. And then from there we said, okay.

Well what do developers really want? What what's gonna get them? What what kinds of things can we smooth out?

How do we make this a frictionless experience for them? Well, we've been working on Spin and that gets you going building the application but developers tend to not want to have to set up a lot of infrastructure before they can try it. Right.

So we said, all right. Well, we think that we can do this for an incredibly high density which means we can operate a cluster at a very minimal cost of fermion, which is important thing when you're a startup right and so we begin building a kind of hosted version of the for me on platform and we called it for me on cloud. We just launched it.

And it's an open Beta right now and it's it's our free tier of the platform. You can you know, sign in with GitHub and within moments again, it's part of that whole blinking cursor to deployed application in two minutes or less Roddy. Who's our CTO says, you know, you really ought to point out that you can do it in 66 seconds or less.

Know but two minutes for some reason just sounds catchy to me, but that's the kind of experience we wanted so it's like, you know spin for the developer for me on platform for the people for infrastructure and platform Engineers who really want to try it out for themselves. But really this hosted fermione Cloud for the developer who wants to say, hey, I built something now. I want to publish it out there and have it on the have it on the web.

I'm guessing with the answer given your target market if I deploy this on the fermione cloud. Is this something It can stay there. It's not a 90 60 day 66 day.

Yeah by something but you know, it's one of those when I get to the point, I really need yeah, the advanced features or whatever might be and and again we're in the kind of open Beta periods. So you're early one of the things we decided not to do in the open Beta period which will introduce you know, coming up early next year was we didn't allow people to map their own custom domains to it because we thought the last thing we want is to introduce something and then have it have it be unstable while people are trying to put it into production the again one of the pleasant surprises, you know, like the discovering that it was faster than we thought it was discovering that actually it is a remarkably stable technology. A lot of that is because of hashicorps nomad project but a lot of it is just because webassembly is a very robust in spite of its newness.

It's a very robust technology. And so even even in this first month we started to say, okay, well our level of concern for for performance and for stability Tea and for the ongoing maintenance of the platform is is going down and down and down and our own confidence has really grown quite a bit. So I think you know early next year.

We'll start saying okay the things we weren't sure about releasing in the early open Beta we're going to start rolling out like being able to map domains and being able to store things in a key value storage and and you know, inject environment variables and things of that nature. So it's been a really exciting time. I think startup life is hard in a lot of ways and when you get a little Victory like that, it just really gets you amped up especially because you can't do all the things you want to do.

Oh, yeah, that's worse. It's all that. Yeah.

That's right. I know that'll kind of progress you down the path. Yeah, what what are the things what is webassembly need next?

What's it missing that it needs to evolve to so I mentioned there were a couple of auxiliary specifications that are making their way through W3 to me the most important one is is called the component model specification and you know sort of at root what it is is a specification for how you can take two webassembly mod. Roles and call make function calls back and forth between the two. So this is sort of like, you know, the next iteration of the microservice model is one way to look at it another way to look at it would be to say, okay.

Well we can compile a library to webassembly. It's like a sort of a library model for microserve. Yeah.

Yeah. And again, you know, what would be ideal would be to get out of that habit where we keep having to reiment the same exact feature set in every single programming language Under the Sun that's JavaScript version of this thing. There's a python version of it.

There's a rust version and instead be able to say, well actually it doesn't matter what the Upstream language for the library was we just built it into webassembly and now whether you're writing JavaScript or python or rust or whatever you can call into that same webassembly module. So the rather than doing like a rest call or something, you know, you're doing yeah and mediating it through the webassembly run time instead of over the network and not that would be I mean to me that's really where web assembly as a technology begins to show not just Study its Java with a different security model, but that it's bringing something nuanced and really exciting to the table and something that really has the potential to change the way that we build software right now and reduce the effort. We put into duplicating things in different languages, which I mean you think about the countless hours that are spent figuring out why the animal I this might be coming from personal experience why the animal process processor in go parses my document this way and the ml processor in Python parses it that way that would never happen.

Never know. Yeah, really? So I think that's really exciting.

There are a couple other specifications that are working their way through and I think probably the most important one is that webassembly doesn't have threading yet and threading you can do a lot without threading. I mean if if you need an example PHP has existed without threading for a very long time and very impressive systems have been ability. Yeah, but but there are things like databases if you want to write a database and webassembly, you're probably gonna need threading and so the threading specification in W3 is working its way through and is looking really great.

There's we're starting to see the reference implementation. So I think you know, they're moving along quickly for a standards body and I'm excited to see those come in because then I think that the end with the component model and threading I think then the entirety of what we can do with webassembly. We're really start to Bubble Up into you know, these different areas where we're starting to see it used.

Well, it seems like whenever we have a new technology or new approach come along it's well that's going to supplant and replace everything else before it which is the exact opposite. That's true all those yeah just forever and this will exist too. Do you see it overtaking some applications of what we do in kubernetes and Cloud native?

Or is it specialized enough for these are things you wouldn't want to do in a kubernetes environment, you know. I was a big kind of zero-sum thinker for a long time and really thinking okay. Well, in fact when I went from the Virtual Machine World to The Container world the way I was approaching it was oh, you know, nobody will ever need virtual machines anymore.

We'll have containers. It's gone. Yeah.

Yeah and ever then, you know, the the scales fell from my eyes and I went. Oh, yeah. Well, both of those are gonna be around forever, right and when so as we got interested in webassembly have with a little bit of that perspective we can say, okay, we don't need to tell the story webassembly is gonna supplant containers or web assembly is finally gonna put virtual machines to rest or something audacious like that.

We could start saying well, what's its strength? And and then what are we going to build with on upon those strengths and some of its strengths, you know, there there are the current way. We do a lot of functions as a service of you, you know, get the behind the scenes view of how Azure functions or how Lambda works.

It's inefficient at the core webassembly might provide the alternative runtime that would do that. So is it taking something away from the VM container kinds of solutions? Well, maybe in some of those cases but those are cases where we've known for a long time.

Those aren't the right long-term Solutions. We needed something that would come and be able to do those more efficiently. I don't see any any reason why this becomes an existential threat to either of those but for certain, you know, we'll start to see a new level of a robustness on the kind of serverless aspect of things because webassembly will definitely serverless sounds like an ideal environment.

Maybe even a K3 k0. Yeah, really lightweight environment where Do you need that infrastructure? Maybe this is a variation of yeah.

Yeah, and I think that's what you'll see the case is particularly cases where you're system is constrained is a small processor or cases where density really matters like cloud and being able to do sort of large scales serverless functions and and packing more and more applications on to a thinner and thinner hard virtual machine profile. Those are cases where I think webassembly will really Excel but you know, we build a game we like to have fun, right and I'm not gonna sing take me on again. I swear but we built a game called finicky whiskers that was a game about one of my colleagues is cats who's was always changing her mind about what she wanted to eat.

So we built this little game about a cat that changes her mind. She wants steak. Then she wants chicken then she wants vegetables and you're trying to click button and steak stay chicken chicken as fast as you can to feed her as she's changing her mind but behind the scenes it was built out of nine different webassembly micro services and a containerized version of Reddit is and the reason we did this silly.

Was to give people sort of like that palpable experience of seeing the webassembly stuff working really really fast while also working side by side on top of a platform that's running containers and webassembly and of course the whole thing is running on a virtual machine. So then we can say look these are complementary technologies that we're going to enable us to build a new kind of application composite application. It's going to be better than what we had before and that's really all we want.

We don't need to talk in language of defeating something or supplanting something. It's just how do we make this stuff better cheaper faster and so on I'm getting the hook, which means we're having a great conversation by the way have it on good good authority that your theme song for the company is trending on tiktok. Haha.

Oh, no bad picture for me on right, correct, and thanks for joining us. Sure. Great talking about webassembly and I kind of bringing it, but we'll be back.

We've got some more great interviews today and tomorrow. So look forward to having more good conversations. Thank you, Matt.

Thank you.