Liam Randall, Cosmonic | DevOps Experience 2022

Good afternoon. My name is Liam Randall. And today I'm going to talk to you about distributed devops and what happens to the Enterprise in an age of distributed applications?

My name is Liam Randall. I'm chair of webassembly day for cncf. I'm the CEO of cosmonic.

I created one of the very first kubernetes companies in 2014 called critical stack later. I ran Innovation for one of the most Innovative companies in the cloud a Capital One, and I'm an all-around a very entrepreneurial person in and around Cloud native specifically around the security space. Is helpful to start with a quick overview of what we're going to talk about.

So we're going to start with how did we get to kubernetes and containers? Where are we today? We're going to talk about some of the challenges in today's ecosystem.

And then we're going to talk a little bit about what's coming down the pipeline in the future. If you've been in this industry long enough, like I have you probably started your career where we had applications operating systems and physical Hardware all tightly coupled with each other and this was a huge challenge because as we wanted to move between specific systems, we would run into challenges of portability challenges of security challenges of operating and managing these images and backups at scale. Now as we looked around the ecosystem, we recognize that there were actually common platforms that every application and every piece of Hardware wrote on top of so in the late 90s, we started to see the rise of virtual machines specifically with VMware and initially other competitors, but this eventually ended up creating the early days of the cloud.

And if you weren't doing devops in the early days of the cloud you would be using things like Hashi packer in order to create images of systems or armies that you were running inside of AWS. Your world was a little bit better. But you still were launching these huge devops processes in order to build operating system configurations and deploy them within the cloud.

And the observation then was that hey the configuration the operating systems themselves. Well, what do we layer on after we deploy the OS what patches do we put in is itself a platform and in the early days of containers there were competing container formats in 2011 and 2012 and we sort of standardized on container format around Docker and on top of oci and our lives were markedly better than they were on the ages of Hardware or the ages of os configuration. But we saw some challenge in tightly configuring the composition and the scaling of those apps to specific places, you know, where we are running them and someone observed that the right Folks at Google and other places in the early days of orchestration, there were multiple solutions that we could look at people were looking at mesosphere out of Twitter and kubernetes definitely came to rise as the dominant technology.

It's standardization to the cncf is the story that everyone is certainly up to speed on And the challenge is is that because of the way containers and the sort of baggage they bring from their legacy as virtual computers. There's a huge amount of add-ons that you need to control these things in order to lock them down or to manage them and it sort of hard to wrap your brainer at it. Unless you're maybe in the modern version of devops a platform engineering or perhaps you your company still calls it SRE but you need to sort of start with kubernetes put your application and container and then add, you know, 20 other service meshes and lots of other ingredients in order to get these things running.

And your devops pipeline is probably exploded at this point. You have things like Jenkins or Argo doing your ci/cd for your applications. You may have another layer doing your platform Integrations and you're testing here and the complexity has absolutely just exploded.

And I think the interesting observation here is is that even with all of this incredible technology. We still are not anywhere close to a Fantasia. If you understand how containers work.

They're just layers of of components that we put together in order to build an application. So you could start with a base layer an operating system layer like Fedora or Boon to or Alpine and you're gonna add in all of your components on top of this. But as we dive into this a little bit further and we've lived through painful episodes like log4j something still doesn't feel right.

It still feels like even if I lock down containers completely or as much as I can there's still something missing in the in the ecosystem. And so to demonstrate this today. What I've done is I've taken a little webshell that I wrote.

It's available here on my GitHub. You can download it. It's to create a virtual exploit in an application because I thought it would start by demonstrating how easily painful it is to be running in today's world of containers.

So we're going to take this webshell and we're gonna put it in an application and we're going to build four different types of containers. We'll start with maybe like a development type of container with lots of additional resources in it something you'd never deploy into prod and then we're gonna look at something with Ubuntu or Alpine something you may actually be running in prod and then we'll even look at the idealized container a container that doesn't have anything in it, but the application itself And the sort of big idea here is is that even if we take containers and we talk about reducing the service weight of the container there still security opportunities and portability opportunities that we have in today's ecosystem that become incredibly painful as we consider the distributed ecosystem of tomorrow. So I've created a quick simple application called person facts and person facts is completely made up.

It is a simple web app here that sells the best person facts and it's a little go program so I can compile it and build a from scratch container. And on top of this we're going to do a demonstration and look at those four container formats that we layered our application on top of so, let me roll over to My Demo here. So you can see what I've done is on each one of these containers.

I've gone ahead and just put a little Mark in here. She know which one we're looking at. So there's the Fedora container and Ubuntu container and Alpine container that has a even less things in it and then a from scratch container where the only thing in this container is just to go binary that's been compiled down with the web shell in it and this website in order to do the demo.

Now the web shell is very easy to use we can take one of these existing hosts and we can visit forward slash exact and then we can send commands and the system will run them and I'm running now in this container environment here. And if I were threat actor and this where your Cloud, you know, I might you know download AWS utilities into your container. I've already done that to kind of save time on the demo today.

I might unzip these and I might start understanding what I am this container has running in your kubernetes and start to try to Pivot around and if you follow some of the mega hacks that have happened over the last few years. This is exactly what threat actors you're doing even on top of your container on top of your kubernetes. Now, what's interesting is is that if we reduce the attack surface down a little bit further and we pull out, you know, all of the operating system layer here, we sort of end up in a place that's a little bit better, you know here the threat actor is going to have a significantly higher bar of complexity that they're going to have to cross.

Because we don't have any of the Legacy craft that we would have with the operating system. So if you're if you're dealing with modern deployment systems, this is probably your goal is to reduce and take as much of the Legacy personal computer Legacy operating system Legacy virtual machine out of your deployment and get you to a place where you have a small security footprint. So the sort of lesson that I hope you picked up is is that containers are great, but they still don't really fix an address the application layer.

That even in our minimal deployment scenario. We still had a vulnerability that was there that did have a chance of being exploited perhaps there was a sequel injection type exploration that that could be available there and there's there's additional challenges because that container due to the sort of inherited Legacy of the way that we have evolved today's modern infrastructure is built for a specific CPU architecture and a specific place. So if you're trying to deploy This application on those new arm CPUs that the gravitrons that AWS offers, for example that are 40% less expensive.

You need to go back in time to the kubernetes and the devops step and you need to build a whole new pipeline to support this new architectures and as the world becomes more and more distributed and we have more and more hardware and software combinations. This problem is exploding dramatically because there's additional problems here wasn't just the portability. It was the security as well as access to all of the common capabilities we would have in that application.

So let's pause for a moment and understand what does the modern distributed Enterprise look like and regardless of what industry you're in if you're in manufacturing or financial services or retail or software? It doesn't really matter what industry you're in there's a large number of common things that you're trying to achieve for every one of your Enterprise apps. You want your apps to be mobile first.

Bring them to the user's pocket and make them available 24/7 The apps should be real time. So I shouldn't have to be dealing with batch architecture. I should be dealing with streaming type architectures so that I always have the latest most up-to-date information and increasingly customers expect your applications to be personalized and proactive and enriched by Ai and ML and many organizations are having trouble Crossing this Chasm because they're just struggling to keep the lights on and manage the day-to-day security which takes away and starves their time in order to develop the features that they really need to be on organizations are spending 60 70% of their developers times on management and the devops parts of the equation and not enough time on the feature delivery a part of the equation.

And this modern world looks even more complex because if we take these features and we play them forward even into their application criteria. It starts to sound like distributed distributed distributed. If you want to be mobile first, you are building your native web apps or your native your native applications for multiple architectures, perhaps an Alexa or a TV, certainly IOS and Android start.

If you need to be real time, you need to be dynamically scalable and you need to have a streaming architecture. So you're building message queues. You're building a message buses.

If you're available 24/7 you're running at an at least multiple regions and a single cloud and possibly even in or across multiple clouds and certainly as we pull the aperture out and include privacy and Regulation and you're an international organization. You are definitely running in multiple clouds to be personalized and proactive. You need to understand the context of the user.

So that is parts of your application executing across your stack and to be enriched by Ai and ml you need to be out where the data is. You need to be developing a solutions for those customers. They need to be writing close to the users in order to deliver those real-time experiences in real time.

You essentially have a 200 millisecond interactivity budget before you use your Spiel that things are starting to slip towards batching. But I think there's something even additional ways that are pushing this distributed theme. It's not just latency and determinism it's that the edge is where the data is for regulatory reasons.

We sometimes don't want the data back pulled to us. We need to just use the data in the customer's local environment for limited or deliber autonomy, you know, your internet connected device better still work, even if the internet is not connected which means we often times have to operate portions of the logic even limited portions of the logic locally in a self-contained way. And it would not underestimate regulatory or privacy and security as additional ways in parts of Asia.

You already are limited in what data you can pull out for processing or storage and that means that if you want to operate in China, for example, you're using a local Cloud which means that your devops and kubernetes pipelines have gotten even more complex. So these this distributed world when it meets containers in kubernetes where we are today, we have vast challenges in system architecture. And how do we architect these applications in system security offline use and then machine learning I've documented this a little bit more in a long form and a new stack article.

I launched you're more than welcome to read about it on that QR code. But this is really what the modern ecosystem starts to look like for the Enterprise. It's not just applications running in centralized data centers.

It's using Regional edges. It's looking at maybe even some of your on-prem stuff. If you still have it or your customers on Prem applications browsers mobile devices and more of the modern Enterprise.

It's just pulled continually towards the edges. I like to think that Docker really helped us to power the great lifting shift into the cloud. But now we find yourself poised at the precipice of a new age of computing and that is where I think the most interesting part of this discussion can have So at this point, I think it's helpful for me to bring up a new standard for the web that's called webassembly, which is very poorly named because it's neither about the web nor is it really about assembly in any language and any format that you may understand a webassembly is really the opportunity for us to rethink about how do we isolate and how do we virtualize individual applications?

And we just hosted a talk recently with Kelsey Hightower and the CEO of Docker Scott Johnson. They got together to have this conversation at an event. We co-hosted with our investors vertex Ventures because they wanted to talk about A docker's new announcement that they're supporting webassembly.

Now, this is the company that is quite possibly one of the most pivotal companies in the age of containers in kubernetes, and we now have them saying that hey there's this new machine that's out there that we think it's is important to support now. This isn't necessarily to a serp containers just as containers didn't assert or replace virtual machines and kubernetes didn't replace cloud apis and none of this replaced physical machines or devices underneath the hood. This is an additional layer here.

So if we think about first understanding what it is the new machine, let's talk about the old machine. Well, the old machine is what we sort of paired away from the initial hardware and all of the Legacy history that came with it. So if you're you know regular console user, you know your console when you're out at the command line is 80 characters wide, it's 80 characters wide because Punch Cards were originally 80 characters long and there's all of these crazy historical elements that we've really carried forward us into applications.

And what does an 80 character terminal have to do with your company's application? Why do we bring along all of these unnecessary things when we really care about delivering processes? Why are we still bringing with us that Legacy from before and it feels as crazy to me.

Sometimes today's containerized world as the virtualized world. Did you know, why would I care about a specific configuration of Hardware or specific operating system or a specific operating system config one also I really care about is that the highest level just my business lodging not even the application libraries that I'm writing on. So webassembly is sort of the successor to flash and Silverlight.

If you will, it was initially designed out of the think tank and Mozilla. Sort of take a step back and say what if we had a common an open format a fourth language for the web the other three languages for the web or HTML CSS and JavaScript? What if we had a little virtual machine we could use a simple little computer.

If you're a CPU nerd. This should be more for risk type computer reduced instruction set computer. It's not you know, it doesn't have like Mindy and Cindy architectures that are in it and it would be an opportunity for us to sort of put first principles first.

So we could make this computer safe and secure we could make it really portable. We can make it really fast. We could allow multiple languages to compile down to it.

Unlike Java for example where I guess Java does have you know, there are multiple languages that can compile down to the jvm now, like kotlin and things like that. But Java is really a super opinionated application runtime, you know less of a virtual machine. So this of virtual machine is really helpful to think about it as a process virtualizer.

It's running today and all of your browsers. It's something a technology that you're likely already using. I'm not likely you are definitely already using in and around your Enterprises.

I mean, you're using it in an increasing fashion and my sort of hot take on this is is that well today people point to companies like cosmotic and they say oh cosmotic, you know is the webassembly platform as a service in three years. All companies will be using webassembly everywhere and my hotter take would be we will really care because webassembly being a webassembly company would be as crazy as being referring to somebody's in HTML company or css company or a JavaScript company will just be one of the technical attributes that is faded into the background and is one of the common things that you just accept Is being used everywhere now as I mentioned it's polyglot and it really is in a compilation Target and what's powerful is is that it's completely portable. So if you start to deliver your applications and webassembly you can take them to any CPU to any Edge device.

And that's important because when we look at the sort of rise and portability challenges as a proxy for that, I would present the sort of CPU manufacturing numbers. There's an increasing diversity of CPU manufacturers, even within the arm sub segment you find hundreds of different types of CPUs that you're likely building and deploying on from dozens of Manufacturers. And even if you're only talking about Cloud native in AWS, you have first x86 the classic Intel you have arm and they even have new CPUs that are targeted for machine learning applications.

So this is not a shrinking area. This is a growing area of concern. Now in today's devops world, you are very painfully coupled to targeting specific CPU architectures, if you're using the old computer and if even from a security perspective webassembly raises the bar dramatically because these CPUs are reactive and completely denied by default.

So if I'm running in one of these virtual CPUs unlike in my previous container demo, I don't have by default the ability to Pivot to a layer below me in my application. I don't have any ability whatsoever. Beyond the capabilities that I get granted and the capability Grant is something you're familiar with and you're using your phone you sort of have your application ask you.

Can I use a camera? Can you some microphone those are capabilities and then a cloud native context capabilities become things like access to a file or access to a directory or the ability to use some specific resource on a computer. And so what we see the world evolving to now is moving away from the idea of container orchestration and opening up a new area of orchestration around processes.

And in this age of webassembly. We are are already seeing massive adoption of this little virtual machine today and we're and due to something called the component model that we'll talk about shortly. We're seeing even lighter weight business logic being deployed in and around applications.

So this world of today, whereas kubernetes and containers have some lower bound on where they can effectively operate and we can't effectively bring containers to places like browsers. Right now, although there are some webassembly projects that essentially do that for you webassembly gives us a deployment Target for our ci/cd pipelines that is portable across the entire Cloud native ecosystem and even into places where you may not consider taking your application. So let me explain.

So while webassembly was initially designed as a browser plugin something that we could use seamlessly across any browser or any Edge if we were to write the criteria for an application Plugin or for an app platform. They would be nearly identical. We would want those things to be.

We would want those things to be seamlessly secure regardless of where we were choosing to run our code. We were running our code in a browser queer running our code as a plug-in to another app, or if we were running it in a cloud native context. And the proof points are everywhere if you're using Google Earth or Microsoft.

flight simulator or Adobe Photoshop on the web or if you've ever used figma the biggest it exit of the year figma is a pure webassembly play not just to build their web application, but they also allow their application to be hosted to be customized by using webassembly plugins. If you were building a secure CPU or secure platform and the way you probably try to build your kubernetes today is is that you don't trust any of the code that's running on it that's saying and webassembly really shines and performance here. If you're using service mesh Envoy for example used to be customizable through the use of Lua.

But today we find that Envoy can be customized through webassembly plugins. There's a cube Warden which is a an admissions controller for kubernetes that allows you to write your admission controls using webassembly plugins. I mean, it's a platforms as well Shopify says that they deliver 80% of the shopping cart experience out of the box, but that last 20% comes from a whole partner ecosystem and Shopify scales and orchestrates those components using wait for it webassembly.

And as an application platform in the traditional microservices context is where I think webassembly offers by far the most promise webassembly is so small and so fast and optimized for just the isolation and starting and stopping processes that it has some unique attributes. The first unique attribute is a five microsecond start time and what that means is that in that 200 millisecond interactivity budget you can now scale your workloads down to zero. These things are incredibly tiny and efficient so you can run tens of thousands of webassembly processes.

Even on a single modern computer where you might be running dozens or hundreds of containers. It's orders of magnitude better which to me is the Hallmark of something that will be adopted widely around the ecosystem and we've already seen this at scale if you're building on cloudflare or if you're building on fastly these are straight web assembly plays now These Are Walled Gardens and they sort of give you a limited set of capabilities like SQL in KV But the future looks more like this than it does the current cloud native context that you're developing in in today's world you simply bring them your functions and the platform provides all of the components and capabilities. Now Amazon Prime is another good use case if you've ever wondered how Amazon Builds an incredible streaming experience that operates across 8,000 unique hardware and software combinations.

My MacBook your Windows machine Linux your Samsung TVs your LG, whatever No, Name Black Friday TV, you're gonna purchase in a week. They did this incredible blog post where they talked about why and how they adopted webassembly in order to scale that experience as a proof point and even across Cloud native today. We still see this continued progress towards reducing the weight of services that we deliver and operate on our clouds many of you if you're on a platform engineering team are likely building application harnesses, so you can take this type of business logic from your users and then combine it with an application framework that runs in real time.

Now, let's talk a little bit more about webassembly because if I'm building these applications it would be helpful. If there were some sort of a operating system that we're running and there is there's a standard that's called wazzy, which is sort of like a posix type environment that we you would use to operate your applications. When we're doing process isolation.

Now on top of that you would use a framework. I'm one of the creators and contributors to cncf laws in cloud and wasm cloud is to webassembly what react is to HTML. It's a framework that helps you get started writing and building Cloud native microservices very quickly.

It reduces your boilerplate by leveraging something called the component model, which means that when we looked at our previous example, and we talked about how containers don't fix applications but the component model does is it helps us to break our applications into Lego blocks that we can see mostly build and connect around our ecosystem. So these components are very high level functional things like SQL servers or KV that actually get wired up to specific capabilities depending on where you're running. So if you are running a simple app with a key value store locally, I might spin up rent a server if I'm running it and AWS.

I might connect it to AWS elastic cache if I'm running on gcp or Azure, I would have the same sort of a context as well. And what that leaves us with then is small pieces of business logic things that don't suffer the sort of embedded vulnerabilities that we tend to find in the huge amounts of code that compose most of our underlying applications. And then together in our framework wasn't Cloud.

We actually then connect these applications together in a seamless distributed way. So you can even start using these capabilities across other clouds. So if you have your business logic running on some Edge device, you could be using a cloud capability like that SQL database or that key value store all without thinking in the Legacy idea of layer 3 vpns or things along those lines.

It has a very magical experience to it much like tail scale. If you had the pleasure of using of their product line and I would point out that these ideas and Adoption of webassembly is our absolutely exploding tomorrow in the cncf Adobe launches a case study around webassembly and awesome cloud and how they've been able to adapt and take existing microservices. That they were built a Java they were running these things on their kubernetes.

They're using around two gig of memory each and how by moving them to the new computer to webassembly these things are now tiny little actors that take up, you know, 20 40 kilobytes of memory each. They're so small. In fact that adobe can instead of running them and paying for them themselves on their servers are able to push them down to the customers browser into the customers Edge.

And in this case you as you read their case study, you'll find that it's not only good. It's the same algorithm. It's faster.

There's no network latency and it's cheaper because Adobe is no longer paying to run the servers. So, thank you so much for your time. I really looking forward to this evolution of webassembly and the continued adoption of webassembly in around and on cloud native, and I look forward to any questions you might have.

Thank you very much and have a great day.