Alexander Lawrence – Extend Falco with Plugins: Trigger Alerts with any Stream of Events

Hi, I'm Alexander Lawrence. I'm a principal security architect here at systic. And today one of the talk to you about extending Falco with plugins and triggering from all sorts of events and event streams and things like that.

So with that in mind, let's talk about a little about what Falco is Falco is a cncf incubation level project. It is the kind of standard Cloud native runtime standard for threat detection against up can things running in containers and kubernetes and kind of that cloud native space for the most part. It is a pretty interesting solution set.

There's a number of ways. You can leverage Falco today and use it in your environments. If you want to know more about Falco itself and kind of get into the Nitty Gritty details of what you can do with runtime security around kubernetes containers and all that.

You can get much much more in-depth information from the cncf. You can find a link down here at the bottom of the slide, but suffice it to say it's a pretty strong program that's out there. Excuse me to help you understand more about the overall solution set.

That is Falco and what can be done? See here. So from Falco itself just to kind of take a quick step back before I go into the the really nerdy elements of the slide Falco effectively is a tool that you install in an environment.

It sets up a driver that instantiates against the the kernel of the nodes or the workloads themselves and it starts pulling a system calls. It uses these two libraries live cap and lip snip to be able to parse events and get information from those data sources and then let you do interesting things with the resulting data sets that come out. So basically you can leverage this ruling and that exists in Falco to do all sorts of detections again things happening in that environment.

So you can look for malicious activity, right? You can look for folks trying to access things. They shouldn't be accessing, you know, reading the password file any kind of and send yourself happening on the nodes itself.

You can actually look into the workloads in context of kubernetes and the containerized stuff that's going on in there and look for other malicious things like shells being opened up. Or folks trying to access things that are happening within those containers themselves that they really shouldn't be doing. So effectively Falco is kind of a pattern matching application against data sets that we pull from the kernel itself.

And so to help kind of understand a little bit more about that. We'll kind of dive into the various libraries and modules that we've got specifically the the lib snip and lib cap modules to help understand where we're going with this plugin architecture to make Falco a little more interesting and useful to other other worlds and other areas of the technology stack. So livescap.

This is a library for system capture. It's basically a library that operates in user space at communicates with that driver stack beneath it against the colonel itself and it reads all of the sis calls and all the events coming from that driver set and puts them into a ring buffer. And then at four is them up to to the next Library lips net which then does kind of other stuff with it to help make that be Reversible by the the Falco engine itself.

What makes this interesting? Is that the kind of key element here as we run in the user space and we are putting the stuff into a ring buffer. And so what makes that useful makes that unique is it allows us to be able to do novel things with the the data we pull from the host itself out slowing anything down, right?

It also means you don't have to drop stuff into containers themselves or into other spots to have to figure out what's what's going on with a particular workload or whatnot an impacting it slowing it down causing it causing grief to the overall note itself. So it's someone of a novel thing but it also allows us some pretty interesting levels of flexibility on the way. We kind of handle that that tool chain of information libsnip again is on the the other side of it.

This is the library for a system inspection. And so again, it runs in user space it receives events from the previous library and enriches those events with the kind of machine States and does filtering so Basically, when you start pulling all the system calls from a node you can imagine that gets pretty noisy. There's a number of things going on across that that you probably don't necessarily care about and this Library here allows us to be able to enrich the things we do care about and remove the stuff we don't care about.

So if you're trying to detect just very specific Edge use cases or specific things within the note of self again looking for shells being opened looking for access to sensitive directory sensitive file type stuff. This is the library that allows us to basically parse through all of that information and be able to hone in on the things that actually matter and enrich it with the things like container IDs namespaces kubernetes other and Sundry metadata, lets us track events two particular point in time and do something, you know novel with that data set. So why why does all that matter right it mattered because of how Falco was evolved, right?

It originally was looking at just system events coming from the kernel, right? What we've described this snip and scap. Uh, most modality against that driver set is very very focused on what we can do in the runtime security space on nodes to solve what's going on in those Cloud native environments.

This probably looks an awful lot like Network analyzer tools like things such as Wireshark or whatnot and that's because a lot of that lineage of how Falco was designed is how founded within that concept of packet inspection, right? If you think about the the old days when we had a castle and a perimeter around stuff if we can instrument that Network we saw everything happening with that infrastructure and we start talking about the cloud. It's much more analogous to like a carnival right?

There's a bunch of different individual things going on a bunch of people's individual spaces. They're all now controlling there's not Central aggregation Point anymore. So, how do you get Data out of a system in the novel way that a packet was well the way we do this with Falco is really about you know, inspecting the workloads themselves grabbing that data from the source and then looking through it like he looked through a packet.

That's great. Right it lets us do a lot of cool stuff against workloads against nodes, but it really isn't looking at the stuff holistically. And so the concept was from the community.

How do we take Falco to the next level? How do we really push it to be able to get more than just that data source on the nodes themselves, right? There's a bunch of other things out there that are very valuable.

And in fact critical to the way we secure workloads in a runtime context that don't involve a system call. And so it was really about how do we extend this engine that we've built? You know, this these two different Library modules to handle more things.

And so the initial instantiation was well, let's let's make a web server that's in front of Falco and then we can take in data sources from you know, different API endpoints and then kind of have it interpreter through the the libs, you know SP module that can take that data and make it conform to a Falco format. It was was very very good for a V1 right it was it was great to be able to Hole in new data sources and enrich it and do something with it. But the problem was is that it had kind of this unclear boundary around, you know, what apis we communicate with and made it really difficult to basically take data sets can form that to a way that worked with Falco and do something with it.

We get how to put a web server in front of Falco, which is never the greatest thing to do. We had to manage TLS. It also made it really complicated to handle these types of things in managed service environments, right?

So things like AKs eks gke other managed kubernetes Solutions. It wasn't the most ideal implementation to to make that work. So great idea really good V1, but how can we make this better?

Right? How can we make this extendable? How can we build out something that lets us interface with whatever we want, you know pluggable data model for Falco in effect.

And that's where this comes from. It was really about taking data sources in a native way from the definitions coming from the providers, right? It could be the kubernetes audit API.

Could be Cloud logs. It could be any number of API sources we want to interpret but we needed it to be standardized. Right?

And so we we took this concept of the the plug-in architecture and applied it to these Library modules, right the the snip and scap modules to basically take them and leverage the power that they're bringing us as interceptors of data and do something new and novel with those. And so this plugin architecture was really about trying to build something that is more of a dynamic, you know, Library Dynamic share of libraries that we can then take and ingest any data source that we want basically turn Falco into something that was just for system calls and really turn it into a standard to be able to interpret streaming data sources. So if you think about system calls, they really are something that is, you know happening in sequential order and happening in a very large magnitude, right?

There's there's tons and tons and tons of calls coming through any given time and as the nodes get bigger we start talking in the millions or tens of millions of events. So if we have a tool like Falco that can handle a ridiculous number of events coming at any given time how powerful could that be if we applied it to other types of data sources and that was really what this plugin architecture was about. If we've got something that can go and accept so much data so quickly and do a analysis against that data set so easily it can be really powerful to do it with other stuff.

And so this this architecture is really about accepting data in its native format and then letting Falco be able to understand that and do something with it. So the plugins really come in two flavors. There is a source plugin and a distractor plugin.

So the source plugin resembles the lib s cap module that we were looking at earlier. Basically it's looking for open and close from event sources. It's getting a batch of events from a particular location events or have UniQue Ideas.

They come through we can eventually extract data from those and that extraction happens on the extractor side, right this extractor plug-in is really the lib snip library. And so we're taking data from the previous location and then we're enriching it again, and then tying it to other things that we see as they come through and and work with the data sources. That's given So what's like an example of that?

Right? What does it actually look like in practice? The cloud trail plugin is one of the first ones that came out and this one is really powerful because it effectively allows us allows you to use the native format of a AWS accounts and read the cloudtrail data as it comes in as it sort of new address three buckets or whatnot and be able to generate event data from configuration change in real time.

Right? So I just like Falco can tell you like somebody opened up a shell in a container in real time with this plugin with the cloud trail set up. It allows us to go a step further and say well, you know, I need to know when somebody logs into my AWS account without MFA a lot of tools could do that but they would be on a pulling interval or they would be delayed.

This is doing everything in real time. So it allows our customers of Falco our adopters of Falco to basically be able to say hey, I need to do real-time analytics against my cloud configuration against my and set up against my you know infrastructure as it may be in the cloud of wherever it is and I can now leverage all of the data. I'm probably already capturing in a new and novel way so effectively it's extending Falco to take an Event Source data in real time from cloud configuration and then bring up the field data that we see like usernames info account access things like that and then be able to build a rules on that data set against things in again real time, really powerful tool really allows us to extend what Falco does best to to other things and do this and kind of a new way A New Perspective.

Um, so what does a plug-in look like if you're familiar with the Falco syntax? It's our rule looked like I should say it looks just like a Falco rule or basically just going through taking the same type of of context of a I think happening. This is a login without MFA in this case and looking for conditions happening within the data source, right?

So if we see a council login and we see that it was a assumed role and they were logging in without leveraging amfa so we can see that, you know, Json dot value additional events MFA uses. No, so if that condition meets on a login access that was successful, we generate a critical alert based off of that condition happening again, super super useful for knowing what's going on in my environment in real time to be able to take action when I need to if you're running a production workload inside of AWS or some other cloud provider MFA is probably a standard you have for that production authentication. So if someone a circumventing MFA, that's something you should know about as soon as humanly possible Right.

Leveraging this type of plug-in architecture against Falco makes that alerting actually a real thing that's tangible that exists that you can do against your configuration and make sure that you're adhering to those particular policies. The Json plug-in. This is the other side of the the equation, right?

So up on this side. We're talking about doing the the collection of events itself. This Json plugin would be the extractor side of it.

So basically we're saying now that we've got the data coming in we use this plugin to extract it out of the data source itself and then push it up into Falco for analysis. So effectively this Json plug-in is the thing that lets us take a standard data source, like cloud trail and convert it into a language or a syntax. I should say that Falco understand itself.

Um, so great, you know, we can we can do these things we can be able to do the alerting in real time. We can take this plugin architecture apply it to all sorts of data sources, but at the at the net of it, you know, why would I do it? Why why is this important to me as a customer?

Why should I invest the time and helping develop plugins for Falco and then leveraging with my environments really it's about it's about this right? It's about costs. It's about doing things in a way that's native to the environment you're working in when I start looking at leveraging cloud trail data.

Most people today would say well, I've got a bunch of data. I'm gonna go ship that to my SIM and let my sim handle doing all of that analysis. It's not a bad way to handle it but it is an expensive way to do it.

Right? It's not native to the tooling that we're in front of so what I mean by that is that when you take your cloud trail data, you've enabled it within your applications your services in your you know, you best environment probably storing it in a bucket somewhere and letting it sit there, right? It's not bad in terms of how much it costs.

It's pretty cheap, but then you have to build your own tooling to be able to look against that data set and do something with it. So most people tend to say well I'm just going to take that ship it off to my SIM. And then let my sim do its job which it's very good at but it's pretty costly right.

You have to pay the egress all of that data out of your environment you basically take and you know structured Json and stuck it into a SQL back in instead that now you have to build queries against and do a fair amount of heavy lifting to get data out of that and you get to pay to store all of that cheap Json data and a much more complex indexed format again, you know, it lets you leverage native tooling that you might already own to be able to do that analysis, but effectively you're buying down your licensing you're having to pay additional costs to be able to support that concept leveraging Falco allows you to use the plug-in set that we've just talked about to do the analysis before it ever actually has to leave your premises, right? So it stays in that cheap as three bucket that doesn't have egress costs. It doesn't have to pay to go against a giant back end.

You can do that analysis and place or even in real time against those data sets and then just send the events to that Sim right? Push the events out that matter of things happening to do aggregation. At that point so look for security vulnerabilities or things like that.

So effectively what this plugin architecture does is it allows us to handle the data set in its native format right where it's coming from. It doesn't have to be expensive. It doesn't have to cost a lot and it gives a sense of full coverage.

Right? I think one of the most critical aspects of this is that we don't have to deal with picking and choosing what we're doing security against when you start looking at the former model since it is fairly costly you end up only pushing your cloud trail logs or your access logs of the most critical environments the most critical applications to it because there's a significant cost impact. I'm leveraging the Falco plugin architecture takes away that expensive notion of things right?

It's not having to to pay for all of those fees anymore, which means you're able to have full coverage across all of your content and your your Cloud environment. It's probably one of the most critical critical aspects of it because it allows us to look for other vectors of attack this day and age in the the cloud world. Most of the time these types of attacks are lateral movement attacks right there people accessing resources inside of your Cloud infrastructure in a place that you're not commonly using so it's not following your best practices, then they're leveraging that access to move horizontally within the infrastructure to go do something else.

So effectively what ends up happening is you don't find the breaches. You don't find the risks. You don't find the misconfiguration problems early on you find them after something about your critical data set has already been exposed and you've had to make that choice because it was expensive to do it any other way handling.

This tooling natively allows you to have that full coverage that full scope of view across the environment regardless of where that data is sitting even if it's in places that aren't often used right it's it's not expensive to do it now and now you can have that full awareness of when things do happen or do change in the environment. You're very well aware of it on top of it on top of all those things that the solution set here is highly highly scalable. It can handle, you know, ridiculous amounts of data Falco was built to to an intercept system calls from nodes themselves, right and those notes can get very large and very heavy in terms of their system call aspect of things.

And so we're we're already our bill with Falco to handle a extremely large amount of incoming data, which means you don't have to worry about overwhelming something with a lot of data coming in because it's built to do that. So it ends up giving you a solution set that's you know, open source. It's pluggable.

You can build multiple modules yourself work with the community on these things and give you coverage across your entire. Application stack as it exists within your your Cloud configuration, you know, wherever it might might receive or reside. So with that in mind, you know what what other things out there what other useful stuff is there that we can look through to manage to work with there's tons and tons of documentation out there.

Excuse me, as you can imagine with with Falco itself, the plugin architectures out there. We're registry of all the different plugins folks have written up to date is out there and accessible to the world. There's a bunch of developer guides out there.

If you want to start building your own stuff, you'll find plugins for all the major clouds right around their own audit trail and stuff. You'll find stuff out there on using Falco to detect OCTA authorizations things like that. There's there's a bunch of really really interesting things folks for doing with this plugin architecture to look at their security posture in a different way.

If you ever want to follow things that do get produced and pushed out by the community. Obviously, there's a announcement blog you can follow then there's the getting started Basics with kind of that whole plugin architecture and all that. org is the the project site for all of this.

There's a great active community on GitHub over it. There's also a Falco slack you can join to be able to type or talk with like-minded people and collaborate on various projects or you know, help get help fixing something if it didn't go quite right and I was always there's a Twitter feed you can go to at Falco underscore org and be able to engage the community that way as well. So, I appreciate your guys's time today.

I know that was pretty quick, but wanted to give you kind of a quick and dirty run through of this pluggable architecture for Falco and how it can be leveraged in real life use cases to secure your environment and go on kind of your your day to care about more important things. So again, thank you very much for your time.