Doing AppSec the Hacker Way | DevOps Connect: DevSecOps 2023

Today we're talking about DevSecOps the hacker way. We'll cover a few topics. I'll give a little bit of introduction, my bonafides, talk about what the problem is for trying to solve, dive into what I mean when I say DevSecOps like a hacker.

And the first time at the end I'll be in the chat for any questions. So first things first. I'm Josh Lure.

I'm the VP of Marketing of Oral Secure, oral Secure. We make a platform called Mayhem. It's a developer first security testing solution, oral secure.

We're a hacker organization. Get into what that means later. Myself, I'm an ex developer, QA engineer, and DevOps engineer.

Over the past decade, I've been a marketing leader at organizations like Puppet, Twistlock and Torque, focused on application security, cloud security, and DevSecOps. com. Let's start with a few core principles.

It's better to find a fix before production. I'm not trying to blow up what DevSecOps is here and say, Hey, it's not valuable. It is security and dev need shared visibility, no playing telephone, single source of truth on what matters most.

Also, last time I gave this talk, someone said, well, isn't what you're talking about really sec DevOps? Look, it's semantics. It's about how do we work together, not how we order these words.

So with that said, let me go into what the problem is and again, you know, when I, when I left the development world, we were shifting things left. Hey, how do we find it and fix before production? We've been doing that for over, you know, 15 years now.

But 41% of AppSec professionals say they spend five hours a day on security issues that should have been caught before something went live. Developers surround a third. So that's a lot of stuff that's not being addressed, even though we've shifted left at the same time, 80% of teams say that they get over a 50% false positive rate in their AppSec tools.

That's a lot of noise to sort through for developers or KPId on shipping new features. Thirdly, look, if I read a survey where 75% development teams say, well, sometimes we skip security steps and 25% say no, we never do that. My takeaway is that 25% development teams are, well maybe fudging the truth on the survey a little bit.

And the reason for this is because speed and security are so at odds, development teams are pushed to deliver and to deploy. And sometimes that means end running around slower steps. So when we think about shift left, like what really shifted left, well, we shifted left a lot of things around posture.

You think about AppSec, static scanning, dynamic scanning, S E a pen testing. You've got kind of this notion of posture. What are the risks I have in activity?

How does my application behave? And are there anomalies or risks in that? Well, what we shifted left was largely the posture pieces.

Find and catalog all the risks. We didn't really shift left the piece that says how should this behave? How does it behave?

And how could it be made to behave in a way that opens me up? Exploits. So you have a lot of noise that got shifted onto developers.

You have a lot of well go scan for the vulnerabilities that have already been cataloged, shifted that left. But we didn't really do anything to take those AppSec practices that used to be the responsibility of a post-production AppSec team and make it better and make those techniques and those tools help developers find and fix things faster. We just said, well if you do it earlier, everything's faster.

But we didn't actually make those techniques themselves any better or invent new techniques. Meanwhile, attackers still have an advantage and there are a lot of reasons for this. You know, attackers only need to be right once there are far more developers pushing code than there are security experts who can assess on AppSec report.

But more than anything else, it's security can do every single thing. Right? Here's my software bill materials, it's all clean, we audited, we did testing, we released, yay.

And then the next day someone finds a zero day in a component you're using and there's an exploit out in the wild. And then the day after that you're stuck doing costly incident response. And this is, this is a fact of life for ABSEC teams.

This is a fact of life for developers. This just happens because attackers still have the advantage from how they approach their work versus how we approach the work of integrating security into the development pipeline. So what is the solution?

And again, we don't have a ton of time, so I don't wanna spend hours on detailed technical implementations, but there are a few concepts that we need to think about when we think about shifting left. When we think about DevSecOps, and these are concepts that don't come from traditional AppSec, but rather from how attackers, how hackers operate to find exploits. And this is really down to the meat of the matter, finding what's exploitable, not what's vulnerable.

Not every vulnerability's exploitable, you know, I talk to security teams, I talk to engineering teams, I talk to leaders, and one of the biggest conflicts they always say is, this vulnerability doesn't apply to how we run. And figuring out what that means for priority security wants it fixed, it's critical. Dev says this doesn't matter.

How do you resolve that? That's a really tough challenge. But that happens because we start and focus on what's vulnerable, not on proving first whether or not it's exploitable.

So look, oh, find the things that someone could use against you. Whoa, crazy idea. It's, you know, this isn't rocket science.

Um, it makes more sense to focus on the things that can be used against you than the things that can't, you know, if it, so then the question's like, why didn't we do this already? What's different now there are a few things. So you think about rise techniques for behavioral testing once upon a time, if you wanna test the behavior of an application, and I remember this in my early days doing qa, you did it manually and that that wasn't really, and that doesn't really scale nowadays.

You know, be it things like fuzz testing, be it things like, you know, fancy algorithms, symbolic execution style techniques, or be it, you know, again, I'm going, I'm going, can I say this? Things like generative ai, things like machine learning algorithms that can rapidly process application behavior and use that to understand how to create new and interesting inputs and test cases for that app. Those types of things are much more prevalent now, let's be honest.

They're being used and they're being used by folks who aren't trying to ship new features. They're trying to find ways to break things. And so the multi of techniques for behavioral testing and the advances in things like machine learning, portfolio algorithms, generat of ai, as an aside, as a cybersecurity marketer, I love being able to actually say generative AI in a way that, hey, guess what?

That term's already been defined in the marketplace now versus a year ago where I'd have to yell about, ah, no one knows what that means. Um, putting that back aside, but advances in how ML and AI work and the scale at which they operate allow rapid creation of inputs and test cases that are targeted towards specific applications. It's not, you know, the blind and dumb fuzz testing of, you know, the early two thousands.

It's behavioral testing that leverages multiple different techniques that's guided by every single pass result. And everything that's known about application architecture, it's all well and good. But the other thing is computes gotten cheaper too.

And so it's easy to spin up, you know, instances in the cloud or local and start running behavioral techniques that analyze applications as they run, run. I could dive into this for hours, but the gist of it is these advances in compute allow for a style of testing in your DevSecOps pipeline, in your delivery pipeline that's oriented around how a hacker would approach breaking into an application in your building. And so what this really means from a, you know, putting into practice is you focus on vulnerabilities with proven explodes.

You don't take a list of 200 CVEs, you use the right technology to figure out which of those CVEs is exploitable the way your application runs. And you ship that to a developer and say, go fix this. It's exploitable.

If a developer has to spend time, one, ordering and prioritizing illicit cvs and two, aligning with security on that priority, that's time wasted and that's time not spent fixing. Two, define your risk based on both posture. Where am I vulnerable and behavior?

How are these vulnerabilities going to impact me when my application runs? It's not just posture. And that's a myth that we often have on the AppSec side with static scanning, SCA and things like that being shifted left.

How can it be used against my application, my architecture. Three, automate testing and triage. Automate testing and triage cannot say this enough.

And this doesn't mean automate a scan. This means automate generation of tests and automate triage of results. Can this result be reproduced?

Can we deliver a stack trace to a developer if it caused a crash? Can we show where in a code anomalous behavior first was seen? Those types of triage activities have to be handled at the machine layer, not the human layer.

Because every minute that a developer spends sorting it out, well, where would I start to mitigate? This is time not spent actually mitigating or remediating. And also don't stop once you've shipped.

This is the thing that I think is really great about some of the new advances in compute power ml. The more tests you run, the more inputs you have for the larger you generate a model of application behavior, the better you're able to find new and interesting ways something breaks. Don't put it in your pipeline, don't slow down delivery, but constantly test your applications as they're in production to find new things to shift back into dev for fixes.

So kind of look, what does that all look like? This looks like using techniques and tools of attackers. You look at things like heart bleeds, you look at, you know, I don't know if anyone saw the times those folks hack to Tesla just by flying a drone nearby it.

They're using techniques taken from fields like testing, pen testing. They're using human intelligence to know where to start and machine intelligence to know where to scale. We need to do the same.

You gotta automatically generate behavioral tests against your applications and you gotta let machines scale that. Stop making developers figure out if things are real. Only validated results.

Go to developers first and foremost. If there's no trust that results real. You're moving slow.

You're not aligned on priorities and you're not focused on actually staying safe. And also, success isn't no critical CVEs, that's compliance metric for security. It's making sure you're fixing what's exploitable and minimizing the risk that your application, your architecture, your deployment has when it's running.

So wrapping it all up, you've gotta shift left. That's a must. But shift left created a security burden on developers that hasn't been fixed to this day.

If you start to move away from traditional AppSec in the development pipeline and start to use the techniques that hackers, attackers use to identify exploits, you're able to actually ship faster and be more secure. If you have aligned AppSec measurement between Dev Nsec, you don't ship false positives or low priority fixes or low priority issues to devs and say, Hey, go triage this. Go spend hours trying to figure out how to fix and you're able to actually ship and release faster.

So why am I saying this? Here's the Shameless Plug Mayhem. It's a platform that takes these concepts, takes these techniques, and puts them right in the middle of your delivery pipeline so that every developer has access to this type of technology.

Thank you so much. I'll be in the chat with, answer any questions or do any follow up. Hope everyone has a great rest of your day.

Cheers.