Chris Scharff, Hackerone | DevOps Connect 2022

Good afternoon. Good evening. Good night.

My name is Chris Scharf. My Twitter handle for those of you that tweets is not my circus. I'm a security architect.

And today I'm here to talk to you about how ethical hackers can increase your application security. So first just quick bit of background about me. I've worked help desk.

I was a systems administrator with on-premises servers managed servers in the cloud. I've been a QA engineer. I've been a scrum Master a product manager slash owner and today I help customers improve their application security.

So as we're talking about this, I'm coming from a broad set of perspectives and background not all this may apply to you in your particular organization, but hopefully you'll find something here that you can use. so you know as we're thinking about Application security the real challenges that our tax service continues to grow if you're an it. Maybe you've inherited some infrastructure constantly adding new infrastructure whether that's on premises or in the cloud new middleware boxes existing stock market technology new cdes come out for those of you that were like me this over the winter holidays, we all got to enjoy log for Jay and just quick show of hands.

How many people had no idea what log4j was before we got notified. We needed to deal with lock for Jack. I had no idea right?

There's so much technology out there. We can't know it all if you're a developer product owners and managers like me are coming to you and saying hey, I need these new features this new application this new functionality. We need to deliver this we've moved to Agile from waterfall.

And so, you know constant kind of turn there if I'm an operations team. My deaf teams are picking up new technologies. Aw.

OS releases something like 15 new applications a week. My my developers want the new shiny they're bringing that in operations is responsible for configuring and securing that and then at the high level the security team itself. We're trying to figure out how do I manage all of this all the risk that's coming in.

Where do I draw the line on what we're gonna use for testing and how we're going to test what tools to use and the challenge that this presents I think for all organizations is that there's a gap between the attack surface coverage that we have and the actual attack surface that we have in the organization. That's always great. When a vendor comes with a handy granny Dandy chart or diagram just to demonstrate a problem to you, but I think at a high level we can all agree that there is some type of a gap between what we're actually covering and what exists out there if you've ever had an overzealous marketing team that applies new domains and Spins up micro.

You know that there's a tax service out there that you're not managing and it comes from a lot of a lot of other different vectors as well. So move the purple bar up or down as you feel you need to to sort of meet your organizations Gap and coverage but I think it I would be very surprised if anybody wants to tweeted me and tell me you have no no gaps and coverage and so sort of what what drives this so part of this is as I mentioned like an incomplete knowledge of the attack surface, right? So we don't know everything that's out there.

We also have the security testing frequency. We'd love to do, you know full security testing on every build but that slows down our Dev teams not just doesn't happen in the past. I mentioned I used to do waterfall development are it was a SAS company and our time between major releases in waterfall was actually 18 months.

And so during that time two standard annual pen test actually got run against existing infrastructure. We had a pretty good idea of what was there then we released new code and and found all kinds of problems. Agile just exacerbates this or every two weeks every day for releases.

It's hard to test with it with appropriate depth and frequency. And then as we look at our security tools, you know, we have SAS and dassed and I asked and all this tooling that we use to try to find vulnerabilities some of these you run them and they light up like a Christmas tree and so we're busy going through and trying to Mark everything that we can ignore. Sometimes we don't understand the implications of those alerts or it just doesn't understand our application logic well enough to really tell us what that problem truly is.

And then as as Defenders we run into this challenge. It's for the American folks on the on the call. It's sort of like we're in the fourth quarter game was tied three three.

We haven't allowed the other team to score the entire game. We just scored a touchdown and suddenly we switch to prevent me which at that point. I turn off the TV because you know, the other guys are gonna score.

We're sort of just playing defense holding back holding back trying to understand what's coming. And and hoping that we can kind of dress all those and it's a little bit of a specific fee and effort. So you know that being said ethical hackers, how can those folks help me?

So just to give you kind of a storage work from I recently got a coupon code for a food delivery service. I'd never used it before but hey free food. I'm gonna enter the coupon code.

Download the app to sign in I don't feel like creating another username password. So I hooked up to my Gmail account login go to payment pop into coupon code and it pops up a box and says, hey we want to verify your phone number. Okay, so click the button, but it never asked me for my phone number and it's asking me did you get the verification?

Code so at this point I start digging into the app. Oh, there's a telephone number in there. It's not mine.

There's an address in there also not mine. Somebody else's credit card information isn't in this account that I just created and I'm I'm not much of an ethical hacker. Well, I'm not much of a hacker.

I'm pretty ethical. So I find this problem I go on Google search try to figure out how am I going to report this problem to the company, right? There's clearly a security vulnerability here.

Spent about 20 minutes doing Google searches look on the custom on the company's website no indication on how to report a vulnerability. Go to their chat app on the phone thinking, you know, maybe they can take something and file a ticket internally. Nope.

They're script for the people on the chat app is you're having a problem with your order or your delivery driver and you're having problems delivering to a customer no script at all for for how to communicate a security issue. So I actually picked up the phone and cold like my least favorite thing to do talking phone. Call them up.

Basically, same group no script to talk from security team. So at this point I've wasted about two hours of my time trying to report a security vulnerability and I just gave up. Security vulnerability still there as far as I know I don't use the product.

But making it easy for researchers to communicate with you and tell you about vulnerabilities get those ethical hackers involved with their company. Is is something I think is really useful. So I work for vendor obviously, but want to keep this high level and talk about some of the options here.

The first is free and easy and that's to implement a security dot text file. org is security text file Builder. So wizard you can walk through for those of you that work in kind of smaller organizations or where you have a lot of authority you might actually be able to go on this website create a security text file put it in place for your organization before we even finish this talk.

Typically this winds up in two play one or two places either off the route in a security that text file or off of the dot well known interface, which is also used for things like SSL certificate issuance. So having this there can tell the researcher how to get in touch with you. This is one of the things that I look for from the from A company that had that vulnerability in their food delivery app and couldn't find so putting something like this in place pretty simple allows researchers to understand how to communicate with you.

Sort of the next thing that you might think about is a vulnerability disclosure program. This may be something that you've heard about before comes up in the news on occasion something maybe you've even talked about internally. The question comes like why should I do this in the organization?

How do I sell this to the siso? How do I sell this to the CFO that this is something that we should put in place. I think first and foremost.

This has been established as a best practice in in the industry. So the National Institute of standard and technology has put forth this as a as a standard the GSA. It's been adopted by folks in the industry leaders that will recognize IBM Toyota GM Goldman Sachs Hyatt there these kind of Large companies are doing it.

This was originally pioneered by folks like Microsoft who put this in place Google and Facebook or also leaning into this heavily as a way to provide researchers a way to communicate your company and availability disclosure program is really sort of a see something say something type of program, you're providing a mechanism for researchers to contact your company and report a security vulnerability. Um, typically with this you're gonna want to have some type of policy page that explains to researchers what you're looking for what the rules of engagement are. So this is this is partly what you want to hear about and things that you don't want to hear about.

So, for example, you may not want to hear about an incorrectly configured SPF record in your environment see lots of kind of low level no ways being reported companies about that type of vulnerability and it's it's just a distraction right? It's not an actual legitimate vulnerability. Most cases you may also not be interested in things like DDOS attacks.

I get it. Our infrastructure can be ddosed everybody scan. Those aren't the types of vulnerabilities that we're looking to understand and then the other thing that you really want to put in place as well is some type of Safe Harbor guideline.

So this is basically a promise not to sue people if they find a vulnerability and I don't at this point people are sort of laughing. Haha very funny. Like this isn't 1987 anymore Chris.

That's not how the world works. But I want to bring up an example from the state of Missouri where the governor threatened to sue a reporter for hacking a government website. Missouri is the show me state.

So you think they would have been okay with somebody clicking right clicking and choosing view source, which is what this reporter did and what he discovered was on a government run website. They're gonna vulnerability since 2011 where Teacher Social Security numbers were exposed in plain text and by reporting that he was threatened with prosecution now fortunately he worked for a company where they have an army for lawyers and they they were there to come to his defense and the prosecutor said, hey, we don't see any evidence of an actual crime here. But that Safe Harbor guidance can really make researchers more comfortable and Reporting vulnerabilities to you because you know, I don't want to get sued that's why I haven't mentioned the name of the company that had that problem.

So so you've got this this policy page. Now, you're gonna receive those vulnerabilities. You're gonna have somebody on your team or in the organization designated the triage those so not all vulnerabilities are created equal.

So it's a critical vulnerability and low low priority. This is something we already knew about and you're gonna engage with the researcher to say. Hey, thank you reporting this vulnerability potentially following up when you've actually resolved it to let them know, you know again, thanks for having reported.

This our team has resulted, you know, if you find anything in the future, please contact us and then you want to track this data understand the types of vulnerability organization is seeing build reporting on it. Some pretty charged and graphs maybe that you can show your manager to show them how hard you're working and use that data to make data-driven decisions about how to improve your overall application security. and infoculture So that's that's one mechanism another pretty common mechanism that folks talk about in the industry as well is a bug bounties.

So in a lot of ways very similar to a vulnerability disclosure program, but rather than typically being see something say something companies will pay researchers for ballot findings. So this incense researchers potentially go poke around a little bit try to find some of that those application logic flaws and report them to your organization in return for you know cohort cash similar. You're still going to have the Rules of Engagement.

You're gonna have some type of bounty table that tells them based on the type of vulnerability or the type of application involved what the potential payout might be to that researcher. You're going to want to figure out a way to recruit researchers to participate in your program communicate with them. So not just necessarily the policy page, but hey, Have a new version of our mobile application coming out, you know, we'd love for you all to test it maybe providing task credentials to a staging or Dev environment for them to test new releases of cloud-based software.

And then, you know still going to want to treat us definitely look at if you duplication because typically you're only gonna in a bug value program. It's pretty common to only pay the first researcher that reports of vulnerability. That way you're not paying out over and over again a couple of sort of interesting points around bug battery programs one.

We typically see this as being Limited in scope. So it doesn't normally cover the entire and infrastructure for an organization definitely exceptions to that rule. And since we've sort of been talking about this Gap in coverage, why are they going with limited scope?

And and the reality here is because they know there are things that aren't currently heavily tested. There's concern potentially about paying out a lot of downies on properties that they want aware of and are being tested. And so that limited scope will often expand over time or maybe divide and do multiple kind of private programs.

Are particular Focus can be brought by a subset of researchers. If you're running this type of program one piece of advice that I would. Also provide is a take a look at the folks that are participating in that program some of those rock stars.

You may be able to find interesting ways to engage with them. So two two examples of this that I that I enjoy telling all the time one is a researcher that was on our platform. His hacker handle is space raccoon because of course why not and he did a lot of work for the government of Singapore and now he actually works for them.

So he was recruited as part of that that hacking talent that was Finding vulnerabilities and is now part of the team that is defending against them. So I haven't met a security company or Security Group yet have more Talent than they needed so it can be a recruiting playground the other the other research destroy. I like to tell Kim across somebody that's a gaining bug family program and he had 10 to 15 bounties.

There's only program he participated. And so clearly played this game a lot was a big Fanboy of the company and his profile says I'm a 16 year old gamer and security Enthusiast. So 16 years old this kid each one of those kid this this young researcher each one of the vulnerabilities.

They found were somewhere between 500 and 3000 dollars. I wasn't nearly as productive when I was 16 years old, but you can you can find a lot of folks there that then invite those people into Early Access beta for your game. They're already in an Enthusiast and they want to play the new game and they find vulnerabilities in the platform before those get out into the wild.

So bounties can be used not only to help strengthen your internal processes and make those same data driven decisions, but can also be a source of Talent OR entertainment. Another thing that some my customers do though invite some of those researchers come in and give talks. Either virtually or in person to their application teams to show them the techniques that they're using in real time to try to exploit those applications so you can improve processes internally.

So, you know again this Gap trying to figure out how to to close that bringing an ethical researchers communicating with them more effectively providing them channels to provide feedback on vulnerability so that they discover to your team can have a huge impact on your on your business and really address that incomplete knowledge of the attack surface, you know, as you expand your bug Bounty programs, you can you can pay down east for assets that people find that you aren't aware of as you get reports in a public vulnerability disclosure program about vulnerabilities on an old version of Wordpress that was stood up by the marketing team for a site that you didn't even know existed. You can bring those things into your normal testing and management interfaces move beyond for that that basic tool testing and and with the continuous security testing that you get Above dining program be more confident that it has those more frequent releases are coming out that you're getting eyes on the application and folks are are testing to try to find vulnerabilities that sort of basic tools may miss and then as these vulnerabilities are being reported your internal security teams your stock can look at the applications and figure out you know, hey, we've got a Sim did it catch this nefarious Behavior? If not, why not?

And if it did had we respond to that internally and and where we doing the right things that the right time and so engaging in these researchers where they're finding vulnerabilities correlating it to your team in this case or sort of acting as a shadow red team for your organization and so helps improve the skill set of your of your application security team. So with that, my name again is to Chris Scharf. My email address is here.

com. I hope that this has been helpful to you all and understanding ways that you can work with ethical hackers to improve your application security. Please find another video here learn some more technology and if you feel like it, give me a tweet or shoot me an email.

Thank you so much.