Brian Reed – Developer First Security for Screaming Fast Mobile Pipelines with GitHub & NowSecure

Hello everyone. I'm Brian Reed from now secure and I'm here to talk today about developer First Security for screaming fast mobile pipelines with GitHub. And now secure you may come to the web world.

You may come from the mobile world, but I'm going to be talking about pipelines in general blending web and mobile together. Now now secure really invented modern mobile application security over a dozen years ago. I myself have been a Mobility going all the way back to Blackberry days.

And on this journey, I've worked for all of the major mobile technology and mobile security companies these days I work at now secure and we work on best practices standards tools technology training and services all mobile. First, mobile only we work with the top organizations that you see here that many of you rely on in your life and we have a strategic relationship with GitHub and I'm going to be presenting on behalf of both now secure and GitHub in today's session. So let's start with the Need for Speed so it would be nice if we were on stage together, but I'm still having fun one in the virtual world when I think about the Need for Speed I think about race cars.

Now. I happen to own a couple of very fast moving vehicles. I don't have to own one of these but we think about modern development today via mobile or web.

It's all about how fast can I deliver new Innovations and new capabilities to Market on behalf of the business. So the developer side is racing to deliver new features, and the security side is trying to make sure the car doesn't blow up hit a wall or otherwise just a racing down the path. And so when we think about how success actually happens when we look in Best in Class companies, like uber and Netflix and Amazon and Google and so on and so forth It's All About a combination of the team and the technology the team working together to get the car on the road to make the app on the road to get the features in it.

It's needs to run in high performance in great experiences. Got the driver in the front. It might be the guy right in code or you can think of that as the user driving the car, but either way tools and Tech come together in order to make it work.

Now behind the scenes you have the engineers, they're building really cool software and you have security team and they're trying to make the software work. But in the end the race car has to race that's to stay together. It's got to stay on the track and it goes as fast as it possibly can in order for you to ultimately win the race in order to win the race.

Not only do you have to have great people operating together. You also have to have a great engine under the hood need the right combinations of Technology. A perfect driver doesn't win with a bad car and perfect technology doesn't win with a bad driver and so the realities of modern mobile technology today and really modern applications today is it's all about speed and we want to go screaming fast.

Around the track, we want to deliver applications and Innovative experiences to Market as fast as we can to win that Checkered Flag not checkered flag means more users more Revenue more growth more mobile dominance for the business that we're in today. So if we want to run fast just like the racing companies we need to put together the right combination of team and people and technology in order to make it work. And so I've been doing mobile and really application development itself since the late 80s and early 90s mobile as I said since the Blackberry days and as I work with our top clients, I'm finding that speed is dominating everything.

How do I get to Market? How do I innovate a market? How do I add new features and Market?

How do I bring speed to my business? And for many businesses mobile is the business or mobile first has become the strategy in the business. There isn't a single consumer business today.

That doesn't have a mobile app. Most Enterprise businesses have mobile AppSec as well. And in fact 70% of All Digital time now is spent in mobile AppSec and last year during covid.

There were 200 billion mobile app downloads around the world to give you order magnitude pre-covid. That number was about a hundred billion. So during covid mobile became further dominant as a leading platform.

Now as part of our business, we monitor the public app stores. And so that gives us a data set to look at when it comes to the security and privacy state or status and trend lines of millions and millions of mobile AppSec and I'm sorry to say that while there's six million AppSec in the App Store about 85% of them have security flaws and vulnerabilities that would fail, uh, gdpr CCPA that would fail the loss mobile top 10 and that's pretty bad. If you've been around security for a while, you know, we have lots of long running security issues in the web world too like cross site scripting for example, But not only do we have some security issues.

We also have the supply chain as we all know. It's getting worse and worse and that'll be a key part of our conversation today. So what we're seeing here is that mobile Innovation is outpacing mobile security and many instances that's happening in the web world and the cloud world and all the other worlds.

We're all operating in today. So what are we going to do about that? Well?

as a technologist the thing that I've learned the most with all the gray hair, you're looking at me is it actually the team is important or maybe very important or how about the team is the reason for success or failure tools are a footnote when it comes to the process of delivering high quality Innovative effective applications and market for whatever business or organization you're working from And so we need to start with how we bring the teams together. Not what tools are we going to buy and let's think about a strategy where yeah, you've got secure by Design where you're also doing like innovate through design, but you also have to have trust but verify you need to think about how to get these different groups between security and the development team kind of all on the same page operating and Rowing together. This is about you know, how does the security team if the development driver is driving that race car?

How is the security team changing tires in 30 seconds when it pulls off for a break, right? Those are the kinds of things we need to think about in terms of enabling those teams to come together and as I've studied this and as we've studied this through the years we found some patterns and those are the patterns we're going to talk about today because these are best practices that you can apply. What's really great about it is you can bring the human part and the technology part together in order to get these screaming fast Pipelines.

So we got to have a pipeline. So let's look at a pipeline. So you probably have a picture like this in your organization and everybody you talk to and every white paper you download will have some kind of diagram.

Sometimes they're an Infinity diagram. But you know, we we establish some policy for the things we're going to do we scale up our teams we start with requirements. We write some code commit build and test welcome to my CI/CD pipeline eventually get to Stage deploy and then into production where you Monitor and you kind of rinse and repeat around the cycle.

So we're going to use this as a framework for today's conversation. now for gonna bring success to this and get a really fast moving Pipeline with really fast moving fast feedback loops, but we want to do it without racing the the tires, you know driving the tires right off the car flying the car off the road. We need to help these two teams get together.

So how do we help developers can mostly live in the coding phase and how do we have security team who traditionally live in the testing phase come together and how can we do that to a combination of behavior as well as technology in order to facilitate their success? Now stud having studied this for about a decade. Now what we have found what I have found is there really are classic friction points in the pipeline.

So there's the classic I'm doing late stage manual testing. There's no security in the early stage, you know development and Engineering or speeding their way through it. They throw something over the transom security team catches it and all heck Breaks Loose if they haven't been grooming Security in since beginning, they're gonna slow everything down because you know, they're gonna find vulnerabilities and they're either gonna fix the security issues that they're gonna release with them and prod and just take the rest, right?

Now that risk actually can have massive Upstream impact. We have seen organizations like British Airways, for example who had gaps in testing coverage. They were the first and the largest fine ever for gdpr 158 million pounds for effectively escaped defect.

There was security vulnerability that's exploitable to steal 300,000 records. Talk about security Escape defect rates. Uh, Under Armor is the largest breach.

We know of with the highest number of Records stolen 150 million Under Armor My Fitness Pal user records were stolen again because of security failure above these organizations will tell you they were racing ahead and security was left in the dust. So we don't have that happen, right? So how do we kind of solve that?

Well, maybe we get some Automation in place, but but too few organizations have full Board Test Automation in place and then they get this big backlog because you're releasing with known defects. Then you have this backlog and that backlog drag trains you down and development and product management have to decide in the release cycle. Do I fix what's in the backlogger really build the new features that the business is screaming for and more often not they build the new features that the business is screaming for.

And so you get this giant growing backlog which leads a lot of vulnerabilities and privacy issues in the wild. But the interesting thing about it is if you actually do longer term root cause analysis what you find is that a lot of the vulnerabilities exist because of two reasons. The first one is the lack of skills from the development team to understand how to write secure code or how to select.

Secure third-party libraries and make sure they're doing the right thing. It's just some ignorance during a hurry. They grab something they use something whatever happens.

They're really not grooming best practices into how they write their code. So they're not writing secure code and that impacts everything Downstream. The other place that's very interesting that we've seen as a number of organizations the expectations for what security even means is not aligned between development product management and QA and test.

And so there may even be standards that the security team uses but the development team isn't bought in then boom. You're going to get a defect rate that's going to go up if the development team doesn't understand a threat modeling approach and what your policies are for like a tier one risk versus tier two or tier three, then when it hits the security phase you're gonna have issues again. So all these different things like slow things down right this longer release cycle and we're gonna talk about a caffeinated company and how they had a really long release cycle and had to figure out how to innovate it's a case study here and in doing that they and we realize that there's a reason this happens.

So if you do a bunch of studies and you go analyze what's actually going on and you look at what's going on in the daily basis and steps in the Us in all the rest what you find is there are chunks of things that slow it down and they're meaningful chunks that you can dive into and address so that lack of policy upfront waste time the lack of security skills writes bad code which in turn has to get tested which in turn finds vulnerabilities which in turn have to be fixed the lack of security embedded in the tool chain itself and the repell itself so is everything down because he's out on tools take more time the lack of test automation of course means manual testing and we all know manuals way slower than automation so what we want is faster release cycles and what we figured out is it's fairly straightforward mathematically to compress the time and impractical real world of press the time if you address these Yellow Boxes So let's look at how we actually do that. So I want to accelerate my Pipeline and I'm going to take a series of incremental steps in order to help my pipeline run faster. So the first thing we're going to do is we're gonna bring security into the repo.

And so this is an innovation that GitHub was first at and the whole idea is now security is going to bake into my repo itself. Not something hanging off the repo. So let's take a look at GitHub.

Right? It's the number one used development tool on planet or some 82 million developers using GitHub and within that what we want to do is we want to have GitHub taking security action to analysis alerting issues automated updates with pull requests automated scanning on push and pull requests. We want that to happen transparently in the background.

Now a lot of developers don't want something in their IDE, but there's no reason autonomous security can't be operating in the repo. If you do that, then you have centralized controls in place. No matter who the developer is.

The repo itself is doing the work on your behalf. And you're going to configure the repo to do the work for you. So it's really great.

Now as you can get automated scanning on a whole variety of schedules on push on pull on schedule on automation on continuous lots of different ways to do it. That means every day is they're checking their code in or pulling their code down you're getting another set of scan. So you're grooming security and as the work occurs throughout that cycle.

Now GitHub bought a company called codeql a while back and they've now built in code scanning within the GitHub environment. So when you're using GitHub Advanced security you get built in code scanning and that works for your web-based applications late last year GitHub partner with now secure to bring the first Dynamic mobile action into GitHub Advanced security. And so now you can get integrated scanning of mobile applications written in any language any developer tool or framework in the repo.

And so what was natively done for web is now also available for mobile seamlessly. And so now I'm push and pull request. The mobile is automatically scanned an issue tickets are automatic generated back so that the developer in that environment can use those tickets and that's what you see in the blackout screen on the lower, right?

What's great about what GitHub has done is they build an ecosystem a partners? And so while GitHub does web announce here does mobile there's a whole bunch of Partners to do cloud and kubernetes containers and all kinds of other component. That are in the development cycle.

So we have this common infrastructure now with GitHub actions and GitHub Advanced security that allows GitHub to do Native security inside the workflow autonomously in the background and pull through their Partners in order to make sure that your code is secure at all points throughout the development lifestyle kind of a very native very built-in way. There's no tools for a developer to learn how to use because it's on the environment. They already know.

Now we need to do the first party code and when you look at like mobile AppSec for example half or less of the typical mobile app is first party code and the rest is third-party code of many kinds and so we got to make sure that while we're addressing the code. We're writing that native code. We also have to address third party code and there are literally are millions and millions of third-party libraries out there.

So dependabot is a really great tool from GitHub that allows you to continuously scan those third-party libraries and repos and automatically update them. So you can identify unknown unlicensed out of date libraries for mobile or web with now secure and GitHub and by doing that then you're able to automatically generate pool requests to update those to the more modern versions. So basically your code stays up to date via the next pull request each time.

So you're now automatically updating. And again, it's running transparently in the background developer doesn't have to take action. The the dependent bod is doing the work for you get up takes care of this bill tin for web AppSec.

They just announced an open up GitHub actions for third-party s bombs and Analysis to plug in to dependabot in populate the dependency graphs and now secures the launch partner for the mobile app side of the house and there are other partners for cloud kubernetes containers and so on and so forth as well. So we've seen how to look at the first party code and we've seen how to look at the third party code and both of these are all about actually using the repo and the automation embedded in the repo is the most ideal place to build security into that life cycle. So we could say okay, we're done we built security into life cycle.

Everything's there that I could possibly need. It's gonna do all the work for me that I need to but developers still write bad code. And so no matter how good your automation is built into the environment and how transparent it is easy for the developer to adopt we need to think about how do we raise the bar on security?

How do we build security champions in software engineering? But also how do we build security knowledge into the larger team across everyone who's writing code? So what we've learned now over the past couple of years in working with organizations is the best of breed model is a two-part embedded or leveraged learning approach.

So the first one is some basic for mobile, especially there's some basic online security training that you can do like 320 minute courses in your done. And within that training what we can teach mobile developers is hey, here are the most common places we find security vulnerabilities having tested millions of AppSec if you can Master these couple of areas, which basically means using certain apis correctly. Then you're going to write secure code most of the time because this is where the most frequent vulnerabilities are found.

So let's go. Look where the bad stuff is and teach them to make sure they don't do the most common mistakes. So the first part is teach them a little bit upfront now some developers hate training.

A lot of traditional training sucks. This isn't about doing a hackathon. It's straight up.

Which apis should you use for which tasks? And so the second way to do it is training in the moment. So as I am going through the experience of building my application and say GitHub is feeding me issue tickets.

Those issue tickets actually should have a better training in them at the moment of need when the vulnerabilities found and as a developer I crack it open and start working on it. So evidence instruction sample code links to IOS and Android development documentation that they should have read before links to training videos. All of these things can be brought in now and that creates a more seamless experience where I get proactive training and Link learning training as I go and ultimately we find invest the class organizations.

They dramatically groom quality in and basically groom Out Security bugs and vulnerabilities because the net knowledge of the teams goes up. So our NASCAR Academy is free. You can actually log in and start using now secure Academy today.

It's free training for everyone on the planet. And then if you happen to be using platform you get the embedded remediation and training link to it as part of your development life cycle. So we've got a repo scanning strategy for first party and third party code.

We've got a training strategy now, we need to set standards and this really gets to You know the seven habits of highly effective people if you're just fan and so when you look at that begin with the end of Mind should be let's have a standard set of policies in place. Well defined and agreed to from product owners development leaders CTO Architects coders, obviously QA security regulatory teams compliance teams, all of that. Let's have a common set of policies that we can all operate to we've had a number of clients where some of the marriage counseling as it were that we had to do with square basically the technical team and the security team and the government's teams hadn't properly agreed on what policies were up front.

So, of course, that means they're not building a compatible application. They're not building an application that's going to fly through because standard warrant set there. So as part of that we are strong recommenders that you leverage owoss.

Oh loss has a series of very strong very effective best practices that are really put together by an industry group over the Years, whether you're using the AVS for the traditional web where you're the masvs for mobile, what you can do now is bring them through the life cycle establish a policy that says this is the parts of os we're going to use in the framework. This is the best practice approach we use for a little bit of threat modeling. So we know what approaches to apply this could be a strategy where you want to take OS driven policy and actually embedded in the tools.

If we go back to that idea of automating stuff in the repo. Well, why not have the automated Security in the repo be aligned to your choice of the owoss requirements from the asps and the masvs that will make you a lot more successful and that will align you with industry standards. They're there for a reason so that you don't have bad vulnerabilities Escape depending on the risk of your application now to go with this the security team product development team should actually partner with product management to make sure clear security requirements are going into that pipeline.

So it's established some policies. Let's use OAS to help us as a standard space approach. Let's make sure we establish Standard Security requirements.

So security stories as it were functional and non-functional requirements and really bake that policy into the pipeline itself. So choose tooling and strategy that says, hey I can configure it so that the dependabot will scan looking for these things that I care about from oasp or the GitHub actions will scan for those kinds of things in a way that says here's a standard threat modeling approach early in the cycle that we're going to use to empower developers to make the right choices. And then the tool chain does the automated work for them as well through the cycle.

So now we can put it all together. We'd say. Hey, look at this.

We really have three ways to make this work. So we're going to establish policies leveraging standards or in a bake that into the tool chain and bake that into our stories and requirements. We're gonna bring security training and leverage learning throughout the workflow.

Then we're going to bring security right into the code repo that makes it more effective for all stakeholders working out of a repo like GitHub, you know to do the work that they need to do. So if we go do this you actually get screaming fast pipelines. And so we move from the super long release Cycles iteratively into a much faster release cycle within our organization.

And every organization we've worked with that is done. This has had this happen when they look at all three steps. They can materially move a needle.

So I said earlier I was going to talk about caffeine in a mall wired up for screaming fast pipelines Caribou Coffee is a great example of a slow moving organization that basically was on a quarterly or slower release cycle. Well Caribou Coffee was all about the coffee experience in the coffee store. What happened when covid hit?

When covid hit they couldn't have people come to the coffee store anymore. So there are simple coffee app of find me a local coffee store had to turn into basically a transactional app where you could buy coffee for delivery or pick up at the curbside and they needed to figure out how to innovate very quickly while they were all at home on covid. and so bringing in this strategy has enabled them to accelerate their development life cycle being able to trigger tests when they promote code is saved meaningful time.

They move from quarter releases to monthly releases to my weekly releases until now they're innovating and in a rapid life cycle to meet the demand to the business while practicing good standards-based policy hygiene while practicing linked and embedded learning through the life cycle. Well automating security testing in the pipeline and the in the skill set of the developers coming together to make this work and it's very exciting to see many organizations like this be successful. So if you're if you're on your team, you're looking at SecOps, you want a solution blueprint if your mobile focused here's a very common blueprint.

We find organizations used to be successful. So set to your policies and controls by a standards that gives you that predictability and consistency through the pipeline that will drive the ability to have predictable development Cycles where Dev and security and governance are all in the same page do some of that training and a minimum do the training on the top five security vulnerabilities always found in mobile app. So you don't make these mistakes that'll move the needle material in your organization.

Make sure you partner with product management to include security requirements based on those policies within each application to threat model. Make sure you're continuously testing the mobile applications or any applications you have frankly and analyzing the upball sbalms Levering leveraging the repo. So do it inside the GitHub repo leverage dependabot for the third party code and make sure there are automatically gener.

Knows issue tickets into the GitHub workflow. So those developers can fix them quickly and that GitHub environment they know and make sure those tickets can speed remediation. So it's not just find fast.

It's about fix fast, right? We talked about all the elements of a successful ticket is remediation instruction code examples embedded training links to documentation and all the rest and that will not only accelerate the initial fix. It'll reduce the likelihood of a repeat issue.

Now sometimes when I talk to security folks are like wait, wait, wait wait, I have these super high risk scenarios and I still have to do pen testing an automation doesn't replace pen testing. And yes, that's true for super high risk application. You may need to do some pen testing.

But guess what if I'm doing everything else here the likelihood of severe security vulnerabilities showing up they'll be found by a pen test is very low. So whatever pen testing remains can be smaller scope pen testing just focusing on areas without Automation and because the quality and grooming in it's much more likely to pen testers don't find anything of material issue. That would slow you down or block you when they release Now mobile has a specific deployment challenge out there, which is you need to be able to make sure you get through the app stores.

So Apple and Google each have with their app stores some requirements. You want to make sure you catch those App Store blockers early on these may not be security or privacy related. But Apple has privacy rules.

Now Google has the data safety labels. Now you want to make sure that you do those parts correctly as well. So you don't get blocked from the App Store you need to use the right Library types.

You have need to have the right minimum versions of the sdks and things of that nature. So make sure you watching for that. And the last thing is monitor in production.

You want to monitor in production for two reasons. The first one is you want to make sure that whatever you actually tested and certified pre-prod is the right binary that made it into prod the other reason the Monitor and prod is any dependent libraries or services that might have been secure early on could become insecure due to updates later so you don't want an existing in production application to suddenly Come in secure because the way in a library is calling a remote service in the background. It's on Monitor and production so you can periodically check them and make sure something hasn't happened since you deployed the build.

I'm gonna take a brief break here and and shift down for people who are mobile Centric everything as I've talked about is possible in the web world and the mobile world. But if you're focusing on mobile, make sure you focus on the right coverage what we have found in testing millions and millions of AppSec is the bulk of vulnerabilities. We find that escape into production are related to data and motion and data at rest.

So teaching your developers on how to make sure they understand Secure Storage and secure networking is a critical component of your success and make sure you're testing tools are testing for those as well. Nothing to test date a motion date and rest. That means you have to test dynamically.

So that means you need Dynamic testing as part of your strategy and yes, it is possible to automate Dynamic testing. You also want to make sure that while you're testing frequently and fast that you have those remediation assistance capabilities and that there's high accuracy in the output. So here's a little checklist for you to help you drive your organization.

Screaming fast. We just saw how to do it, right? This is actually about continuous Improvement leveraging Three core strategy types and the foundational elements are aligning the team together to operate as one around the race car as it were and empowering them with the best tools in order for them to operate and you can get this fantastic win-win that allows you to race that car faster and get those releases out the door faster.

If I could bake it down into three things, the first one is continuous policy establish a behavior and a practice in your team of using policy to drive that behavior leveraging into standards to do that. Make sure the devs understand them all the stakeholders understand them and you fake it into the tool chain. Continuous security should be built into the repo.

It's the most ideal place to have security operating autonomously in the background push and pull requests off our scanning could all run independent of any individual developers Behavior continuous learning is critical need to do some upfront learning but more importantly make sure you have that link learning where the issue tickets you're generating has the learning in there. So you not only fix it fast, but they learn a skill so they don't write bad code again in the future. So with that just to close out we have a ton of free resources here.

You have the embedded links as well. These links will be in our portal for the event. You can see what the App Store looks like.

There's a link here to Academy. There's a link to get a free report. We'd also generate a free s-bomb.

There's a link to set up your own GitHub actions and your own GitHub dependabot actions as well. And so again, my name is Brian Reed. I'm from now secure here on behalf of now securing GitHub to talk about really developer First Security for screamingly fast pipelines.

I hope you've learned something here today. You're welcome to hit me up on Twitter or through email. Please stop by the now secure Booth or the GitHub area to learn more and I wish you the best and driving really fast with your mobile AppSec in the market.

Have a good day everybody.