Mark Lambert – Scaling Application Security to the Speed of DevSecOps

Hi, my name is Mark Lambert VP of products at Amaco. And this session we're going to talk about how to scale application security to the speed of devsecops so that you can ship secure software and ship it fast. So let's start with in the beginning in the beginning.

We had waterfall. It was a nice prescriptive process. We first of all Define our requirements and then from a security perspective really defined what the security requirements were and we then went into design and architecture and at the same time we could run parallel security and architecture reviews.

We then would start doing implementation. So the development teams would start implementing software writing code and to assist with this we would develop secure coding practices. That would be very fine through basically first generation static analysis.

We'd also be running some kind of pen testing to kind of trying to uncover the security vulnerabilities during verification and then we would switch into publishing live and having maintenance mode and then we'd respond based upon when we saw potential vulnerabilities in production. And and we do iterative maintenance as we went. Now but then Along Came agile and turned this all upside down as we started changing the way we deliver software so we can meet the business demands.

So what you know, what what would the fundamental things that we've done to change our software development process? Because what I mentioned as I mentioned a moment ago, what we're trying to do is we're trying to really meet business needs right delivering software delivering functionality delivering Innovations as quickly as possible to Market so we can really Drive competitive advantage. So we've done a transformation of our software development process.

We've transformed from this waterfall model to Agile where leveraging microservices so that we can do independently Deployable and component releases rather than having to wait for deliver a deployment of one big monolith. Well leveraging open source to get a jump start leveraging Frameworks that do common functionality and but we're also leveraging third party services in this world as well. So we have an increased dependency upon external entities.

Well also leveraging Cloud platforms to scale the the web deployment and the infrastructure that powers our applications. At the same time as increasing the complexity associated with application delivery and application security. We've also got an increased risk within the market.

We've got a lot more commercially and politically motivated attackers. We've also got a lot more government regulation where really privacy and data protection is the tip of the iceberg. So really before we jump into what we need to do, we need to get an understanding of what the current state of the market and the state of the industry is and this is where recent report published by the purple book Community the state of application security operations, really highlights a few key elements that we'll talk about today and use that to extract what leaders are doing to really ensure that they can ship their software secure.

So the first thing within this report is that we see that the majority of respondents agree that shipping fast takes priority of a shipping secure meaning that the meaning of the business demands misses needs for competitive Advantage outweigh any security risk and what that results in is that because we're pushing functionality hours quickly as possible. We end up having unmanaged risk within the portfolio. Now, you're always gonna have some risk in your portfolio.

The question is is it managed. Do you have Insight? Do you have the ability to make informed decisions as part of yourself by delivery process?

But wait, there's more. It's not just about how we build the software today. It's actually what's embedded within our software.

So really security isn't a point in time anymore. It's not a case of delivering the software and then just kind of like monitoring it because we're leveraging open source. We may have embedded security issues within our application.

We don't even know about yet. So things like log4j solarwinds. Hope lead all of these big security events can really be built down into saying.

Okay. What does my pipeline look like and what technology is around embedding in there and these high profile security events. I really just not a point in time either.

The impact of them is really very long-running and organizations are still struggling with heartbeat lead that was over six years ago now, So the bottom line is we need to identify and we need a respond and we need to respond fast. So how are teams typically putting in the security measures or the security guard rails in place so that we can achieve that well. According to the survey not surprisingly the number one initiative to really am we build quality into your process is to take security tools and embed them in the pipeline.

But we have to think about that. It's not just about the code anymore infrastructure Security application security these two things are really coming together. We have a traditional vulnerability and management approach which is primitive based infrastructure-based.

And then we have our application security approach which is really looking at the pipeline and how we embed the technology into the pipeline. But as we start leveraging Dynamic cloud-based infrastructure, we're leveraging infrastructure is code this really brings these two things together and your infrastructure is always changing. It's on demand.

You don't have a server that you can simply patch you need to be monitoring the containers and being able to tie vulnerabilities that are identified in production back to those original container images infrastructure really is part of your application now, So what this means is you have all of these different tools and our traditional approach to application security is painful and not scalable. We have Point tools integrated into our pipeline as well as infrastructure related tools all generating massive volumes of data in a fully automated way. But this massive volume of the data is overwhelming our teams with alerts findings and vulnerabilities lots of automation of the generation of data, but not a lot of automation for their mediation the triang basically what you need to do with that data and This is really impacting multiple teams within your organization and teams are important because it's not just different development teams is your security team and your development team.

They need to collaborate and the disconnected tools and processes really that's not generating finger pointing and frustration within the organization and at the root causes you don't have enough Security Experts in your organization. Typically speaking when I talk to organizations, they've got an out number of a hundred to one application security Engineers, which really overwhelms the staff. So what does that look like?

Well, we have all these tools embedded into the pipeline generating massive volumes of data. That's overwhelming our app sack development and devops professionals. Our website teams are trying to crush correlate this information bringing it all together so we can be all in the same page.

We can communicate and often times Well leveraging Excel, especially it's emails slack Communications just follow up on where we are on things and this really does not scale. We need to cross correlate this information in a really efficient way. Otherwise, we're just really chasing our tails and we're never going to be able to capture all of the vulnerabilities before they go out of the door.

Now again, looking at data in the survey and really organizations have the data so over 71% of those surveyed said that they have sufficient security data to understand the risk to the development organization. However, visibility is still a problem about 60% and realize that they have average or below security posture visibility and over 60% name it as a top three priority for improvement over the next six months. Sorry next 12 months.

So that's so good problem statement, but what are the leaders in the industry doing to ensure that they can succeed in ship secure software and ship it fast. So let's look at defining what a leader looks like. A leader is those that never release a critical or never or rarely release a high vulnerability into production this within their survey represented less than 10% and interestingly more than 25% of the survey said every release or even worse didn't know.

So let's look at what the remediation times look like when we're talking about these type of vulnerabilities. So critical and high you can see the majority of people fix a critical between one and five days and a high typically between you know, between five and ten days depending upon that level now, of course depending on where that vulnerability is identified and how frequently I'm releasing is going to dictate if I'm actually releasing that into production so when we look at our releases What leaders do well the leaders really never release more often than weakly gives them the time window to remediate within the time frame that they have also the most common actually within the leader category was actually every two weeks to ensure that it really not releasing those things into production. other interesting survey is looking at how respondents actually You the tooling and and really the vast majority of people view the tools as in to changeable is the processes the most important now, what are the tools that people are leveraging and what's the approach that they take?

Well the vast majority and really this was very very evident in the best of read the vast majority leverage a sorry with the leader group The Best of breed approach to Tool selection. Meaning I'm gonna pick the best tool for the job and then I'm going to build my process around that so I'm not relying on just a pure open source, or even just a pure single vendor approach. Now the different techniques that are being leveraged this the this report is actually there's a second report.

There's due out and about a month. I believe and leaders are leveraging SAS dashed and penetration testing as core capabilities. But those that really are excelling in their delivery also are starting to bring in infrastructure Cloud container security as well as Source composition analysis into the equation.

Now when we look at what are the things that these folks are all gonna focusing on and what are the things that they identify as the challenges when it comes to a successful abstract program. The first is hiring an app second engineer being able to find people that understand the problem is a really challenging and a point within the industry. Disconnected systems and processes.

We already talked about that. That's number two and then Paul collaboration between the teams is a number three now as interesting when you dug deeper into this looking what the leaders do that poor collaboration is gone away. They've focused on addressing that because that's a key element for you to be able to scale the impact of your team.

Now if we look at the focus over the next 12 months from that survey and automation of critical apps Tech workflows, which you know, if you look at this you're thinking about releasing faster, you're thinking about resource constraints obviously automation is gonna be a top priority. Visibility of the application security posture again, we all saw that the data was there to show that we have the we understand our visibility. We don't have the visibility.

So identifying that that's an area for improvement as well as that collaboration between security and development teams because if you can eliminate that friction, if you can figure out a way of scaling that that's the thing that you're gonna really be able to leverage to overcome the bottlenecks. And this is what an application security operations or an app SEC helps platform really helps you achieve it helps you you bring order to this chaos improve the operational efficiency and scale the impact of the team. So what do we do with an application security operations platform?

We're ingesting data from across all of these different tools and Technologies where normalizing did duplication and correlating those into a centralized view so we can provide actionable insights to the team so that the security team can very quickly triage and identify points of prioritization so that can then collaborate with with the development between through automated workflows. Those workflows should include the ability to be able to have request approval workflows and check some balances so that you can really scale the impact of your team through the use of something like a security Champion or security liaison program. You also want to unify the service level agreements so that as a team, I'm all looking in one place to understand what my remediation times what my targets are because if I'm having to go to multiple Point tools at different points in time, we're not going to get the operational efficiency required.

And then third but not least. We need knowledge base. We need some education capabilities within our platform so that we can enable the team to not only educate themselves.

But furthermore, we need to identify ways that we can Elevate the tribal knowledge within my organization so that when I have a security vulnerability, that's come to me and I don't understand this. I can look at some general information, but it can also identify who within my organization has remediated this type of problem before and potentially also look how they remediated with it. If I have integration with Source control.

I can identify fixes within my organization that have been applied and I could also identify other tickets or with I have Integrations with something like a jira so that I can identify the different ways that other teams have very immediated similar problems. So in absecopics platform allows me to scale my teammate. It really enables me to achieve four key use cases.

First of all application security posture management getting a complete 360 degree view of all of my software assets integrated with my desktops tools. I also want to be able to unify vulnerability management traditionally or traditional approaches to vulnerability management Focus very much on infrastructure and perimeter defense. We need to now start looking at how we can take the data coming from our abstract programs, which is going to be a lot more frequent and a lot higher volumes and bring that into the same process and it's typically easy to go the other way around.

It's typically easier to start looking at my appset program and saying, how can I bring in the infrastructure? So unifying both software and infrastructure assets into one workflow. So the development team can help and work on this because ultimately when we're looking at application security Security team can't perform.

The remediation tasks. It needs to be the development team and the development team don't necessarily have the security context that the security team have so we have to build collaboration and this is where devsecopes orchestration comes in improving both the human collaboration effort leveraging things like jira and slack the ecosystem that the development team are using today unifying the slis that I talked about earlier but also orchestrating execution of some of my pipeline tools as well. So tools such as sasta Source composition analysis static analysis security testing.

Those are gonna be embedded within my Pipeline and executed as part of a Jenkins or Circle. It's a CI for example, but something like a dashed or penetration testing or an infrastructure Scan, they are not necessarily going to be done as part of the pipeline because I need to be able to make sure that everyone's is quickly as possible. So that my development team can really get the information that they need to validate the functionality in the work that they've done.

So orchestration is a key component. As well as important. And then last but not least compliance and compliance really is the last bottle making the pipeline.

We've got techniques which allow us to shift left security from an analysis perspective and testing perspective. We've got techniques which allow us to move performance and and site reliability testing earlier in the pipeline compliance requires us to have a mere real-time view as we're going through our development process. So compliance is not the last checkpoint or puts the brakes on the pipeline.

This is where if we can get a near real-time view into our compliance position, we can eliminate drift we can also streamline an optimize auditing and processes. So the really five steps that we can go through to scale our application security program and unify our vulnerability Management program so we can meet this the needs and the pace of delivery the devsec helps brings in. First thing it's unified.

We want to get everything on the same page. We want to be able to unify severity reporting. We want to be able to unify workflows across all of my tools giving us a single source of truth that gives the teams the ability to leverage information from Technologies across the Parkland and not across the infrastructure that they may not be experts in if we have something can bring it all together.

I can connect together Concepts from a sneak for example with a qualities or a rapid servant and bring all of that together into one Central source of truth, but also unify my workflows and my sla's around that The next is that correlation. So not just correlating the the data from the individual tools, but correlated them together leveraging something like a cve for example and assigning risk using a vulnerability database like for example nvd and bringing together the data from as I said before both code and infrastructure together and understanding who owns those vulnerabilities and findings and alerts and assigning them appropriately having to sign defined roles. So I understand who is the business owner who is the security lead.

Who is the development team members that work on this? Third step is to Define our contract and this is where service level Agreements are critical you want to be able to make sure that you've unified the first step the findings across the different tools so that I have a uniform view of severity for example, but then I also want to unify the service level agreement based upon that severity but correlating that with the type of application. I'm a developing.

So for example, if I've got a front-to-end web application that has the personally identifier full information inside of it that needs to be that's a high risk and application. I need to have a rapid response associated with that. So I want to Define what my sla's are for first response.

So my triaging efforts what my sla's are for developer response. So my resolution time for example my remediation time and then the end to end resolution time and that's going to be different than a back end data science application where I'm just gonna Using internally, it doesn't actually get have any customer facing applications. There's risk there, but it's a different level of risk.

So to make sure that we can meet the needs of the business certainly in the case of those high-profile front-end applications will also going to want to be able to automate the processes and the critical workflows to make sure we can meet the slas that are required to really mitigate the risk for the business. We then get into collaborate. I mentioned this a moment ago.

This involves teams for us to be able to get together and actually solve the the problem of scaling abstract or just scaling Security in general to the speed of desktops. We need to collaborate leveraging the systems that the development team allow already using today. So integrating with gear teens slack and as your as yours are ticketing management system as well.

Being able to leverage those and when we're creating work for the development team to perform remediation. He goes directly into the systems so they can immediately bring it in to their Sprint planning efforts. We then want to be able to collaborate on the triaging workflow.

There's going to be discussion and development will say that this is a false positive given the context of the code or this doesn't apply we need to accept the risk, or maybe this isn't a vulnerability. It's a bug it needs to go to a different team. This communication needs to be facilitated.

It needs to be tracked and they're also needs to be some workflow that promotes approval of those changes as well so that we don't just have in essence development team, you know going rogue per se or make you or a junior developer making a reprioritization that really is incorrect. We also need to provide a framework for our security teams to really change from if you like the security guards to being more of a mentor or a coach and this is the key for being successful in scaling the teams leveraging a program like a security Champion or a security liaison program where you have representation within the different development teams, and we're facilitating the collaboration between application security your security champion. And then your security Champion is the one that's working within the team that the expert that the conduit for information and they're also able to promote the security best practices so that we can get better not just fix the things that we're seeing today.

And then last but not least. We need to monitor the process. We need to be able to monitor the service level agreements much more important the remediation time remediation time.

You would need to further segment by criticality and also segment by the times of application that we're talking about if we're focusing on the sla's we can clearly identify where the breach is happening which teams need help which teams we can hold the role models within the organization. We also need to Define Gates or guardrails within my cicd pipeline so that I can ensure that no critical vulnerabilities are me getting making their way out into production so that I can provide warnings to the team when we are actually hitting the ends of the the corner cases of rslas or that we've got for example higher vulnerable issues. I high severity vulnerabilities and inside of the the deliverable that need to be remediated quickly and being able to integrate that into the pipeline put also communicate that through slack and email for example, so that we can get and a communication and awareness to the teams.

We also need to have Discovery being able to identify when a new source appears within my environment. So for example leveraging. We're repository analysis of something like a GitHub to identify Repose that are new or being stood up within the environment that don't have security tools embedded within them and be able to set up automated onboarding workflows so that we can notify team members set up that visibility.

So we in essence don't have a backdoor go through our process. So when we're looking at how to enable these things there are really four key areas that we want to focus on when we're talking about enabling app Cyclops. The first is that you want to everything in one platform.

There's a lot of needs that you have there. You need to be able to make sure you're not having to go to a number of different tools. That's the fundamental problem.

We're trying to solve here is we want to bring everybody together bring it together into one system, but it's important that you're leveraging the infrastructure you already have in place. So different teams are gonna select different security tools. It depends upon their needs.

It depends upon, you know, the the validity of that particular tool within that pipeline. It might even just depend that it's particular development team and likes snake versus why source and you want to be able to have an ecosystem that can enable that flexibility in the tools, but also put the infrastructure in place so you can get the visibility from that and really Leverage The collaboration across the teams. Workflow automation is essential for us to be able to scale to the speed of apps and if to speed of types that cops without automation we're going to have so much manual efforts not just in collection of the reports, but also the follow up with the sla's the assignment of tasks.

This is what Automation and unification becomes critical. And last but not least. We need to identify areas to boost the team's productivity.

This is where we want to leverage knowledge bases of Industry best practices can be able to customize that for my organization but more important have a way of not just making it a library of stuff that people kind of look out but kind of don't understand but building in a framework where we can Elevate the tribal knowledge. That's within your organization. So you can quickly identify who would maybe understand this problem better and that could help me remediate my my issues.

So it really when I'm leveraging an abstract platform. There's really three things that it gives me the ability to do it gives me ability to Crow control that apps that chaos that we talked about at the beginning that spider web of information we can get a very quick view so that we can do better we can identify things that we need to improve. We then can improve our operational efficiencies both eliminating the manual task from a triaging effort, but also facilitating better costing communication and collaboration so that we can scale the impact of our app SEC team so that we can do more so the bottom line is do better do it faster so that we can scale and do more this is actually what we do at Alma code and we have a number of clients that really are free being able to achieve these goals by providing better visibility scaling the team so that they can do everything quicker and actually enabling something as I said like a security Champion process.

So with that, I like to say thank you very much for your time today. And if you would like to check out the survey in more details, you can access it here. Yeah, here's also my LinkedIn information.

Definitely. Yeah ask you to reach out and connect and I hope you have a great rest of the conference. I'm here as well.

So I'll be answering questions in chat. Don't do it direct message to me as well if you wish and thank you very much for your time. Have a great day.