Evolution of DevSecOps – Anuj Kapur and Tim Johnson, CloudBees

This is Textron TV. Hey guys. Thanks for the throw.

We're here with Tim Johnson who's director of product marketing and the newly appointed a news report CEO for cloudbees. We're talking about security and devsecops and the overall state of the world Tim. Welcome the show.

Thank you. Glad to be here a news. Welcome to the show.

Thanks. Bye good to be here. Tim.

You guys have a new survey talking about what it is folks are doing where we are. We've been talking about devsecops forever in a day. What is the level of progress?

Is anybody's guess so maybe you can who is in on some of the results and more importantly what surprised you in here if anything? There's a lot to in fact there the you know day of secops has been around for a long time. The concept of shift left has been around for even longer and what we're seeing as organizations.

Progress through their their devops journey and you start off with maybe doing CI and then CD and getting on to you know, more complex pipelines and so forth. Security and it's it's close cousin compliance to been kind of the laggarts in that and it's now getting to the point where it's a significant source of friction an organizations preventing them from getting to that that digital transformation that they're looking for. And we went out and and with the survey we asked the whole bunch of c-suite people, you know, how does this affect your organizations and some of the biggest things that we found is that security and compliance work is a major major source a barrier to Innovation because it slows things down.

And and that wasn't really that much of a surprise. I think it was a surprise at how severe it was to us and some of the numbers and one of the other things that came back is about 80% of the the respondents. Are doing shift left or moving to shift left, which is for those aren't that familiar with it?

It's it's do more testing more Security checks and so forth early on and in the development cycle, which is the right thing to do. You catch bugs early you fix them there. It costs a whole lot less than than having your customers do your testing for you?

But the problem is again, it kind of goes back to the first statistic about 60% of those people that said that they're doing ship left. They're saying that it's a burden to them as well. And you get you have to take a step back and look at what we're asking those people to do those developers to do early on.

and there's now a whole, you know the batch if you will a huge number of security testing tools that that Developers are supposed to run test people are supposed to run. Since it's a whole bunch of them and then I have to figure out what did it what is the results mean? And then they have to try to figure out what does you know, what's how do I prioritize my work?

I have these six different security scanners coming back with all these priorities. They're all they're different or whatever we call those alert storms. This is burying people.

And then the security and compliance people come along and say, well you have to do this and we're not complying this because they have to change that you have to do all these other things and it's just the at the end of the day the the software delivery ecosystem is just so complex that it's it's beyond your developers to understand how complex it as it's beyond your developers skill set and actually what you're paying them for. To understand how this all works together and what is compliant and what is secure and so forth and and the effort that we you know, we see people expending on that we call that the compliance tax because you know security compliance work you got to do it. Nobody wants to do it but you got to pay for it and it doesn't add anything to Innovation and actually gets in the way.

So that's that's the summary of what we found from the survey. Is it your sense that we want to shift left all the way to the develop or are we just trying to lean left and put more guardrails into the devops platforms that address security issues without requiring the developers to all become Security Experts because the probability assessment of that happening is almost there. Exactly.

It's it's more along the lines of Shifting the security compliance everywhere into the right people at the right time and giving them the tools from the visibility to to make the right decisions. And so and that takes Automation in and we call it shifts left done. Right?

So, yes, you do have to do more testing early. But how do you take that burden off of those? How do you Declare it to really State for the organization what is safe and secure in a way that people can understand it and it can be consumed by your automation by your software delivery life cycle.

And then one of the other issues, especially when it comes to compliance because somebody's got to raise their hand and say based on the data. I have we are compliant to whatever standards you know, you have to adhere to Most of the effort that we see is point in time or well after the facts assessment. Let's go rebuild.

Let's go look at these logs from three months ago. Or well we ran and it's security scan two days ago. Are you still compliant now?

Right. So has anything changed and and does anything change after that particular point in time in your delivery the lights like only answers. Yes it can.

but do you know if it did and so You need this. Like I said this this corporate catalog of it's safe and secure is but you also need something that treats the security and compliance process holistically and in real time in continuously, right? and then Take that and fill throughout all the noise and give people the information.

They need to meet the right decisions. So developers say I need to stop on this feature because I need to fix that bug or I can delay fixing that because I need this feature for this this particular bug or this particular finding it is not as important and Dev environment versus something that would hit him in in production. Michael maybe just just add to what Tim said.

I think there's there's active debate inside most organizations as to the choice awards that you used between a shift and a lean and a big part of that is the maturity of the organization the risk tolerance of the organization the capabilities of the developers and where they stack rank in terms of regulatory scrutiny or in general sort of geographic base of operations. And that's something that I think as an industry will likely going to figure out which is what's that happy medium in terms of how far back do you go how far left do you go versus you know, does that sort of start to ultimately impede your primary goal, which is you want to basically ship production software with reasonable velocity with high levels of security and and that Balancing Act I think is as much of a debate inside most organizations at this point, and I don't think we've arrived at a happy medium across industries that can be used as a benchmark and I think that's in some ways. The with the survey also demonstrated, right and usually both have a little Touch of Gray going on so we both been around the Enterprise block a few times.

How is their relationship between devops teams and the rest of the IT team and the business is changing and evolving especially as we start paying a lot more attention to security compliance. All these things used to be things and somebody else's job. So how is that kind of changing and how should people be thinking about the role of devops?

Absolutely, look devops as a word was pointed in 2009. And so, you know as much as we talk about it, we were sort of barely entering the second decade and in terms of sort of mainstream awareness and adoption and I was gonna talk about this in my keynote General Orlando which is you know, even the company that sort of came up with the slogan move fast and break things very quickly pivoted to move fast with stable infrastructure. And that was obviously Facebook.

So this notion of the lost city that is paired with operational resilience that is paired with security is is something that is very much and evolving question inside most organizations. And that's sort of really the relationship vectors are changing Beyond. You know, what velocity used to be the primary kpi that everyone was targeting to experience to now resiliency and security and I think those attributes change with the level of sophistication of both Teams and the Ops teams in a previous discussion.

We were sort of talking about, you know, do developers actually want to embrace Ops if they do do they want to embrace security or is that being imposed upon them? Right? So this notion of Dev to devops to devsecops is very much an industry term, but how pervasive is it inside most organizations?

I think depending on you know, the development teams you talk to they will either Embrace that or they will push back against that then you have the role of the platform teams, which are basically driving a level of standardization or level of compliance because the argument is that you know, if you do velocity at the expense of security, then you will ultimately pay for it in terms of the vulnerabilities that you're creating in production code and the debt that you're putting out there because you're going to have multiple versions of software operating in any given environment. So I think it's a very very live thread honestly inside most organizations with regards to what the lion is and what the attributes are that that you ultimately value. Think what's clear though?

Is that the Brute Force attempt to embed security into the sdlc is creating significant frustration amongst the people that you have empowered to actually drive high quality and high velocity code. And that's the developers right the security Persona inside most organizations typically has a very different caliber a very different tolerance and sort of very different view of what the right answer is that is not consistent with what a 25 year old developer that just wants to basically develop This Cloud native containerized up on a serverless environment is so I think Bridging the philosophical difference as to what success looks like and then breaking the job description Evolution between all those organizations is something as an industry. We need to get right and it's kind of vary between Industries between companies with different maturities within those Industries and then obviously the geographical sort of bias of your state of operations and more importantly just what you believe your vulnerability is That is tolerable, right everything.

Every question about risk has to do with what is the appropriate level of risk you want to take on to be able to actually achieve the outcomes that you believe you need to drive and that's that's a threshold that I think is very much influx inside most organizations and depending on if they headline in the Wall Street Journal is tied to the next log for J vulnerability or the headline in the Wall Street Journal is tied to the you know, depreciation of Sterling relative to US dollar. I think the sentiment shift changes so that's sort of I think where we are and we're gonna have to work our way through it over the course the next two years, but I think what the survey demonstrates is that there's a Hut and level of anxiety around security apps being something that has not been as food nice inside most organizations as being a kpi that needs to absolutely be upheld. yeah, and if I could if is that I mean Tim does that mean we're having something that feels like a Ralph Nader moment where we're unsafe at any speed and is there some way to have our cake and eat it too?

Can we be safe and still build applications quickly? Okay. Yeah, I know and actually have enough gray hair to know who you're talking about as well on that.

the the I mean what we're seeing kind of leveraging off that in your question, I mean the yes, you can go faster safer. And in fact if you are safe and secure if that's if that's embedded in there. That's your way to go fast because You know what would devops and and really fast pipelines you can get to where you don't want to be faster than you ever thought imaginable.

But where I think some of the issues are running are coming up is is you know, newsman security folks are different from Dev people which are different from management people and supports there's people who are talking different languages and they need a way to kind of rationalize that and one of the other things we looked at in the survey was how you how organizations Define the work that they do. And some of our customers and from the survey there's there's innovation. There's risk, there's technical that and there's bug fixing and you got to spend time mixed on some or all of those all the time.

And one of the scary things was is that companies spend about 75% of the time 70% or more? Not on Innovation, they spend it on fixing things and Technical that and stuff that can arguably be said that it doesn't add any value. It may prevent, you know, it inhibits value if you've got these things so if companies start looking at how they do work.

then that gives them a language that they can decide that helps them decide where to invest or where you know how to choose to spend time on this and the other and if that's too complex another another customer that we we deal with they have the concept of above the line and below the line and above the line is stuff that adds value and below. The line is everything else that you got to do. And there and again they're at about 30 to 40 percent.

above the line and now you've got a language that business users can understand and a language that the developers and the security people can understand to help drive those conversations on how do we make a change? How do we justify the cost of making these types of changes to making these types of decisions because you tie it back to Innovation as your as your value generator. then at that sweeps away a lot of that friction as well and there's two Tim's point What's your best advice to sea level Executives who are confronted with this?

Right? They've got developers in the one hand speaking one language. They got a bunch of security people speaking another language all they understand is there's some friction.

The Temptation would be just to throw them all our room and lock the door and see what happens. But is there some you know more intelligent way of going after this? Yeah.

I mean they probably is but you know, some of the examples that we've seen in terms of best practices is really driving a discussion almost beyond the CIO to say what is the primary purpose of the application right four years ago. I used to sort of have this statement, which is your application is your business and application experience is the only kpi that's really matters. So I think for a lot of Corporations, they need to really deconstruct the role of applications and basically driving the primary differentiation that they see whether it's employee facing whether it's partner facing over whether it's customer-facing so Important as the application and within the application how do you sort of value the attributes that are represented by each of those constituencies, right?

How much of it is velocity of feature development and/or ui/ux Improvement. How much of it is resiliency and scale and frankly how much of data security and you know for a lot of our customers what they will tell you is that somewhere between 90 to 100% of their revenues are actually driven through applications. So for all intensive purposes that will tell you there are technology company just in the case of financial services with a banking license.

So applications and application uptime is effectively synonymous with business resiliency and business up time. So if you're a major financial institution and your systems to actually process International wire transfers to the extent that goes down. Well, that is absolutely Revenue generating.

So the extent you have a wealth management app and that goes down. That's absolutely Revenue generating. And NPS generating so, you know Israel resiliency important.

Our feature is important or are both important but really ultimately ensuring the integrity and privacy of data in a secure and compliant manner the most important thing and I think stratifying your application Universe across those I think gives you an intelligence starting point to say that we need to move Beyond sort of a high level platitudes are on everything matters all the time to where we think we have a risk tolerance that we can manage today based on existing processes and a risk tolerance that just maybe aggressive based on you know, where we find ourselves and the other sort of reality is that you know, there is a randomized aspect to security vulnerabilities. But this also a very purposeful aspect different organizations get targeted differently either proactively or accidentally right if you decided to basically use log4j or some of the underlying applications where love for J was a vulnerability, then you have sort of a systemic issue across your entire application here, but if you had Excited as a development team to perhaps use a different class of capabilities as the underlying supply chain component of your production software. Then you were less impacted.

So luck is a sort of element in in this because development decisions of the past result in vulnerabilities in the future. So I think that's stratification that understanding of sort of the risk that is tolerable and sort of what the key attributes are that customers value. I think is a way of sort of driving the discussion to say, let's reduce the aperture on what truly matters and what's above the line below the line and make sure that we all agree that there's a threshold that we can't violate the reality those sort of the other dimension that I think we talk about is the sophistication of the tooling is just not there right in the security World.

There is a sense that the more tools you put into an environment the safer you will be that's just not how developers think. They don't want to deal with sort of five different repos, but it's very normal inside a typical it environment to find 40 to 50 different security tools. To basically cater to the surface area of protection that you need.

So I think driving sort of philosophical alignment between those two teams begins with understanding that they had very different jobs and their approach on sort of third-party technology. They use to actually accomplish what they need to do on a day-to-day basis. It's also very different and you can you can't just apply a principle from security into development and a principle from development into security because I think they're coming from two different worlds and probably won't see the alignment.

So I think a lot of this does come down to as you said there is a absolutely a human Dimension to this because both sides believe they're sort of on the right side of history in terms of what they want to accomplish, but we know they absolutely need to work together to drive the only outcome that a CIO or a CEO cares about which is my application up time what it needs to be to drive the outcomes that I need to with my customers and employees. Marine function hurt in here first and vulnerability is only the tip of the proverbial security iceberg in the conversation gets bigger all the time Tim. Thanks for being on the show.

Thanks for having me. And usually thank you for being on the show. Thank you so much.

Stay safe out. All right back to you guys in the studio.