Integrating DevSecOps and AI in Software Security with Paul Davis at RSA Conference 2024

This is Techron tv. Hi, I'm, it's Ashley here at RSAC 2024 in Las Vegas, Nevada. Talking with a gentleman from Jfr Field CTO.

Really wanna find out more about that, but really talking more about how we think about DevSecOps and security more comprehensively in this environment of security and emerging software people. I'm pleased to be joined by Paul Davis. Welcome, Paul.

Good meet chat. Well, Thank you. Thank you.

Great to be here. Great. So field CTO sounds like a fun job.

Tell me what a field, CEO and CTO does. CCTO, CS O-C-I-S-O, um, in this role, it's really about helping companies be more effective at securing their supply chain. Mm-Hmm.

Because you can have as many tools as you like, but unless you have the processes, the framework, the operationalization, horrible work, but it works. Mm-Hmm. If you don't have that in place, then it's not gonna work.

And in today's world with regulations, they're asking us to be accountable for the entire lifecycle of building software. Exactly. And it's gotten very serious in the regulatory front about that.

Oh, yes, yes. With the executive order from the White House last year through to the European Union, to Canada to Asia Pacific. They're all putting in regulations now because they're realizing the risks associated with software are not going away.

And they, you know, regulations help us enforce that. Yeah, I know. It's not either one of our first, uh, RSAs that we've been to sacs.

Yeah. I've always kind of waited for the conversation to start to mo include software. Yeah.

Right. Application security APIs. Very much part of the conversation landscape today.

Yes. Don't say that it's dominating it, but it, it very much is, is in the fabric of that. And, and as we were talking before, I think of DevSecOps.

I like to use that term both horizontally, all the way through the lifecycle software delivery into the environments it's deployed in, but also vertically, the AppSec we're creating, the infrastructure that it's running on and the tool chain processes, software supply chain Yes. About creating all of that. It sounds like you're kind of a similar thinking.

Yeah. With, the thing for me is, is that, um, the developers and security teams have been separate and our tools have being separate. Mm-Hmm.

And the problems for it to work, we have to link them together 'cause they're both generating information, which is valuable to both teams. And also the benefits of both strategies mean they both get faster and more effective. We have less incidents, less outages.

So I, when I'm doing my CO role as strategizing and advising customers, it really is sort of, first of all giving that, that first visibility about what they have in place and where the gaps are. Mm-Hmm. And then we can talk about regulatory compliance and the world of SBOs, which is a whole journey onto itself.

Um, but having that infrastructure for both those teams. And I actually break it up into three teams. 'cause it's devs, AppSec, security, and then sometimes product managers.

'cause they're also defining, Hey, we wanna be FedRAMP certified. They're the people that put the onus on the test to make it compliance. Kind of compliance wrapped around all of That.

Oh yeah. Oh yeah. Yeah.

So very, so, you know, it, it, it's interesting, uh, it seems like one of the approaches you're describing is the old adage of we have more in common than we have a difference. Right. Let's really talk about application security.

And I was just, uh, just as an example. Yeah. When we talk about cloud native, I'm like security people, you know this already.

It's APIs, it's protocols talking to each other. Used to be systems now it's, you know, microservices. So it's not that different than what you already know.

I I mean it's different in some respects. It's like one of the stats we have is like if you're writing software, I heard somebody saying this week, 3% of the code's actually written in house. The rest of it's imported through open source.

Mm-Hmm. Right. This is as an unknown.

But the fun thing is, is that it's like machine learning. We've got data scientists now, data scientists, coding, Python. So we've got a new branch of coders now there Mm-Hmm.

Who have the developers who are now being asked to, Hey, worry about security. But the key thing for us, and especially in the way Jfr thinks about, we want to keep them in the environment that they're used to working in. I don't want them looking at another ticketing system.

I want to either Jira or their IDE for the developers. The security team doesn't wanna look at Jira. I've been there, I've got the scars from that.

But they wanna be in their sim ticketing systems, their ServiceNow's, whatever they're using. And you wanna keep them there. But trying to build the bridge is interesting.

And many organizations are struggling with this issue about how do I get the two teams working together? I've done for many years talk programmers to how to be bad. As I say, you teach them how an SQL injection works, how a buffer overflow and cross site scripting, which you gotta plug a bit.

State of the union report still indicates even today there a problem. Still is. Yeah.

It still is. We, where's the solution? I dunno.

But Mm-Hmm. It still there, but they're still there. But trying to teach them that gives them an understanding.

But the thing is, we're not trying to turn 'em into security to Wes. We're trying to empower them so that they can get faster at delivering their software. They have, 'cause software developers like artists, they wanna build stuff.

They don't wanna fix stuff. Right. Right.

And if I can make it easier so that they don't have to go and revisit and touch up their packages like painting, they're gonna be happier. 'cause they can build more, more exciting things. Yeah.

I, I share a lot of that too. Um, do you think sort of the, the move to a platform approach, um, is part of the solution of this, is that part of bridging all the tool gaps and data sources and all of that into a way we can share that information? I think you talked about regulations earlier.

I think this is mandatory. If you don't have a consistent platform from beginning to end, like almost a framework that you hang the data off, you are not gonna get that. And if you don't have that platform perspective, there's gonna be gaps.

So I, I really do believe that it's, it's key to have that. And I use the analogy, um, in the past we had logs and all the firewalls or the servers, then we came with log management. Great.

Brilliant log management. Now we can go to one place and see the data. We are going to need to do that with all the security tools.

So we have one place to go to see the full life history of how our package built that particular build was built. And you can't do that, like plugging pieces together. It, it reminds me of kind of how we used to do system architecture and I see like, here's the boxes of all the things and here's all the pairwise and wise.

Yeah. It looks like a breadboard motherboard. Oh yes.

Somebody constructed back in the day when we did those things, that's not a platform. That's, that's the other, let me connect all the dots and just show how complicated, messy, and it's like my wiring closet right. Back in the day about that.

But, you know, platform approach gives you kind of a way to build security in Yeah. As opposed to bolting it on through trying to figure out where all these things are. It is that we have had the traditional sort of knee jerk reaction on going do buy a point solution.

Now, I'm not saying you have to like pick one thing. Yeah. It isn't just living in one thing at all.

Yeah. Yes, yes. That, that, that doesn't happen either.

But you do have to have that core framework to hook everything together. Mm-Hmm. And then so having that in place, um, you know, that's one of the things why I'm here at Jfr is we have that, but basically the ability to actually support other tools and have that framework, that consistency means you get efficiency and it's, you've got that single place where you're getting all the data.

Mm-Hmm. Yeah. I, my first exposure to implementing DevOps was using Artifactory back Yeah.

Back in the day when that was. Yeah. J Rog, if you will.

But I think that influences a lot of your direction. I always believe that companies are what they start in. So if you're, uh mm-Hmm.

You know, information, content, you know artifacts that drives a lot of your thinking and that goes across the entire life cycle, right? Yeah. Yeah.

Yes. Yeah. I mean, but it also surprises me.

I mean, in this world, how many organizations are building software and don't realize it, especially for the security team, we don't think about that. A we are worried about services and servers and patches and things are now all of a sudden we've got to a software to worry about and how it's being built. And that's a big stress for CISO.

I mean, I've seen a lot of conversations at this conference about personal liability insurance. 'cause they're worried about the stress of protecting against something happening. That's a big, big, big nasty thing happening at the moment.

I, I, I don't know for sure, but I'm pretty sure somewhere in those click boxes to sign up for RSAC was a requirement that we talk about ai. Aha. Ha ha ha.

Yes. You knew we were going. Oh, yes.

Sorry. Let me talk to my robot. Let me turn, excuse me.

Cyborg that in there. Yeah. Tune in, tune on.

Yes. Um, so I'd love to hear your perspective about, if we're thinking about ai, how does it, I, I know the adversarial and the kind of good and bad guy part of it, right? There's that, how does it help us deliver better software?

It's really interesting to see the evolution. So I've been doing AI for many years through natural language and methods, but now we have this chat GPT, but there are other engines and the different qualities, et cetera. The thing for me is how you regard it.

It's like, we talk about generative ai, it's generating stuff. I mean, I heard recently we've run out of text to train the AI engines. Mm-Hmm.

So now it's gonna have be 'cause it can recur on itself and it Yeah. Oh yeah. It's kinda scary.

Scary. So the thing is, is that if we actually, I look at AI as like an augmentation tool to help, it's part of your toolkit. So you could say, I've written this code, what's wrong with it?

Or I, I've seen some loaded presentations. You've got the CVE. How do you fix it?

Using AI to augment yourself is brilliant. Right. And I, I use it.

So I'll be writing Python code even though I hate Python. I said it, I like Python, but, okay. Oh wow.

That's, we Agree to disagree there. Captain spaces. Yes.

What? Yeah. Which one?

I dunno. Semicolons, what do you need? Well, That's C Okay, let's one go there.

Okay. So sorry. Anyway, back to, so, but the thing is, back to the ai, it's, it's how you use it properly is effectively.

And the thing is, is that you, there are different aspects of protecting you against ai. We have the malicious LLMs, as I call weaponized LLMs, stopping people downloading, just like the recent docker report we have, which showed we've got nearly 2 million sort of weaponized docker images, which aren't really images at all. Yeah.

Right. But you've got that side of it. You've then got the side of it of when in generating code.

And I think people are realizing that it's not good for that. I've seen some great sort of evolution in that marketplace where hey, generate a release report or generate a threat model. Those That's great.

Things like that. And then using AI to augment by seeing anomalies in huge amounts of tech or data is very important as well. So what I'm doing is, I'm not relying on it.

We could get there one day, but for the moment, I value it for its insight and for it to give me a foundation, a growth path. But I don't just rely on it. I mean, we did a recent survey and 25% of companies are going to be using AI to generate source code.

That scares me. If they said augmenting, I'd be much happier. Take the 80, 90% of software that's being brought in.

That means, what is it 30% of that's gonna be generated by ai? That worries me. And if you are CISO and you are worried about compliance, how do you attest that it's safe?

How Do you, and how do you know what's generated and what's not? Yes. Would you go to test it or do security testing or whatever in the Yeah, I'm, I'm seeing a lot of activity where organizations saying we only want to allow authorized LLMs real ones not the ones with a, You just put up the challenge for every developer to find a way around authorized LLMs.

Right. Well, no security people. We, I say verify.

Verify and then baby trust. But you know, we have to have a control. So we, we don't wanna get in the way the creativity of the developer, but we have to have the controls in place to make sure we stop the best stuff happening if we can.

It seems extremely simple. Too simple to me. But it seems like, don't we want to train LLMs that generate secure code as opposed to just code?

Is that not, is that too much to ask? Yes, it is. Okay.

I tell why I find somebody to finally push us back. Please tell me why I can't have secure code out of my own. I'm gonna say something terrible controversial.

If you ask, um, um, someone, a data scientist, how did it come up with that answer? They won't able to tell you. No, we don't Know.

So, you know, you go in front of the judge and it says, so is this AI safe? And how did they decide this? I don't know.

That's not gonna work for you. So from that perspective, yes, hopefully we get there one day, but I think we've gotta put more controls on, or not more controls. 'cause controls just greater understanding or something about AI to make it more reliable.

What about thinking of it as kind of a control plane, not controls control plane? 'cause AI can be introduced in so many places in software. Yeah.

Yeah. It could be generating codes. Yes.

It could be automated processing. Yes. Could be testing, could be Yeah.

Test cases all over the place. Yeah. So it isn't like controlling, every time we put in controls, it either slows down or p**s people off or both.

Right? Yeah. So why not think about as things are going through the process, what are the right instrumentation or controls or other process to say, okay, great, let's this needs to go through this security process.

Scanning or analysis or Yeah, Something. I mean the, the Thing maybe that's too overhead. I don't know.

Um, you, you know, they talk about, you know, you can hack Chachi pt mm-hmm. I call it socially engineering it 'cause you're having a conversation. That's What it is.

You can sit here and bang against Chachi PT and find out what the model knows. Right? Yes.

I mean, we did a CTF showing how you can actually hack yourself way outta the controls and get admin rights. Mm-Hmm. Brilliant demonstration about how you can give structure, use a Skill for all of us.

Yeah. If no, we got to be good. Okay.

Right. Okay. See you bring out the dark side.

Oh, No. Well, you see, I'm the Y yang, I'm the white hacker. You're the, somebody's gotta play the batty here.

So, But, but you talk about putting in the controls. The thing is, if we, if we don't understand it, how can we measure it? How can we control it?

And that, I, I don't, I haven't heard of an answer yet too is a problem. All I know is it can definitely improve the efficiencies and highlight information. You know, in the past I used to have the threat intel teams that would look at these graphs, massive data volumes, and produce these relationships graphs.

And you used to look for the anomalies. Mm-Hmm. Right.

We can do that with ai. Now, a friend of mine said, all I said, a lot of all machine learning is this glorified counting machine. It's counting instances, connections.

So it's great for that. And it can handle fast volumes and it can do it repeatedly without getting bored. Mm-Hmm.

Yeah. Not getting bored. No doubt.

I think it's a good, there's a, there's a toil metric right there Also. What, what, as you think about kind of as we're progressing, you know, you're talking to people, we're on war, so I always thinking kind of ahead, what's, oh yeah. What's coming?

What do we do? What do you think the conversation might be next year when we come back? You and I sit down, we're we're meeting next year, right?

We're gonna talk and chat and say, you know, we, you might be, we should be talking about this. What do you think some of those topics might be? I think some of them are still gonna be there from this year.

You know, I think, I think Buffer overflows. Yeah. Buffer overs.

Greg bras ice scrap thing. It's gonna be there. But I think we're still gonna be struggling with, um, the role of a ciso Mm-Hmm.

Defining it. Making sure you feel less like you've got a target on your back. I think we're still gonna be talking about compliance, but I think also we're gonna see some great evolution in the way the tools.

I think we're gonna see more tools that will evolve to help us with that journey. Mm-Hmm. Because it's like we've opened, I'm not gonna say a Pandora's box.

We've opened a new world and the industry's gonna respond and give us, there's something gonna be good, something bad. But I think we're gonna see additional abilities for us to control this new world. Uh, you Know, you know, I think of it almost, not to get too metaphorical about, but it's like opening another dimension.

But we're, you know, in some ways we're still thinking about two dimensional world in a three dimensional model space. Okay. You know, like, okay, so what can, what things do we have to kind rethink and what things advance the way we're currently Yeah.

Progressing. Right. I, I mean, the thing for me is to think of it, um, as a world, as a 360 world, you have to consider all the aspects around software supply chains.

Mm-Hmm. So whether it's business impact, uh, risk, speed of delivery, innovation, all those things there. So I think when you are looking at it, don't, don't focus on one problem.

Step, step back and look at the whole picture and get, make sure you're seeing all the pieces. And when you think you've seen all the pieces, step back again, as I said, a lot of times people just focus on the developer tools. They forget the security tools.

Right. Or the security tools. And they don't think about the integrations I spend a lot of time talking to.

This was about how they need to have the flow data flows nicely, smoothly through so that, you know, when a Jira ticket's created, something's monitoring for the Jira ticket to be closed so that the poor security team doesn't have to keep checking Jira to if it's closed. That's sort of symbiotic agent broker system. Um, but I, you know, for me the key is step back, understand what you really need to do.

Um, understand things like SBOs and how you should really use them. Unfortunately, I've come up some new variants of SBOs. I got told off more acronyms now, but, but you know, there's certain stages where the SBO M is useful.

Um, a lot of people just produce the sbo, but we dunno what to do with it. Um, it's actually got some very definitive stages in it. And it's also part of it is a living document, which not many people realize.

Right. So handling that and addressing that. So, and you've gotta bring in the business owners.

You've gotta bring security. They've product your risk council, whatever. And everybody understand this is what we're trying to do, but be certain that you're not in the business of doing compliance for compliance, but you're improving the business and you've got metrics that show improved resilience, improved customer trust, those sort of metrics which make a difference if you're delivering service anywhere in the world.

And maybe said it another way, I think what part of what you're saying is don't think of software is when we work on improving how we secure or create software, it isn't improving that there's, we're actually working on how we improve the business. Yes. Right.

Because That's, that's the reason context. We're doing all this. We think about it systemically, right?

Yeah. Systems think. Yeah.

Yeah. I mean it's, it's, it's, you know, improving business. You know, when you're building a business, you wanna have innovation, but you also be efficient.

You know, whenever I've run organizations, I've had an efficiency target how to, how to, for example, I remove all the false positives from the vulnerability alerts and actually real deal on the real problems. That's a key thing for me. Right.

So your efficiencies, it, it is running a business and you shouldn't disassociate the two. I know we don't, but it's a key thing We have to think about. Very good systemic thinking, right?

Yes. Look at the bigger, and it takes more than one view perspective. Yes.

I construct that together. Yes, it does. Yes.

It's good. That's, we need to work together. It's a team sport.

Exactly. It's, it's, and it's multiple teams, not just one. Well, fantastic talking with you.

It's a pleasure. And uh, I am hopeful that we'll get to check in not before. Yes.

Certainly by next RSA kind of see what we're talking about then. Yes. Thank you.

Then we'll, guess what, we'll talk about the year after that. Brilliant. Thank you so much.

All right. It's a pleasure. It's Been a fine pleasure.

Brilliant. I Appreciate you. Okay, Paul, Dave is field CTO with j Rog.

The great thing about field CTO is talking with customers, working with customers, understanding the challenges that they have, not just in your product space, but also more systemically trying to help them improve things, move the ball forward down the field. So thank you Paul. Thank you.

We will be back with more great interviews from RSAC 2024 in San Francisco. Thank you for joining us to go anywhere. We have some more great things for you in store.