Is AI in Security a Revolution or Just an Evolution? – Techstrong Con 2023

Hello everybody. Mike Rothman here general manager of texturedome research. She strategy officer of texture on Group Security gadfly irritant and the like I hear techstrong con.

Welcome to Tech strong con. So I'm it extremely excited, you know be and Host this this next panel about AI in cybersecurity obviously AI has been, you know, kind of the topic of discussion for the last, you know, couple of months trying to understand what the impact is. And again, I've just assembled a fantastic panel for us to really get a sense of what's possible what folks are doing how it has to impact your leadership how you have to think about it from a risk management and really risk kind of modeling standpoint.

We'll have tons of perspective. All of these folks are old friends of mine. So we'll be bettering a little bit back and forth but enough for me, let's hear from the panel.

Let's go around. Introduce yourself couple minutes. We are who you work for, you know stuff that that's interesting Andy Ellis my my old friend Andy.

Why don't you start us off? Sure. I always love trying to guess which way you're gonna what order we're going with guys.

It's by last name alphabetically, perhaps so Andy Ellis, I'm the author of 1% leadership, which you can go by right now. I'm also operating partner at Wild Ventures The Advisory siso at Orca security and I was the head of security in CSO at Akamai Technologies for about 21 years and I was inducted into the CSO Hall of Fame two years ago. Wow, so I'm going to go a little bit counterintuitively Andy and not go alphabetically.

Oh Caroline Wong again. Another old friend of mine. Welcome me.

I failed. I know it introduced yourself a little bit. Thanks.

I think it's always fun to mix things up and keep folks on their toes. Even if those folks are us the panel. My name is Caroline Wong.

I am the chief strategy officer at Cobalt. We are pentest company. I am super thrilled to be here today.

I started working in information security in 2005 leading security teams at eBay and zinga in 2011. I published a book called security metrics beginner's guide with McGraw-Hill in 2022. That book was inducted into they cyber security Canon Hall of Fame.

I did be Sim assessments for a while with digital and I've been with Cobalt. I also do a podcast the podcast is called humans of infosec. All right, fantastic and last but not least another old friend of mine, which he gets me to think.

I'm either really old or I have a lot of friends or maybe a little bit of both Adam Shostack. Let's say you're a little bit about what you're up to. Hey, I'm Adam Shostack.

I'm the president of showstadkin Associates. We are a security consultancy training company that really grew out of my second book threat model and design for security and I just published my third book. I am super excited to share with folks.

I don't even have the cover on the wall yet, but it is threats whatever engineer should learn from Star Wars and a lot of my work over the years has been in application security from Banks to Microsoft for a decade. And really these days I'm helping people do that. I'm on the review board for black hat where I help pick content and I'm in affiliate professor at the University of Washington where I also teach cybersecurity.

So I am clearly the least distinguished on this panel, which is how I like it, right, you know, Wise people have always told me if you're the smartest person in the room. You're in the wrong room. So clearly I am in the right room on this one and I am excited about that.

So let's you know kind of dive in right? First of all, I'm not gonna let my ego get in the way that I seem to be the only one without some kind of Hall of Fame distinction, but you know, the day is not over yet. So maybe at some point somebody will bestow that upon me but more seriously.

Yeah, I mean listen we are at this weird transition point in the security space really in technology General, but certainly within security a lot of things have happened, you know, obviously Adam and Caroline. I've been doing application security, you know, probably forgotten more than most people know Andy's, you know, kind of managed just you know, significant programs for for global companies for a long time. We're at this transition point right where we want the machines to do.

You what the machines are good at right? We don't want you know to squander our very limited resources our human capital that we have to you know, really deploy most significantly. We've really kind of hit this, you know, really generational upgrade to generate generative AI right where now it's it's actually generating stuff that you can kind of use.

It may not be perfect, but we can kind of use it so I'd love to get everybody's opinion. I know anything. I'd love to get everybody's opinion a little bit about you know, kind of how do you see AI really starting to impact the practice of security right the stuff that we have to do day in and day out Caroline Wong want to take a first cut at that.

so I think for aeon like GPT is purely for entertainment. I think it's like the number one use for it actually a good friend and colleague of ours at Daniel meesler as chat GPT. What made the 90s so awesome and Chat GPT gave a really good response.

I don't however think that. Chat gbt can help us to solve The World's cybersecurity problems. I think that for decades we've used software and pattern matching and computers to help us find some types of security vulnerabilities and I think there are things that cannot be found by Machines of any sort including things like race conditions business logic flaws chained exploits.

I think that Ai and chat GPT is not and cannot be a stand-in for human creativity human Innovation human judgment human opinions and human decisions, which are needed to drive the right outcomes in this industry now if I'm a scammer And I don't speak the language that I'm trying to scam someone in then. Chapter GPT may become valuable that it may help me to become a better imposter. So those are just some International thoughts interesting Andy any other thoughts to Caroline and that's a kind of a counterintuitive perspective.

Most people like oh my God, it's Gonna Change the World and Caroline's like yeah, it's good at parties. So I he so I'm mostly with I'm mostly with Caroline. And first we should make sure that when we say AI we should recognize that chat GPT is actually a really bad AI it's a large language model.

It's basically nothing intelligent on the easiest way to think about, you know, large language models like chat GPT. Is there like that b******* or at a conference that you walk up at the social event and there's somebody who can talk about every single topic faster than you can fact check them. And that's what Chad GPT is because it lies all the time and but it has no self awareness.

It's not lying. It's just saying things that sound truthy. We should separate that from things like, you know NLP, which is a reductive Ai and things that are doing pattern matching and you know autonomous decision making which I think there.

Use case in the security world around like how do you weed out false positives by detecting that it's the same false positive all the time but to come back to chat GPT. I actually use it all the time. I actually use it as a rubber ducky.

Those do not familiar with rubber ducky problem solving you put a rubber ducky on your monitor and whenever you have a problem you ask the rubber ducky. And in talking to the rubber ducky it sometimes gives you back like that process gives you an answer. I use chat GPT that way I send out a Weekly Newsletter if I'm struggling with a section of my newsletter, like I'm doing a leadership topic.

I will literally take my words in and hand them to chat GPT and say Here's a sample newsletter. I wrote that I like please rewrite these words in that style. Awful at it like I would never publish what it wrote for me, but it gives me this model of oh like here's the form like Chad CPT.

It's a perfect ninth grade English student. It writes exactly in the forms that it is asked to do but it doesn't do great research. And so as long as you can have that filter, it's a useful tool.

Great bottom anything to add to them. Yeah, so I I actually have been using track GPT for over a year. I know some of the people over at openai and they gave me access when I was working on a keynote.

About 25 years in app SEC and so I asked a year apart. What is the future of appsec and a year ago it gave me. A bunch of really bad bullet points and went off into nonsense really quickly and a year later.

Was when I first started seeing these chat GPT screenshots, I thought they were fake because of the difference between what I had seen and what it was doing a year later. And so I think it's important to think about chat GPT in the space of a rapidly evolving thing and I don't disagree about where it is today, but I have seen experiments done where you ask it something and then you ask it why it's wrong. Like how many times was the Golden Gate Bridge carried across Egypt?

And it'll give you an answer. And then you ask it why it's wrong and it'll say of course the Golden Gate Bridge was never carried across Egypt. And so I think that there are things we can learn about how to make use of it that limits some of its weaknesses and I think that where it's going to really start to shine.

Is not in the form of chat GPT, but in the form of automating things, we're actually a ninth grade English student might be okay. Yeah, and I think there's a lot of a lot of jobs that maybe are not a ninth grade level. butter within sight of that and I think that some of the things that we talk about when we talk about the the skills shortage the jobs Gap the unavailability of entry-level jobs.

I think chat. GPT is coming for some entry-level jobs in say sock analyst. In say pen test.

I'm I'm sort of surprised. They haven't hooked it up to bug bounties yet. Seems like you could fund it OpenAI on bug bounties.

Those sorts of things might be things where we can make use of it and it needs supervision. We need skills and using it but it's I I wouldn't be quite so skeptical. Yeah, I'll give you a place where we're it is being used and this is one of the companies I'm associated with Orca is actually using it for generating remediation instructions, which is functionally fits that you know, ninth grade student of like we know programmatically how to remediate something but giving a human readable text that says here's what you do in what order and what the commands are is something that GPT is amazing at you handed a CV and say tell me how to remediate it.

It does a good job of actually creating that text then you supervise it like any ninth grade student you make sure there's a human who then reads it and says, yes, this is good. But when your engineering team has English as a second language. Then this is a really helpful way to produce content that's boring repetitive content that nonetheless needs to be useful every single time.

You know what that sounds like Andy policies right policies. Oh my God, like ask it to write you and I so compliant policy I better yeah, so there's stuff and that's kind of what I don't really want to get kind of conflated with, you know, we're only talking about generative AI here because can I agree with all of you guys right for entertainment purposes? It's gonna be great at a party right a couple of beers in there and who knows what's gonna happen?

I mean that could be kind of scary but you know, I think that the point of having it do some very rope things that you know, we don't have to have high level very skilled resources, you know focused on we'll be really helpful. I want to dig in a little bit towards, you know, really the detection aspect of stuff and and you know kind of being able to use again whether it's NLP whether it's you know different Types of AI being able to really kind of process just the scads of data that are coming out of all of our applications all of our systems now and really use that to find whether their patterns whether they're anomalies all of the above, right? You know, where are we today with that capability?

Everybody talks about it, but you walk around, you know, any of the conferences you're like, all right. Liar liar. That one's a liar too and maybe they're just manipulating the truth a touch.

Nah liar, right? So where we really, you know from the standpoint of of leveraging some of these Technologies to help us detect things better Andy, that's kind of what you do, right, you know, at least what Orca does is, you know, kind of both misconfigurations than other activities. So, how are those guys, you know kind of using AI to better the mission, so I sort of hesitate to call it AI because it's really is more, you know, low-level analysis done at scale so I don't want to Have anybody jump in and say El b******* Andy, you called something AI that's really a low level expert systems, but there's two different places where you see this benefit and one is in building attack graphs and connecting the dots.

You're very complex systems and you want to be able to say I see there's a system and it's internet facing and that internet-facing Port hasn't open vulnerability on it that system also has an identifier that can be used to move somewhere else. And so almost all of these are rules being handed to an expert system that then goes in and can Auto generate and say oh by the way, I just discovered a possible path an adversary you could take it starts from internet facing using only open source tools and adversary you could end up with control of this database like me, you should remediate one or more of the steps along the way like that's a fantastic use of a very basic AI capability which is let's connect those in and then on the other side now you have this mass of these You know thousands and thousands of paths and in a sense, what you want to be able to do is tell people which ones to ignore and I don't mean that they don't matter at all. Right what I mean is the human is going to ignore them.

Anyway, if I give you 3000 problems, you're gonna ignore 3,000 problems if I ignore 2,954 of them for you and tell you deal with these later. Here's 50. And here's the three that are most important like that's a great categorization step that AI is very helpful with of letting you focus on the things that really matter right now.

Yeah. So so Carolina, you know, you're you're looking pensively at the at that screen and you know being in its pen test business I suspect you may have some opinions right about how to leverage some of these capabilities right to make our people more effective. Are you guys working on some of those Technologies as we speak?

You know my comment is going to be a little separate from pen testing. And from what Cobalt does. We're not using AI in any sort of fancy way right now.

And that's very much on purpose. What we do is teams of manual pen testers and The reason my face is kind of scrunching up the way it is is because I'm not by any means an AI expert. I do have a degree in computer science, and I kind of wonder if because in our industry There's security work that you do and then there's security work that you talk about.

And I think that if you're like a security talk about her. Then we use these words right and to me. I'm just like some of the things that we call AI like isn't that just Computing like is it like isn't it just like pattern recognition just like straight up and like, you know, so, you know, I think the the other side of this comment is the we as an industry for example.

I will compare our owasp top 10 list from 2021 with the OS top 10 list from 2003. Hilariously. I don't know if I went to Chad GPT and I input and I said hey Chad GPT.

He was the oauth's top 10 for 2003. What do you think? It looks like in 2021 surprise.

It's like the same list to me. That actually happens to be an indicator. That like we're kind of stuck like we can have new fancy names for new fancy stuff.

But like Computing is Computing and you do pattern matching with different data sets. Cool, right like and that's cool, but I don't know. That it's new.

I don't know that it's that exciting frankly and maybe I'm just a naysayer. Um, or maybe my understanding of computing and AI is just really simple so, you know when I hear about, you know different uses for ML different uses for AI, like certainly there are some that are very cool and some of them I'm just like that is just like Computing that we did 20 years ago with like a fancy name attached to it. Go ahead Adam you're dying.

I don't but no no, I was trying to think what I want to say Bill Gates used to say that it's easy to underestimate what you can get. It's easy to overestimate what you can get done in one year and underestimate what you can get done in 10. And when I think about some of the things we've achieved, you know, we no longer have Summers of worms, which was a thing when in 2000 and 2002.

Where these things would just take over the Internet for a little while? And I think that when Google recently announced that most of the new code in the newest version of Android is written in memory safe languages. And so I I agree with you.

There is way too much Groundhog Day. There is way too little lesson learning. And actually I'm going to troll Landy here just a little bit.

One of the things I liked about the new National strategy to secure cyberspace was the focus on the last page on Lessons Learned and how do we get smarter? And I think that we may see. Yeah, there's a lot of hype there's a lot of things that we that AI isn't going to solve for us, but I don't want to underestimation.

The possibility if we start to bring it into our engineering things I was I was in a group discussion recently. of ctOS here in Seattle and somebody said all of our senior people are independently using copilot and all of our Junior people are not and what they said is you have to think about what tasks co-pilot is going to do the boilerplate sort of code the how to fix a phone. It's really actually pretty good at and if you're not paying attention to what co-pilot is going to do.

It's going to reintroduce CVS from 2015 into your code. It's really good at that too. And and so I do think we can use these things to make progress.

And to scale some but not all of the things that we need to do to avoid this Groundhog Day that you were talking about and that the tricky bit the bit that we as experts need to be thinking about. Is how do we intelligently grapple with how to use the tool? So that we do that that's that's my sort of that's actually that's fantastic.

Right? I mean, I I think that really puts a lot of stuff into perspective. I use that Bill Gates quote all the time because you again I think all of us For Better or Worse have been through a number of those decades cycles and you know, we've seen just miraculous kind of changes and and advancements over that time frame surprisingly, right?

I mean surprising the amount of progress that we've made over that kind of time period but I also agree that you know, it does feel like it's the same, you know, same stuff different day, you know kind of when we look at it and those, you know little micro type things. I do want to as we kind of start to move into the home stretch, right? I want to hit on something you brought it up as we were, you know, kind of chatting about what we wanted to talk about before during the panel and that's the idea of adversaries using some of these Technologies, right and and what I found and again having just been an observer for a long time not having to do Operational stuff anymore is that that tends to be the Catalyst right the adversaries using the stuff against us tends to be the Catalyst to get us to start, you know, kind of becoming a lot better at what we're doing right automated attacks started to get us to have to automate detection in a much greater way and make a lot more Investments that way.

So what are we going to see? You know, whether it's from you know, a threat modeling standpoint or something that I want you to start with this one since it was kind of your idea, but you know, how does it really kind of change how we as Defenders have to think about this knowing that our adversaries have these same technologies that they can use to maybe automate the simple stuff or maybe get a little bit more sophisticated if they've got the technology to do that. So I think that we absolutely are going to see adversaries doing this someone mentioned the the language you're not skilled in.

and I think that a lot of the things where we've been relying on it being expensive for the attacker to do this or that Are just gonna start happening at scale and so to take Andy's example. If you've got 3,000 attack paths and your attacker has a tool that lets them experiment with all of them. And then it's using chat GPT and say and tell me which one of these looks like.

It actually worked. that that sort of thing is is a shift and I don't think a lot of folks are ready for it. Or not Caroline thoughts on that.

So I'm a little bit terrified. And it's because I think so highly of Adam and I think he's got a really interesting point. You know what what can be done at scale now, that could not be done before on both sides.

My contribution to this part of the discussion is not nearly as interesting. It is simply that one of the things that I think is being overlooked actually is the inputs to things like chat GPT and how is that data being protected or like maybe not at all, you know kind of a simple analogy is you know. Who has access to what I type in to Google search?

And if I had if someone had access to my Google search for the past week. they might learn some interesting things about me, you know, but if we look at something like Chachi BT then what are Caroline's or anyone for that matters inputs to chat GPT what sorts of confidential information are people putting in there what sorts of sensitive information are people putting in there that I think is something that may be overlooked in this particular phase of like lots of excitement and lots of uptake Wow. Yeah, um, it's so again two sides of the of you know, kind of a similar risky coin right one is when the adversary start turning this technology against us the other is and I'll call it the customer service rep problem, which is they're just trying to do their job, right?

They don't realize that they're putting corporate data at risk because they're just trying to do their job, right? Hey chat gpt's new thing. It can maybe help me do my job better.

So I uploaded a file of you know accounts and asked it, you know, what would be different about this yikes? Right? But they're trying to do their job, right?

So they're being Innovative, right? You know, and and so we have a training issue ahead of us too. Right?

We've got, you know again a threat modeling issue and that's really Adams area of expertise. But I do feel the same way that folks just don't see it until you see it and that was a similar type of thing. Right and I've been doing Cloud stuff for as long as I can remember because I'm kind of losing my my short-term memory for the most part but that was one folks started compromising, you know, aw.

Accounts starting to use them to launch DDOS attacks, right? It's like oh God, they started this great thing we could spin up an infinite amount of stuff. Oh and it's turning against me, right so that kind of changed the game in terms of how we had to start thinking about Cloud security.

So and Andy, why are you kind of you know, at least start to get us into that, you know kind of coming in for for landing perspect. So I think anytime we're talking about a new technology right you do have to question. Like is it actually useful I think Caroline is absolutely correct that sometimes it's just marketing buzzwords.

Like AI has been the buzzword at RSA for like six years now, so we clearly know that that first year not very interesting but at some point what are the actual useful pieces and I think we've talked through some then it's how our adversaries going to use this or use the fact that we're using it against us, you know consider the fact that what all machine learning does is lets you make the same decisions repetitively at scale. Means that any wrong decision that you get that's going to be made gets made over and over and over again with zero introspection. And so how do you find that because trust me adversaries are going to find an exploit that And then of course the question like how do we use it?

And how do we start to use these as business enablers and maybe it's just by taking little pieces, you know, maybe you take some NLP to do your semantic understanding what humans are asking match that into your machine learning, you know pattern analysis to say. Oh this is the question I'm being asked on my website and then I'll hand it to chat GPT to generate an answer and send it out like that could be a pretty useful and simple thing but it also means that if somebody asks you carefully tailored questions, you might create outputs that you would be embarrassed to have against your corporate brand and you might be recreating Tay from Microsoft who I think we all remember as being an early chat bot instance that went very badly South when they allowed it to learn from new input. That's right.

So and again, I wish honestly guys first of all, thank you all so much for your time today. I mean, I think it's been really an Illuminating discussion. So Adam Caroline Andy again, I really do appreciate you taking some time with us.

We're not gonna solve this problem today, right? We're not really gonna understand what the true impact and the ripple effect of generative Ai and a lot of the other, you know, kind of pattern matching and reductive technologies that we're starting to introduce into our environment. We're not going to understand what those Ripple effects are.

What we do have to do is pay attention, right we have to be in touch with our product folks we have to be again as Security Professionals and as Defenders, we've got to understand how our attack service is changing on an ongoing basis and we have to experiment with tools that may be able to change the the vernacular I may be able to change how we do things. Because what you don't want to be is the last one at the party, right? You know, you're there everybody's already, you know, kind of three sheets to the wind and you're like, wow, this isn't fun.

These folks are way ahead of me from that standpoint. So get out there find a cocktail find some friends start playing around with these things right start figuring out how they're gonna impact your day in and day out existence because they will have an impact. We just don't know what it is.

And with that we are out of time. So again, thanks so much Adam show stack check out his new book Caroline Wong check out her podcast. I know it's fantastic humans and information security and Andy Ellis 1% leadership right over his right shoulder as as I can see it Go by it.

It's gonna be fantastic. The guy's been there and done that and I'm Mike Rothman and not a hell of a lot more value than just being able to assemble a bunch of great folks to you know, really really help us understand what's going on in the environment.