Beyond the Buzz: Let’s ChatGPT About All That Matters with GenAI | Predict 2024

Thanks so much for having me here today, Casey. My name is Diana Kelly and I am the CISO at Protect ai and also, uh, very proud to be a program committee member at RSA conference, uh, and, and working with Andrea and Rom. Yeah.

Hi everyone. I'm Andrea Lumbago. I, uh, an SVP of research at Interros.

And I'll just echo what, what Diana said. It's been great working, uh, with this group and very proud to be part of this committee and looking forward to this discussion. Yeah, and, um, I'm paving the way to the, to our two fearless leaders, Diana and Andrea, uh, who have been like helping me with my program committee, uh, responsibilities.

Uh, I am the lead for the AI red team at Microsoft, and also a fellow at the Berkeley Client Center at Harvard. Awesome. And we are so thrilled to have you all here today.

Um, missing from today's call is also the fourth member of, uh, the program committee, uh, Lamar, um, Kesem, ke Cassam. Thank you. Thanks Diana.

And, um, she was unable to join us today, unfortunately, but she is, uh, the fourth member of this strong committee of, uh, really passionate leaders. So it's been really interesting to sit on their blue sky calls and listen to their selections as they've reviewed the pro the submissions that came in. So, Diana, maybe if you could walk our listeners through the program committee process so that they understand what happens between that, you know, call for speakers, which opened in September, and the delivery of the notification letters, which will happen at the end of this month, and then the launch of the agenda.

Yeah. Um, happy to, uh, so I know it can be, when you're trying to get accepted for the first time, I always used to wonder how on earth does this process work? But there's, there really is a huge amount of thought and process that goes into it.

The first thing is that blue sky call that Casey was discussing, because we all get together and we think, well, if we were going to be attending talks on ra, uh, at RSA conference on AI and security, what's really important for this audience to understand and learn? So we first try and understand what a good track would look like for you, the audience. And then it gets infused and changed and grows, uh, organically as we get all the submissions coming in, and we receive a spreadsheet from the RSA team with all of the submissions and all the information that you put into your, your submission talks.

We get spreadsheets and then we individually go through those sheets ourselves with that blue sky in mind. And also, as I said, I I, I certainly, I suspect you guys do too. I, I get inspired and, and awed by some of the creativity of what the submitters you, you, you come up with.

The community comes up with so much of great, of interest. And then we go through and we pick out what we think are, are, what are the best talks, and then what are alternative talks, and then the talks that really maybe not don't fit in that track for one reason or another. Then we all get together as a group and we negotiate with each other.

And I can tell you, you know, it, it, there were talks that I had, I hadn't seen the value in them. And then rom he would explain why he loved it and he picked it, or Andrea would explain why she loved it, and she picked a talk or lamo and I went, oh my gosh, I missed this. So that's the one of the wonderful things about having all four of us is that we are, you know, again, it's organic, it's dynamic.

We are helping each other understand and trying again, to always think out what's gonna be the best experience for the people that come to the conference. And then with that, we bring our suggestions to Hugh, who is the chair of the conference, and Brita and Casey, who do the final assessment of, of deciding what's gonna go on. Um, and sometimes we also have some horse trading.

We may wanna talk that another track wants, so we may have to pitch why we really think it belongs in our track. And with that, uh, it comes out with hopefully a very balanced track within our area, but also a balanced conference across the entire presentation set. Thanks for that overview, Diana.

It was very thorough and, and I think you touched on all the fun aspects of it too, right? The horse trading that happens and, uh, there are some tracks that are super competitive and like, no, we must have this session on our track. And, uh, it is, it is fun to, to be a party to all of that.

So thank you for all that you've done to get us this far. And again, notification letters will go out at the end of January. So we won't talk specifically today about sessions that you selected for your track, but we will talk topically about things that you're interested in looking forward to.

Um, shifting gears a little bit, we often talk about AI as if it's, you know, this one thing, but there certainly are far reaching tentacles that have serious business and social implications, right? I was just reading this morning, um, uh, an article in Fortune magazine, um, that was just talking about the potential impact of, uh, misinformation if cyber criminals use generative AI to create, uh, election campaigns, right? And so it's not even much the infrastructure security of election technologies, but the social ability to manipulate minds through misinformation with ai, right?

So there's just so many implications Ram what aspects of AI are most important to you? Yeah, that's an excellent question. I really love your, you know, vivid imagery, Casey, at this far reaching tentacles.

You know, um, there was a scholar, there's a content moderation scholar who wrote, like, um, back in 2012, like anything the light touches is a content moderation issue. Now. It's anything the light touches is an AI issue.

There's hardly like a field that has not been affected by how, you know, this, this machine learning revolution has spawned, and it happened in waves. Like, you know, back in 2012, it was all about like big data. You see like different iterations of it come through, but this time it feels a little different because the ones who were croaking about, oh, look at like big data, look at machine learning back in like 20 13, 20 14, were mostly folks who were like, you know, the industry, the highfalutin, you know, uh, but now it is really changed to what can it do for the common man.

I, I think that the thing that I always, the concept of image that I have in mind is how during the writer's strike, they kind of said, no, no, to using generative ai. That's like, that's like crossing boundaries. That's not like, you know, Pepsi versus Coke.

This is like, what does it mean for a person on the field who's like writing scripts? What does it mean for them? And I'm talking about writing like, uh, television scripts, not like PowerShell scripts, which also is affected by this.

So the first thing I think very broadly is it has now pervaded our collective conscience, um, since last year as how this, how this revolutionized everything from like how to insert like a c theater to like how to write compelling scripts to how to sell a product and, and homework. Um, you know, turnin was like now was now trying to find out if something has been plagiarized by chat GPT. So there's this whole wide variety of a panoply of effects.

And I am still trying to wrestle and wrangle what does it mean to secure, what does it mean to do, like, to work with inherent failure, but it's really beyond the collective conscience of one person or one company, um, per se. And I think you summarize, I think, really well that tho those extended tentacles because that it's exactly, um, the way that I kind of look at it as well and thinking about like how Casey and RSA team pulled together, you know, this committee, we each focus on different areas. I think one of the interesting things in our discussions was that what seemed like very common knowledge for one of us was not for the other three mm-Hmm.

And that just kind of shows how, not siloed, but just how it's just nuanced. Uh, the whole AI discussion is right now, you know, ROM does a ton you focusing on, you know, it's hearing ai. I, my my area is more on looking at the geopolitics, like the elections that you talked about, Casey, we have more elections this year than in any time in history, potentially in 2024.

And so it's, depending on how you look at it, 50 to 70 countries across the globe have elections. AI's gonna have an impact on all of those. And so the impact of that on democracy is just, you know, astounding and exactly what Rah was talking about.

It's the democratization of these capabilities that I think often gets overlooked because, you know, with chat, EBT, it made it so everyone, you know, my kids can go and, you know, use it for their homework. Everyone, you know, it's accessible now. Whereas, you know, like a decade ago, the big data revolution really wasn't accessible to everyone.

So I thought that that was a really interesting point. But, um, I think across this group really highlights the, the wide range. And this, the, the, for me, it's the role of the human computer interaction and the, the impact on, uh, you know, democracy across the globe.

And, and then the, and the policy frameworks are starting to evolve significantly. I mean, it's really just, it's amazing just how nuanced different areas of it are right now. Diana, I wanted to hear from you and then Andrea, I have a question I wanna go back to you in your comment.

Yeah. So I mean, as, as Rom said, we do, we all had very different lenses. And, and I tend to be the person who looks very much, you know, practical and applied.

That's sort of what my, my deal is. You know, how are we actually gonna make this happen? So I get very excited about the, what's happening in the ml SecOps space as we talk about AI and ml, and a lot of people sometimes really wanna create this huge division.

AALS AI is entirely different than ML A is a super set. But ml, if you look at, I think it was ssa, uh, or maybe it was NIST, put out this really beautiful, um, visual representation of how ML and AI and, you know, it wasn't even a, a Venn diagram. It's like a giant circle of ai and almost 80% of that circle has ML feeding it.

So ML is just as critical feed into a majority of AI that we use, including large language models in chat GPT. So that's why looking at it from a i, if people followed my work, I was very, very a strong advocate of, uh, shifting left and DevSecOps when that became a reality. And so that's why I, I really am interested in, I think it's very important that we look at ML SecOps and try and bring the same security rigor and goodness to ml.

'cause we have had ML for years and years, and many companies have thousands of models deployed. It feels very different than that. As, as Andrea was pointing out now, that like all of us can just go to a, a, a chat bot and prompt whatever we want and have it write things for us or, or create images for us, it feels very different and, and very tactical.

Um, and, and very, you know, it's accessible, but machine learning has been driving a huge amount of what we as, as people interact with, whether it's your streaming service suggesting which movie you might wanna watch next, or your retailer suggesting what you wanna buy next, or even the robo-advisor managing your 401k, deciding which, uh, which stock is, is, should be sold. So ML'S been driving a lot of that. And so, yeah, that's why I am, I'm really excited about some of the practical aspects that we saw coming into this, into the, um, into the, the, the talk track for this year, looking at that, that how do we shift left?

How do we create good security within the ML lifecycle by creating mls SecOps. And we'll come back to, um, to you in a little bit, Diana, on the MLS SecOps, because I want, I wanna dig a little bit deeper into that. But while we're talking about, you know, what, what is most important to you?

What sort of is front of mind for you? Um, I am fascinated by the geopolitical, mostly because it's the, the piece that I can understand, right? Anything about the human element and, uh, the social impact is something that I can wrap my head around.

And so, Andrea, you had mentioned that, you know, this year there are so many a record number of elections that are happening this year alone, and the impact that that's gonna have on democracy. What, what do you find is most concerning as it relates to the potential use and misuse of ai? And then what do you find most promising?

Yeah. No, and it is, I mean, honestly, it's very easy to get, um, to go down very dark paths when thinking about this, because I immediately go to digital authoritarianism and, you know, governments that may not be, have the best interest of their society in hand and using it for everything from, you know, targeted genocide. And we, we have seen some of that going on.

Um, two other forms of bias and, and control being leveraged by governments, uh, without, you know, there's a good discussion going on right now in the European Union, a little bit in the United States and some other democracies about what policies should be there and what kind of guardrails need to be in place for ai. And I'm glad we're having those discussions. There's a lot we could go down on on that as well.

But on authoritarian regimes, don't even consider having guardrails and want to see what, how they can best leverage AI to basically maintain regime control and impact the narrative. And, and what's worries me the most is what those powers and the capacity that they have also can then be used, you know, externally outside their own borders as well as domestically. So it really is far reaching.

Then you have the accessibility to other, you know, uh, you know, malicious actors as well across the globe that can use similar kind of capabilities. And so that, that is what, that, that is probably one of the bigger things that keeps me up at night and that I worry about. But also what gives me hope is that, you know, like technology know, it's a dual use, dual purpose can be used for good.

And there's a lot that we can do for using it for good to help safeguard, uh, civil liberties and help to safeguard, you know, freedoms and democracy. And so I think we need to continue to push on being more creative on that front and how we can apply it in that sense. And then also, just the fact that we're having discussions about data privacy right now.

And I kind of feel like in the past we, we had these technologies come in and didn't really, data privacy was an afterthought, and we're actually thinking about it as a society. Uh, but earlier, I know the technology's moving very fast and it's gonna be very hard to keep up with it. But we're having some really good discussions now in areas where the, that that didn't normally happen in some other, you know, big technological shifts.

And so that, that does gimme some hope. Mm-Hmm. Um, I think there's a lot we can do with that to, to leverage it for good.

And I, and I think we're probably gonna see a lot of really interesting defensive uses of AI against the malicious actors during the elections, for instance. So, so I think there's, there's a lot that we're gonna be able to see, uh, on the positive front as well. Yeah.

And you know, interestingly about, you know, privacy, I mean, one of the, the, these systems we need to continue to truly test and test again. And because sometimes people think, oh, well, the training data was fully anonymized, but it's possible to reconstitute data, uh, and, you know, do, do, you know, layer back. And to be able to actually in infer understand who that information was about.

Some, uh, some of these systems, the data that it was trained on, it was anonymized, but you know, the users, the attackers are able to get to actually get the system to return it. And in some ca there's a really interesting attack, which is sort of, it's almost race conditioning, I think, which is the, the poem, poem, poem, attack. I dunno if you guys have heard about this one.

But, uh, so in that case, the prompt was, say the word poem indefinitely, which I, I immediately thought was gonna be some kind of denial of service. 'cause if you're using the, if the system is is just saying poem, right? It can't do anything else, potentially.

But, um, what was interesting was that it, it was saying poem, poem, poem indefinitely, but then it started to actually release sensitive data, uh, phone numbers, for example. So it's gonna be really, these privacy issues are gonna be complex both from what the, the known, the top part of the iceberg that we know, which is it was the anon, was the data anonymized cy, we made sure that it can't be reconstituted. But also some of these other additional things that we're finding out with, you know, prompts, being able to extract sensitive data that, that we may not have expected it being able to extract.

And, you know, I asked this somewhat in jest, but somewhat like out of curiosity, um, in, in, in that, is it a legitimate analogy, but is it somewhat akin to the advent of the printing press and the ability to sort of, you know, spread information and then have to discern the integrity of that information that you're receiving? Is this just gonna be like a natural cycle of human development as we continue to innovate in different ways? I love the printing personality, um, because I think it's, people said that about when social media came in.

So I wonder, I, I would love to get Diana and Andrea's perspective on this, especially from the geopolitics side. You've seen the printing personality come in many, many different technologies. How do you rationalize that from a policy perspective?

Do you wanna do geopolitical Andrea before I Sure, yeah, I Do. Okay. Okay.

Yeah, Real fast on that. Um, so I think there definitely are analogies there because it's, it's access to information in ways that had never happened to wide parts of the population. And I think, I think it's an equal size disruption, however Mm-hmm.

The speed and the scope and the scale just are so enormous. Like before, it's still was in pockets of areas. And I know, and you know, back in the day that that still was an, an enormous shift.

And so I think the magnitude of shift is, but just the breadth of it now is just you, you can't really, that that, that's where, you know, basically it's taken to the, the modernization component of it. And I think still what we saw in both cases was the use of it for disinformation and information. And so in that regard, you know, it's, it's, you know, human nature and so that that part's not necessarily new but's, the ability to spread it so much faster now and the speed of it.

And so I do think it is a informational access, similar kinda blip in society and in development. Um, and now we're seeing it though at it, but at a extremely, extremely exponentially different scale and used in, in for both cases for, for good and evil. Yeah.

I, I, I agree. I mean, it's, it's, it's like when you said printing press, I was like, wow, is it that impactful? But it is fairly impactful and that now people can generate massive amounts of content that looks and sounds pretty good.

I'm not talking necessarily about accuracy. There's been a lot of issues about the, you know, a lot of of ink has been spent on the hallucinations, which sounds kind of really like, oh, sweet. But I mean, it's, it's an inaccurate response from the, the generative.

It generated something that wasn't accurate. For example, like a a, a case, you know, cited a case law, um, you know, previous, a previous, uh, previous finding in a ruling in, in, in, in law, for example, in a, in a legal brief. And then it's not, that actually didn't exist.

So there's a lot around the hallucinations, which is one big issue I think with disinformation and LLMs is just the bot told you the wrong thing. So there was no, and to, to rom's wonderful point. And Rom has done fantastic work with Harvard on taxonomies, around failure modes, intentional and, and unintentional.

Um, so, you know, thank you rom for that work. 'cause I, I found it just so helpful as I think about, uh, the risks. But, you know, so there's that, that unintentional risk, which is the, the chat bot just was generating something that's statistically probably sounded right, which is what it's supposed to do.

It doesn't know if it's right or not. It's statistically probably right. Um, so there's, there's that for disinformation, which could just lead somebody who's a, a general user to get something that they believe is right, but isn't actually right.

It's not the same as going to Wikipedia and seeing something that's peer reviewed. Uh, but there are, there are, you know, with things like Rag al augmented, um, you know, generative, uh, AI that I, you know, so you're going to a specific data source rather than the whole internet that the LL m's trying to figure out what's right and what's wrong. So I think that there's gonna be positive work there in information disinformation, but it is a problem until we get it solved.

But I can see that path on solving where it's gonna get kind of a little wild westy, I think is with this ability to just generate images and communications and videos that are either look like some, you know, the deep fakes, but also just I writing a ton of content that couldn't be written as well before. And then who wrote that content? How do we attribute that content properly?

I think that there's gonna be a lot, we're gonna have sort of a wheat chaff problem when it comes to misinformation. And in that case it is, it's kind of mind blowing in that the printing press locked oral history in place. 'cause now we had something to go back to that wasn't just scribed and maybe rewritten by somebody.

And, uh, you know, but the printing press really kind of locked data in place in some ways. And the chat bots are interestingly exploding data and data for a variety of different aspects. So how this ultimately comes into it, how it ultimately impacts our world, is gonna be really interesting.

But I do think that that disinformation and at least making sure that we're helping our users understand what LLMs can and can't do, and how to find those trusted sources for the information that's coming out. So there's gonna be some interesting, again, wheat versus chaff work to, to be done. Mm-Hmm.

I love that. Um, I love that framing because I loved how Andrea was talking about speed of dissemination, and I loved your spin, Diana, of how the information itself is exploding. Um, I just wanna add one more liner note to this, um, music.

Um, I remember like watching this video by, uh, Salman Khan from the Khan Academy, and one of the earliest ways that he was using, um, generative AI was to teach kids. So, for instance, um, I'm sure you've all seen this thing, it was the Great Gatsby, they ingest the book Great Gatsby as one of, in the 32,000 context window. And now students can ask questions to the book, you know, like, why did the green light come at the end?

You know, uh, what's the significance of that? So Casey, tying this back to your printing personality, I think, I love Andrea's point about speed. I love Diana's point about how it explodes.

And now books were like one way street, right? You, you just like, get what you need, but now you can have a conversation. Now entire startups are like built around this character AI where you can have a conversation with, um, Kylie Minogue, so like personality, um, you know, and, and people are licensing like a million dollars for the likeness of image.

So there's all these new ways that is opening up, which I think is pretty exciting. Yeah, imagine. And, and it's blowing up in different ways, right?

You can have a conversation about your security logs that you put them to your scene. You can have a conversation about your medical records, you know, your bank statements, you know, your balance sheets. And, and now think about all the problems that Diana said, which is like, Hey, these systems can fail.

They can hallucinate. And what if there's a big difference between telling this kid that, Hey, this green light comes because of flash, you know, comes into the Great Gatsby, the end to like lying about, you know, your not lying about, but hallucinating about your bank statements. So anyway, something to think about.

That's why I really like this richness of conversation. Yeah. But even just something like a book analogy.

I, I agree. You know, and the, the, the, you know, what does, what is Jay reaching for with the green light? And, and Gatsby is, is a great example because the first thing that everybody went through with the LLMs was like, well, the kids are never gonna write the essay.

So they won't do the critical thinking of what that, 'cause it's not your teacher knows what the green light means to Jay Gatsby. It's you that needs to, you know, that they want you to write the essay so that you have had that critical, the ability to do the critical thought. But as Rom's pointing out some of these, what they're you, rather than just giving you the answer, these systems are now able to, in a guided conversation, help to advance that critical thinking so that you ultimately begin to understand.

But it's almost like a hint system when you see some of these, these working, right? It's almost like a hint because, you know, they, they, they, they don't give you the answer. They kind of help you understand it's money, it's aspiration, it's, you know, but asking in the right way, which I think is a, a really beautiful way to use these systems.

So teachers were initially worried it's gonna give them the answer, but as, as Rahm's describing, there are ways to reframe that interaction to now you can help to learn hint through to get to the answer yourself, which is great. Yeah. And I'd say, and we saw a lot of examples of that, right?

During the review committee, I thought was ways of applying in various kinds of gen AI and ML as an assistant for that, like exactly that in the security space. And that, that's where I think it was like back to both Ram and Diane. Like the, the positive aspect of all this really is, is like, how can it make us all better at what we're doing in our job?

We hear a lot about, you know, it's gonna replace our job. We don't hear as much about how it's actually going to assist our job, make us all do it better and work together as a community better. And there's just a lot of examples that popped up across, again, like nuanced aspect of it across everything from dev, you know, DevSecOps to threat intelligence, to writing executive reports.

Like we saw, you know, just discussions and examples across different industries and different areas of cybersecurity showing how this can be an assistant in, uh, in improving security across the, you know, across the entire landscape. And I thought that was really fascinating. And what's interesting is that it helped for me, it sort of underscored.

Um, so the work, when I was at IBM security, which would be about 20 16, 20 17, we were training Watson for cyber, and Watson was an early ai, um, you know, training Watson for cyber. And we knew that, and it was to be human assist. It was to help the analysts, help the human.

And when we started, when we got it out there and, and folks were using it, and we had analysts coming back and saying, this report that would've taken me four hours, not the writing per se, which we all think it was Chet, but the investigation to understand which systems were touched, what it meant to the organization, that kind of signal to be able to bring it together. We were getting them coming back with, you know, it used to take me a couple of days, it's taking me a few, four or five hours now. And then we were like, that, that is what we were trying to get humans being able to get, go through all this vast amount of information, bring it together, tie it together, and analyze it more quickly.

Love that. So, am I the only one that's sitting over here thinking, dear God, I don't want anyone having a conversation with mind comp. I mean Oh, Yeah, Great point, Casey.

Yeah. No, and, and, and that's, it's, it's a training data, right? And that's, you know, one thing that I think also, you know, gets overlooked a bit, but the data is just so, so essential.

I mean, we just saw the New York Times now introduced a new lawsuit Mm-Hmm. 'cause of how their data was potentially, um, used on, you know, without permission. And so that's where we're gonna see a ton going on there.

And just the, everything from data quality, huge, huge component and what it's actually trained on for those responses to copyright and infringement, uh, lawsuits to the hallucinations that, that Diana talked about. I mean, it's you as always in this, it's all about the data. And uh, I think we forget that sometimes.

Yeah. Yeah. And so, you know, I like to think about the whole, you know, I feel like everything with AI right now is moving so quickly, right?

And when we had this conversation initially about like, Hey, we're gonna have this new track and, um, it's gonna be the intersection of AI and security. What, what would you expect to see, right? We have this conversation about our expectations, um, versus, you know, what actually came in.

I know, Andrea, you've mentioned a few topics that have come in through the interview I, the review process. But what, for each of you, what were sort of the expectations going in? Like I think that we're gonna see topics on this, this, and this and that.

We should see topics on this, this, and this. Mm-Hmm. And then what were, um, were those expectations fulfilled?

Like, ES as expected, we saw this, or surprisingly, you know, rom's sort of geopolitical moment, like, oh wow, I never would've expected that matters. But that I, that's a great topic. What were some of the, those expectations and surprises for you?

Yeah, I'd say you, for me, and I'll go real quick because I did covered a bunch of the, what we have covered and, and seen, I actually was a little bit concerned just so there's so much marketing right now around ai. And so my concern was more so that we're gonna see a lot of AI as a silver bullet for everything. And that you just push button solves every problem just like that.

And, you know, quick and easy to use. Nothing wrong. And, you know, it basically like all, like literally all the stuff that we see in the, and how it's marketed right now, um, 'cause we see that, you know, really spreading across you.

We even hear you some various executive statements and they're expecting their teams to then implement ai, you know, in the next week because you can just, you know, push a button and implement it in, in a week. And it's, so I, I was kind of concerned we were gonna see something along those lines. Um, not say we really didn't.

Uh, and that's, so I, I was, uh, pleasantly surprised. It really was just very, uh, multifaceted across, you know, many of the different areas that we talked about. And even as far as the area of causality, you know, AI securing, you know, our systems as well as, you know, what we can do to then secure AI as well.

So it's looking at, you know, the need for security and, and you know, it's both directions as well, which I think was really interesting. We, we saw a lot on that. Um, then you, the other broad ranges, you know, everything from the script, we had DevSecOps aspects of it and how to actually implement it correctly.

Uh, I think we actually saw, um, good coverage on that. I, it's, it was great to see coverage in those areas, um, as well as some of the policy and landscape and just other, other really just, I think focus on the implementation and practical practicality. That's, I think that's, that was what I really liked seeing, and that's what I was hoping to see, uh, going into it.

Yeah, same. I, I always wanna say, I'm always, you know, 'cause I'm, I'm a pretty hands on person. I always wanna know, how do we do this?

Um, why do we do it? How are we gonna do it properly? So it was great to see there were, you know, a number of, of really practical focused, um, uh, that, you know, submissions that were there.

Um, I too was really happy that there wasn't a whole lot of people just AI's magical fairy dust in, it'll solve all you. I was afraid that was gonna be, like, it was just gonna be buzzword, buzzword, buzzword. These were really thoughtful submissions.

What surprised me was the, the ones that were focused on, you know, intersections, intersection points. So I thought that was really wonderful. We had a number of, of either co missions or panel submissions, but people are very much thinking about this in ways that has been recommended.

Get a whole lot of stakeholders to discuss mm-Hmm, why you're gonna use it, how you're gonna use it, who's gonna, you know, clean the data, train the data, make sure things aren't biased or drifting. Um, you know, all this. But the, you could see it in the submission.

So it was, you know, government, private sector, tech companies, companies, you know, financial services or big pharma that use a lot, lot of a IML, uh, you know, lawyers, you know. So it's really interesting 'cause, you know, the ethicists coming together and wanting to have this conversation with the RSA audience as a, as a group. I loved that because that there, it is really important that we've got, we need to have multiple different viewpoints in the, the planning and the, the operational life cycle of machine learning and ai.

And it was really great that it seems like that's, there's a lot of that happening. And the way it was happening, a lot of it was surprising. 'cause I was like, oh, I never thought of that angle as it would intersect that other, you know, audience or that other cohort.

And so yeah, that was really a lot of like, really just bright pops of aha, these are really important areas to illuminate. Absolutely agree with both, uh, Andrea and Diana framing this is, I had no expectations. I had some expectations.

I had know what to expect. That was my only expectations going into this case. So, and, and again, I wanna sort of touch on this idea of like, the speed at which things are changing and evolving and rim this is specific to you.

You were on a panel last year. Um, this session was titled Security as Part of Responsible AI at Home or at Odds. And, um, we see, I think more and more frequently this idea, this concept of responsible ai, ethical ai.

mm-Hmm. Um, I'm wondering for you who, you know, that conversation was so, um, individual to you, like you sat on that panel, so you, you know, um, more than anyone the conversation that was had there, but what's changed over the last several months in regard to the complex connections between responsible AI and traditional security and what continues to be a challenge in finding common ground? Yeah, I, you know, I always like to think back whenever I am part of a panel, the book, uh, super forecasters, like how the, uh, how the worst people to forecast something are the experts.

Uh, and I love that book because that's the first thing that comes to my mind when it comes about the panel. 'cause um, I remember starting to joke, I, uh, opened the panel with a joke like, who among you thinks there's gonna be AI's gonna take over the world? And, you know, we were all laughing about it in a killer robots, ha ha ha ha ha.

And in, in a year, Casey, it feels like everything has changed in terms of the dynamics. We, you know, the White House came up with like, if you're releasing a foundation model, you need to evaluate that model for sea burn threats. Like, you know, chemical, bio radiological, nuclear weapons threats.

Imagine that. Like the White House telling a bunch of like ML folks, like, Hey, we are worried that your ML model can generate like nuclear weapons. Something that, you know, my panel last time the panel was like a, uh, was a joke.

So this time I feel with the UK AI Safety Summit, with the White House executive orders, with the EU AI Act, um, there's a lot of like regulatory moves. Um, and I also think the zeitgeist around, uh, this topic has changed even with the ML researchers. You've got very prominent ML researchers, uh, who have like, who stepped out of the woodwork and been like, oh, I actually think this is gonna pose existential risk.

And if anything, it's, it's even more polarizing. Um, these views, there's some factions who believe like, this is nothing. This is just going to, you know, all these foundation models are just predict the next word.

And then there's another faction that thinks that, um, this is gonna kill humanity and the problem that we all face. And, you know, like Diana, Andrea and I, and, and Leemore was like, where's the truth in this? And somewhere in the middle, but where is it?

How do we tease that apart and give it to our audience in a, in a, in a way that is informed, that's not sensational, but grounded in evidence. And for me, that was like the tough part with this year. Yeah.

Yeah. I, I, I agree with that. It's, it's like, you know, there's a lot that we need to think about that's grounded, that's here right now.

Uh, you know, of all that, right? RAM was talking about foundational models. A lot of organizations use open source, I think synopsis, uh, the company did some research instead of 80% of analytical type tools, including ml, are open source at this point.

Um, so a lot of companies are, are downloading open source foundational models. That's great. We're all using, we're not repeating the, we, we've invented a wheel, let's use that.

You know, they're training it to what they need it for, but they're downloading these, running them. They have high privileges when they run and they're not scanning 'em before they run it. So there could be a backdoor, there could be a malicious calls.

And that, that model, we, it's really important that, that we're doing scanning on the models, that we're downloading this really kind of down at the, at the practitioner level, important things that we need to do. But, but as Ram's talking about, so we needed that. Of course we needed that in the, in the track, but we also have to deal with it.

This is moving how people are responding to ai, how we're using it is moving very quickly. So in addition to this very practical scan, your models kinda information that we need to impart, we also have to have those conversations when we're looking for that. What's coming next?

What are people that are really looking forward thinking about? And not just forecasters that are saying the most outrageous or soundbitey thing they can, but truly thoughtful consideration of where might this take us? Because we are adopting these technologies, the next gens of these technologies, like the LLMs and the chatbots very rapidly.

So yeah, we did, it was a, we had to, to balance those two things. Diane, I know I mentioned we'd come back to this, but, um, you know, I, I would love if you could dig a little bit deeper into this concept of ML SecOps and particularly as it relates to what you would define as responsible ai. Yeah.

So, um, responsible AI is, is a, oh, um, responsible AI is is ai that's gonna be resilient. It's gonna be accurate, pur fit for purpose. It feeds the audience that is supposed to be fed with it.

You know, there's a whole lot related to responsible ai. But one of the steps to get there is you have to have security built into the process. You need to have a secure by design.

If you're a AppSec engineer and you've been deep into DevSecOps, this is gonna sound really familiar. But if you're not, then, and if you're, if you've been working as an ML engineer or a data scientist, coming into some of these concepts may be a little bit new, but it's, it's really about taking that lifecycle of ml and it looks a little different than the lifecycle of DevSecOps. 'cause DevSecOps starts with requirements definition in machine learning.

Sometimes it's really just scope, can we do this with machine learning? Should we do this with machine learning? You know, they're like questions that it's, it's different from Right, because We came in doesn't mean we should.

Right? Exactly. Yes.

And then, you know, as you move through, which models are you going to use? How do you train those models? Is the data you're using to train those models?

Has it been cleaned? Is it from a a, a reliable source? Have you checked to see if it's bias or not?

'cause if you train on bias, you'll have a model that that outputs bias. So it's, it's weaving the security into the steps of the machine learning lifecycle around scope model definition, and, and, and model use training data. Um, and, you know, data, data source, uh, acquisition and training the model, and then making sure that you deploy it in a way and test it before deployment, obviously, and deploy it in a way that's gonna be monitored and also protect the information that was in there.

So it's this, this whole lifecycle, um, and MLS SecOps is just really infusing security into the full machine learning lifecycle. So hopefully, um, as organizations are, are looking at adopting that, that's gonna help them get more, just as we got more responsible resilient software by putting security into the lifecycle, we will bring this. And the other thing that's kind of interesting is that another way that responsible software was, was, uh, you know, supported and pushed forward was through software bombs or bill of materials, you know, the recipe of what's in the software, IE what are the dependencies in the stuff?

So, you know, do I have locked four J do I have to patch machine learning bombs? Uh, again, it's a little different. Same concept as a software bomb, but now you wanna look at the artifacts and the data and information that's important to the model.

What's its name? Where did it come from? Who trained it?

What data was it trained on? These sorts of things, which again, will help us to practically implement the, the concept of responsible AI as we build security in through the process. Uh, thank you.

And another topic that I wanted to return to is this, um, mention of legislation and, um, work that governments are doing both in the US and abroad. And maybe let's talk a little bit about how these efforts, first of all, what are these different efforts? There are lots of them.

And how will these efforts impact the technologies moving forward? I can kick off with, with a couple of them. I mean, I think, and ro you know, addressed one of the ones I think that's making the biggest headways in the eu, uh, with the EU AI Act and really looking at, you know, the risks to AI and helping, um, consent guardrails around the health and safety, um, and rights that are within those.

And there's the US executive order that's been out there. Um, I think what's interesting at the same time, there also are lawsuits. So it's, there are the policies that are going on there.

And as these policies are starting to get crafted, we are seeing lawsuits pop up across the globe, like from, from Australia and Asia and the United States as well, that are then informing some of the policies. And so we're actually interested, it's an interesting world that we don't normally see in the policy space. We're seeing policies in the lawsuits actually kind of informing each other a bit right now.

Um, but we're still, I'd say, fairly nascent technology still is moving a lot faster. And what's I think great though in some regards is that the governments are also getting ahead in the absence of a policy or a legal framework. They're providing recommendations.

And that's where I think we're, I, I've seen, you know, some just really, really great work for, you know, for our, you know, people listening right now. If it's, I mean that all this can seem overwhelming, like, well, what do I do? There's some really great documents put out by the governments on this front too, address, uh, what to do.

And, you know, NIST just had a new one come out on adversarial AI and highly recommended reading. And, um, hire Anderson can contributed to that, that Ram and I both worked with. Um, and so that, that's one place to look at you.

And NIST also has an ai, um, risk management framework as well. And so there are good frameworks starting to pop up in, in the absence of policies. And that's, it's almost in the absence of something with a compliance mechanism.

There are, there is guidance out there for what companies can be doing now, but those policies are starting to pop up, take shape. Um, and again, it almost goes back to geopolitics. It's different depending on different areas of the world, because in some other areas, uh, it's own, it is the wild west and will continue to be.

So, uh, depending on what, you know, the, the government's intentions might be, and I'm sure I missed some so wrong, or Diana. Yeah, I was, uh, Rob. Okay.

Um, I, I was gonna, yeah, add in that I, I agree. It, it's really nice to see that governments are, are trying to get involved in a, in a responsible way. They're having a lot of hearings, they're listening to the industry.

I think they're trying to really provide good guidance here. And I think that that's a, I, I'm happy to see that because these systems do have a lot of potential for good and for bad. So making sure that we're thinking about what the bad could be in advance so that we can kind of constrain and restrict, I think, um, is, is that's really, it's really a, a good positive step.

I agree that tax, if you haven't looked at the adversarial attack taxonomy that came out of nist, anybody listening, it is a really fantastic document. I've been sort of just vibing on it all week, just loving it. So, and thank you for your work on that.

Um, and then some other things at Mitre, if you're an attack person, a Mitre attack, if your company has adopted that, MITRE has Atlas where they're starting to look at, you know, where as we, as we take an attack, kind of how, where in the, the process and attacker might get in, uh, making that, looking at that for MLAI also, OASP has done some really, really good research work here. They've got top tens both for LLMs, um, gen AI and ai. So, you know, looking at these two top tens so that people machine learning so that you can understand what the risks to the different systems are.

So a lot of work, it's still, obviously everything's kind of, you know, jelling out. We're a little early, but all of the, the, the work that's been done and the quality of it, and I think the intent is, is really, really wonderful and encouraging. And like I said, it, you know, props out to, to sist, um, a wasp, uh, and Mitre because they really have done a, a lot of foundational groundwork for us.

Absolutely. And Paul, hopefully that answers your question. Paul was asking about industry standards and resources and framework, um, would be helpful to understand best practices.

Uh, we have a ton of questions that have come in and I wanna get to as many of them as we possibly can. Starting with one from, um, Coupa to all the panelists. What is the best way to begin learning about and contributing these working groups and committees?

That's a great question. I think for Mitra Atlas, you can, um, you can pretty much just email them and say, Hey, you know, I have this case study and they're very responsive. Uh, OAS top 10 has need apps as well.

But, um, if you're just like broadly thinking about, Hey, what is one thing I can do after this webcast to just come up to speed on large language models? Um, what I would suggest is, there's a really good video, uh, by Andre ti it's called, funnily enough, a Busy Person's Intro to lms. It's one and a half hours.

I mean, it's one or 10 minutes, but it is a fantastic introduction. You can like do it, you know, between like, as you're eating lunch and it's packed with information and it's, if you're, if you're ever like, no, I get nervous too, sometimes I'm like, oh my God, the field is moving so fast, how do I keep up? Um, so it's a, it is okay to feel this anxiety.

B there's a lot of content out there especially, but, but if you can do one thing, I would say go listen to the Busy Person's intro to LLMs. Love it. Thank you.

Um, okay, so what about potential constraints of using models in one country and not in another for multinational companies? Andrea looks excited to answer that One. It's a key question, right?

Because you don't wanna be at a competitive disadvantage. And that's, so one of the biggest concerns about the regulations on, you know, coming out of the more democratic countries is are they gonna basically, you know, minimize innovation by putting in some safeguards? And so you, it's that that, you know, innovation versus regulation you trade off.

And that if there's some governments and companies elsewhere that are basically ha have zero, um, restrictions on anything that they can do, will they have the competitive advantage? And that is know that that is the ongoing discussion and you know, it obviously we will, we'll see how it all plays out, but I think at the same time we can see that having some of these restrictions in place actually may provide more of a competitive advantage if, if done well. Um, especially for, you know, fostering ideas and innovation and, you know, creativity and collaboration is actually how the, if the regulations are done the right way, they could actually help foster greater innovation in that regard.

Whereas in other areas it might, because there are, you know, there's less of that and because of, um, the way the governments might be applying it, you know, it may create more of a, you know, civil society that's less prone to innovation. And so I think, you know, we, we will see, I guess. But, um, that definitely is a concern.

That's definitely something that's being talked about at, at, at, you know, various levels, um, and across the globe. Awesome. Um, Rob, thank you for sharing that, um, that link, I will go ahead and, um, asked John to copy it and put it into the q and a for our listeners.

So thank you so much for that. Um, Diana wanted to go back to, someone had a question about your, your point of scanning for models, scanning your models, um, and she says, yes, sure, or I, sorry, Krupa said, um, however, what kind of training do my teams need to determine which of the flags from these model scans we should really worry about versus informational low risk items? Well, that would be your, your model scanner, right?

Should be telling you this is, this is a problem. And why is it? Does there, is there, is there a call in there that's maybe getting an attribute or pulling down a dropper?

You know, it's a basically pulling, it's a dropper and pulling down a bigger piece of malware. So that would be your, your model scanning tool would indicate just like your, your virus checking tool. It's gonna tell you, is this a, this a big risk or not a big risk?

Your, your model tool, your model scanning tool can tell you that. Thank you. And, um, another just comment from one of the attendees, we do need discussion without a doubt, but at some point in the very near future, we're going to have to take action and quit just discussing this problem is growing at a rapid pace and we're running out of discussion time.

I fear we may be on the tipping point as we speak of being too late to act and reel it back in. Any thoughts on that? We're we're we're still around.

We're still, yeah, so I mean, you know, it's, I I think the, the, I don't think it's too late. I, but I do think that people have to take action in the area where they're the most interested, you know, and where they feel they can make the biggest impact. Like Andrea, you know, out there setting the pace and, and getting people to think about the really hard geopolitical issues, for example.

So if you, if you know, if you're a lawyer, if you're a geopolitical expert, we need you to be involved. If you are, if you're a hacker research kind of person who just loves to, to find out where things are broken, um, start looking at bug bounties around, there's a platform Hunter, HNTR, which is a, a bug bounty specifically for MLAI. A lot of the MLAI companies have their own bug bounty programs.

Look at the bug bounty programs, look at the kinds of things that they're trying to find out. So at a very, you know, sort of practical hands-on level, if that's the kind of thing you'd like to do, start helping us figure out whether it's you start, uh, you know, pro prompt injection attacks or looking at the supply chain related to I ML that will help us understand the inherent risks so we can be better about protecting and practicing. And then, you know, absolute leadership around taxonomies that you, the work that that ROM has done so that we can start to really, instead of it feels like it's this and I'm amazed that rom gets, feels overwhelmed, that makes me feel a little bit better because, um, because it's, it feels like there's so much coming at us, but we need to start thinking about this categorically and how we can actually start to, to put these into, you know, something that's, that's sort of, uh, that's manageable in our own brains about how we, how we start to address these problems.

So, um, I don't think it's too late, but we do need a lot of people to get involved and, and to help us, uh, all of us, I mean us the world, the humanity, to make sure that as we continue to adopt, we're adopting safely. Nice. And last question, I'll give you each 30 seconds to respond, um, because I do think this is important for Bobby to have an answer to which jobs in cyber are most affected by AI in cyber, or which areas should we focus our skills to be relevant in the future.

Ram, you wanna start? Yeah. 5 million jobs that have been posted in the US for cyber and have not been filled yet.

I don't think AI is going to take away, um, any of your analyst job. It's so logical. It's, it's highly structured.

There's one thing that I, um, this is coming from a bias perspective of ai, red teaming. There's one skill set that I think you can, you can practice today is, um, think about prompting strategies and, you know, if you can look up, hey, start building things with these like APIs. So whether it's like cloud anthropic, open ai, Azure, open ai, don't worry about it.

You know, get your hands dirty and build a small app. It should take you like 20 minutes with no coding experience. So that will pull the whale away and kind of make you realize where, where you can see this.

So get your hands dirty and don't feel that any of your work is gonna get cannibalized. I'll pass the Patel on to Diana. Okay.

Um, yeah, I, I, I completely agree with that. Um, I would add that I, I think that we're, we can see some of the biggest wins from AI early for security is to do the things that maybe humans, uh, shouldn't have to do or is really tedious. You know, is this a fish or not looking at, look, image classification is this, is this not acceptable image or not, you know, they can be really wear on people's brain.

So having machines help us do that, this is actually a plus and I see this as getting some of the stuff that humans would like to be relieved from doing and doing the jobs that are, as ROM said, you know, more thoughtful, higher level. Um, but do absolutely educate yourself on what ML and AI are because just saying, I don't wanna do that, that's new. I think that then you could risk, you know, not being, having your skills ready for the next generation.

But I wouldn't be so scared that there aren't jobs. It's just gonna be the jobs that are gonna be different and hopefully AI's gonna do some of the jobs we really don't wanna do. Awesome.

Yeah. Yeah. Andrea, anything to add?

We're at the top of the hour. Well, it's super fast. I, I would just say for those interested on the sort of the human element side and that there's plenty of you need for this kinda expertise in the policy and regulatory space to make sure those, those laws are done well.

And so if you tend to be more on the tech side and think you don't have anything to contribute on the policy side, please cross that path. There's a bunch of different, um, collaborative efforts out there. I, I'm on fellow at NSI, which does a tech policy, um, you know, fellowship.

So like look into those kind of things and find some work there to have a good contribution even to the broader way this all this is evolving. Um, and if you want, don't wanna do quite that, you know, high level, but one within a company. You working in the product management space and having the AI and ML background and having that understanding, that aspect as well as implications of what those products are, that's a great area to go into as well that I think, um, will only grow in relevance.

Yeah. Awesome. Thank you so much, Diana, Andrea Ram, thank you so much for being here today.

Listeners, thank you for joining in. com/marketplace. Here you'll find an entire ecosystem of cybersecurity vendors and service providers who can assist with your specific needs.

com for new content posted year round. Until next time, thank you.