Limitations of ChatGPT for Business Incident Response – Alex Waintraub, CYGNVS

This is Techstrong tv. Hey everyone, welcome back to techstrong tv. Um, we have a, you know, this is the first time this gentleman's been on Textron.

So let me introduce you to Alex Wayne job. Alex is a, uh, digital forensics and incident response engagement leader at Sickness. So Alex, I think that's the first time I've ever introduced anyone with that title.

You're gonna have to do some explaining here. Tell us about you and your title and how you got here. Sure.

Thank you so much, Alan, for having me here. Uh, my name is Alex Waintraub, as you were saying, and I am the, uh, D F I R expert engagement leader or expert or evangelist. Essentially, all I do is help GNA define and deliver a path of instant response, uh, nerdiness over to our customers to allow them to and enable them, uh, to start planning, preparing, practicing, and ultimately responding to cyber incidents.

So that's basically who I am. I've had 12 years in the industry of, um, building out some of the biggest security operation centers and incident response operations. Everything from big banks to, uh, biopharmaceuticals.

And, uh, really excited to be here with you, Alan, to talk and discuss artificial intelligence and discuss Cignas. Absolutely. Well, you brought it up.

Let's, let's discuss Cignas, that's the company you work with. Why don't you maybe give us a little background there. Sure.

So after 12 years and at least, uh, 1700 cyber incidents that I've worked through, uh, what I've realized is that small, medium and enterprises really struggle with being aware of the cyber crisis. And back in June of last year, I came on board to help design the instant response operation for Cignas. Ultimately, we are a guided crisis response platform.

We're helping the small all the way to the enterprise, understand, be aware of the cyber crisis, prepare for the cyber crisis, and then start responding to the cyber crisis and, uh, making it a lot faster for them and remediating and recovering, uh, within those 48 hours post-incident rather than waiting a week or two weeks. So essentially it's a secure out of band communication and collaboration area, which allows you to invite your internal and external, um, retainer companies, legal departments, your IR team, your security operations team, your executive team, everyone into a war room essentially, and giving them the access that they need. There's granular access within the rooms, granular access within the work streams, and essentially giving you a place to start, uh, working through all the task workflow.

I love it. You know what, um, for people who wanna get more information, can you give us a website? Sure.

com and that is NUS with A V C Y G N V S. You, you anticipated me there. Yeah.

And that's an important thing, guys. C Y G N V S, but it's pronounced Nu um, Hey, Alex, where are you based? I'm in Bergen County, New Jersey.

Oh, okay. Saddle River area there. We have a lot of friends up in Bergen.

Yeah, she, Rubinoff just came. Sherry? Yes, she's up.

She lives, uh, she blocks away. She's actually in DC today, but, um, that's many blocks away. True.

Yeah. She, well, I don't know if she's there yet, but anyway. Fantastic.

Fantastic. Um, so look, you know what, Alex, you're in tech. I'm in tech.

We're both in cyber. You can't walk more than three steps without tripping over if someone talking about AI and generative AI and large language models and what it's gonna mean. Yeah.

And, and what is it gonna mean? It could be the greatest thing that ever happened to mankind, to an extinction le extinction level event. Right.

And everything in between. Um, I haven't seen this much hype around something I don't know, since maybe the cloud in 2005, but I, I do genuinely believe that AI and generative AI and all of this will have a greater impact on civilization, if not technology on humanity than, than the cloud did. Certainly.

Um, that being said, though, all is not, you know, as doom or gloom as they say, nor as it as rosy as others say. Let's talk about, you know, how generative AI can help hurt or not make much of a difference when it comes to incident response. Sure.

And I think this brings up a topic. You bring up the cloud, the cloud was a big impact on technology. Yeah.

Cybersecurity. Um, just literally the tech field took a big impact on how are we gonna learn this? How are we gonna grow within this new field of, of risk.

Mm-hmm. And how do we take on the risk? What, what are the risks?

Artificial intelligence is taking that on. But from a global stance, we're seeing the every single industry utilizing AI in some sort of way. Everything from healthcare to cyber, everything from the small business to, um, Vimeo just came out with a artificial intelligence solution to build out scripts for their videos.

I know. And so we're, we're Vimy, this is being done on Vimeo, so Oh, perfect. They've already reached out, out where's script?

Yep. Mm-hmm. Yeah.

So it's just, there you go. I wish they did, but no, they don't. Oh, well, at least.

Uh, but, but you Know what? That brings up A whole nother thing, Alex, which is, Yeah. And, and this happened in the cloud too.

Everybody, you know, when you went to raise money, all of a sudden every VC said to you, well, what's your cloud story? What, what's the angle with the cloud? How does the cloud play into you?

Here? Everybody wants to know, well, what, what, what, what's your AI angle? What's, are you incorporating ai?

What's AI doing for you here? How are you going to use AI to make your product better, your customers happier? All of these things.

And I do agree with you. I, you know, I've been in tech a long, long time, and there are certain tech technology things that revolutionized the tech industry. I think the cloud's a perfect example, right?

But the internet itself revolutionized humanity, right? Think about how your mom, grandma, whatever aunt is using the internet today versus what their life was like 35 years ago. Um, I think AI eventually has, has the potential to have that kind of impact.

I, I believe so too. So from a pros and cons perspective, let's, let's balance out the scale here. There are a lot of good things that come from artificial intelligence, and there are a lot of cons.

Just like the cloud gave a lot of, a lot more pros than it did cons, but it did have pros and cons from a risk perspective, from a GRC perspective, uh, there are a lot more cons on that side because where's the data being stored? Is the data being stored by us? Is it stored by the Linux server in someone else's, uh, environment?

You know, there are a lot more, uh, fun memes that came out when the cloud started becoming a thing than artificial intelligence. But let's just talk top five biggest, uh, pros and then we'll talk some, some cons on, on artificial intelligence. So the first thing that I see is that from a pro perspective, the artificial intelligence AI is going to allow the teacher to the healthcare individual to process a vast amount of data very quickly and make easy, uh, emails, uh, send out, uh, reports, make it simpler for them to just pop in a few pieces of information and it pulls out a full report for, you know, your kids, um, quarterly review.

Now that coming with that, also, there are multiple tools out there that could help us out with, uh, productivity and with the, you know, processes and with just making things a little bit more scalable than what it was before. Because now you could push out automated emails at three in the morning from a, from a teacher that says, hi, this is Miss June and I'll be back online at this. And this time it essentially makes things a little bit simpler.

Uh, we see a lot of personalization, automation, and then the language translation being a really big thing on the pro side, right? Yep. Absolute, absolute.

Any other pros? I we'll stick with that now cuz we only have 15 minute interview. We'll go on all day.

Yeah, I know. We can go on this for a few hours. I I go over beers and we can talk for this for, for a few hours.

Absolutely. So then comes the, the, the cons. The cons are, there's a lot of susceptible bias that we're gonna see within the data.

Uh, there's a lot of something called data poisoning where you can actually take the data and push, input other information on top of it and say, no, this is incorrect, this is correct. And essentially change the, the legitimate fact tool piece of information. There's, uh, a lack of contextual understanding.

So we don't really understand how tools like Chache BT can generate these coherent responses or give that deep understanding of context. And we don't know how old that context is. So when people are relying on this, it can cause a lot of risk because people are inputting secret information and hopefully using that to manipulate it and create something new.

And that can be used for a misuse or, uh, you know, irresponsibility of data. We saw that with GitHub back in 2006 and 7, 8, 10, 15, almost every year. That's where these brand new tools came out, where they're searching for secrets or programming language or data related to organizations and saying, Hey, your passwords are everywhere.

Your secrets are everywhere, and they're publicly accessible. And that's where GitLab came into place. So essentially we're gonna see a lot of, um, exposed pieces of information that's gonna lead to my next con, which is ethical concerns.

Uh, it's technology. And we, we as security professionals are going to see this being used in a manipulative way very, very soon, if not already. One of the things already that I've seen already.

Yeah. We're seeing this on bcs, the business email compromises are very sophisticated now, and they are manipulating tools like this to convert their Russian or, you know, beru language into, uh, English and then sharing it, or the, we're seeing other places like the North India areas utilizing tools like Chacha, BT to translate their, you know, broken English into really sweet looking emails. Um, yep.

And then there's a lack of emotional intelligence. This is something that I've been seeing where you want it to generate it a response, but then it lacks that empathy that a person might have. And so people might struggle to understand how the, the email or the message was supposed to be, because you're lacking that actual human being writing that message.

So that's basically the cons of it. So you're gonna see pros and cons on both sides. Absolutely.

And, You know, but that's life too, right? Nothing's all good and all bad. Now let, let's dig in specifically the incident response though, Alex.

How, how is this, you know, I mean there's, we're talking about the dangers of it, but there's, there's good to be had from it as well. Oh yeah. So I went and I did a almost a three project on artificial intelligence back in February and March where I tried to build out an incident response playbook and triage plans in Chachi, bt, because someone from a, someone who went through a ransom incident actually generated their response playbook for ransomware through Chachi bt Now, when, when just you and I, Alan, Alan, you're very technical, I'm very technical.

When I looked at this playbook, I, uh, did that emoji where you put your hands over your face because it was horrible. It was, you know, four steps for each one of these. Uh, you know, here's preparedness, identify your team.

Great. What teams? Who am I identifying?

And as a non-technical person, it's like, yeah, we get to check a box or, or you know, even from regulatory, you can check a box, but from a technical sense, there's no personality to that response playbook. And it missed a lot of steps. And so what I did was I tried to define hundreds of different playbooks.

It maybe maybe a thousand or so of these different business email compromise and ransom ones to see if I can input randomized data and see how it would, uh, output the data to me. And the output was always missing something key, either in the containment steps or in the remediation steps. It was old school, it wasn't new school.

You're missing, uh, that person, the personalized, uh, playbook. And it's gonna cause a huge risk. Uh, over time we're seeing, uh, predictive analysis, like how is, how is this gonna work if we, if, you know, we, if L statements, you know, how are we gonna be able to predict the attack vector based on X, Y, and Z vulnerabilities that might come out in the next year.

You're not gonna be able to detect that with these types of playbooks. Yeah, no, it's a game changing thing. So look, Alex, our audience, they're technical people too, right?

They're not the business finance team, they are the cyber team, the development team, the IT team, there's leadership as well as practitioners. What can they do? What can they, what can, what can they do to protect themselves, to lessen their danger to, you know, manage the risk, right?

Because at the end of the day, that's what, especially our cyber friends are doing, managing risk. It's awareness. You have to be aware of this.

I, I see it with business email compromise. People still clicking on links even though they've been trained not to. There needs to be more training and awareness of these risks and showing people what the risk is if you were to input your information into a tool like Check G B T or any of those other tools that have, uh, artificial intelligence behind it.

Yeah. I, I, I think that's the biggest thing. You know, I, I, I, uh, read an article this morning on the way in, uh, you know, to open AI and Google are actually negotiating with a lot of media companies to actually pay them like a royalty or a license because, you know, so much of what these, uh, generative AI things are, right?

Uh, you know, uh, programs are writing come from material that they've consumed from, you know, mainstream news sources or media sources, for instance. And I think that is really something that we need to drive home with our audience, which is, hey, when you put stuff into chat, G P T and ask it to, hey, write something, that stuff you put in there then becomes part of it's, you know, it's like the borg, it assimilates it and it becomes, you know, part of, of what it's doing, and the next person who uses it kind of has access. Well, the chat, G p t, the, the AI has access to that information and giving the next person their information or their response.

Yeah. And You know, we mentioned earlier Shira, right? Adjoining text strong.

I wrote a press release and just for kicks, I cut, cut and pasted it into chat g pt, and said, do better, you know, right. Write a version of it. And it did, it wrote a very nice release, don't get me wrong, but soon as I did it, I, I said to myself, oh crap, if this was really, we were a public company and this was really something under wraps, I just put it out there.

Right? Right. And, and that's how easy it could happen.

I mean, with the best of intentions, not even, you know, talking about sensitive, really sensitive kind of data and we haven't even touched on ability to generate code and so forth. Right? We're seeing that too, a a lot of it where people are just dropping in their code like they were on other tools on Google.

But this is much worse because now anyone can access that. We're gonna see probably in the next few years, tools like GitHub when GitHub came out and all the risk of all the secrets being exposed publicly. We're gonna see tools for artificial intelligence that are like the GI guardians and the site codes out there that were searching GitHub for doing it.

Good. It's a new industry. Yeah, no, it, it's what's old is new.

Hey man, we are out time, but Alex, you know what? Pleasure, we'd love to have you back on. Let's you know, don't be a stranger here on Text Strong tv.

I want to thankly you for coming on today. com. Alex sw Chv here.

All right, Alex, thank you so much. We'll be in touch. Thank you, man.

We're gonna take a break on, okay? We're gonna take a break on text on tv. We'll be back in a moment.