Cybersecurity Concerns of AI Platforms – Sam Crowther, Kasada

This is Techstrong tv. Hello, I'm Amanda Ani with Techstrong and I'm excited to be here today with Sam Crowder. He's the c e o and founder of casada.

How are you doing today? I'm good, thanks Amanda. How are you?

Doing well, thank you. Can you talk about Cassada and the services you provide? Yeah.

Cassada helps organizations stop the problems that bots and malicious automation calls. So anything from bots trying to steal customer accounts, you know, run stolen credit cards through payment gateways to make sure that, you know, our customers digital experiences are for their human customers. Excellent.

Well that's a great segue into the topic that we're discussing today, which is cybersecurity concerns regarding chat G P T. So with the rise of ai, you've noted a more recent cybersecurity concern regarding chat g PT hackers. How are fraudsters exploiting GitHub repositories to gain unauthorized access to chat GPTs APIs?

Yeah, what we've seen is obviously a, a massive adoption of the technology and folk who are, you know, maybe doing not so good things, realize this, you know, can help them achieve far more more scale than they otherwise will be able to. And what we've seen is there's been quite a few public repositories, you know, bots or scripts if you will, that attempt to bypass the security controls of, uh, the chat G P T APIs in order to let people integrate it directly into backend applications that maybe it shouldn't be integrated into. And so this is effectively meant that it's ended up inside different phishing toolkits and other fraud tools, uh, which now effectively have the power of this amazing l l M behind them.

Mm. So for what are the main reasons that hackers are doing this? What is their benefit?

Their benefit is a key limitation to a lot of attackers and fraud's disability to scale are the human components of their operation. Right? So a good example may be phishing.

Uh, someone has to write an email, even if a lot of it is automated, you know, someone needs to sit there and validate you know, what it's going to say and it's usually not super accurate. So now all of a sudden they have the ability to generate very human sounding text without ever having to have a human interact, you know, with really anything. And so it really unlocks an ability to scale like they've not seen before and that's why they're so attracted to it is all of a sudden the potential returns on these fraudulent operations sort of through the roof.

That's pretty scary when you don't know if you're, uh, who you're really talking to. So what are the guardrails, if any, that were meant to prevent this unauthorized access and how were they broken? Yeah, so there's really been two main things that organizations have done and the first one is restricting the sorts of prompts that can be used.

Now the issue here is that there's actually a real use case on the defense side to be able to get Chachi VT to do some of this stuff. However, you, they need to balance it, right? It's how do you figure out if someone's doing this for good or for bad?

And I think for the most part they've tried to mitigate people putting in prompts that may be used for nefarious purposes. The other side is then how you access it again, Chuck j BT is a good example, or it's meant to be accessed through all browser, you know, buy a legitimate customer. However, because the APIs that the browser uses when it talks to chat GPTs backend are actually able to be functionally accessed by unauthorized third parties.

It gives them another way in which they shouldn't really have. Mm-hmm. And is chat g t the only platform or are you seeing this across many similar platforms?

Yeah, we've seen this across others. So Google Bard, when that was released had similar tools that were built on top of it. Gotcha.

Can you provide any specific examples of fraudulent activity carried out through jailbroken versions of chat G P T? Yeah, so one of the ones which we reverse engineered and dove into a little bit more was actually using Facebook Messenger to effectively scam people. So they would, you know, have chat G P t, pretend to be someone you know, clearly doesn't exist, and actually start to engage with and interact with folk on Facebook Messenger.

And I believe also Telegram and Signal, um, what exactly they were doing was a bit shady, likely trying to get money somehow, right? But again, they no longer need to actually have a person sit there and interface and try to scam someone manually because they can, they can have chat G P t do the, the human part of their operation for them. And you mentioned earlier too that, uh, chat g PT accounts being sold on the dark web.

What are the implications for users? Yeah, so it's really valuable for a lot of these attackers to get their hands on the pro accounts because they're less restricted. They're able to use the, the more powerful models and some of the plugins, uh, the real implication for, you know, Mia say a legitimate customer of open io one of these other organizations, is that if my account is being used to enable cyber crime, yeah, it's going to be shut down.

I, I will lose my access. Uh, so that, that's really an implication for me. And then obviously for the company itself, the problem for them is now they're actually potentially having to turn off customer accounts that are paying the money, right?

And that's never a, a good look or a good conversation. Definitely a big concern. What is the solution, if any, to preventing and blocking additional hacks?

Yeah, I think it's making sure that both channels are as tight as they can be, right? On the, on the guardrail side, making sure that they have as much control of PO as possible around, you know, what these models will respond to, what scenarios they will, you know, partake in versus just flat out refuse. And on the flip side is the security of the actual accounts and the APIs, right?

That needs to be as solid as it possibly can be to dissuade people from, you know, from using them maliciously. What Alternative methods can companies employ to protect themselves from bot attacks and fake mimic human behavior? Yeah, and this is like why businesses like ours exist, right?

It, the reality is it's a very difficult problem to solve. We've spent many years, you know, getting our heads around it and building tools to deal with it. Um, but upfront the best thing that most organizations can do is actually have a look at what, what their consumers are doing on their digital platforms and making sure that expected consumer behavior, expected traffic actually lines up with maybe revenue or other cycles that, that they would expect to see from their customers, right?

Humans are really, really predictable on mass. Really practical example is if you have a lot of, you know, US customers, you should expect to see most of your revenue, you know, during the US day and then cycle up at night. If things don't sort of match that pattern, maybe something to look into more.

Got it. So last question. As rapidly as this technology is advancing at the government level, what do you think can be done or is already underway to better control AI use?

Well, I think they're starting to have the right conversations, which is good. It's good to see them have it this early. Uh, you know, obviously Sam Altman and other folk who are very, very senior in these organizations are gonna be very influential there, but as long as they continue to understand how it could be abused, they'll, they will make the right moves.

I have, I have no doubt. Well, thank you Sam, for coming on our show today and giving your insights about this issue and hopefully we can put some regulations in place and, uh, help prevent these attacks in the future. Oh, appreciate it.

Thanks for having me, Mandel. Thank you.