Intensifying Identity Management with Stytch’s Reed McGinley-Stempel

This is Textron tv. Hey guys, thanks for the throw. We're here with Reed McGinley, Stempel, who's CEO of Stitch, and we're talking about the need to protect these AI models from phishing attacks.

'cause it turns out that bad guys are pretty smart and they figured out that they can poison these things with the right amount of manipulation. Reid, welcome to the show. Thanks for having me.

Why don't you describe the nature of the threat? I mean, I just gave it a cursory description, but it seems to me there's a lot of things that can go wrong here. So what do we need to be aware of?

Yeah, so maybe I'll take a step back, um, and just talk very broadly first at kind of like, I guess security strategy and security security theater when it comes to applications. 'cause I think we're seeing a lot of interesting elements where AI has kind of changed the game theory of how attackers are, are working online. Um, so virtually every app has kind of two primary security goals.

One, you want to secure the application and its perimeter against abuse. So stopping things like DDoS, SQL injection, uh, botting, things like that. And the second, uh, primary security goal is typically protecting your customers.

And that typically is, uh, both protecting them in terms of making sure that their account can't be credentialed, stuffed phished, things like that. But also protecting them from themselves when it comes to things like social engineering and what we're seeing with ai, uh, as you know, our vantage point in the market just a bit, the context stitches a customer identity and access management platform. And so, uh, we provide authentication, fraud, fraud prevention tools to developers.

And the areas where we're seeing AI have an impact is much more efficient. Botting and headless browsing attacks on websites, uh, both to attack non-AI sites, but also particularly for that use case that you just mentioned, where it's, uh, reverse engineering AI APIs in order to get free ai, free AI compute, or to poison models, uh, to abuse them that way. The second piece that we're seeing is that there's much more sophisticated and a wider net of phishing.

Um, the third is that there's a rise in social engineering and more sophisticated social engineering. So think, you know, you receive a phone call that you know, it sounds like it's from your, your child or your grandson or your grandkid. Uh, but it's actually an AI clone.

This is something that we've seen over the last couple months is particularly on the rise in terms of kind of, uh, social engineering. And the final piece that we're seeing is apps being reverse engineered at a much higher rate. Uh, and so maybe I'll, I'll start there first with kind of the example you shed about like, you know, uh, an app and the LMS that they use maybe being reverse engineered for abuse.

There's kind of two different, two different motivations that we've seen. One is people that just want to abuse a free resource. So imagine if you're an attacker that wants to get free access to open AI's APIs or anthropics APIs.

One way that you can actually do that right now is you can go to any website or application that exposes AI compute, uh, on client side and what you can, uh, pretend to be if you're botting it or if you're headlessly browsing it, you can pretend to be a real end user, but instead you're typically just running, uh, you know, puppeteer or, uh, selenium or one of these tools that's trying to get past its typical bot detections. And then you're making API calls through its, uh, AI compute so that you can skimm those answers and get free access. It's kind of like if somebody exposed AWS compute client side, uh, you'd imagine there'd be a lot of people that would try to get free resources.

But the second piece, which is more malicious and nefarious and usually is somebody actually trying to harm the applications use of AI itself, rather than just, uh, piggyback or free ride on on it is when somebody will try to give, uh, you know, poison the model, whether that's through, uh, injection or if you have some type of feedback mechanism for how you're using AI in an application. They'll try to, uh, reverse engineer it, bought it in a way that makes you think that your AI is actually getting smarter, when in reality there's actually an attacker on the other side that is trying to make it dumber, uh, so that it's less effective. And so we can go into more detail on any of that, but that's kind of a, a broad view from what we are seeing in the market.

It seems like the common theme here is that somebody is trying to impersonate something. So how do we kinda identify these issues and prevent them from happening? 'cause we've been talking about the shift to identity and zero trust for a while now, but do you think that this whole issue will force that shift?

I think so. I think it mostly accelerates, um, a problem we've always had on the internet, but that one that we've never had as much of a need to solve so urgently, which is, uh, you know, since we've gotten the introduction of online forms, uh, we've also, you know, shortly after that got the introduction of capcha, right? You know, prove to me that you're not a robot when you're interacting with my site.

Um, and that's always been something that's important in order to stop, you know, abuse of signup flows, login flows, CR credit card, uh, validation flows. Uh, but what we're seeing right now is that it's become much harder to, to, uh, determine whether something is a human, uh, or a bot. And an example of this would be, you know, CAPS has been around for a couple decades now and it's fairly ineffective for two reasons, uh, in the post AI age.

One of them is directly related to ai and one of them is just a natural kind of cybersecurity, um, uh, introduction that we've seen recently. The first one is that if you're familiar with GPT-4, um, open AI's most popular API, uh, you may also be familiar or or may not yet have heard of this, that they have a feature called GPT-4 Vision where if you feed a capcha into G into GPT-4 Vision or any of the other similar tools on the market that do something similarly, the AI can now solve caps, uh, and actually typically better than a human. So if you've ever gotten one of those really complex captures that like, that's like rotate the shape or identify two similar objects, um, captures have historically started to become harder for real humans, but are still theis are able to pass them at actually a very high rate.

Uh, so that's one issue that we're seeing is that, you know, the typical tools that we use to determine human versus bot, uh, once you have a reasonably smart bot that can think for itself like ai, uh, they become much less effective. com, where you can actually go to this website if you're an attacker. And for about three to $4, you can buy a thousand solutions to Google Capture or ARCOS capture or hcap, whichever capture provider you're using.

And that's less of an AI attack and more of a mechanical Turk attack. Where what's happening is when you're presented with a capture challenge to prove that you're human, uh, it's actually possible to pass that challenge on to, uh, developing world countries for low cost labor where somebody's actually completing the capture challenge for you and then passing it back. com, which is more of a mechanical Turk attack, Who's ultimately gonna be in charge of, uh, fixing these issues.

And I asked the question because most of the data scientists I know don't know a whole lot about cybersecurity, and most of the cybersecurity people are not even aware of what their organizations are up to with AI development yet. So who's gonna step up From what we're seeing, the identity teams are playing a really key role at companies. And if you don't have an identity team, typically you still have some semblance of an identity vendor relationship, whether that's you're using a customer identity access management platform, or you're using a fraud detection platform, really, I do think the vendors are gonna be the ones that have to step up.

But also companies need to realize that they need to incorporate some of these better toolings to identify if, you know, if, if GP Tvo vision is trying to, uh, solve a cap on your site, the way that you actually solve that is through really good bot detection. Because even if they are solving it with ai, they're still programmatically submitting the solution to, uh, that cap. So you can still get deterministic readings on whether this is a human or bot, but you need to invest in that.

I do think, um, to your point, your average engineer is not gonna be, uh, the right person given kind of the complexity of these, uh, of the rise in these risks. Cybersecurity teams are gonna be really helpful in guiding companies internally on what they need to do. Um, but of course, uh, the fraudsters are getting better, uh, each year.

And so I do think it's gonna be a combination of companies realizing there's a lot to protect at stake using internal data and internal signals that they have, but typically layering on external valid, uh, validation strategies like using a better bot detection service or a better, um, AI resilient service when it comes to authentication and fraud prevention. Some of the risks sound like they're really aimed more at the service provider versus the enterprise organization and others are more serious. How do I kind of distinguish between, you know, that which is an existential threat to me in my business and that which is more of a nuisance crime that I might not necessarily get immediately impacted by?

Yeah, so the way that we typically talk about it with folks is, um, there's a few different levels of risk. There are some risks that are being created by AI that a hundred percent of externally facing apps and enterprises that run those apps need to think about. Um, and this is an example of this would be more sophisticated phishing, uh, and also more sophisticated botting where somebody's maybe credential stuffing, where even if you're not incorporating AI into your company's roadmap, uh, some of the data that we're seeing, like, uh, Darktrace, which is a cybersecurity firm, has found that, uh, the complexity, linguistic complexity of phishing attacks has increased about 20% since the release of chat GBT.

And the reason I, you know, the reason for that is that the attackers are actually using chat GBT, but also the GBT APIs in order to make their phishing campaigns more complex, more believable, more linguistically sound. And so this is a problem where any company that has an externally facing app has always had to worry about things like phishing and credential stuffing. Uh, that's now much easier in a AI based world because the attackers, we don't just have these great AI tools.

The attackers also have these AI tools. 5, which is what chat GBT used when it took the world by storm a year ago. Um, the biggest difference is that if you look at the benchmarks of different tests that the AI had to complete, it went from completing most benchmarks of intelligence at like a 20th, 25th percentile intelligence rate to completing most tasks at an 80 80th to 90th percentile.

So it went from failing the LSAT to getting an LSAT score that was good enough to get into a top 15 or 20 law school as an example. And it kind of replicated this across most disciplines. So coding, um, linguistic complexity, math, uh, et cetera.

And the reason i I say that is really what that means is that now that fraudsters have these tools at their hands, it's not that it's, um, made necessarily the best hacker in the world that much more productive 'cause they may still be better than the ai, but it's cha it's changed the distribution of where that median hacker sophistication has likely jumped one or two standard deviations to the right because they have the assistance of this tool that is able to be linguistically very intelligent, mathematically intelligent, a great coding, uh, assistant. And so people that historically were not gonna be super sophisticated when it came to phishing or reverse engineering or botting websites now have really complex tools at their fingertips, which they're, they're using to a high degree. So does this mean there'll be more of the bad guys because the level of complexity or skill required to launch attack is even gonna be less than it is today?

I think so I think you can argue that I think for sure what I know is that it will feel like there were more bad guys because the, because the sophistication level increases applications that historically had thought they were doing enough like on the bar of preventing attacks are gonna find that attackers, uh, sophistication, as it increases enough of those people are able to jump over that bar. And so it'll certainly feel like that. I think there's a good point to argument to be made that it will also, uh, bring in more fraudsters and attackers 'cause more folks can be competent attackers that historically may, may have not had the software engineering skill or the cybersecurity expertise.

Um, I think that's still to be seen as there's not great data on whether it's actually bringing more people into the fold, but there is very good data on that. It's making people that are already fraudsters or attackers much more productive, efficient and complex in the way that they attack. So what are we all supposed to do about this?

Do we need to upgrade our identity access management systems? Do we need to go build a large number of AI models to secure our AI models? I mean, how complex does this get?

Yeah, so the two biggest pieces of advice that we give to security teams when they're trying to deal with this and the new risks is one, uh, ensure that you have a very, very deterministic and comprehensive bot detection layer in place. Because if you're able to discern between the real humans on your site and the actual bots or headless browsing that are, that makes the rest of your job significantly easier, uh, because that means that you don't have to worry about whether somebody trying to enter their password or somebody trying to hit your ai API, uh, is a real human or a bot. And most of the abuse is still coming from bots because attackers want to be efficient and programmatic.

So that's the first thing is like what we found is actually just having CloudFlare, which is a great solution for DDoS, uh, has not been great at actually stopping these attacks because a lot of them are using headless browsing attacks, which CloudFlare is really good at, at determining is this programmatic language. But it's not great at determining is this an automated browser that's trying to hit this API endpoint or is trying to log in. And so that's the first thing that we say is make sure that you've invested in and have taken a look at the landscape of the bot detection offerings that you have.

Um, typically you can get this from your customer identity access management platform, uh, someone like a stitch, uh, or others. And so that's one where if you are not already using a provider, you could consider using them even just for this. The second component though, uh, and second biggest piece of advice that we give is as phishing is on the rise more complex.

So social engineering is occurring. The single easiest thing that you can do for both yourself and your users is invest in what we call phishing resistant authentication factors. So, uh, what are the types of authentication factors that aren't phishing resistant and what could you move to?

So the ones that aren't phishing resistant are things like passwords. Um, and even though there's passwordless methods like email onetime passcodes phone onetime passcodes or email magic links, those are still phishing, um, vulnerable, uh, even if they're more secure than a traditional password because a user that's phished can still be convinced to erroneously or accidentally forward along those codes to an attacker. So what does that mean then?

What is phishing resistant authentication? Uh, the best example of this in the market that I would advise everyone if it's not already on their 2024 roadmap to find a way to fit into their 2024 roadmap is implementing, uh, Fido's pass keys, uh, authentication technology. This is something where Google's announced that they've just ingra integrated it for Gmail.

Uh, Amazon's come out that they're, uh, beta testing it. WhatsApp just made it the standard. You're seeing a lot of these large companies that care a lot about cybersecurity move towards it.

The reason being is that pass keys are effectively cross device biometrics where it uses your, you know, iCloud or your, your Google Cloud account in order to sync, uh, these private keys of your biometrics for face id, touch ID across, uh, devices so that you can sign up for an app with face, ID then go to your laptop and log in with touch ID without having to take any other action. And the reason it's valuable outside of that just being a great user experience that we're familiar with biometrics, is that it cannot be phished. There is no way for somebody to get a man in the middle of what you are doing and then be able to pass it on as if they are the true user because the check happens device side.

Uh, and so unless you have my device, my mobile phone or my laptop, and you're able to emulate my biometric, so my face or my fingertip, which is not impossible, right? So there's still some risk here. Um, but it, it, it makes it much harder because then you have to have the physical device in order to actually orchestrate a phishing attack, which is not how most phishing attacks occur in the world today.

'cause most attackers are remote. They may be in, you know, Russia or they may be in some other country and they're trying to convince you to do something. And that's where phishing vulnerable authentication types are becoming, um, less and less useful in this modern age.

So ultimately we have seen a lot of new regulations come down the pike that are a little more proactive than they have been. But do you think the rise of AI is just gonna force us to revisit that whole, uh, paradigm because we are gonna need to be a lot more stringent about security because AI platforms can do things at a level of scale that were unimaginable before? Yes.

Yeah, definitely. And I think you're starting to see it first at the government level where obviously the government is taking particular caution and we've talked to a number of government agencies that have, uh, been trying to catch up with a new regulation and advisory from, uh, the federal government to incorporate phishing resistant authentication by 2024. Uh, and so there already are some regulations that are putting, being put in place for some of our most vulnerable or high risk, uh, targets.

I would not be surprised at all in the next few quarters or uh, year or two to see that flow down to other parts of the internet, whether that be consumer companies, B2B companies. Um, but I think, you know, already we're seeing this in the government space that people recognize that, uh, you know, whether it's a nation state actor or just a traditional attacker, the kind of game theory has flipped, but also the risk factors have flipped. And so they're pushing for regulation and uh, for phishing resistant authentication to be a requirement.

Alright, Well folks, you heard it here. We're on the cusp of something brand new here. A lot of us don't know exactly what to expect next, but I got a funny feeling that everything that we have experienced up until now with cybersecurity is gonna feel like childs play real soon.

Hey Reid, thanks for being on the show. Thanks for having me Back to you guys and.