Huntress Director Jamie Levy on How Adversaries Use AI to Launch Cyberattacks

Hey guys, thanks for the throw. We're here with Jamie Levy, who's director of adversary Tactics for Huntress, and we're talking about, well, an interesting adventure they had where a cyber criminal downloaded their software, and that gave them in kinds of interesting visibility into how that hacker was using AI to drive some outcomes. But Jamie, welcome the show.

Well, thanks for having me. So, walk us through what happened here, because we've been speculating that cyber criminals are using ai, but speculation is not proof, but maybe we have proof now. Yeah, so I mean, totally they are using ai, it's just that we haven't really seen it play out.

Um, but this time we actually got a little bit of insight and saw that we, that there was an attacker who was using it. And so just to preface how this came about, uh, oftentimes attackers will install security software just to figure out if there's ways to get around it, um, you know, on their own side or just what makes it tick. And we had some particular event that actually happened like this where an attacker installed our EDR agent.

Um, and as a result of that, uh, they had a lot of malware on their machine for some reason, probably they're doing research, but there were some things that were running on there that were malicious. And since we are a managed EDR, uh, we are obligated to re to basically, um, triage those alerts and figure out what's happening and do an investigation. And so an investigation was kicked off just from the malware.

And so as the analyst was digging into it, eventually they realized that this actually was a bad actor. Um, but we had the files from their machine so we could do a little bit of investigation about what they were up to, uh, in the course of this. And, uh, as we were looking at some of the browser history, history, we realized that they were actually using AI and some of their workflows.

com to kind of tie together these telegram bots and some other things using various APIs and basically have a nice phishing workflow to target people and, and do this at a grander scale than just, you know, somebody doing this on their own. Is it your sense that they're pretty sophisticated or are they like most of us trying to hack their way through this thing to kind of make it work and kind of bend it to our will? But are they maybe a little more advanced than we are?

It's a little hard to tell. I they're probably somewhere in the middle. Um, there were some things that they were doing that I wouldn't necessarily call advanced, uh, but they were, they obviously had some kind of a workflow going, and so they weren't just completely flailing around, but there were some fail moments that we saw.

Uh, and, and then we put some of these things in the blog, like where they were trying to run executables with a python, uh, you know, interpreter, which this is never gonna work. But, um, you know, so there were things like that. So I would say there were somewhere in the middle, but they weren't, they, they definitely had a little bit of technical, uh, expertise, but, but they weren't like, uh, nothing impressive really, uh, that we saw from, from their outputs there.

Hmm. Um, well, looking into your crystal ball though, how quickly do you think they're gonna be moving down the AI learning curve as we go forward? And I imagine that, you know, once one knows something, they'll share it with somebody else.

And so, you know, as you think about where we might be six months from now, how dire could things get? Well, this person was obviously putting in a lot of hours during the day. We, we did a chart where we saw like, sometimes they put in like 14 hours a day or more, you know, just plugging away at this.

And anybody who's determined, it just keeps at it. And plus, plus AI is getting so much better and easier to, to use. I mean, yeah, they could be a force to be reckoned with, uh, pretty quickly, I would imagine.

Like if they just kept up with it, What are we gonna need to do to defend, I'm assuming this is one of these, you know, we need to fight fire with fire kind of scenarios. But as you kinda look at how cybersecurity might evolve to thwart these adversary tactics, what should we be working on? Yeah, totally.

Um, I mean, if, if people aren't also looking at AI as a way to use for defensive mechanisms, I think you're already behind the curve. I mean, the, the defensive side in general is typically a bit behind the attackers. You know, the attackers are figuring out ways around things and, and just constantly plugging at it.

And we're usually kind of catching up to what they're doing, uh, at least like as a broad, uh, view of cybersecurity. It, that's, that's the way it seems. Um, but yeah, we, we definitely need to be using AI to our advantage.

We need to, uh, figure out like, what are these vulnerabilities? And, and, and, you know, places where attackers could get in ahead of time need to be a little more proactive about these things. Um, but yeah, just watching what the adversaries are doing anytime that you get a chance where they've tipped their hat, like learning from that, taking advantage of that, um, talking to each other.

Like this could be a community effort even, uh, building, um, relationships with other companies and seeing what they see because like for instance, we see a lot of things on the, um, in smaller companies that some of these other cybersecurity companies don't see, but a lot of this nefarious activity tends to happen in our customer base. And so think about, um, the attackers, uh, threat landscape. It's kind of like this iceberg, like there's parts of it that none of us see really.

And there's parts of it that, you know, we might see, but other people see other parts of it. And so if we just kind of share all this, then we might get a bigger view of what this, uh, iceberg looks like. Right.

Um, I guess, is this a unique set of circumstances where somebody who is, uh, malicious is download your software and you get some visibility into that and or does this happen all the time? I'm, I'm fairly certain it happens all the time. Uh, I mean, there are probably people abusing our trials right now as I speak.

Uh, just trying to figure out ways around things. And in particular, this attacker had a bunch of different security software installed. It wasn't just ours.

They installed Bitdefender, they installed Malwarebytes, they were looking at something, um, from FireEye, I think, which I don't, I'm not even sure like that they're around anymore, but they were doing something with something named that. Um, so they were just going around like to all these different security vendors and trying to start, uh, trials and just, you know, figure out how can they use the software or get around it or whatever. So, and this is just one person and we, we know for sure that other attackers or even security researchers will try to download other people's software and then see are there ways around it.

I mean, I don't know how many times I see somebody saying they have a new EDR bypass for CrowdStrike or something like that. Right. And they're, and then, you know, they get their 15 minutes of fame just on that.

Um, so yeah, if security researchers are doing it, we know for sure the attackers are doing this as well. Right. So largely they're probably using this stuff to, to research, but they say there's no honor among thieves.

So maybe they're using your software to protect themselves from other thieves who are trying to steal practicing techniques. Well, in, in this case, some of the things that, uh, this attacker had installed seemed to be that he, it was for preventative measures, like he had, he had different browser extensions installed that that would protect him from various types of threats. And so yeah, it was there, there, there's that side as well.

Mm-hmm. So as you kinda look forward to where we are, um, clearly we're gonna need AI to combat these threats as these guys use ai, but what might that look like in your mind, if I'm gonna be creating my new defensive team? Is that gonna be a mix of humans and AI agents and how will they all kind of come together to function as a team?

Yeah, so, uh, we're not at the point where AI can just do all the work for us. Really, there, there has to be humans in the mix. Um, because AI does make mistakes, it does hallucinate some, it might just do something that you totally wouldn't want it, you know, it's not intended to do.

Um, but it can definitely help you automate a lot of things and scale things. And if it's properly trained, um, I mean, that helps reduce burnout. That helps, um, it helps you come to better conclusions faster.

It, it helps you get past some of the mundane parts of, of the job, uh, which, which in all, uh, fairness actually helps you be a better defender. If you're not bogged down doing these, um, you know, random tasks or whatever that take a lot out of you, then, then you're able to do the more effective things to help protect your, uh, your enterprise or network or whatever, you know, you're, you're dealing with. Um, so that's, that's where I see AI being the most effective.

And then also on the proactive side, people are using it for, for figuring out vulnerabilities and testing their networks and all these other things. And I, I see AI being a big factor there. Um, and, and, uh, you know, the quicker that you can get to, um, to the, you know, finding these vulnerabilities and these weaknesses in your own infrastructure, uh, the safer that you could be, because it'd be better if you find it as opposed to an attacker finding it.

Mm-hmm. You know, and I'd love to get your input on this, 'cause there seems to be a lot of folks talking about the color purple these days, and I'm not talking about the book. Um, there's red teams and blue teams historically, and now people are melding all that together and saying, you know, you can't really learn to be a good defender unless you know how to attack and you can't really create a interesting attack unless you know what the defense is doing.

So is is this whole conversation about how we approach, um, defense changing? Um, yeah, I mean, I, I, I guess in some ways, I know purple teams have been ar around for a while, but may maybe some people have been reticent to adopt them, but it, it doesn't make any sense for red teamers to do things in a silo. And it doesn't make any sense for blue teamers to do things in a silo because there's so much you can learn from the other side.

If we didn't have, um, red teamers building out new trade craft and, and, you know, honing it and sharing it, blue teamers wouldn't really know what these other attacks could look like. They wouldn't, they wouldn't know what to look for if they didn't have that. And then on the red teaming side, they can only improve, uh, if they know how they're gonna get caught, right?

So, and to enable to avoid it. And so that's what they get from the blue teaming. It's a purple teaming actually is the sweet spot where you're, where you have these two sides that you, they basically have their focus, but they're sharing everything across that purple team.

And, and that's where you figure out like, what are the gaps in your technology and what are the things that you need to fix and, and how, you know, both of these sides are collaborating And yeah, I I think that purple teaming is something that everybody should consider if they, if they have the resources to have an internal security team. So this interaction that you had with this hacker who is doing all this stuff, is that all now working its way into some sort of training module somewhere? Or how do I kind of look at that or how will the greater community benefit from this observation?

Yeah, so we did write up a pretty long blog about it, um, and we had some findings in it. So that's, that's one way to start. Um, and then there were some conversations.

People have kind of talked about it off, you know, outside of the blog, like in the greater community about what they've learned from it or thought about it. Um, it's, and so yeah, as far as like training modules, um, internally, like we've learned a lot about it, um, externally, we'll probably there probably probably will be some derivative, um, things like some other blogs or, or, you know, sequels to it at some point. Um, but yeah, I, I, I'm not exactly sure what the timelines on any of these things are.

Yeah. So you've been doing this for a while, but what's that one thing you see folks doing out there that makes you shake your head a little bit and say, folks, we need to be a little bit smarter than that? Uh, I think the biggest thing that we see at, at least here, um, from, you know, my day to day, uh, the customers that tend to have, um, attackers get into their infrastructure, it's because they don't have visibility on all of their assets.

And so what I'm, what I'm seeing is that they'll install an EDR agent on their, um, servers, but they won't install it on their laptops for some reason. And so what happens is an attacker gets in on somebody's laptop and then they figure out how they can move laterally across other people's laptops and eventually make it maybe to one of the servers or something. But basically the compromise is happening on machines that we have no visibility into.

And I think that's the biggest mistake is just thinking, you know, that these other laptops couldn't possibly, like, if that one gets compromised, it couldn't possibly have an effect on the rest of the company, but that's just a foothold in, into the rest of the company basically, and you're just leaving yourself exposed. Um, I think the other biggest mistake is that people think, well, I'm just a small, you know, company and nobody cares what I'm doing. But, um, the thing is like criminals are opportunistic.

They will take any opportunity that that presents them, or maybe you actually are more interesting than, than you perceive because maybe you're doing business with, with some other target that the, that the attacker's interested in. Maybe you're doing consulting for, you know, some other like government entity or something that, that the attacker's interested in. And so yeah, nobody's too small to fall and, um, yeah, just make sure that you know what your assets are, if you can, and, and make sure everything's covered.

All right. Hey folks, even in the age of ai, there's no substitute for fundamentals. Hey, Jamie, thanks being on the show.

Thank you. All right. And back to you guys in the studio.