Smart Bot Defense: Blocking Threats, Not Traffic – The Last Great Cloud Transformation EP11

Hey everyone, it's Alan Shimel, and welcome to another episode of the Last Great Cloud Transformation, today's ep. Today's episode is entitled, protecting Against Malicious Bots. Now, as I was telling our panel in the green room, uh, I think every bot can be malicious at some time or another depending on, you know, how it's affecting your site's performance.

But, um, before we jump into that, let me give you a quick little background. The last Great Trout Cloud transformation is a, uh, video series and podcast, as well as some live, live round table webinars produced in partnership between us here at Techron Group and our forensic Cloud Flare. Uh, for those of you who may not be familiar, CloudFlare yeah, about 21, 20 2% of all the internet traffic in the world goes through Cloudflare's network.

And, um, we've been exploring this now for about six months. This last late tr last great cloud transformation as we saw people, you know, we moved from uploading from data centers to a public cloud, and now we have these big hyperscaler public cloud, you know, uh, dumps, you know, AWS and Google and Microsoft, et cetera, Oracle. But that's not enough.

We've also started moving our data to the edge. So to be closer to the consumer of the data, some data stays on the consumer's devices, right? What's better and faster than that other data we've either kept or moved back to the data center, to the closet or server closet or what have you.

The bottom line is our data lives everywhere. And so the idea behind the last great Cloud transformation is it's not just a migration from a data center or a hyperscaler or the hyperscaler to the edge. It is how do we distribute, manage, connect apps and data that are literally everywhere?

And that's what this transformation's about. In CloudFlare language, they call it the connectivity cloud. And you may hear us, uh, referenced that today and, and, but that's what we're talking about.

Let me quickly introduce you to our audience, or excuse me to our panel. First of all, joining us from Austin. She's a frequent, frequent guest on the Textron gang and, uh, very well known in the world of SEO and, and Web and everything else.

That's our friend Ann Ahoward. Hey, Anne, how are you? Morning.

Nice to see you. Good. Well, it took me a little while to get here as we all, as we've been through this morning, but yes, you made it.

I'm here and thank you. I made it. That's what counts.

Um, joining in with me is Michael TreMonte. Hello. Am that better?

Yeah, That was a perfect pronunciation. Very happy Okay. To be here with you all to talk about bots.

Absolutely. And Michael is a, uh, senior director of CloudFlare and product management. And Michael, thank you for joining us.

And then joining me is my co-host here on the last great cloud transformation is Futur Analyst VP Mitch Ashley. Hey, Mitch. How are you?

I'm doing really well. My favorite topic bots and how to kill 'em. Well, you don't wanna kill all bots.

Well, They're good bots. They're good bots. They're good bots.

Wasn't there one nominated for the Oscars last night? That was a really The wild. Oh no, that was a robot.

Robot. Yep. Okay.

Well, robots are sort of bots. Yeah, I mean, but anyway, with AI agents you'll have robots commanding bots, right? So Yeah, you certainly will.

You certainly will Sooner than we think, I suspect. Anyway, so, so guys, let's, let's first set the table. What is the why?

You know, are all bots bad? I mean, even, even bots with the best of intentions can wind up being somewhat malicious is, is kind of my take. But, you know, and in the SEO world bots can sometimes be your best friends, Absolutely.

A or not? Well, a lot of tools we use, uh, utilize bots. I mean, we depend on Google's crawlers to crawl sites to make sure we're in compliance.

Uh, when we initiate crawls, um, you know, generally in the old school way, we handled it with robots txt, and that was good and done. And that's obviously not the world we live in anymore. Um, I think as an SEO the thing I'm kind of interested in now is how these bots are probably gonna start beating caps.

Uh, so our lead generation is gonna get a little thorny, uh, because they're gonna be able to mimic human browsing patterns if they haven't already, uh, filling out forms for us. So I think it's gonna be very interesting how we are are able to combat that. Michael, how big is the pro?

How big of a problem is this for? Yeah, CloudFlare. Yeah.

And, uh, so I I, my day today I'm dealing with a bad boss more, more than the good boss. We definitely see a lot of the good bots flow through the network. Um, it, it shifts really quarter by quarter.

We, we have a, a solution that's able to sort of differentiate at our best of our knowledge what, what traffic is human driven. So as, you know, someone using a browser versus what traffic was bot driven. And, uh, uh, I remember, you know, 5, 6, 7 years ago, we had a stat that we started following and it, it was about 30% of all internet traffic was automated.

And then you look today and that number has actually gone up to 40 50 on some days, even up to 60% of all traffic through the network is automated. Some of it is expected, right? Um, as more and more companies are just building and interconnecting the services on the internet, that's the whole point of it, machine to machine communication.

Um, but I'll tell you that, uh, the internet feels a lot more lonely lately if you're navigating. 'cause a lot of the traffic around you is, is no longer human at this point. For sure.

Absolutely. I mean, uh, well I, you know, so they say that 52 to 57% of all the traffic on the internet today is API, basically API to API traffic, which Yeah, that's right. It's not human to begin with.

Yeah. By design. So API traffic is, uh, is something that we've seen grow a lot and it's not slowing down, right?

So I'd actually expect the number to go higher, um, to the point that vendors and not only CloudFlare have started building solutions to manage and control API traffic and all of that is automated, which actually causes, uh, it's good, right? 'cause we're inter interconnected more, but actually makes it harder than to di differentiate the good and bad 'cause by default, you're expecting a machine to connect to you. And then what's the difference between a partner doing an order on your e-commerce site versus someone doing inventory hoarding on your, on your e-commerce site, right?

Which is definitely malicious behavior. And the distinction between those two things, again, just mentioned this, right? You have bots now being able to complete captures as well, right?

So not all the historical layers of defense robots, OTXD captures that you throw at bots nowadays very often are no longer actually that useful anymore. 'cause it all gets bypassed out of the box. Aren't, uh, API bot attacks more subtle, like a little harder to detect than traditional web attacks?

Like Yeah, for for sure. Yeah. I mean the, the, the api.

So the, that's another interesting thought. The evolution of the web. I remember I started as a penetration tester in my career.

And, uh, you know, the, the thing to focus on was always you got a website and you're gonna try, if you get permission to do so, try to hack into it. And what you see is what, where you start from and you, and then, and then you take it from there, right? But APIs are somewhat invisible to most users, right?

They're only, the development teams are building these APIs. Even security practitioners don't see the APIs as you would see a web application. And, uh, and because of that, I feel there's a lot of, uh, unknowns when you would refer to the term shadow APIs.

But even the behavior against those APIs is a lot more shadow than you might expect. And, uh, and yeah, differentiating becomes even harder 'cause it's not browsers to begin with, it's just scripts. Yeah.

Of question Michael that I have is, is it getting tougher to differentiate between what are bots talking to APIs and what are legitimate uses of APIs given we have, you know, so much happening, you know, API first design of applications and Yeah. And It's a little more nuanced than it was maybe two or three years ago even. I would actually say that's now the bleeding edge of security is API security today because of that, because of the nuanced definition.

Whilst, whilst arguably, let's go back a couple of years, being able to differentiate a fully blown blown browser, internet explorer, even going further back. But even, you know, if you stick to Chrome and, and Microsoft Edge and Firefox versus someone building an integration with our custom tooling was, was quite a distinct sort of signal to go off. Um, as soon as you go to API first, the browser environment very often just disappears out of the box.

And the, the tooling that developers have nowadays to integrate with APIs is a lot more diverse. And as soon as you take that into account, if you are responsible for, let's say, integrating with your supply chain with partner, uh, and all you're seeing is API traffic, the where do you start from that, the problem becomes a lot harder. There's a lot of security sort of policies that customers and, you know, folks at large will implement just allowing specific IP ranges, allowing, you know, only the connection is coming from company X, but, uh, Alan, you mentioned this earlier, right?

Everyone's moving to the cloud and when you, and when everyone moves to the cloud, which IP ranges is the legitimate one at that point, right? Because AWS could switch up and switch in a new IP at any moment, and you don't want that to cause like a side effect to your business. So IP allow listing, for example, very often is no longer, no longer a tool to, to defend against that.

Um, so it's a, between VPNs, It's a hard problem. Personal VPNs, and as you mentioned, IP addressing changing, it's, it's a tool, but it's not as reliable maybe as it once was for sure. I've actually, oh, I'm sorry.

I'm just so excited. No, no, you go away. I just, I'm like, ooh, an expert.

I wanna ask Michael stuff. I've actually, I wa I wonder if this is actually true, but I've heard rumors that there's a bot as a service offering where people can actually buy bot attacks. Yeah.

Like, it sounds like something out of a movie to me, but like, is that becoming more common? Is that like a real thing that we need to worry about? Well, as, as well as with all things computer related, uh, uh, they repeat after a certain number of years, bot as a service.

Uh, companies, even SAS bot as a service companies have been around for at least 10 plus years. So if you were to go on the dark web and you were to find the right spots in the right forums, uh, you will find companies that are offering botnets as a service, right? And you can go there and essentially, uh, sign up for two hour usage or how many whatever requests you require, and they give you an endpoint where you can control.

And it's a time limited endpoint, right? So they want to control their business and, uh, you can spin up all the bot traffic you want. It goes even further than that, right?

You can find capture, capture solving farms as a service. I'm sure some of you may have seen the videos that you see sometimes on TikTok where like there's someone sitting behind thousands of little phone screens solving captures, uh, 'cause ar arguably good captures are still hard to solve. Uh, but then you just outsource that single step to humans, and then the humans will give the key, the token to the bots and the bots performs the rest of the task.

Um, where whereas, uh, you know, someone trying to get into something and there's value because it saves time, you'll find a, you'll find a business that's, uh, doing that, be it legit or not. So to me, here's where you separate the, the pros from the amateurs in this though. Yes, there are all these malicious bots.

Yes, there are all these API related calls that, you know, potentially security. I think what makes this crazy hard is today's good bot is tomorrow's malicious bot, right? Because today it's doing a function and it, and the way your traffic is and the way your site is, it's good.

We want, I wanted to index it, I want it to do whatever it's gonna do. But you know what, that was yesterday. Today I've got a different set of priorities and this bot is now, you know, I'm deeming it malicious.

Maybe it'll be good again tomorrow, but today it's bad. Michael, how does CloudFlare kind of account for that? Yeah, well, we're, so we're two, two things to note here.

First of all, we're constantly evolving what we define good and bad. The other thing we notice quickly is it depends on customer, by customer, company by company, right? Yes, it does.

Some companies want to get crawled by Google, lean Andex, uh, Microsoft, you name it. A w even even Amazon, right? Has their own crawlers.

Facebook and some companies have the resources to be crawled as fast as you can, right? So out of the box, the more, the more they get crawled, the better. 'cause the listings are more up to date.

Um, but as soon as you have a company that may be struggling a little bit with their cloud transformation and they're still relying on some old box, sitting in a data center that's, uh, starting to, uh, be a little bit too loaded, that behavior from legitimate quickly gets defined as malicious by some in the business, right? So the, the difference becomes very subtle. Um, the way we think about it, first and foremost is wanna provide the, the visibility, right?

That's that number one is understanding your traffic. I actually say even outside of cybersecurity, undersell your traffic should be something every, every business should put as, as a priority. And then once you know what traffic is automated, what traffic is human and out of the automation, what traffic is potentially classified as a verified bot coming from the big names, uh, versus some other services, right?

Uptime monitors, um, you know, automated AI agents you call it. Then you just, we just provide the tools for the customer to make the decision what they want to block or what they want to allow under the hood. We're constantly evolving that intelligence engine, right?

Um, and, and talking about ai, this is the other thing I wanted to say is how things have transformed over time. When, uh, the AI sort of era came up a couple years ago, everyone was like, great, this is interesting. Let's look into what use cases this is allowing.

And the AI companies started crawling the web and immediately no one thought of the side effect of that. Um, fast forward to today though, um, arguably these AI companies, unlike search engine crawlers, are using the content to monetize traffic to their own agents and not giving anything back to the content creators in the first place, right? And even in the short span of two years, the definition of what is good or bad behavior has changed.

Um, so, uh, there's no straight answer to that question, Alan. It's, it's a, it's a game where we're gonna be playing, I think for the foreseeable future. For now, we're giving the tools and the visibility and then we let customers decide what is good or what is bad.

Dang. And in your, in your experience, do customers know good from bad? Absolutely not.

Uh, no people, That's the problem. I, I mean, the thing is, I've been identifying this for many, many years, right? So is this traffic, Google Analytics does a fairly decent job filtering out known bot traffic, but we still see it.

Um, I'm obviously having used analytics since it was urchin and, and for many, many years. Uh, I can spot it from a mile away. Oh, is it coming from a subdomain?

Do they stay zero seconds? Even if a human doesn't like a page, they're only gonna, you know, they're gonna stay three seconds, five seconds. Like it's, they're just very clear signs, uh, of where it comes from.

Um, but definitely like, you also kind of have to apply a zero trust principle, uh, and just assume, you know, from from the outset it's bad until you know that it's good. Um, and generally I don't get too excited about it 'cause we don't see it as much. I am also a CloudFlare customer, I should say.

Uh, so we do have tools in place, um, but it is a problem and it's becoming more and more of a problem. And we're starting to see search engine traffic, or not, sorry, not search engine traffic. We're starting to see, uh, search GPT, we're starting to see traffic coming from LLMs and it, it's, so that's, that's gonna be very interesting.

Um, the quality of that traffic remains to be seen. Yeah. Yeah.

Michael, what about kind of like search, llm, llm, refer to them? How's, what's CloudFlare doing on that front? Yeah, so, so we provide, we, first of all, we've, we've class, we've got a belt bot directory, right?

So, uh, in addition to the visibility, we, we are opinionated, right? So we do provide suggestions to what we think is good or bad. But, you know, whenever we make default suggestions, uh, customers often tell us, ah, you're a bit too aggressive sometimes.

But the bot directory is the first tool. Most of the large LLM providers, uh, OpenAI being the top one that comes to mind, right, are, are actually mostly well-behaved, right? So we know where they're coming from most, I say mostly, uh, we know where they're coming from, we know what their bot looks like.

Um, and the same actually hosts rule for all of the other big providers building sort of LLM based engines. And, uh, again, the visibility is step number one. So if you have your traffic proxying through the cloud over network, we can tell you via our AI audit dashboard, um, this is the traffic coming from open ai.

This is the traffic coming from meta, and it's gonna be used for training in LLM because that's how the bot is advertising themselves, right? Um, and then, and then of course we have an opinionated view, right? If the bot traffic is causing, for example, increased latency, you may argue it's starting to be a little malicious 'cause your legitimate user cannot access your content anymore and therefore you can block it.

Maybe you're a content producer and you don't want your content to be ingested by the lambs and you can decide to block it out, right? Um, uh, the, Alan, the interesting thing about the broader AI topic, I also think besides the LLM creators, is this idea of AI agents. com and buy something as an example, but I don't actually do that anymore.

I've got my little chat, GPT or whatever it is, and I tell her, go to Amazon, go do com and buy me some, you know, some new shoes, whatever it is. Wait, Wait, you are not doing that right now yet? Not Right now, but I, I don't think, okay, not far off.

I'd argue that some people, I Was, I was about to get jealous. I was gonna start searching where I could get that. And, uh, and uh, the, the behavior is legitimately coming from a human, but it's, the action is being performed by an agent, which will show up in bot management tools as a bot, right?

And then we were discussing earlier the difference between good and bad, but, and that's where it starts becoming very hard to distinguish. And that's where we're think putting a lot of our thoughts and our detection systems in place, and in some cases even proposing new standards to be able to differentiate across the too. So we're gonna try and keep, uh, in our control, in our, in our web admins and content creators hands as much as possible.

Now, Michael, it seems that that's one of the things that you're in a key position to do that most of us aren't, which is you have such massive amounts of data from traffic traversing networks, um, yeah. You know, AI lives, feeds, eats, needs data, right? And that's how we improve both LLMs as well as machine learning.

This Is, this is The, this is gonna advance a lot continually. The web is a not great data though, for LLMs. Well, I'm thinking traffic information though.

Yeah. Yeah. Well actually you're both, you know, both of both.

You two very good points. Number one, to be able to provide that intelligence, you need to see a lot of traffic, right? And arguably it's, it's good for CloudFlare that we in that position, right?

It's very hard if you're a single player trying to solve this problem alone. 'cause you don't have access to the, to the, to the baseline. Um, uh, but then, and to your point, a lot of, a lot of content out there is, is is not necessarily good, right?

And I, and that's more of a challenge for the alarm providers. Um, there's, there's already a lot of LLM generated content online, and you can see how that can be a vicious cycle moving forward where a model starts train training themselves on incorrect content, and then people will echo that content on whatever platforms they have, and then the model finds that new content and retrains, and then you get in this spiral of like, just made up hallucinations. Um, and I, I don't think we haven't seen the full effect of that play out, really.

It's gonna, it's gonna be interesting in the next couple of years. It was an interesting article I came across and an employee of Disney was kinda self-learning ai and one of the ways of, you know, getting infected, that person did end up infecting the business, but, um, malicious content was in an LLM that he downloaded from probably hugging face or something similar. So it too is, you know, as a supply chain concern, um, it, it bots can infect other areas, not just attack us, uh, that we consume when it comes to software and supply chain.

Yeah. You know, something though, I was sitting here and I'm reflecting on our own experience at Techstrong with this subject. You know, we mentioned, uh, botnet as a service, right?

I wonder, I, 'cause I think we're pretty typical of companies our size are probably a little more tech savvy than let's say a non IT tech related company, though all companies are tech, right? How, how many companies have the wherewithal absent a CloudFlare to decide good, bad, let it through, don't let it through, you know when to let it through. Well, who's put it out there?

You know, it used to be a very simple thing. I wanna start a business. I op I built a website, I, I put it up there and that's my open for business sign to the world.

I didn't have to worry about whose bots, what road, you know, I had robots text, and for many of us that was like a self-generating file depending on what you put in there, right? Is this beyond the norm that most organizations can deal with? And so they have no choice but to outsource it.

Michael, I think we know your answer, right? You, you have what we call a vested interest. I do have it, but Ann, what do you think?

I don't think most organizations or SMBs small and, you know, I don't think they're equipped at all for this. I think they need tools. I also am very hesitant to say this, but I think it might, we might one day see some regulatory compliance here, maybe as part of an IT audit.

Um, we're gonna start to see organizations that are gonna incur regulatory penalties if they don't take steps, uh, to adequately protect their sensitive data. Um, I mean, we see these bot attacks all the time and I, I don't think that the skills are there within most organizations to do this alone, Perhaps. Yeah.

You know, are you, are you guys familiar with the term swatting? Yes. Mm-hmm.

Could we see botting? Right? So, you know, all, so, and what I mean by that is not the botnet as a service where clearly it's malicious and they're trying to do denial of service or whatever, but the, the misappropriation of otherwise legitimate bots Oh, Yeah.

Thinking that they're doing for sure, you know, the right thing here. But someone's kind of manipulating that, you know, manipulating that. I've already seen that happen with competitors Comp, really Competitors clicking on, I uncovered a case, uh, while ago of a competitor.

I have, have only ever had one client max out the maximum cost per click bid in Google, which is a thousand dollars. And so a thousand dollars a click. If your biggest competitor is doing that, they're, they were using tools to basically drive It, Use game it.

Yes, exactly. Exhaust their spend. And so I didn't investigate audit and discovered, okay, well all this traffic is coming from this IP address.

They've got an office there. And we were able to get that money back, but it was a huge ordeal. And so when there's competition that's cutthroat in certain industries, financial industries, there's a vested interest in that.

I could see on a personal level, people using it to, you know, using these tools to stalk, you know, we've seen it within big tech. We've seen big tech employees use this information to, to stalk people and harass them. It's unfortunate.

Yeah. And, and it's also kind of ruined social media in some ways. Well, it's why we can't have nice things.

Yes. That's what you mean. Yes.

You think about cyber bullying. So Michael scl, think About cyber bullying, you know, just as one example, set your bots about creating deep fakes images of whatever and posting it on social media and, you know, spreading misinformation, you know, on the web about a person or an organization. You Know, it's, it's any tool will get misused by someone.

Absolutely. The, especially if it's a software tool. Sure.

Um, Michael, what, how does CloudFlare help there? Because I mean, you almost need inside knowledge to know what the hell's going on. Yeah.

This is where actually we've been focusing a lot of our detection efforts on the behavior of the bot. There's this idea of, you know, put aside the fact it's automated or not for a moment and just focus on what the action they're trying to perform is. And, uh, and we are trying to highlighting within the dashboard, you know, this, this connection is trying to access all of your product list in pages and adding them to your shop, to a shopping cart, right?

Which may result in inventory hoarding, which means your legitimate users cannot buy them, buy the products. Um, this bot seems to be only scraping your pricing pages. Uh, chances are, uh, you know, they are a price scraping bot and they're trying to match or just undercut you on their, on their own website.

And, uh, and actually a lot of our new sort of detections are just gonna be starting highlighting behaviors. Again, we're gonna have a very opinionated approach, right? 'cause scraping inventory, hoarding are all things that I think everyone can agree are malicious and should be blocked out of the box.

Um, and looking at it from that lens, some cases very helpful to identify, you know, your term, uh, a bot that's being misused with bot thing. Uh, and uh, appears to be a search engine crawler, but in reality is, is, you know, doing something completely different or, or misusing the information. Um, and all of this of course, can then be used to build your security policies, right?

'cause from a cybersecurity perspective, then you wanna be able to block or, uh, in some cases, which is very interesting, potentially trick the bot by serving altered content. Um, I've seen a couple of customers do that. It's really fun when it works really well.

'cause you then you can identify who, who is doing the malicious behavior, um, where you have a price listing and you only deliver a slightly adjusted price to who you think is the malicious bot. And then you see that price show up somewhere else on the internet and you know exactly who's behind your, uh, your bad, bad bot. Well, that's the kind of forensic stuff that Anne works on, right?

Yes. Yeah, it is. I love a good audit.

Excellent. Yeah. That's awesome.

It's crazy stuff that goes on out there. But you know what, there, there's probably some segment of our audience out here, guys who are saying, I don't know, I never had this problem. You just didn't know you did because Right?

You do. You just didn't know. Exactly.

Exactly. Because if you're not monitoring how well your site performs, how, you know, what's that user experience at your website, you may not realize how big a an input impact this has. That's why step one is always visibility, understanding what's coming on.

Yeah. Michael, we gotta wrap up here at top of the hour. com, but Yeah, we, we publish Any particular Yeah, go Ahead.

Bot trends in general. com. You can see what's going on, how many bots are crawling the internet at any given moment.

com, uh, we publish very often some very interesting technical details as well on how we're improving bot detection and potential attacks. We've seen, we haven't talked about den denial service a lot today. Um, there's some really big botnets out there doing some really horrible things to, to online apps.

And, and when we can we publish that, those details online, it's very interesting. com, of course, for our product piece and how we can help, how, how we can help companies solve their bot problems. So yeah, bot bots in, in use for DDoS attacks is, we didn't really talk about it, but it's kind of, that's old hat already, right?

And, and the numbers like the amount of megabits, gigabits per second ParaBit that these things generate terabits per second. It's nuts. Anyway.

And for people who maybe wanna find out more about your service, 'cause that may be something that says, Hmm, I could use that. Where would they, what's their best place? com.

C-I-R-C-L-E-C-L-I-C-K. And then I'm Anne Bot. Uh, I've been and bought, uh, has been my nickname since late nineties.

And so I'm, and Botted everything on the internet, so you can find There you go. And bot, but not a malicious bot. No, I'm a good bot.

Just an and bot. All righty. Mitch, you want to take us home?

No, I just think about bots and attacks and, and not only what we see today, what we saw yesterday, but the, uh, inno face of innovation is, uh, increasing greatly. Working on some research and advising companies of any strategies put to you put in place. You have to think about an increasing level of innovation that's happening, whether it's AI or security, all of the above.

So we have to play a really good game to keep our organizations and our employees and customers safe. Absolutely. Folks, thank you for joining us.

Thank you for joining us on this, uh, episode of the Last Great Cloud Transformation. Many thanks to CloudFlare for sponsoring and to our crew here for, for, uh, producing. Until next time though, this is Alan Shemel for Textron.

We're outta here.