EP1 – Unbridled Bot Attacks: How We Mitigate Their Unwelcomed Rise

An Introduction to Edgio’s Beyond the Edge podcast Episode 1: Unbridled Bot Attacks: How We Mitigate Their Unwelcomed Rise, hosted by Nathan Barling, Vice President of Product & Solutions Marketing at Edgio.

Nathan Barling: Hi, everyone, and welcome to the first episode of Beyond the Edge, a podcast dedicated to exploring the dynamic challenges faced by modern digital businesses. So what we’re going to talk about is we’re going to talk about real world examples and historical trends. We’re going to different examples of notorious outages or notorious data breaches, marketing missteps, and just general zero-day exploits to better help businesses anticipate and prepare for the future of technology, especially when the future of technology is being on the Web. Together, we’re going to explore different innovative solutions that businesses have come up with in order to streamline workflows, cutting edge technology techniques for enhancing the digital experience, and just robust strategies that businesses can take on to help combat some of these evolving threats.

So I’m your copilot, Nathan Barling, and today we have two industry experts. First, I’d like to welcome Brian Pillsbury. He is the director of Solution engineering here at Edgio. Hi, Brian. And we also would like to welcome Andrew Johnson, Senior Product Marketing Manager, also here at Edgio.

Okay, gentlemen, so today we’re here to talk about why we’re seeing such a rise in malicious bots. What is a bot? How is it impacting businesses? How is it impacting the industry, and what generally businesses can do to help combat this new sort of evolving threat?

Let’s go beyond the edge!

Today’s Rapid Digital Transformation and the Increasing Cost of a Data Breach

Global organizations are engaged in digital transformation. There’s all sorts of various different types of digital transformation. McKinsey recently had a study that showed that ten years of digital transformation was accelerated and compressed in, like, six months at the start of COVID.

All of this rapid, sort of unbridled digital transformation, what we saw was this massive expansion, and everybody kind of racing and trying to get to market as fast as they could.

• So how do I get my app?
• How do I start to replenish some of these various different types of commerce conversions?
• And how can I start to focus in order to deliver the experience that my customers kind of demand?

And with that, they sped through it. They brought their online presence really quickly, and digital security really wasn’t top of mind. According to a study by IBM just last year, 83% of US companies have experienced a data breach, and this is costing them over $9.4 million in the US, which is more than double the global average. So what can businesses do and how are these bots starting to play a role in this, right? How are bots even opening up these types of attacks for businesses?

So Okta, which is one of the largest single sign on identity businesses in the world, recently published a report that 34% of all login attempts globally were by bots. Bots just trying to get into networks.

So another study from Kount showed that one in four organizations have lost $500,000 from just a single bot attack. So that’s massive amounts of issues for businesses and massive kinds of financial damages to businesses.

What we want to talk about today is how can we start to understand that not only is it just about the money that’s lost, but it’s also how user trust is diminished and really oftentimes irreparable inside of a business. So how can we start to better position your business, to be able to react, to protect, be proactive, and do what it ever can to protect themselves against this type of threat.

So let’s talk about some of these different types of examples. Andrew, I’ll go with you first. So what’s a notorious example that you can think of that comes to mind as it relates to bots?

Notorious Bot Attacks (Taylor Swift Eras Tour/Ticketmaster)

Andrew Johnson: Yeah, I think probably one that a lot of our listeners have heard of recently was the whole Taylor Swift fiasco. I think this happened last fall, fall 2022. Taylor Swift was coming back on tour after five years off, and Ticketmaster was distributing the tickets. They had even anticipated, of course, high demand and high bot demand visiting the site, but it was actually three times as high as they actually forecasted.

The site was unavailable. A lot of the tickets got picked up by scalpers, resold for over $20,000. Even the president of their parent company, Ticketmaster’s parent company, was hauled in front of the Senate Judiciary Committee. A lot of brand damage, unfortunately, was done by bots.

That’s a good example from recent history, but I think bots for years have been in the spotlight. We hear, especially when Elon was buying Twitter he highlighted the bot problem, when he was trying to get out of that deal. There’s other bots that are used for election information and disinformation campaigns. So I think there’s been a lot of examples, and I think it’s really been in the general population’s consciousness the last couple of years.

Notorious Bot Attacks (Shoe Drops)

Brian Pillsbury: Yeah, I think anytime you’ve got inventory scarcity, you hit maybe an airline ticket sale, or maybe it’s things like in the UK, home gyms, or actually everywhere, home gyms, home gym equipment during the pandemic. Really anywhere where you’ve got limited inventory and a lot of demand, the way that you can be able to acquire these things at scale is to use bots. I think one of the more interesting ones these days really is the sneaker copying or they’re called shoe drops, basically. So you can think of limited edition shoes, maybe it’s the new LeBron James’ Nikes. Bots can go and scoop up that inventory, sell it for six times the price. The way that these businesses are run, for example, you could look at something like nikeshoebot.com or bot broker. These are very polished businesses that allow people to rent the bot that they’re looking for to get the inventory that they want. Pricing is going to fluctuate based on how effective they are. They really operate as fully legitimate end-to-end businesses that say, “Hey, we can defeat bot solution. We’re the best for Shopify or something like that.” The profit margins tend to be really high even if a bot is getting blocked 98/99% of the time, there’s enough profit margin where costs of computing are so low, that there’s a lot of incentive to keep at it and to keep deploying botnets to buy up what you need.

Notorious Bot Attacks (Nintendo Switch)

Nathan Barling: It’s interesting because it pretty much, as you said, anything with scarcity. So it’s anything that has demand. If it’s Nintendo switches, same problem. They were snatching up Nintendo Switches faster than anybody could put them on their website. And even last year, it was graphics cards. During the Crypto boom, everybody was buying graphics cards and marking them up 6 to 7 times the value that they were for the MSRP value. And what they were doing to the point where entire websites shut down their e-commerce operation because the bots were able to evolve faster than they could deploy any security measure to get around it. And they were just snatching up as much of their inventory as possible.

And I think that’s what’s incredibly important about it, too. We talked about credibility damage when you have a data breach. And 83% of customers will stop doing business with a brand when there is a data breach. So the bad bots breaking through and taking information brings tons of credibility loss for a brand. But it’s also damage when bots are snatching up all of the inventory and people who want to get a Taylor Swift ticket or a Nintendo Switch or want to buy anything, the new Jordan, the new Airs that drop, when they know that the brand isn’t doing enough to get the product in the hands of the fans and their diehard brand supporters, that also creates tons of business damage and credibility damage and ultimately reputation damage.

So we’ve established some interesting ways that aren’t necessarily an attack. They’re just ways of automating purchases on websites. It’s just I’m going to write a script that’s going to go through and just be able to automate the checkout. So oftentimes there are legitimate purchases. They’re just done at such a volume that it just snatches all of the inventory in less than what Taylor Swift sold out in less than half a second or something weird like that.

Brian Pillsbury: Right – more than any human could possibly ask to check the card and put their credit card in.

Why Are We Seeing More, Larger Attacks?

Nathan Barling: There’s an other side of it that’s far more malicious. These attacks are growing in size and they’re growing in frequency, quarter and quarter, year on year, basically every over metric you can possibly get her out. So can one of you elaborate on why are we seeing more attacks happening now? And what are the kinds of things that we’re seeing? What are the types of attacks that we’re starting to see as well?

Andrew Johnson: There’s several reasons I think that we’re seeing proliferation of bot attacks. I think you guys both touched on them earlier. First, the incentive has to be there. So when it comes to scalping and picking up shoes and PS5s, the incentive to flip those is pretty great. Secondly, it’s a lot easier to pull off these attacks. There’s even bad bots as a service. So you don’t have to be super technical. You can just go out there and buy some kits and run bot attacks. So it’s a lot easier, with a lower barrier to entry. I think overall, like you talked about during COVID, people are online more. There’s been a rise of buy now, pay later. E-commerce has grown during the pandemic and post pandemic. I think those are a lot of the factors.

I think the second part of your question was about what things are they doing. That really depends on the industry. I think one of the most important or biggest challenges for across industries would be preventing account takeovers, credentials stuffing. Bots are automated programs, basically. They can test a lot of usernames and passwords. That affects all industries.

For scalping and denial of inventory and price scraping, that affects retail pretty bad. There’s scanning bots looking for vulnerabilities in software and web applications, and that’s a door to data breaches and things like that. Depends on the industry, there’s a lot of challenges. But we can’t block all bots. Some bots are good. SEO bots are vital to people’s websites, too. So it is a challenge, definitely.

Differentiating Between Good Bots & Bad Bots

Brian Pillsbury: I think that’s really where a lot of the emphasis is, is determining that there’s classifying the good bots from the bad bots and the unknown bots. There’s new user agents, there’s new bots out there. And so understanding what they’re doing and differentiating between malicious behavior and benign behavior is really what a lot of the providers like us are out there trying to do. The more advanced bots – they can use modified web browsers, they can imitate human like mouse movement and clicks. So collecting the telemetry is super important because they can change IP addresses quicker. It is a constant game of cat and mouse to determine what’s good from bad. I think in the other category of good bots, that can be a voice or a chat bot. There are some really useful bots out there that really help companies drive more revenue and drive their business strategies forward. But it’s about trying to figure out which ones are the bad ones.

Nathan Barling: Now, how can you start to determine that? Because as Andrew touched on, SEO bots are a big deal. A lot of these sites, these good bots, it’s how brands are discovered. It’s how Google understands your content. They’re coming in and scraping your content so that it can be indexed and understood. It sounds easy enough just to say, I want to block all bots. I’m just going to stop bot traffic as a whole, block 100%. But then what you’re doing is giving up a large part of your business, a large part of the partners, and just a large part of the way the internet works – 40% of total traffic on the internet is bots. With that in mind, what are some ways that we can start to think about how we start to be more prepared for it? What are some of the systems or some of the methods that brands can use or policies, or processes, or systems that they can start to think about to protect against these types of activities?

Effective Security Solutions & Practices

Brian Pillsbury: I think number one is you need a solution that combines both some signature-based defenses. So looking at what is the bot? What is its DNA? Some of its digital DNA. And that can be things like the JA3 hash. That can be things like, where is this bot coming from? A lot of times we have customers that get hit with spoof bots. So the bot says, “I am Google bot,” but based on using a variety of methods to see where they’re coming from, we can say, “Well, actually, no, you’re not coming from Google.” So that is going to be malicious.

And so combining some of that signature-based defense along with behavioral-based as well. So looking at session intent, looking at what pages is the bot hitting and how fast is it doing it. Being able to combine those two detection techniques is really critical. And then going from there, having a whole way to be able to not just mitigate bots, but be able to manage them, because sometimes you might want to be able to dial up security on a particular section of your application for maybe your cart or your checkout or your password reset pages, for example. And other times you don’t need such a strict or very stringent security policy on a certain other part of your application or website. So really having a solution that gives you that flexibility to dial up and dial down different configuration policies and being able to use different layers of the security solution to be able to deal with automation threats is really critical.

From there, having that be native on the edge. So if it’s an attack or having a content delivery network, for example, we are closer to the attacker and you’re extending that perimeter defense. And that’s really key from keeping attackers out of infrastructure and from being able to breach infrastructure systems. So that’s a really big factor. And that provides better performance as well, because if you’re using a more holistic solution, you’re not proxing traffic out to a different provider. That can affect performance of a particular application, introduce latency, things like that. And then a large platform that gives you an incremental approach to modernization. So using a well known platform gives companies the ability to modernize their tech stack with a platform that you don’t need to worry that you might break your application by deploying a holistic solution like that.

Andrew Johnson: I think those are great, Brian. I really like one of the points you touched on that it’s about bot management. There’s good bots and there’s bad bots. So I think in the solution, I would definitely look out for something that provides high levels of observability and visibility into bots visiting your site. I think along with that, you want to choose solutions that are API first and really integrate with DevSecOps so you can rapidly respond to bots. Let’s say you’re a retailer doing a Black Friday sales event, having that observability information in real time is key, as well as being able to update rules like allow lists and block lists in real time is definitely very important on closing the window on bad bots, especially during high volume times like Black Friday.

Brian Pillsbury: Right. I think customers, they want control. They want to be able to deploy something very quickly. When there’s an attack, you’re losing money potentially every minute. And so being able to deploy a new policy or a rule and push that out globally very quickly is a really critical importance.

Nathan Barling: I think that becomes the balancing act that you have to do as a business. I think before the edge was really what it is today, it was more “do I want to tighten security or do I want to have the fastest website possible?” Because it was a distributed architecture where you had your security vendor on one side and you had your application on the other side, and often it wasn’t connected. Now with the edge, you can have all of that security without modifying anything about your application at all. It can be part of it where it becomes where it just naturally fits towards the solution itself. As you said earlier, it can actually speed up your application because now all of that demand and all of that processing and all of that load that would normally have been on your app servers being stopped at the edge before it even gets in. It’s an interesting change now that we’re able to stop all of those attacks before they happen. Now the attacks are just getting smarter, and they’re evolving, and they’re changing, and they’re getting better at hiding what they’re doing, and spoofing and acting like other people.

I think that becomes why it’s so important because the attackers now aren’t just attacking the largest companies in the world. There were government leaks, there was General Motors. I could go down the list of everything from betting sites to getting car purchase information to pretty much anybody that has customer data effectively, people are trying to get in.

But now what seems to be the big trend is almost half the attempts are now targeting those small to midsize businesses because to Andrew’s point, they’re online now. And through third-party media providers they are just buying a product out of the box. My whole business is now online. Well, now they often don’t invest quite as heavily in security. So nearly half of those attacks are now in those small to midsize businesses because the volume is where it’s at and not necessarily just getting into ones and just getting out in the world.

I think we’ve talked through a lot of the threats and some of the problems and how now customers know full well personal pain from both their credentials being linked down into the Dark Web, but also just not being able to get Taylor Swift tickets.

Edgio Client Examples and Success Stories

Nathan Barling: How about we end with you guys talking through a couple customer examples, if you can, about how customers have found success with deploying security, and how they’ve actually helped drive a better customer experience without necessarily sacrificing that security.

Brian Pillsbury: One of our customers is a large shoe retailer, Shoe Carnival, and they are able to block upwards of eight, nine million malicious request in a month with some of our ML/AI bot protections really benefited their business, and really reduced the security exploit mitigation and the time to resolution, we cut that down dramatically, and then being able to deploy rules really rapidly when something evolves, when something changes. Being able to do that very quickly, I think, was a real difference maker for them. Same thing with Mattress Firm.

Bots are always changing. I don’t think there’s any bot vendor that’s going to tell you, We’re going to guarantee you there’ll be no false positives and we’ll catch everything. It is a cat and mouse game where there’s innovation from the attackers. But being able to, for example, create whitelist exceptions to if maybe there is a false positive, to be able to exempt that and be able to just iterate really quickly is really, I think, a real difference maker for businesses out there. And then one of our larger customers who is one of the largest shipping companies globally, being able to use our entire security stack to incorporate the bot management piece into the other layers of the security stack and do that in a really, really intelligent way to fine tune and be really surgical about how they implement their security, I think they got a lot of dramatic positive results out of that with our platform.

Wrapping up

Nathan Barling: This has been great. I certainly appreciate both of you guys joining us today and sharing your points of view. I know I learned stuff, so I’m hoping everybody that’s listening did as well. I think in closing, talking through just knowing that it’s happening. Bots are attempting to log in the sites even if you focus on the security or not. I think while you can’t always prevent 100% of the attempts, you can at least have systems in place to be aware of them. That observability, Andrew, that you talked about earlier becomes critically important because as the attackers become smarter at their attacks, so does your defense need to evolve along with it and having the system that’s using the same types of techniques to get ahead of it and to prevent those things and make zero-day like every day. That our systems can just act and respond accordingly and be as frustratingly brick wall to those attackers as possible.

I certainly appreciate your time today and look forward to talking to you again in a future episode. Thank you everybody for listening. Stay tuned for the next one!