Botnet attacks on APIs: Why most companies are unprepared

As companies move applications to the cloud and expose functionality via application programming interfaces (APIs), criminals have been moving quickly to take advantage of this newly exposed attack surface. By using botnets, they can dramatically increase the reach and effectiveness of their attacks. As with many new technologies, security is lagging behind.

The problem is that companies must be strategic about where they spend their security money, says John Carey, managing director in the technology practice at AArete, a management consulting firm. Investments in anti-bot technology are usually invisible to customers. “Tools and skills are in demand and increasingly expensive,” he says. “Similarly, the threat landscape is expanding, as it’s a lucrative crime area.”

Botnet attacks on APIs a growing problem

According to a report by security firm Radware and Osterman Research released earlier this year, 98% of organizations saw attacks against their applications in 2020, and 82% reported attacks by bots. The most common types of bot attacks are denial of service (DoS), experienced by 86% of companies, web scraping, seen by 84%, and account takeover, reported by 75%.

API security was a “top priority” for 55% of organizations surveyed, and 59% said they want to “invest heavily” in it during 2021. Only a quarter of companies said they used bot management tools. Over the next year, 59% of organizations said they planned to invest heavily in API protection and 51% planned to invest in web application firewalls, but only 32% said they planned to invest in bot management tools. In addition, only 52% of companies fully integrated security into continuous delivery of APIs, compared to 63% for web applications.

The situation is only getting worse. According to a March report from the Council to Secure the Digital Economy (CSDE), the Consumer Technology Association, and trade group USTelecom, the destructive potential of botnets has increased exponentially as they leverage IoT devices, which are estimated to reach 80 billion in number by 2025, or ten times the size of the world’s human population. APIs are a juicy target, since they allow enterprises to expose back-end data and functionality to trusted partners, customers, and the public. The CSDE recommends API gateways to help protect them against botnets.

According to data from security firm GreyNoise Intelligence, during the past three months, more than 6,800 IP addresses have been scanning the internet for ENV files, which are configuration files that are used to store things like database logins, passwords, and API tokens. Of this traffic, 1.4% was known to be benign, says Nathan Thai, research lead at GreyNoise. “Some security companies will scan for these files,” he says. “They have no malicious intent, just doing surveys or reports.”

Another 23% of traffic is known to be malicious, because the same IP addresses were engaging in additional suspicious behaviors. The other 75% falls into the unknown category. It could be harmless research, or it could be criminals doing passive surveillance before doing other things with it through other channels. “Typically, they’ll do it all in one, because they don’t care if they get caught,” Thai says. The biggest traffic sources? Cloud hosting providers Amazon, Linode, Microsoft, Alibaba, and DigitalOcean.

The level of activity is growing, Thai says. Activity volumes of botnets doing opportunistic ENV crawling have doubled over the past six months. According to Imperva’s 2021 Bad Bot Report, malicious botnets now account for a quarter of all website traffic, up 6% from last year, and a third of all login attempts are malicious.

Worse yet, the bots are getting smarter. “Sophisticated bots, those that are harder to detect and stop, were the majority of bad bot traffic last year,” says Edward Roberts, director of strategy for application security at Imperva. These are the bots responsible for high-speed abuse, misuse, and attacks on APIs, he says. “As the volume of APIs multiplies and grows each year, it means bad actors have more paths to access sensitive data.”

How botnet attacks on APIs are used

According to Sandy Carielli, principal analyst at Forrester Research, the bots are commonly used for credential stuffing attacks. They’re also used for inventory hoarding. “When desirable merchandise like limited edition sneakers, concert tickets, or the latest gaming system goes on sale, bots swoop in and grab the inventory before legitimate human users can,” she says. Then the bot operators resell the merchandise for a nice profit.

Businesses also use botnets, Carielli says. “Unethical companies use bots to scrape prices from competitors’ websites and then set their own prices at just a little lower, or they scrape product information and pictures of high-end products and use them on their own site to sell counterfeits.”

DDoS tools and web application firewalls won’t protect against all kinds of bot attacks, Carielli says. Enterprises need dedicated bot management solutions. “Remember that bots attack legitimate business logic,” she says. “You don’t want to block everyone from logging in or buying a product—you just want to block the bad bots.”

How one bank battles botnets

At one medium-sized financial institution, 85% of all blocked traffic comes from malicious bots, according to Jeff, the company’s manager of cybersecurity technology. The other 15% are either geoblocked logins or legitimate human users who had too many login attempts or are using obsolete brokers or applications.

Not all botnet traffic gets blocked. Some comes from good bots, Jeff says. “We work with other financial institutions and aggregators like Quicken and Mint,” he says. “Those are botnets in the sense that it is an API running from multiple sites doing multiple functions.” The bad bots can do quite a bit of damage if they get through. “Worst case, they are able to impersonate a user and gather that person’s financial information.”

Botnets can also be used by cybercriminals in other ways. For example, they can use web scrapers to find out which banks offer the best rates, create accounts, and use them to launder money. “You’ll have botnets constantly moving money, real accounts, but moving the money in an automated way,” he says. “They’ll also use botnets to get around restrictions. They might be based in a restricted country and have the botnet based in a cloud provider in an allowed country to bypass compliance and regulations.”

To spot bots, the company looks at the bot’s user agent name and IP address. If it’s a known bad IP address, those can be blocked right off. Then, it looks at how the bot is interacting with the API, for indications of cookie or session replay, unusual behavior patterns, and other suspicious activity.

“If the first page requested is account status and not the login page, then something isn’t looking right,” says Jeff. “If we know an account holder is a 22-year-old college student with a $200 deposit every other Friday, and they now start depositing large sums in cash a couple of times a week, then something isn’t right.”

Jeff declined to say what tools the bank is using internally to spot bad behavior. At the edge, they’re using Salt Security. Its AI and machine learning significantly reduce work for the internal security teams, he says.

Once a bot attack is spotted, often all the requests in that attack will have something in common, such as a similar pattern in how requests are structured, or a common origination address, or the same proxy being used. “If it’s a legitimate request, it comes in a certain order,” says Elad Koren, chief product officer at Salt Security. That common parameter can then be used to identify other traffic that is part of the same attack or to flag targeted accounts for additional security. “A botnet is usually just part of an attack,” he says. “With account takeover, once they get the credentials, they can go in with a more sophisticated tool and take the money out.”

Most common botnet detection technologies

According to the Radware and Osterman Research survey, web application firewalls (WAFs) are the most common technology used to detect bot traffic, used by 48% of companies. In addition, 47% looked for known malicious IP addresses, 43% used CAPTCHAs, 34% used rate limitation, 26% built their own solutions, and only 24% used a dedicated anti-bot technology.

“CAPTCHAs are very effective if implemented properly,” says Andy Thurai, vice president and principal analyst at Constellation Research, though he adds that the original CAPTCHAs are easy to solve. “Bots are known to have a 90% success rate against it. Generally, visual processing challenges are very effective [and] require a human brain to solve.”

Rate limiting and WAFs can be effective as well, Thurai says. “A properly implemented API security should rate limit the use of APIs on a per user, per location, and per identity basis, or limit if the volume is suspicious, or block unsupported protocols, invocation methods, or suspicious headers or content.” Dedicated bot defense solutions will also monitor for traffic patterns such as volumes, signatures, geographic frequencies, and traffic content.

It’s important to differentiate between good bots and bad bots, Thurai adds. “For example, most of the customer communications have moved toward bots such as chatbots,” he says. “So, there is a sizable amount of good bot traffic flow through the network at any given time.” Pattern recognition can help distinguish between the two.

Top botnet management vendors

According to Forrester’s most recent report on the topic, released last year, the market leaders in the bot management space are Netacea, PerimeterX, Akamai Technologies, and Imperva. Other significant vendors are Alibaba Cloud, AppsFlyer, Cloudflare, DataDome, Radware, and Reblaze, as well as Insart, which has since sold its assets to Akamai. Shape Security has since been acquired by F5, and White Ops is now Human Security.

The best tools, says Forrester’s Carielli, collect data and perform analysis to detect both simple and sophisticated attacks, and can block attacks or make them more difficult for attackers, raising their costs and, hopefully, making the attacks economically unviable.