A bot is a software program that performs automated, repetitive, pre-defined tasks. Bots typically imitate or replace human user behavior because they operate much faster than human users. Good Bots make the Internet work - From search engine crawlers that bring the world to your fingertips to chatbots that engage and enhance the user experience.
Bots can also be used to scale automated attacks which can result in account takeover (ATO) and fraud. Motivated cyber criminals leverage a sophisticated arsenal of bots, automation, and evasion techniques. They also perform ongoing reconnaissance to identify security countermeasures and constantly retool their attacks to evade detection.
Infrastructure Costs - Infrastructure needs to scale to deal with unwanted and/or undetected bot traffic
Competitive Intelligence - Web scrapers collect important data to help competitors adjust their pricing strategies
Wrong Business Decisions - Bot traffic distorts web site analytics which could lead to making wrong business decisions
Sneaker Bots - Bots are buying limited editions of certain products before regular buyers and then sell on black market
Account Take-over - Credential stuffing leveraging stolen accounts purchased on the Darkweb providing access
Fraudulent Transactions - Fraudulent transactions with large financial consequences as a result of account take-over in the finance sector
Figure 1. It begins with unwanted automation and ends with account takeover and application fraud
Credential Spill - A cyber incident in which a combination of username and/or email and password pairs becomes compromised.
Date of Announcement - The first time a credential spill becomes public knowledge. This announcement could occur in one of two ways:
Date of Discovery - When an organization first learned of its credential spill. Organizations are not always willing to share this information.
Stage 1: Slow and Quiet
Figure 2. Slow and quiet stage. Attackers use credentials in stealth mode from 150 to 30 days before the public announcement
Stage 2: Ramp Up
Figure 3. The ramp-up stage. Attackers ramp up use of compromised credentials 30 days before the public announcement.
Stage 3: The Blitz
Figure 4. The blitz stage. Script kiddies and other amateurs race to use credentials after the public announcement.
Stage 4: Drop-Off / New Equilibrium
Figure 5. The drop-off stage. Credentials no longer have premium value
The simplest level of user simulation contains tools that make no attempt to emulate human behavior or higher level browser activity. They simply craft HTTP requests along specified parameters and pass them along to the target. These are the simplest, cheapest, and fastest tools. Sentry MBA is perhaps the standard tool of this type.
Figure 6. Sentry MBA, a standard user simulation tool
Until 2017, PhantomJS was the most popular automated browser in the market. When Google released Chrome 59 that year, however, it pushed forward the state of browser automation by exposing a programmatically controllable “headless” mode (that is, absent a graphical user interface) for the world's most popular browser, Chrome. This gave attackers the ability to quickly debug and troubleshoot their programs using the normal Chrome interface while scaling their attacks. Furthermore, just weeks after this announcement, Google developers released Puppeteer, a cross-platform Node.js library that offers intuitive APIs to drive Chrome-like and Firefox browsers. Puppeteer has since become the go-to solution for browser automation, as you can see from its growing popularity in web searches.
Figure 7. Google trends graph showing interest in PhantomJS versus Puppeteer between 2010 and 2016. (Source: Google Trends)
The next level of sophistication above simulating a browser is simulating human behavior. It's easy to detect rapid, abrupt mouse movements and repeated clicks at the same page coordinates (such as a Submit button), but it is much harder to detect behavior that includes natural motion and bounded randomness. While Puppeteer and the Chrome DevTools Protocol can generate trusted browser events, such as clicks or mouse movements, they have no embedded functionality to simulate human behavior. Even if perfect human behavior was as simple as including a plug-in, Puppeteer is still a developer-oriented tool that requires coding skill. Enter Browser Automation Studio, or BAS. BAS is a free, Windows-only automation environment that allows users to drag and drop their way to a fully automated browser, no coding needed.
Figure 8. Browser Automation Studio User Interface
As attackers grow in capability, they succeed in creating automated attacks that look more like human behavior. In some contexts, it actually makes more sense to just use actual humans. "Microwork" is a booming industry in which anyone can farm out small tasks in return for pennies. These services describe their jobs as ideal for labeling data destined for machine learning systems and, in theory, that would be a perfect use. In reality, the tasks the human workers perform are helping bypass antibot defenses on social networks, retailers, and any site with a login or sign-up form.
Figure 9. Data labeling “microwork” using humans to help bypass antibot defenses
Depending on the attacker sophistication level and motivation there are a variety of tools ranging from basic automation to leveraging real humans to attempt to bypass bot defenses and perform account takeover actions. No matter the skill level, most attackers (at least, most cybercriminals) will start off with the cheapest, that is, least sophisticated, attacks in order to maximize rate of return. Able attackers will only increase sophistication (and thereby cost) if their target has implemented countermeasures that detect their original attack, and if the rewards still outweigh that increased cost.