on 13-Apr-2020 15:42
While estimates vary, it is believed that more than half of the Internet traffic is being generated by bots, out of which unwanted or malicious ones (like spam or malware bots) account for more than half of the traffic, the remaining traffic being generated by “good” bots (like crawlers or feed fetcher bots). It is therefore important to differentiate between different classes of bots and treat them according to site-specific security policies.
The Unified Bot Defense profiles, first released in TMOS version 14.1, package bot protection features like Bot Signatures and Proactive Bot Defense previously found in L7 DoS profiles and Web Scraping protection found in ASM policies.
Configuring Unified Bot Defense profiles through BIG-IQ ensures configuration consistency over the centralized managed BIG-IP estate and enhanced reporting capabilities.
This article will guide you through the configuration of Unified Bot Defense profiles using BIG-IQ CM User Interface.
It is assumed that the BIG-IP device where the Bot Defense profile will be deployed is currently managed by the BIG-IQ cluster, at least one BIG-IQ Logging Node / Data Collection Device is available and the Virtual Server to be protected is already configured (in the example below, VS_12BOX) - the configuration of these elements will not be part of this article.
This article covers:
1. Go to Configuration->LOCAL TRAFFIC->Pools, click Create and fill in the settings:
-Name: Pool_DCD
-Device: select the BIG-IP device
-Health monitors: gateway_icmp
-New member: - Select "New Node"
- Address: Type the Log Node / DCD IP address
- Port: 8514 (this is the port that Web Application Security Service is listening on the Logging Node / DCD)
Note: Ensure that the Logging Node / Data Collection Device has the Web Application Security Service activated and the managed BIG-IP has LTM, SSM and ASM services Discovered/Imported.
2. Go to Configuration->LOCAL TRAFFIC->Logs ->Log Destination, click Create and fill in the settings:
- Name: Log_dst_HSL_DCD
- Type: Remote High-Speed
- Device: select the BIG-IP device
- Pool: select /Common/Pool_DCD
3. Go to Configuration->LOCAL TRAFFIC->Logs ->Log Destination, click Create and fill in the settings:
- Name: Log_dst_Splunk_DCD
- Type: SPLUNK
- Forward to: select /Common/Log_dst_HSL_DCD
4. Go to Configuration->LOCAL TRAFFIC->Logs ->Log Publishers, click Create and fill in the settings:
- Name: Log_pub_DCD
- Log destinations: select /Common/Log_dst_Splunk_DCD
5. Go to Configuration->LOCAL TRAFFIC->Pinning Policies and select the BIG-IP device - Filter the available Local Traffic Manager (LTM) objects by selecting Log Publishers from the dropdown menu - Check Log_pub_DCD and click Add Selected button
6. Go to Configuration->SECURITY->Shared Security ->Logging Profiles, click Create and fill in the settings:
-Name: Log_bot_protect_demo
-Bot Defense:
-Status: Enabled
-Local Publisher: Enabled
-Remote Publisher: /Common/Log_pub_DCD
1. Go to Configuration->SECURITY->Shared Security->Virtual Servers and click on VS_12BOX VS
2. Select the Log_bot_protect_demo log profile for Logging profiles
1. Go to Deployment->EVALUATE & DEPLOY-> Local Traffic & Network, create a new Deployment. Once the evaluation has finished, click on Deploy.
2. Go to Deployment->EVALUATE & DEPLOY-> Shared Security, create a new Deployment. Once the evaluation has finished, click on Deploy.
Go to Configuration->SECURITY->Shared Security ->Bot Defense-> Bot Profiles, click Create and fill in the settings:
-Name: bot_defense_demo
-Enforcement Mode: Blocking
-Profile Template: Strict
-Browser Verification:
-Browser Access: Allowed
-Browser Verification: Verify After Access (Blocking)
Note: As per K42323285: Overview of the unified Bot Defense profile the available options for the configuration elements used in this examples are:
Enforcement Mode: Select one of the following modes, depending on the readiness of your application environment and requirements:
Profile Template: The template you select determines the default values for mitigation and verification settings. However, you can customize these settings to meet your application security requirements.
After the system saves the profile, you can't change this setting.
The following list contains descriptions of the available templates:
Browser Verification: Specifies what and when the system sends challenges.
Device ID Mode: A unique identifier that BIG-IP ASM creates by sending JavaScript to get information about the client device. The default value for this setting is determined by your selection in Profile Template (under General Settings). F5 recommends you use the default values set by the Profile Template you selected unless you have specific application requirements.
1. Go to Configuration->SECURITY->Shared Security->Virtual Servers and click on VS_12BOX VS
2. Select the bot_defense_demo profile for Bot Defense profile
Go to Deployment->EVALUATE & DEPLOY-> Shared Security, create a new Deployment. Once the evaluation has finished, click on Deploy
To monitor Bot Protection operation, check the Monitoring->DASHBOARDS->Bot Traffic Dashboard and Monitoring->EVENTS->Bot->Bot requests logs