Forum Discussion

Blue_whale's avatar
Blue_whale
Icon for Cirrocumulus rankCirrocumulus
Dec 19, 2024

How to block specific User-Agent in ASM Policy

Hi Experts ,

We are getting many requests from specific IP with the User Agent libcurl .We would like to block this user agent containing curl . Could you please help to configure the rule in the existing ASM Policy?

I would like to apply the Policy for the URI - /bluewhale/api/ProdSearch .

 

Dec 19 12:08:29 F5-ASM-PROD-P1 ASM:"2024-12-16 12:08:28";"213.X.X.X";"20179";"192.168.30.35";"443";"/Common/PRD_ASM_SSL";"GET";"passed";"9232836799849750123";"301";"/bluewhale/api/ProdSearch/Search";"N/A";"N/A";"0";"N/A";"N/A";"N/A";"N/A";"Host: www.example.com\r\nUser-Agent: libcurl/8.10.1 r-curl/6.0.1 httr/1.4.7\r\nAccept-Encoding: deflate, gzip\r\nAccept: application/json, text/xml, application/xml, */*\r\nX-Forwarded-For: 213.X.X.X\r\n\r\n"

  • Hi, you can do this in two ways.

    One, any signature can be defined, like I described below...

    • Define a Custom Signature:
      • Go to Security > Application Security > Policy Building > Request Signatures.
      • Click Create to add a new custom signature.
    • Create a User-Agent Blocking Signature:
      • Name: Provide a meaningful name like Block_libcurl_UserAgent.
      • Description: Add a description for documentation purposes.
      • Attack Type: Select or create an appropriate attack type (e.g., Abuse or Bot Traffic).
      • Signature: 
        • Rule Content: Specify the signature rule to detect the libcurl user-agent.
          • Example rule --> makefileUser-Agent:.curl.
              •  
      • Save the custom signature.
    • Enable the Custom Signature in the ASM Policy:
      • Go to Security > Application Security > Policy > Policy List.
      • Select your policy and ensure that the new custom signature is included and active.
    • Verify and Apply Policy:
      • Save and apply the updated policy.
      • Test the policy by simulating requests with curl to ensure they are being blocked.

     

    Or, using irule...

     

     

    I hope this help you.

     

    HT

    • Blue_whale's avatar
      Blue_whale
      Icon for Cirrocumulus rankCirrocumulus

      Hi HarunTuna ,

      Thanks for the detailed info ..I would like to apply these policy/rule to only to the path  /bluewhale/api/ProdSearch . It should not block any other url's or path which gets the connection with usersgent : curl .

  • when HTTP_REQUEST {
        # Check if the User-Agent header contains "curl"
        if {[HTTP::header exists "User-Agent"] && [string tolower [HTTP::header "User-Agent"]] contains "curl"} {
            log local0. "Blocked request with User-Agent: [HTTP::header "User-Agent"] from IP: [IP::client_addr]"
            # Send an HTTP 403 Forbidden response
            HTTP::respond 403 content "Access denied."
            return
        }
    }

  • Keep in mind, the sender can simply change their User-Agent header to get around such filtering.

  • To block a user agent containing "curl", you can typically implement a rule in your web server configuration that identifies any user agent string with the word "curl" and denies access to your website from that source; depending on your server setup, this might involve creating a firewall rule or using a web application firewall (WAF) to filter based on the "User-Agent" header containing "curl". 

    Key points to remember:
    Identifying the "curl" pattern:
    Look for the exact string "curl" within the User-Agent header, as it's usually the most reliable indicator of a curl request. 

    Implementation methods:
    Web server configuration: Most web servers like Apache, Nginx, and Microsoft IIS allow you to set custom rules based on the User-Agent header to block specific patterns. 

     

    WAF (Web Application Firewall): If your website uses a WAF like Cloudflare, you can create a blocking rule specifically targeting User-Agents containing "curl". 
    Example rule (Nginx):
    Code

    location / {

        if ($http_user_agent ~* "curl") {

            return 403;

        }

        # Your regular website content serving code here

    }

    Important considerations:
    False positives:
    Be careful not to accidentally block legitimate users who might have a browser that happens to include "curl" in its User-Agent string (although this is uncommon). 
    Alternative methods:
    IP blocking: If you suspect a specific IP address is using curl for malicious activity, you can block that IP directly instead of relying solely on the User-Agent. 
    Rate limiting: Implement rate limiting to prevent excessive requests from any single source, including curl-based bots.


    HTH
    F5 Design Engineer

    (Please rate and mark as solution if this is helpful.)
    (F5 DevCentral is a free help community, but it requires time and dedication to help others, a thanks or rating would not hurt or cost anything)