Forum Discussion

Steve_87971's avatar
Steve_87971
Icon for Nimbostratus rankNimbostratus
Mar 30, 2012

Using ASM logic in iRules

Hi all,

 

 

I have a HA pair of 3900's running v10.2.3 LTM and ASM so I have the web scraping blocking capability.

 

 

I’d like to be able to identify traffic from search engines so I can tag the URI sent to my IIS application and log an additional value for reporting purposes.

 

 

As ASM can identify known robots, is it possible to reference this logic from an iRule?

 

 

I won’t embarrass myself putting up a clumsy iRule but in English, this is what I want:

 

 

Client connects to VS

 

iRule fires

 

iRule uses ASM engine to see whether this is a search engine

 

(ASM checks user-agent and IP address as per web scraping technique)

 

ASM returns yes/no for search engine

 

If yes, iRule adds a variable to the URI for the application to pick-up. If no, iRule completes with no action taken.

 

 

 

Has anyone ever tried to do something like this?

 

 

Cheers, Steve
app sec
BIG-IP Access Policy Manager (APM)
security
  • hooleylist's avatar
    hooleylist
    Icon for Cirrostratus rankCirrostratus
    Mar 30, 2012
    Hi Steve,

     

     

    You could try testing the following:

     

     

    Put the web scraping check set for alarm and learn but not block in the ASM policy's blocking settings.

     

    Create an iRule which checks in ASM_REQUEST_VIOLATION for ASM::violation_data's Violation list containing VIOLATION_WEB_SCRAPING_DETECTED

     

    Insert an HTTP header in the request or modify the URI to append a query string parameter using HTTP::header or HTTP::uri

     

     

    Here are a few related wiki pages:

     

     

    https://devcentral.f5.com/wiki/iRules.ASM_REQUEST_VIOLATION.ashx

     

    https://devcentral.f5.com/wiki/iRules.ASM__violation_data.ashx

     

    http://devcentral.f5.com/wiki/iRules.http__header.ashx

     

     

    If you get something working it would make a great codeshare example. Else if you get stuck add debug logging to your iRule and reply back with the iRule you're testing, details of the issue you're seeing and debug log from /var/log/ltm.

     

     

    Aaron
  • Steve_87971's avatar
    Steve_87971
    Icon for Nimbostratus rankNimbostratus
    Mar 30, 2012
    Hi Aaron, thank you for your response.

     

     

    I'll take a look at the links though I think there may be a couple of problems straight off the bat:

     

    1. We already have scraping in blocking mode so only Google and Bing etc can spider our site (or we will have - got a bit of a support issue with it)

     

    2. If the request we want to mark is from Google or Bing then surely they wouldn't be marked by ASM as violations? - ASM is supported to let those guys through.

     

     

     

    Cheers, Steve