Disable web scraping detection per geolocation?

James,

 

 

Yes the best way to request an enhancement is to open a support case and work through that chain.

 

 

Basically the way you are seeing traffic behave is correct, when you have two classes attached to a VS you have to have something defined in them to pull traffic towards one or the other. If they are both configured the same then traffic will always fire to the first class process through that policy and exit back to ltm. If I had to compare it to something it would be firewall rules once you match to class you are done it will not check lower classes in the list.

 

 

Aaron had made a creative suggestion in his last post about how to do just one policy by doing an http:respond in an ASM_REQUEST_VIOLATION event. Basically what he is saying is to set Web Scraping to Learn, Alarm but not Block, then build your iRule to fire on ASM::violation_data for VIOLATION_WEB_SCRAPING_DETECTED. I think something like this would work, you may want to re order the flow based upon whether you want to whitelist or blacklist countries, but I believe this will work. The only thing I am not sure about is exiting directly to the pool if you match the whitelist or do not match the blacklist, as I am not sure if all of the policy checking will have completed when this event fires, so please test to make sure you are not bypassing other checks in the policy that you want done.

 

 

 

when ASM_REQUEST_VIOLATION

 

{

 

set x [ASM::violation_data]

 

 

for {set i 0} { $i < 7 } {incr i} {

 

switch $i {

 

0 { log local0. "violation=[lindex $x $i]" }

 

1 { log local0. "support_id=[lindex $x $i]" }

 

2 { log local0. "web_application=[lindex $x $i]" }

 

3 { log local0. "severity=[lindex $x $i]" }

 

4 { log local0. "source_ip=[lindex $x $i]" }

 

5 { log local0. "attack_type=[lindex $x $i]" }

 

6 { log local0. "request_status=[lindex $x $i]" }

 

 

}}

 

 

if {([lindex $x 0] contains "VIOLATION_WEB_SCRAPING_DETECTED")

 

and ([whereis [IP::client_addr]] contains "US")

 

}

 

 

pool pool1

 

 

else {

 

HTTP::respond 200 content {

 

 

 

Blocking Page

 

 

 

Your Requested has been blocked please contact your systems adminstrator

 

Your support ID is: <%TS.request.ID()%>

 

 

 

}

 

}

 

 

}

 

 

}