Forum Discussion

MihirJoshi_3137's avatar
MihirJoshi_3137
Icon for Nimbostratus rankNimbostratus
Mar 14, 2017

Allow user agents to scrape my website

I am trying to write a iRule which allows facebopot to scrape my webpage   when HTTP_REQUEST { if { [HTTP::header "User-Agent"] contains "Valid_Agents_for_Facebook" } { ASM::disable } els...
application delivery
ASM Advanced WAF
BIG-IP
security
  • MihirJoshi_3137's avatar
    MihirJoshi_3137
    Mar 14, 2017

    Finally got it working,

     

    when BOTDEFENSE_ACTION { if { [class match [string tolower [HTTP::header "User-Agent"]] contains "Valid_Agents_for_Facebook"] } { BOTDEFENSE::action allow } }

     

Vijay_E's avatar
Vijay_E
Icon for Cirrus rankCirrus
Mar 14, 2017

Are the values within the datagroup case-sensitive and have you added them as they are expected to show up from facebook ?