For more information regarding the security incident at F5, the actions we are taking to address it, and our ongoing efforts to protect our customers, click here.

Forum Discussion

Bill_Rowell_102's avatar
Bill_Rowell_102
Icon for Nimbostratus rankNimbostratus
May 18, 2006

Reading User-Agent field for spiders

Trying to emulate the article by network computing:

 

 

http://devcentral.f5.com/Default.aspx?tabid=29&ArticleType=ArticleView&ArticleID=40&PageID=47

 

 

I've narrowed the code down to this for my install, where I just need the spiders to be redirected to another machine for optimized spider content:

 

 

when HTTP_REQUEST {

 

if { [matchclass [HTTP::header User-Agent] contains $::useragents] } {

 

pool spiders [HTTP::header Host]

 

}

 

else {

 

pool www [HTTP::header Host]

 

}

 

}

 

 

For the useragents field I'm using the following in the Data Group List as a String:

 

Googlebot

 

googlebot

 

 

So far no worky. Any help or insight is greatly appreciated.

 

 

Thanks
application delivery
dev
devops
iRules

4 Replies

  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    May 19, 2006
    The error is in your pool statement, you are missing the connector between spiders/www and the output of [HTTP::header Host]. I'm not sure how your pool is defined, but following the other rule it would look like this:

    
when HTTP_REQUEST {
  if { [matchclass [HTTP::header User-Agent] contains $::useragents] } {
  pool spiders_[HTTP::header Host]
  } else {
    pool www_[HTTP::header Host]
  }
}
  • Bill_Rowell_102's avatar
    Bill_Rowell_102
    Icon for Nimbostratus rankNimbostratus
    May 19, 2006
    Ok, wondered what the underscore was for in the article I referenced earlier. Rules is not working right now, receiving Page Reset error. I've looked through some posts and saw that putting log local0 will put the output of the rule into the ltm log. How would that best be implemented in this rule?

     

     

    when HTTP_REQUEST {

     

    log local0. if { [matchclass [HTTP::header User-Agent] contains "Googlebot"] } {

     

    pool nsa_[HTTP::header Host]

     

    }

     

    else {

     

    pool www-l3_[HTTP::header Host]

     

    }

     

    }

     

     

    Thanks for any assistance.
  • Bill_Rowell_102's avatar
    Bill_Rowell_102
    Icon for Nimbostratus rankNimbostratus
    May 19, 2006
    ok, so I'm getting the following in the log:

     

     

    TCL error: Rule spiderrule - no such pool Failed Tcl_pool_GetFromObj: pool (line 1) invoked from within "pool spiders_[HTTP::header Host]"

     

     

    Hate to ask a stupid question but here I go anyway! The pools I created are just normal pools pointing to the different devices for this rule. Is there a unique string or something I'm missing in the pool setup for the pools referenced in the rule?

     

     

    Thanks again.
  • Bill_Rowell_102's avatar
    Bill_Rowell_102
    Icon for Nimbostratus rankNimbostratus
    May 19, 2006
    Thanks. Yeah I just tested out removing the following from the pool statement and the traffic passed as expected:

     

     

    _[HTTP::host]

     

     

    I'm using static pools and don't fully understand how that statement works within the rule, ie the [HTTP::host] in reference to the pool.

     

     

    I ended up seeing the error in the logs for the irule. Is that just dumbluck since the error was the pool didn't exist? I have a pool with the name but still not understanding the [HTTP::host] option seems to be my achilles heel in this rule.

     

     

    Reading up on tcl but any advice you can give on where the log option would go in this rule for future reference would be great.

     

     

    Thanks