Limiting Duplicate HTTP GETs

Thanks for the response Aaron. Here are some answers:

 

 

If we knew we could apply the new rule to all URIs without causing a performance hit, I think we would do that. But there's only two URIs that are causing us problems, so it's probably safer to limit this rule to just those. As for IP ranges, we've seen these HTTP floods from several different customers, so we'd prefer to not restrict this by IP.

 

 

I should tell you that we're not setting a session cookie on the application in question, as persistence isn't a requirement. Rather, what we are setting is a unique workstation-specific cookie. The first time the user visits our site, we set this cookie (i.e "workstationid=xxxxxxxx"). We then can track that user/workstation anywhere on our site. What happens when we see a flood is the duplicate GETs all contain this same workstationid cookie, so it's either the actual workstation/browser resending the requests, or it's a proxy in the middle that is stuck in a loop and sending the same request over and over. But to answer your question, if we receive a GET without that cookie, which is rare, I'd just assume we allow them access. In all of the cases of these floods, the cookie has been properly set.

 

 

The clients are not malicious. These are our own customers (schools) that have paid for our product. Therefore, I'm not worried about them deleting or manipulating cookies.

 

 

The flood is always from a single client to a single URI. For example, we’ll be seeing normal traffic for days or weeks at a time, and then we’ll receive 30,000 duplicate requests, within the period of maybe 20 minutes, to: www.mysite.com/index.html. But again, because we’re dealing with schools that are using one-to-many NAT, we can’t block the IP because there may be a few hundred other students using our program behind the same IP.

 

 

LTM-1600 v10.0.1 (build 283.0 final) in an active/passive HA configuration. Not sure what other info you’re looking for.

 

 

Lastly, I should provide a little more information on the issue. I recently captured a trace during a flood, and what I found is that each duplicate GET is actually spawning a new, separate TCP connection each time. So aside from the additional overhead of the HTTP requests, our servers have to participate in the three-way handshake for each duplicate request. This may or may not have any bearing on the iRule, since we still need to dive into the layer 7 portion of the packets and identify a cookie, but I just thought I’d mention it. Thanks again for your help Aaron.