HTTP connection limit + cookie insert from BigIP

What are you actually seeing happen? Persistence shouldn't have any impact on the rule, as it's not specifying which pool members a request goes to.

Here's a version with added debug:

rule HTTP_session_limit {
when RULE_INIT {
   set ::debug 1
   set ::total_active_clients 0
   set ::max_active_clients 100
   log local0. "rule session_limit initialized: total/max: $::total_active_clients/$::max_active_clients"
}
when HTTP_REQUEST {
   if {$::debug}{ set uri [HTTP::uri]}
   ; test cookie presence
   if {[HTTP::cookie exists "ClientID"]} {
      set need_cookie 0
      set client_id [HTTP::cookie "ClientID"]
      if {$::debug}{ log local0. "client [IP::client_addr] with cookie $client_id -> $uri, count: $::total_active_clients"}
   ; if cookie not present & connection limit not reached, set up client_id
   } else {
      if {$::total_active_clients < $::max_active_clients} {
         set need_cookie 1
         set client_id [format "%08d" [expr { int(100000000 * rand()) }]]
         incr ::total_active_clients
      if {$::debug}{ log local0. "client [IP::client_addr] with no cookie, under connection limit -> $uri, count: $::total_active_clients"}
      ; otherwise redirect
      } else {
         if {$::debug}{ log local0. "client [IP::client_addr] with no cookie over limit, redirected  -> $uri, count: $::total_active_clients"}
         HTTP::redirect "http://sorry.domain.com/"
         return
      }
   }
}
when HTTP_RESPONSE {
   ; insert cookie if needed
   if {$need_cookie == 1} {
      if {$::debug}{ log local0. "client [IP::client_addr] inserting cookie: $client_id"}
      HTTP::cookie insert name "ClientID" value $client_id
   }
}
when CLIENT_CLOSED {
   ; decrement current connection counter for this client_id
   if {$::total_active_clients > 0} {
      incr ::total_active_clients -1
      if {$::debug}{ log local0. "client [IP::client_addr] closing connection with cookie $client_id -> $uri, count: $::total_active_clients"}
  }
}
}

Aaron

Thanks for the debug lines, I was playing around and got real close, bu you closed the loop for me.

 

 

I have my max_active_clients set low (5 to 10) to test the redirect..what I see is if I set it too low, all the images don't load, as those are subsequent HTTP connections, but I should have a cookie so why is it not loading them?

 

 

I also see this in the log from time to time

 

 

Sep 6 15:02:11 tmm tmm[22188]: 01220001:3: TCL error: Rule rule_http-session-limit-3 - Operation not supported. Multiple redirect/respond invocations not allowed (line 1) invoked from within "HTTP::redirect "http://sorry.domain.com/""

 

Sep 6 15:02:11 tmm tmm[22188]: 01220001:3: TCL error: Rule rule_http-session-limit-3 - can't read "client_id": no such variable while executing "log local0. "client [IP::client_addr] closing connection with cookie $client_id -> $uri, count: $:

 

:total_active_clients""

 

 

 

Any ideas?

 

-L

 

here is some more debug info for the complete page load and close. I have my max set to 15, it appears that my session is getting several cookies, sometimes it lets me in other times it redirects

 

 

HTTP_REQUEST>: client 10.2.47.82 with no cookie, under connection limit -> /, count: 11

 

: client 10.2.47.82 with no cookie, under connection limit -> /path/search.gif, count: 12

 

: client 10.2.47.82 inserting cookie: 82050019

 

: client 10.2.47.82 with no cookie, under connection limit -> /path/logo.gif, count: 13

 

: client 10.2.47.82 inserting cookie: 14673727

 

: client 10.2.47.82 with cookie 82050019 -> /path/images/menu.gif, count: 13

 

: client 10.2.47.82 with no cookie, under connection limit -> /images/123.gif, count: 14

 

: client 10.2.47.82 with cookie 82050019 -> /path/images/icons/cart.gif, count: 14

 

: client 10.2.47.82 inserting cookie: 21333030

 

: client 10.2.47.82 with no cookie, under connection limit -> /path/common/icons/a.gif, count: 15

 

: client 10.2.47.82 inserting cookie: 44244085

 

: client 10.2.47.82 with cookie 82050019 -> /portal/images/divider.gif, count: 15

 

: client 10.2.47.82 with no cookie over limit, redirected -> /Static/images/portals/products/a.jpg, count: 15

 

: client 10.2.47.82 with no cookie over limit, redirected -> /Static/images/portals/products/b.jpg, count: 15

 

: client 10.2.47.82 with no cookie over limit, redirected -> /Static/images/portals/products/c.jpg, count: 15

 

: client 10.2.47.82 with no cookie over limit, redirected -> /Static/images/portals/products/d.jpg, count: 15

 

: client 10.2.47.82 with no cookie over limit, redirected -> /Static/images/portals/products/e.jpg, count: 15

 

: client 10.2.47.82 with cookie 44244085 -> /Static/images/common/icons/plus.gif, count: 15

 

: client 10.2.47.82 with cookie 44244085 -> /Static/images/common/icons/e.gif, count: 15

 

: client 10.2.47.82 with cookie 44244085 -> /Static/images/common/icons/p.gif, count: 15

 

: client 10.2.47.82 with cookie 82050019 -> /path/portal/images/icons/c.gif, count: 15

 

: client 10.2.47.82 with cookie 44244085 -> /Static/images/common/icons/r.gif, count: 15

 

: client 10.2.47.82 with no cookie over limit, redirected -> /Static/images/common/c/divider.gif, count: 15

 

: client 10.2.47.82 with no cookie over limit, redirected -> /path/web/Portal/US/All/Products, count: 15

 

: client 10.2.47.82 closing connection with cookie 44244085 -> /Static/images/common/c/divider.gif, count: 14

 

: client 10.2.47.82 closing connection with cookie 82050019 -> /path/portal/images/icons/c.gif, count: 13

 

: client 10.2.47.82 closing connection with cookie 44244085 -> /Static/images/common/icons/r.gif, count: 12

 

: client 10.2.47.82 closing connection with cookie 44244085 -> /path/web/Portal/US/All//Products, count: 11

 

Does anyone have any thoughts here...from what the log gives me, it looks like there are multiple cookies being issued to a single client. The log was generated by a single session. It makes sense that the logic in the rule is looking at all the cookies, but I need to be able to limit the cookie generation, 1 per client browser....please help if you can, this functionality has become more necessary with our production site.

 

 

Regards

 

-L

It looks like the client is making multiple requests before getting back the initial response containing the set-cookie header. This could either be pipelined on the same TCP connection or separate TCP connections.

Also, from this snippet of the log output, it seems that the client isn't always presenting the cookie even after it's been set:

: client 10.2.47.82 with cookie 44244085 -> /Static/images/common/icons/r.gif, count: 15

: client 10.2.47.82 with no cookie over limit, redirected -> /Static/images/common/c/divider.gif, count: 15

You could get a bit more detail by logging the client TCP port [TCP::client_port] and the number of requests a client has made on the same TCP connection [HTTP::request_num]. You might also try using an interception proxy or browser plugin to record when the cookies are received and when they are sent.

Are the requests going through a proxy server, or is this truly one client making requests?

If the client is making multiple requests over the same TCP connection, you could not increment the counter and send back the same ClientID. This might work to ignore subsequent requests on the same TCP connection:

   if {[HTTP::cookie exists "ClientID"] or [HTTP::request_num] > 1 } {

If the client is opening concurrent TCP connections, I'm not sure how best to handle it. One option might be to use the session table to track the requests per client IP. This would add memory overhead compared with using cookies though--and would be specific to the IP address as opposed to the browser.

Anyone else have ideas on how best to handle this?

Aaron

Are the requests going through a proxy server, or is this truly one client making requests?

 

 

No proxie, just me using one instance of IE 6 to test the VS

 

 

I added the [HTTP::request_num] > 1 in this line

 

 

if {[HTTP::cookie exists "ClientID"] or [HTTP::request_num] > 1}

 

 

 

in the test cookie presence line, do I need to address this somewhere else? What is happening is that once my max has been reached, it only loads some of the page leaving out images, etc..as these are indeed multiple TCP connections to the VS from the same client.

 

 

I am not sure tha client IP tracking would work, as any users behind a proxy (which we know is a large number) could still overwhelm the app servers.

 

 

Looking forward to more feedback.

 

-L

Are you seeing the requests redirected, or are you getting a TCL error?

 

 

Can you add logging of the client port and request_num values and post the log entries for a series of requests where some of the images don't load?

 

 

Also, if there are proxies making requests on behalf of multiple concurrent clients over the same TCP connection, this method for ignoring the restrictions might not work very well. I guess the chances of this being an issue are low though.

 

 

Aaron

I added some additional logging to track client request number and server response number, and there are multiple requests from the same client

 

 

 

: client request number 1

 

: current active clients 1

 

: http request from new client access granted. cookie set.

 

: current active clients 1

 

: new active client

 

: current active clients 2

 

: client request number 1

 

: current active clients 2

 

: http request from new client access granted. cookie set.

 

: current active clients 2

 

: new active client

 

: current active clients 3

 

: server response number 1

 

: client request number 2

 

: current active clients 3

 

: active user with cookie making http request

 

: server response number 2

 

: client request number 3

 

: current active clients 3

 

: active user with cookie making http request

 

: server response number 3

 

: client request number 4

 

: current active clients 3

 

: active user with cookie making http request

 

: Client accepted

 

: active clients 3

 

: server response number 4

 

: client request number 1

 

: current active clients 3

 

: http request from non active connection denied

 

***note, this should be a redirect to my sorry server, but it is redirecting to the app itself***

 

: redirecting to host.domain.com /Static/images/splitter/177.jpg

 

: attempting to close connection

 

: current active clients 4

 

: client closed

 

: 10.20.47.82

 

: decremeting active clients

 

: current active clients 3

 

: current active clients 3

 

: client closed

 

: 10.20.47.82

 

: decremeting active clients

 

: current active clients 2

 

 

 

still stumped...

 

 

-L

I see that the client port is changing for the pages where the images don't load

 

 

: client request number 1 client port 1125

 

: current active clients 3

 

: http request from non active connection denied

 

: redirecting to host.domain.com /Static/images/splitter/a.gif

 

: attempting to close connection

 

: client request number 1 client port 1126

 

: current active clients 4

 

: http request from non active connection denied

 

: redirecting to host.domain.com /Static/images/splitter/177.jpg

 

: attempting to close connection

 

: client request number 1 client port 1127

 

: current active clients 5

 

: http request from non active connection denied

 

: redirecting to host.domain.com /Static/images/splitter/4l.jpg

 

: attempting to close connection

 

: client request number 19 client port 1080

 

: current active clients 6

 

 

-L

Any ideas on the multiple client ports/client connections?

 

-L