Help with URI string-based session persistence

i might be lost but can't we just use persist command?

e.g.

 configuration

root@(ve11a)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm virtual bar
ltm virtual bar {
    destination 172.28.24.10:80
    ip-protocol tcp
    mask 255.255.255.255
    pool foo
    profiles {
        http { }
        tcp { }
    }
    rules {
        qux
    }
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    vs-index 20
}
root@(ve11a)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm pool foo
ltm pool foo {
    load-balancing-mode least-connections-member
    members {
        200.200.200.101:80 {
            address 200.200.200.101
        }
        200.200.200.111:80 {
            address 200.200.200.111
        }
    }
}
root@(ve11a)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    when HTTP_REQUEST {
  set uri [HTTP::uri]
  set key [URI::query $uri strID]
  if { $key ne "" } {
    persist uie $key
  }
}
when HTTP_RESPONSE {
  log local0. "URI: $uri key: $key server: [IP::server_addr]:[TCP::server_port]"
}
}

 /var/log/ltm

[root@ve11a:Active:In Sync] config  tail -f /var/log/ltm
Jun  6 11:28:19 ve11a info tmm[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb22332&TimeLimit=180&User=Joe&request=submit key: Asb22332 server: 200.200.200.101:80
Jun  6 11:28:20 ve11a info tmm1[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb22332&TimeLimit=180&User=Joe&request=submit key: Asb22332 server: 200.200.200.101:80
Jun  6 11:28:20 ve11a info tmm[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb22332&TimeLimit=180&User=Joe&request=submit key: Asb22332 server: 200.200.200.101:80

Jun  6 11:28:26 ve11a info tmm1[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb12345&TimeLimit=180&User=Joe&request=submit key: Asb12345 server: 200.200.200.111:80
Jun  6 11:28:26 ve11a info tmm[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb12345&TimeLimit=180&User=Joe&request=submit key: Asb12345 server: 200.200.200.111:80
Jun  6 11:28:27 ve11a info tmm1[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb12345&TimeLimit=180&User=Joe&request=submit key: Asb12345 server: 200.200.200.111:80

Jun  6 11:28:33 ve11a info tmm[30284]: Rule /Common/qux : URI: / key:  server: 200.200.200.101:80
Jun  6 11:28:34 ve11a info tmm1[30284]: Rule /Common/qux : URI: / key:  server: 200.200.200.101:80
Jun  6 11:28:34 ve11a info tmm[30284]: Rule /Common/qux : URI: / key:  server: 200.200.200.111:80
Jun  6 11:28:39 ve11a info tmm1[30284]: Rule /Common/qux : URI: / key:  server: 200.200.200.111:80

Jun  6 11:28:47 ve11a info tmm[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb22332&TimeLimit=180&User=Joe&request=submit key: Asb22332 server: 200.200.200.101:80
Jun  6 11:28:49 ve11a info tmm1[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb22332&TimeLimit=180&User=Joe&request=submit key: Asb22332 server: 200.200.200.101:80
Jun  6 11:28:50 ve11a info tmm[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb22332&TimeLimit=180&User=Joe&request=submit key: Asb22332 server: 200.200.200.101:80

Jun  6 11:28:56 ve11a info tmm1[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb12345&TimeLimit=180&User=Joe&request=submit key: Asb12345 server: 200.200.200.111:80
Jun  6 11:28:57 ve11a info tmm[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb12345&TimeLimit=180&User=Joe&request=submit key: Asb12345 server: 200.200.200.111:80
Jun  6 11:28:57 ve11a info tmm1[30284]: Rule /Common/qux : URI: /some_uri?strID=Asb12345&TimeLimit=180&User=Joe&request=submit key: Asb12345 server: 200.200.200.111:80

Hi Nitass

Thank you for your input!

I did originally have a similar iRule setup to yours but what does your show persistence persist-records output when you change the string?

Ie. I would go to:

/some_uri?strID=Asb20322232343&strTimeLimitInSecs=180&request=submit

then, I would change the string and I would try going to

/some_uri?strID=Asb20321&strTimeLimitInSecs=180&request=submit

The output below shows that a persist record is formed on the first request, but the second does not show up. I'm not sure why it doesn't create a new persist record that is re-load balanced? Does it matter if the source IP or browser session is the same?

tm@(test)(cfg-sync Standalone)(Active)(/TEST)(tmos) show ltm persistence persist-records
Sys::Persistent Connections
universal  Asb20322232343  10.64.114.155:80  10.64.115.151:80  (tmm: 0)

I hope I'm explaining that clearly.... I think maybe I'm misunderstanding how universal persistence is supposed to work.

Thank you!

The output below shows that a persist record is formed on the first request, but the second does not show up.

did you use oneconnect profile?

sol7208: Overview of the OneConnect profile

I did originally have a similar iRule setup to yours but what does your show persistence persist-records output when you change the string?

i sent requests with two different keys (i.e. Asb22332 and Asb12345). you can see key in the logs.

Since I am newer to F5 I have been using the f5.http iApp template for setup of the testing. I set the OneConnect setting to "Do not use OneConnect" so I am not as far as I can tell. If you want, I could paste my configuration here for you as well.

I can see the key in the logs as you've indicated, and I also receive the same - here are the logs when using two different strID from the same IP & browser session (just changing the value and sending another request, did this many times in a row and the node never changed):

Jun  8 10:25:16 test info tmm[11258]: Rule /Common/URI_LB_Tomcat <HTTP_RESPONSE>: URI: /some_uri?&strID=acb20311211&strTimeLimitInSecs=180&request=poll key: acb20311211 server: 10.64.115.151:80
Jun  8 10:25:30 test info tmm[11258]: Rule /Common/URI_LB_Tomcat <HTTP_RESPONSE>: URI: /some_uri?&strID=acb211&strTimeLimitInSecs=180&request=poll key: acb211 server: 10.64.115.151:80
`</pre>

When I do the "show ltm persistence persist-records" command, I still receive just one record for the output. I guess I would of expected to see two entries, despite the fact the source browser session and IP are from the same - but because the strID is different, it would instead create a new entry for the new key.

-- From further testing --

I notice that when I send in different strID values from different source IPs it creates a new record - which is more what I would expect whenever the strID value changes.

If I send in the same strID value from two different source IP addresses, it doesn't use the existing record of the first request and instead seems to load balance to a different node - as seen by these logs where I used the same URI from two machines:

Jun  8 10:45:03 test info tmm1[11258]: Rule /Common/URI_LB_Tomcat <HTTP_RESPONSE>: URI: /some_uri?&strID=acb211211&strTimeLimitInSecs=180&request=poll key: acb211211 server: 10.64.115.151:80
    Jun  8 10:45:23 test info tmm[11258]: Rule /Common/URI_LB_Tomcat <HTTP_RESPONSE>: URI: /some_uri?&strID=acb211211&strTimeLimitInSecs=180&request=poll key: acb211211 server: 10.64.115.150:80

The node IP is different, which isn't what I would of expected. I would expect that the user's request would be directed to the same node to be processed since the strID value matches.

I feel that maybe I am misunderstanding something fundamental on universal persistence and its functionality, but I can't put my finger on it. Thank you again for all of your help!

Edit: sorry for the messed up ampersands etc in the logs - it seems the "edit" functionality on the forum here can blow up my original post!

The more I think about this the more I think maybe I'm confusing session persistence with the idea of tying a request string to a node.

ie.
String ID ABC comes in, load balancer directs to node A
String ID BBC comes in, load balancer directs to node B
String ID ABC comes in again, request is sent to node A
String ID BBC comes in again, request is sent to node B
String ID  comes in, load balancer directs to A or B and so on

The more I think about this, it seems more of a String to Node lookup table type solution. The problem here being the string is dynamic and generated for any given request, and follow up requests are essentially polling the status of the original request identified by the String ID. For speed, the hope was to direct these to the same servers so the request is in memory and can be responded to as fast as possible...

Just wanted to update that I discovered where my confusion lied: I was seeing persist records drop out of the persist table and not be re-added because my TCP sessions were still active when I was sending follow up HTTP Requests. So long as I opened a new browser window, it would then create a new persist record due to a new TCP connection.

I tested and verified that two source IPs will get stuck to the same node when a persist record exists for a given strID value. So I realized where my testing was invalid, and this seems to be acting as I would expect now!

The only potential use case that I will need to look into to verify it isn't needed is where a request for a different strID value comes in on the same TCP session. If this happens the LB just sends the request to the same node that the TCP session is tied to - but the iRule still logs as seeing the new strID value. I was confused and expected this to create a new persistence record and load balance the request, but something tells me I would need more logic in my iRule to detach the connection and re-load balance it with a new persistence record.

I'm not sure the above situation needed, so I am going back to our development staff for further clarification on how the requests are coming into the VIP with the application.

Thank you again for your help!