Load Balancing based off TCP Payload entity

For 1 yes the entityID appears in the same TCP session initially it shows up and will again further in the sessions once data is sent from a GUI. during the initial login phase the GUI sends the un/pass which contains the entityID, once the F5/server sends all the reference data back to the GUI , the GUI will start to send the entity ID in all subsequent packets. you will just see the ACK packets from the GUI until the login phase is completed.

For 2 I can use SSL sessionID because i need to have all users from entityID AAAA to be persisted to the same pool member. these users can be scattered all over the globe but all from AAAA must be persisted to the same pool member. we do allow ssl renegoitation but usually a disconnect in the GUI will require a re-login attempt and new ssl session

Okay, so once a load balancing decision has been made, and unless you attempt to alter it, packets will persist automatically to the same node within a single TCP session. It's what happens between TCP sessions that matters. You could add some additional logging to your iRule to show that:

when CLIENTSSL_HANDSHAKE {
    log local0. "[IP::client_addr]:[TCP::client_port]: SSL handshake completed, collecting SSL payload"
    SSL::collect
}
when CLIENTSSL_DATA {
    set payload [SSL::payload]
    set entityID [findstr $payload "@" 1 ","]
    log local0. "entityID: $entityID"
    pool xxx_yyyy_Pool
    log local0. "using persistence for entityId ($entityID)?: [persist lookup uie $entityID]"
    if { $entityID ne "" } {
        persist uie $entityID 64000
    }
    SSL::release
    SSL::collect
}

Not 100% certain about the last SSL::collect, so you may need to comment that out if it doesn't work. Watch the log and the wireshark capture closely within a single application session. The first request shouldn't have any persistence information in the lookup, but then subsequent requests, assuming the entityID is available, should.

Thanks I will try this again @ 5PM EDT. keep you posted

It turns out there are two connections made by the GUI... the 1st one contains the necessary entity ID while the other does not.... Is there any way to work around that?

Jul  9 17:19:24 bigip01 info tmm4[13134]: Rule /Common/Entity_Irule : 10.65.59.202:63341: SSL handshake completed, collecting SSL payload
Jul  9 17:19:24 bigip01 info tmm4[13134]: Rule /Common/Entity_Irule : QA-xxx-CUST2: QA-XXX-CUST2
Jul  9 17:19:24 va-bigip01 info tmm4[13134]: Rule /Common/Entity_Irule : using persistence for entityId (QA-xxx-CUST2)?: /Common/_Pool 10.36.6.34 9820
Jul  9 17:19:27 vbigip01 info tmm2[13133]: Rule /Common/_Entity_Irule : 10.65.59.202:63344: SSL handshake completed, collecting SSL payload
Jul  9 17:19:27 bigip01 info tmm2[13133]: Rule /Common/Entity_Irule : :
Jul  9 17:19:27 bigip01 err tmm2[13133]: 01220001:3: TCL error: /Common/_Entity_Irule  - attempt to use empty persistence key (line 1)     invoked from within "persist lookup uie $entityID"

I stand corrected there is 1 TCP socket after the SSL handshake. I am not sure why the Irule is picking up a second connection and then showing a blank entityID record.

its odd that the UIE persist table actually shows the entity being persisted but the GUI connection break's.

i@(-bigip01)(cfg-sync Changes Pending)(Active)(/Common)(tmos) show ltm persistence persist-records virtual xxxxxx.com Sys::Persistent Connections universal QA-xxx-CUST2 10.195.127.9:443 10.36.6.35:9820 (tmm: 2) Total records returned: 1

OK i ran a capture on the SSL/outside and see there are indeed 2 SSL connections made exactly 3 seconds apart and this is why we see the second message in the LTM logs relating to an empty persistence. the 2nd SSL connection sends a packet with payload but its not anything I am expecting. but its obvious this is impacting the Irule interaction.

I am assuming there is no way for irule to ignore the second SSL handshake ?

I am going to run another capture with the Irule removed to see what behavior occurs.

The inside/decrypted should only be 1 tcp session.

confirmed that the working GUI does indeed open 2 ssl sockets... the second socket does not have a entity ID in it anywhere so this explains the Irule behavior.

Does the app per chance use an ephemeral port, like an FTP command channel? If so, does the first channel negotiate the second channel on startup?

Well, this has turned out to be a tough one...

So I think we're left with two possibilities:

1. Continue to investigate the ephemeral port option. Maybe take a closer look at the payload of the first session to see if there's ANYTHING in there (request or response) that indicates what that second session will be.

    

2. Use source address persistence. In the absence of ANY information in the second session to tie it to the first, you may be limited to the only information that will be consistent to a single client.