Forum Discussion
Kevin_M_182964
Nimbostratus
NimbostratusIs my persist record getting ignored?
I posted a similar question a few months back, but I still seem to be having trouble with using an iRule to manage persistence between clients and servers. Basically, the situation is the servers provide a transaction ID that I use as the key for a uie persist record. I seem to be capturing the transaction ID correctly, and I am able to add and delete the entries from the persist records table, but for some reason the LTM is occasionally forwarding some of the requests to the wrong server. Here is the iRule that creates the persist record:
when HTTP_RESPONSE {
make sure Location and transaction exist
if {[HTTP::header exists Location] && [string match *transactions* [HTTP::header Location]]} {
get everything right of /transactions/ up to next /
set Transaction_ID [findstr [HTTP::header Location] "/transactions/" 14 /]
if persist record doesn't exist, create one
if {[persist lookup uie $Transaction_ID] equals ""} {
persist add uie $Transaction_ID 300
log local0. "[IP::client_addr]:[TCP::client_port], transaction ID: $Transaction_ID, created persist record: [persist lookup uie $Transaction_ID]"
} else {
log local0. "[IP::client_addr]:[TCP::client_port], transaction ID: $Transaction_ID, existing persist record: [persist lookup uie $Transaction_ID]"
}
}
}
And here is the iRule that watches for subsequent requests to either use the persist entry or to remove it:
when HTTP_REQUEST {
if { [HTTP::uri] contains "/transactions/" } {
get field from /transactions/ to next /
set Transaction_ID [findstr [HTTP::uri] "/transactions/" 14 "/"]
check if persistence record exists for this transaction
if {[persist lookup uie $Transaction_ID] equals ""} {
log local0. "[IP::client_addr]:[TCP::client_port] [HTTP::uri], $Transaction_ID persist record does not exist"
} else {
check if transaction is closed by PUT with ContentLength=0
if {[HTTP::method] equals "PUT" && [HTTP::header Content-Length] equals "0"} {
log local0. "End of transaction $Transaction_ID method: [HTTP::method], length: [HTTP::header Content-Length]"
persist delete uie $Transaction_ID
} else {
use existing persist record
log local0. "[IP::client_addr]:[TCP::client_port] [HTTP::uri], transactionID: $Transaction_ID, existing persist record: [persist lookup uie $Transaction_ID]"
persist uie $Transaction_ID
}
}
}
}
I can see in the LTM log that the records are being added and deleted correctly, but a tcpdump on the servers shows that sometimes the requests are being sent to the wrong server. Am I using the persist statement correctly? Does "persist uie $Transaction_ID" tell the LTM to use this persist record? Is there a log I can check to see why it might be forwarding the request to the wrong server?
Thanks, Kevin
amolariCirrostratus
It should be display using this command format in your iRule
[persist lookup uie "$Transaction_ID any"]
Kevin_M_182964I think there are a couple of things going on here. First, I've figured out that I can simply combine the two event handlers into a single iRule, and assign that iRule to a universal persist profile. I also figured out how to create a rule that shows which member of the pool has been selected (LB_SELECTED and LB::server addr). I can see that my persist records are being honored correctly, which is a relief.
The problem is that I have two servers in the pool, and they use an incrementing sequence beginning with 1 for the transaction ID. The transaction IDs occasionally coincide, which is why I put "persist delete uie $Transaction_ID" in the request check. However, once the persist record is deleted, the LB want to use the other server in the pool and sends the final request of the transaction to the wrong server. I've found that if I don't include the delete statement, the transaction IDs of the two servers overlap and an existing persist record is reused. This sometimes causes the LB to send the requests to the wrong server.
Is there a way to delete the record after the decision to forward based on the persist record has been made? All I have to go on for the final ACK back from the server is a 200 response (i.e. no transaction ID to tie it to), so I have to act based on the request.
Thanks.
- nitass_89166
Noctilucent
i might be lost but does deleting persistence record after forwarding really help because request with the same transaction id (from another server) may come before (since transaction id is independent on each server)?
just wondering whether persistence method that does not utilize persistence table, such as carp, helps.
sol11362: Overview of the CARP hash algorithm
https://support.f5.com/kb/en-us/solutions/public/11000/300/sol11362.html- nitass_89166sorry, please ignore my comment. i was wrong. carp persistence method won't help.
nitassEmployee
i might be lost but does deleting persistence record after forwarding really help because request with the same transaction id (from another server) may come before (since transaction id is independent on each server)?
just wondering whether persistence method that does not utilize persistence table, such as carp, helps.
sol11362: Overview of the CARP hash algorithm
https://support.f5.com/kb/en-us/solutions/public/11000/300/sol11362.html- nitasssorry, please ignore my comment. i was wrong. carp persistence method won't help.
- Kevin_M_182964
I think I've figured out how to do it. Before I delete the persist record, I grab the node that the packet would have been sent to. After deleting the record, I force the packet to be sent to that node:
check if transaction is closed by PUT with ContentLength=0 if {[HTTP::method] equals "PUT" && [HTTP::header Content-Length] equals "0"} { log local0. "[IP::client_addr]:[TCP::client_port], transactionID: $Transaction_ID method: [HTTP::method], length: [HTTP::header Content-Length], deleting existing persist record: [persist lookup uie $Transaction_ID]" delete the persist record, then send packet to correct server set server_addr [persist lookup uie $Transaction_ID node] set server_port [persist lookup uie $Transaction_ID port] delete the record persist delete uie $Transaction_ID send to server log local0. "[IP::client_addr]:[TCP::client_port], forwarding transactionID: $Transaction_ID to $server_addr:$server_port" node $server_addr $server_portThis seems to do the trick. I think there's still some risk that the same transaction ID could be used by both servers at the same time, but I don't see anyway around that.