Forum Discussion
Mark_Melin_6298
Jun 14, 2008Nimbostratus
I solved this for the most part, and have successfully loaded this with 400K connected clients and over 1000 concurrent messages delivered to those clients.... as follows:
iRules Procedure:
- Collect TCP Payload, Inspect TCP Payload for String
- MD5 hashing string TCP payload concatenated with each node ip address in foreach loop
- binary scan the MD5 hash value for each node and return the first byte's integer value
- compare each score and send the connecction to the winning node
- repeat the process for the second virtual server, the same node is selected for both connections
Handling Node Failure:
- create custom monitor for each service, set as monitor for pools
- set 'manual resume' to yes in both monitors, avoids assymetric conns if host is restored without intervention
- configure pools for 'action on service down' as 'reject', sends RST to all clients on that node, they reconnect, re-calc and relect new winner
- configure tcp profiles for 'infinite' keepalive interval' which disables keepalive ACKs from lb to clients
- configure same tcp profiles for 'zero time wait' 'zero fin wait' and 'zero close wait' and low 'idle timeout'
- clients elect a new node on fail, and timeout after specified interval thus re-electing on specified interval
Caveats:
- communications are assymetrical, and thus broken, for the specified interval above, when a node in added to pool
- sending RST for 1M x 1/nodes is less than optimal, would like to send RSTs on a known failure rather than simply status change
Under Consideration:
- build a secondary vs for msging which has exact same characteristics as vs's mentioned above
- do not restore the pool member on the second vs yet
- apply same iRule that scores the winner, but this time it picks the second server (the winner before adding new member)
- have the sending msg server 'retry' to the second vip, the message should succeed, THEN
- application logic to have the msg sender send a reconnect instruction to client app
-OR-
- create persistence record on the TCP payload string above AND
- iRule that queries persistence table, returns foreign ip and port, sends TCP RST to client
- client reconnects and elects correct winner
iRule 1 - Client Rule for MD5 Hash Node Election
Loadbalancing Based on TCP Payload String, Node IP and MD5 Hash Score
when CLIENT_ACCEPTED {
TCP::collect 2
}
when CLIENT_DATA {
set msg [findstr [TCP::payload] "" 0 "\r\n"]
set sid [findstr $msg "" 9 "\r\n"]
set S ""
foreach N [active_members -list myClientPool] {
set I [lindex $N 0]
binary scan [md5 $I$sid] c1 NMD5
if { $NMD5 > $S } {
set S $NMD5
set W $N
} else {
set S $NMD5
}
}
pool myClientPool member [lindex $W 0] [lindex $W 1]
}
iRule 2 - Msg Server Rule for MD5 Hash Node Election
Loadbalancing Based on TCP Payload String, Node IP and MD5 Hash Score
when CLIENT_ACCEPTED {
TCP::collect 2
}
when CLIENT_DATA {
set msg [findstr [TCP::payload] "" 0 "\r\n"]
set sid [findstr $msg "" 9 "\r\n"]
set S ""
foreach N [active_members -list myMsgSrvPool] {
set I [lindex $N 0]
binary scan [md5 $I$sid] c1 NMD5
if { $NMD5 > $S } {
set S $NMD5
set W $N
} else {
set S $NMD5
}
}
pool myMsgSrvPool member [lindex $W 0] [lindex $W 1]
}