Forum Discussion
Shu_97237
Nimbostratus
NimbostratusDispatch messages from a single connection to pool members
Hi,
I'm trying to write irules to do message-based load balancing. I am not using SIP/radius/diameter, so cannot use an existing profile. The request messages are all sent over a single TCP connection. Each message has a 8-byte header, of which the last 4 bytes store the length of the message body. I want the load balancer to forward messages to pool members in a round-robin manner. Currently, the problem is after "TCP::notify request" is executed, the USER_REQUEST won't fire. The irules are pasted below:
when CLIENT_ACCEPTED {
log local0.debug "Get a connection"
TCP::collect
}
when CLIENT_DATA {
log local0.debug "in CLIENT_DATA"
binary scan [TCP::payload] II head rlen
if {($head & 0x3) == 1} {
if {[TCP::payload length] < $rlen} {
TCP::collect $rlen
log local0.debug "a message is received"
TCP::release $rlen
TCP::notify request
log local0.debug "requested"
}
}
}
when USER_REQUEST {
Dispatch to the pool
log local0.debug "in USER_REQUEST"
pool my_pool
}
Thank you for your help!
Shu
nitassEmployee
this is my testing. client sends 2 http requests in 1 tcp connection. each request is load balanced to each pool member.
in irule, i determine end of each request by looking for \r\n\r\n.[root@ve10:Active] config b virtual bar list virtual bar { snat automap pool foo destination 172.28.19.79:80 ip protocol 6 rules myrule profiles { mblb {} tcp {} } } [root@ve10:Active] config b pool foo list pool foo { members { 200.200.200.101:80 {} 200.200.200.102:80 {} } } [root@ve10:Active] config b rule myrule list rule myrule { when CLIENT_ACCEPTED { log local0. "client: [IP::client_addr]:[TCP::client_port]" log local0. "virtual: [IP::local_addr]:[TCP::local_port]" TCP::collect } when CLIENT_DATA { log local0. "request: [TCP::payload]" if {[TCP::payload] ends_with "\r\n\r\n"} { TCP::release TCP::notify request } TCP::collect } when SERVER_CONNECTED { log local0. "node: [IP::remote_addr]:[TCP::remote_port]" TCP::collect } when SERVER_DATA { log local0. "response: [TCP::payload]" TCP::release TCP::notify response TCP::collect } when SERVER_CLOSED { log local0. "" } when CLIENT_CLOSED { log local0. "" } } /var/log/ltm Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : client: 172.28.19.251:37382 Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : virtual: 172.28.19.79:80 Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : request: GET / HTTP/1.0 User-Agent: ApacheBench/2.0.40-dev Connection: Keep-Alive Host: 172.28.19.79 Accept: */* Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : node: 200.200.200.102:80 Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : response: HTTP/1.1 200 OK Date: Sat, 14 Jul 2012 08:35:19 GMT Server: Apache/2.2.3 (CentOS) Last-Modified: Tue, 08 Nov 2011 12:26:29 GMT ETag: "4183f1-30-47e02740" Accept-Ranges: bytes Content-Length: 48 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=UTF-8 This is 102 host. Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : request: GET / HTTP/1.0 User-Agent: ApacheBench/2.0.40-dev Connection: Keep-Alive Host: 172.28.19.79 Accept: */* Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : node: 200.200.200.101:80 Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : response: HTTP/1.1 200 OK Date: Sat, 14 Jul 2012 08:47:25 GMT Server: Apache/2.2.3 (CentOS) Last-Modified: Fri, 11 Nov 2011 14:48:14 GMT ETag: "4183e4-3e-9c564780" Accept-Ranges: bytes Content-Length: 62 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=UTF-8 This is 101 host. Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule : Jul 14 16:35:00 local/tmm info tmm[5111]: Rule myrule :
Shu_97237Thank you nitass!I tried to follow your example, but have not succeeded yet. Using "\r\n\r\n" to find the end of message may not work in my application, which does not use HTTP. Is it standard for HTTP messages? In your example, you don't handle USER_REQUEST event. Is there a default action for USER_REQUEST event?
My understanding is whether I use length field found in message header or use "\r\n\r\n" to find the end of a message, as long as I call TCP::notify request, USER_REQUEST should be triggered. In my case, it has never been triggered. What prevents it from happening?
Below is the latest configuration I used:
datastor {
low water mark 80
high water mark 92
}
deduplication {}
shell write partition Common
configsync {
password crypt "****"
}
route default inet {
vlan external
}
monitor MYMBLB {
defaults from inband
}
node 10.124.66.35 {}
node 10.124.66.48 {}
pool MYMBLB_pool {
monitor all MYMBLB
members {
10.124.66.35:cslistener {}
10.124.66.48:cslistener {}
}
}
snat snat_one_ip_test {
translation 10.124.6.35
origins {
10.124.66.35
10.124.66.48
}
vlans external enable
}
rule MYRULES {
when CLIENT_ACCEPTED {
log local0.debug "Get a connection"set client_closed 0TCP::collect 1000}
when CLIENT_CLOSED {
set client_closed 1}
when SERVER_CONNECTED {
log local0.debug "Server_connected"after 1000 -periodic if {$client_closed} {TCP::close}TCP::collect}
when CLIENT_DATA {
log local0.debug "in CLIENT_DATA"
binary scan [TCP::payload] II head rlen
if {($head & 0x3) == 1} {
if {[TCP::payload length] < $rlen} {TCP::collect $rlenlog local0.debug "a message is received"TCP::release $rlenTCP::notify requestlog local0.debug "requested"}}
TCP::collect
}
}
virtual MYMBLBLB {
pool MYMBLB_pool
destination 10.124.6.35:cslistener
ip protocol tcp
rules MYRULES
profiles {
mblb {}
tcp {}
}
}
- Shu_97237I read the wiki of TCP::notify again trying to understand the meaning of "
theUSER_REQUESTevent will not be raised until the server-side TCP connection is complete". Maybe the problem is virtual server cannot establish a connection with the pool member. How can I test if the virtual server can talk to pool members correctly? - nitassUsing "\r\n\r\n" to find the end of message may not work in my application, which does not use HTTP. Is it standard for HTTP messages?i used http for testing since i do not have your application. in my case, \r\n\r\n is used to find an end of each message. in your case, you have to parse header to get payload length.
In your example, you don't handle USER_REQUEST event. Is there a default action for USER_REQUEST event?you do not need to have USER_REQUEST event. TCP::notify is used to tell bigip it is an end of each message.
additionally, TCP::notify is not working in 11.1.0. it is tracked as ID 383853. if you are using 11.1.0, please open a support case for verification.
ID 383853 - Need a synchronous event to signal end of message from TCP rule event - nitassnot sure if code is correct. anyway, would you mind trying it? i assume only one message per packet.
[root@ve10:Active] config b rule myrule list rule myrule { when CLIENT_ACCEPTED { TCP::collect } when CLIENT_DATA { binary scan [TCP::payload] @4I len if {[TCP::payload length] < [expr {8 + $len}]} { TCP::collect [expr {8 + $len - [TCP::payload length]}] return } TCP::release [expr {8 + $len}] TCP::notify request TCP::collect } } - Shu_97237Thank you very much nitass!
The irules you wrote works. My F5 still doesn't work well, it should be caused by network configuration issue but not irule. The irules to do simple MBLB turned out pretty straight forward.
- Shu_97237Posted By nitass on 07/22/2012 05:46 AM
not sure if code is correct. anyway, would you mind trying it? i assume only one message per packet.
nitass, I don't understand why you need to assume only one message per packet? Our message is bigger than a TCP packet, e.g. 2830 bytes. Is there anything to pay attention?