Forum Discussion
Paul_73795
Nimbostratus
Jan 06, 2011HTTP forward to Pool not working as expected
Hi, I am attempting to set up an iRule that forwards requests to our pool of sorry servers when there are no active members in the load balanced server pool.
As you can see by the logs if no active members are in the load balanced pool it redirects to the sorry server pool we get SERVER CONNECTED and displays the outage page. If I then turn bring the servers back online and refresh the browser the iRule looks to be doing the correct thing by directing the traffic to the load balanced server pool but in actual fact it is still being directed to the sorry servers and there is no SERVER CONNECTED response.
Have I done something incorrectly?
iRule
when HTTP_REQUEST {
set DEBUG 1
if { [active_members [LB::server pool]] < 1 } {
if { $DEBUG } { log local0. "[IP::client_addr]:[TCP::client_port]: using sorry server Pool: [LB::server]" }
pool pool_http_sorry_server
} else {
if { $DEBUG } { log local0. "[IP::client_addr]:[TCP::client_port]: using server Pool: [LB::server]" }
pool [LB::server pool]
}
}
when SERVER_CONNECTED {
Debug logging only. Remove this event once done testing
log local0. "[IP::client_addr]:[TCP::client_port]: server: [LB::server], [IP::server_addr]:[TCP::server_port]"
}
/var/log/ltm
Jan 6 18:28:58 local/tmm3 info tmm3[21819]: Rule irule_forward_outage_pool : 10.133.27.10:50651: using sorry server Pool: pool_http_bbdev 0
Jan 6 18:28:58 local/tmm3 info tmm3[21819]: Rule irule_forward_outage_pool : 10.133.27.10:50651: server: pool_http_sorry_server 203.2.32.66 80, 203.2.32.66:80
Jan 6 18:29:26 local/tmm3 info tmm3[21819]: Rule irule_forward_outage_pool : 10.133.27.10:50651: using server Pool: pool_http_bbdev 0
Jan 6 18:29:26 local/tmm3 info tmm3[21819]: Rule irule_forward_outage_pool : 10.133.27.10:50651: using server Pool: pool_http_bbdev 0
17 Replies
- Chris_Miller
Altostratus
Are you using OneConnect? Also, are you using any persistence methods? - Chris_Miller
Altostratus
In your case, the user's connection might already be established to the sorry server.
Also, if I recall, LB::server pool either returns the pool to which they're connected or if they're not connected, it returns the default pool for the Virtual. Since they're already connected, I would expect it to be checking whether there are any active members of "pool_http_sorry_server" and since there are, your rule says to go to pool LB::server pool (which again, since they're connected to the sorry server pool, is the sorry server pool)
Hoping that makes sense? I'll whip up a rule quick. - Chris_Miller
Altostratus
Give this a shot:when CLIENT_ACCEPTED { set default_pool [LB::server pool] } when HTTP_REQUEST { if { [active_members [$default_pool]] < 1 } { pool pool_http_sorry_server } else { pool $default_pool } }
Insert comments as you wish.
In the client_accepted event, we're setting the variable "default_pool" to the VIP's default pool. Since a user hasn't been mapped to a pool when client_accepted is triggered, this will return the proper default pool.
Then, we're checking whether the default pool has any active members from HTTP_REQUEST and if not, we're sending it to the sorry_server pool.Finally, I expect you'll need to use OneConnect here to make sure every HTTP Request is evaluated.
- Paul_73795
Nimbostratus
Hi Chris
thanks for your reply and yes that is making more sense to me now. I have edited the iRule to set the default pool when the client accept event is triggered.
In answer to your question OneConnect is not turned on but we use a Cookie Hash Default Persistence and a source_addr Fallback Persistence Profile.
I guess that explains what I am seeing below. I ran three tests in succession to test the failover and fail back between the pools.
The first two tests operate as expected however the third test established a connection to the sorry server and maintains that connection even though the iRule is directing it to the default pool. You can see that there is no event for the third test.
Is persistence the cause of this? Is there someway to alter the behavior of the connections when they are being diverted to the sorry server pool?
Thanks
Paul
1. Active members in pool
Jan 7 10:16:15 local/tmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:63208: setting default pool Pool: pool_http_bbdev 0
Jan 7 10:16:15 local/tmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:63208: using default Pool: pool_http_bbdev 0
Jan 7 10:16:15 local/tmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:63208: server: pool_http_bbdev
2. No active members in pool
Jan 7 10:17:17 local/tmm3 info tmm3[21819]: Rule irule_forward_outage_pool : 10.10.27.10:63279: setting default pool Pool: pool_http_bbdev 0
Jan 7 10:17:17 local/tmm3 info tmm3[21819]: Rule irule_forward_outage_pool : 10.10.27.10:63279: using sorry server Pool: pool_http_sorry_server
Jan 7 10:17:17 local/tmm3 info tmm3[21819]: Rule irule_forward_outage_pool : 10.10.27.10:63279: server: pool_http_sorry_server
Jan 7 10:17:17 local/tmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:63280: setting default pool Pool: pool_http_bbdev 0
Jan 7 10:17:17 local/tmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:63280: using sorry server Pool: pool_http_sorry_server
Jan 7 10:17:17 local/tmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:63280: server: pool_http_sorry_server
3. Active members in pool
Jan 7 10:18:11 local/tmm2 info tmm2[21816]: Rule irule_forward_outage_pool : 10.10.27.10:63362: setting default pool Pool: pool_http_bbdev 0
Jan 7 10:18:11 local/tmm2 info tmm2[21816]: Rule irule_forward_outage_pool : 10.10.27.10:63362: using default Pool: pool_http_bbdev 0
Jan 7 10:18:11 local/tmm2 info tmm2[21816]: Rule irule_forward_outage_pool : 10.10.27.10:63362: using default Pool: pool_http_bbdev 0 - Chris_Miller
Altostratus
You could add a "persist none" statement. I can't recall whether it'll work properly from HTTP_REQUEST without OneConnect since we've already selected a pool member and persisted but you can give it a shot.when CLIENT_ACCEPTED { set default_pool [LB::server pool] } when HTTP_REQUEST { if { [active_members [$default_pool]] < 1 } { pool pool_http_sorry_server persist none } else { pool $default_pool } }
If that doesn't work, we'll try different methods for persistence. - hoolio
Cirrostratus
If you're using cookie persistence you should definitely enable OneConnect on the virtual server as Chris suggested. If you're using SNAT then you can use the default OneConnect profile with a 0.0.0.0 source mask. Without SNAT on the virtual server, you should create a custom OneConnect profile with the source mask set to 255.255.255.255. See the article Chris linked to for details.
Also, if you set persist to none for one case, you should explicitly set persistence in the other case. This is because the persist command takes effect for the duration of the connection.
And just a small edit on your rule, Chris. This:
if { [active_members [$default_pool]] < 1 } {
should be:
if { [active_members $default_pool] < 1 } {
Aaron - Paul_73795
Nimbostratus
Ok, I have enabled OneConnect with the default profile because we are using SNAT.
The iRule has been edited and now has persist none added for the sorry server pool. However the behaviour remains unchanged. Once I am redirected to the sorry servers and then re enable the default pool and refresh the browser I stay connected to the sorry pool.
when CLIENT_ACCEPTED {
set default_pool [LB::server pool]
}
when HTTP_REQUEST {
if { [active_members $default_pool] < 1 } {
pool pool_http_sorry_server
persist none
} else {
pool $default_pool
}
} - Chris_Miller
Altostratus
Let's add some logging to troubleshoot.when CLIENT_ACCEPTED { set default_pool [LB::server pool] } when HTTP_REQUEST { if { [active_members $default_pool] < 1 } { pool pool_http_sorry_server persist none log local0. "Used sorry pool because there were [active_members $default_pool] members in $default_pool" } else { pool $default_pool log local0. "Used default pool because there were [active_members $default_pool] members in $default_pool" } }
This will log to /var/log/ltm and the "Local Traffic" logs section of the GUI. - The_Bhattman
Nimbostratus
Hi Paul,
You might have already done this but did you clear the browser cache when you marked up the node? I have seen this behavior happen several times with my clients sorry pages were being cached by the browser.
I hope this helps
Bhattman - Paul_73795
Nimbostratus
Ok, I have added some debug logging. Here is what is occurring.
At point 3 the HTML content from the outage page is being delivered to the client but the image in the page isn't.
The browser URI is:
http://defaultdomain.edu.au/webapps/login
This is the image path.
http://defaultdomain.edu.au/outage/index_files/logo_full.jpg
The way it is setup is that requests are being forwarded to an Apache web server and it will perform a rewrite on the URI based on the domain. We host a number of outage files for different domains on this server.
1. Active members in the default pool
Jan 11 17:15:08 local/tmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58421: setting default pool Pool: pool_http_bbdev 0
1. Active members in the default pool
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58421: setting default pool Pool: pool_http_bbdev 0
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : persist cookie profile enabled on virtual server
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : persist cookie profile name is: cookie_hash_blackboard
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Name of the cookie used for persistence is: session_id
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Cookie expiration is set for: 0
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58421: using default Pool: pool_http_bbdev 0
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Used default pool because there were 2 members in pool_http_bbdev
Jan 11 17:15:08 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58421: server: pool_http_bbdev 203.2.32.00 80, 203.2.32.00:80
2. No Active Members in default pool
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:58488: setting default pool Pool: pool_http_bbdev 0
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : persist cookie profile enabled on virtual server
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : persist cookie profile name is: cookie_hash_blackboard
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : Name of the cookie used for persistence is: session_id
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : Cookie expiration is set for: 0
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:58488: using sorry server Pool: pool_http_sorry_server
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : Used sorry pool because there were 0 members in pool_http_bbdev
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : 10.10.27.10:58488: server: pool_http_sorry_server 203.2.32.01 80, 203.2.32.01:80
Jan 11 17:17:43 localgtmm info tmm[21809]: Rule irule_forward_outage_pool : Server 203.2.32.01 has closed the connection
3. Cleared cache in browser. Active Members return to default pool
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58549: setting default pool Pool: pool_http_bbdev 0
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : persist cookie profile enabled on virtual server
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : persist cookie profile name is: cookie_hash_blackboard
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Name of the cookie used for persistence is: session_id
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Cookie expiration is set for: 0
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58549: using default Pool: pool_http_bbdev 0
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Used default pool because there were 2 members in pool_http_bbdev
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58549: using default Pool: pool_http_bbdev 0
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Used default pool because there were 2 members in pool_http_bbdev
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : 10.10.27.10:58549: server: pool_http_bbdev 203.2.32.00 80, 203.2.32.00:80
Jan 11 17:19:57 localgtmm1 info tmm1[21813]: Rule irule_forward_outage_pool : Server 203.2.32.00 has closed the connection
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects
