Forum Discussion
smp_86112
Cirrostratus
CirrostratusReverse Monitor and Pool Member Recovery
I have constructed a monitor to check a simple page and look for the string "up" in the response. Here's the sanitized definition:
ltm monitor http reverse {
defaults-from http
destination *:*
interval 5
recv "up"
reverse enabled
send "GET /monitor.html HTTP/1.1\r\nHost: app.mydomain.com\r\n\r\n"
time-until-up 0
timeout 6
}In normal mode (i.e. "Reverse" is not enabled), I can change the text and the monitor goes up/down as I expect. Then when I enable the "Reverse" parameter while the page contains "up", the member gets marked down as I expect. However when I then change the HTML to "down" in Reverse mode, the member does not recover.
The doc doesn't really specifically describe what happens when a Reverse monitor recovers, but it seems logical to me that a monitor in Reverse mode should recover when the test fails. Am I not understanding now Reverse mode works? The only think I could find is SOL10443, but I am running 10.2.0.
nitassEmployee
mine seems to be ok. by the way, is timeout 6 correct?[root@iris:Active] config b version|grep -iA 1 version BIG-IP Version 10.2.0 1707.0 Final Edition [root@iris:Active] config b pool foo list pool foo { monitor all mymon members 10.10.70.110:http {} } [root@iris:Active] config b monitor mymon list monitor mymon { defaults from http reverse recv "up" send "GET /foo.html HTTP/1.1\r\nHost:\r\nConnection: Close\r\n\r\n" } [root@iris:Active] config curl http://10.10.70.110/foo.html WELCOME!!!! This's 10.10.70.110. up [root@iris:Active] config b pool foo|grep -i pool\ member +-> POOL MEMBER foo/10.10.70.110:http inactive,down [root@iris:Active] config curl http://10.10.70.110/foo.html WELCOME!!!! This's 10.10.70.110. down [root@iris:Active] config b pool foo|grep -i pool\ member +-> POOL MEMBER foo/10.10.70.110:http active,up
smp_86112I chose 6 seconds just because I was testing and I didn't want to wait.
Thanks for posting your config. I copied your test exactly, and it behaved the way I expected. Then I started tweaking it a bit, and found that the Connection: close header is the key. When I remove it from the monitor, the member does not recover when the monitor is in Reverse mode.- smp_86112
I did some more testing by adding random characters to the html page (and validating that the receive string was not contained). I found that without the Connection: close header, a response containing less than 1705 bytes (including the headers) would not allow the member to recover. If I add one single caracter more to the page (a total of 1706 or more bytes), the member gets marked up/down as expected.
Can someone else verify?
- nitassmine seems working fine.
[root@iris:Active] config b monitor mymon list monitor mymon { defaults from http reverse recv "up" send "GET /foo.html HTTP/1.1\r\nHost:\r\n\r\n" } [root@iris:Active] config curl -i http://10.10.70.110/foo.html|grep -i up % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1418 100 1418 0 0 328k 0 --:--:-- --:--:-- --:--:-- 0 up [root@iris:Active] config curl -I http://10.10.70.110/foo.html HTTP/1.1 200 OK Date: Tue, 25 Oct 2011 05:19:31 GMT Server: Apache/2.0.59 (rPath) Last-Modified: Tue, 25 Oct 2011 05:18:59 GMT ETag: "66a9-58a-ad3cf6c0" Accept-Ranges: bytes Content-Length: 1418 Vary: Accept-Encoding Set-Cookie: testcookie=helloworld Content-Type: text/html; charset=UTF-8 [root@iris:Active] config b pool foo|grep -i pool\ member +-> POOL MEMBER foo/10.10.70.110:http inactive,down [root@iris:Active] config curl -I http://10.10.70.110/foo.html HTTP/1.1 200 OK Date: Tue, 25 Oct 2011 05:21:00 GMT Server: Apache/2.0.59 (rPath) Last-Modified: Tue, 25 Oct 2011 05:20:45 GMT ETag: "66ae-58c-b38e6540" Accept-Ranges: bytes Content-Length: 1420 Vary: Accept-Encoding Set-Cookie: testcookie=helloworld Content-Type: text/html; charset=UTF-8 [root@iris:Active] config curl -I http://10.10.70.110/foo.html|grep -i up % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 1420 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 [root@iris:Active] config b pool foo|grep -i pool\ member +-> POOL MEMBER foo/10.10.70.110:http active,up - smp_86112Thanks again for posting your test. I calculate the length of your headers to be 290 bytes. So even when I add in the 1418 bytes of content, the total length is 1708 bytes. I would be curious to see the behavior if you removed 100 or so bytes from the content, just to be sure you are well under my theoretical threshold. But I understand if you don't want to do any more - thanks for responses. You were able to help me isolate this further than I would have alone.
- nitassjust a quick question. how come 290 bytes? is it a number of char? i count it 265 bytes. is there anything i missed?
by the way, don't worry i can test it. just wanna make sure we are at the same number. ;-) - smp_86112You have to think about this in terms of what's transmitted on the wire, so characters count, spaces count, and so do CRLF characters. I count 291 bytes in the header below, if you include a standard blank line which seperates the headers from the content. I pasted into a text file and ran the wc command.
HTTP/1.1 200 OK Date: Tue, 25 Oct 2011 05:21:00 GMT Server: Apache/2.0.59 (rPath) Last-Modified: Tue, 25 Oct 2011 05:20:45 GMT ETag: "66ae-58c-b38e6540" Accept-Ranges: bytes Content-Length: 1420 Vary: Accept-Encoding Set-Cookie: testcookie=helloworld Content-Type: text/html; charset=UTF-8 - nitassthis is when it is not working. if i add one more char in the response, it starts working.
[root@iris:Active] config b monitor mymon list monitor mymon { defaults from http reverse recv "up" send "GET /foo.html HTTP/1.1\r\nHost:\r\n\r\n" } [root@iris:Active] config curl -I http://10.10.70.110/foo.html HTTP/1.1 200 OK Date: Tue, 25 Oct 2011 06:42:24 GMT Server: Apache/2.0.59 (rPath) Last-Modified: Tue, 25 Oct 2011 06:41:35 GMT ETag: "66a9-57a-d4a385c0" Accept-Ranges: bytes Content-Length: 1402 Vary: Accept-Encoding Set-Cookie: testcookie=helloworld Content-Type: text/html; charset=UTF-8 [root@iris:Active] config curl -i http://10.10.70.110/foo.html|grep -i up % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1402 100 1402 0 0 292k 0 --:--:-- --:--:-- --:--:-- 250k up [root@iris:Active] config b pool foo|grep -i pool\ member +-> POOL MEMBER foo/10.10.70.110:http inactive,down [root@iris:Active] config curl -I http://10.10.70.110/foo.html HTTP/1.1 200 OK Date: Tue, 25 Oct 2011 06:43:03 GMT Server: Apache/2.0.59 (rPath) Last-Modified: Tue, 25 Oct 2011 06:42:57 GMT ETag: "66ae-57c-d986be40" Accept-Ranges: bytes Content-Length: 1404 Vary: Accept-Encoding Set-Cookie: testcookie=helloworld Content-Type: text/html; charset=UTF-8 [root@iris:Active] config curl -i http://10.10.70.110/foo.html|grep -i up % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1404 100 1404 0 0 173k 0 --:--:-- --:--:-- --:--:-- 0 [root@iris:Active] config b pool foo|grep -i pool\ member +-> POOL MEMBER foo/10.10.70.110:http inactive,down - nitassoop sorry. i forgot to say thanks for the byte's explanation. ;-)
- smp_86112
No problem, thank you for all your independent test work. So it seems your threshold is 1695 bytes (1404 content + 291 headers), and mine is 1705 bytes. But in my mind, that's close enough to convince me this is a bug. It's not exactly the situation described in SOL10443, but it's evident there was some type of problem with that part of the TMM code.