Forum Discussion
HTTPS Monitor not working as expected
I have created an HTTPS monitor as follows:
GET /idauth/isAlive.jsp\r\n
The receive string is set as:
ALIVE
There are 2 servers and when i curl them i get the following:
First Server
curl -k https://server1.domain.com:8443/idauth/isAlive.jsp
OpenAM
Server is DOWN
Second server:
curl -k https://server2.domain.com:8443/idauth/isAlive.jsp
OpenAM
Server is ALIVE:
However. Both servers are being flagged as unavailable.
If I change the receive string to DOWN - both servers are flagged as available.
I have changed curl to verbose to check for separate words showing ALIVE/DOWN but this doesnt show anything:
curl -k -v https://server2.domain.com:8443/idauth/isAlive.jsp
* About to connect() to server2.domain.com port 8443 (0)
* Trying 172.31.151.143... connected
* Connected to server2.domain.com (172.31.151.143) port 8443 (0)
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES128-SHA
* Server certificate:
* subject: C=GB; ST=Scotland; O=STANDARD LIFE; OU=IS; CN=server2.domain.com
* start date: 2014-11-11 21:48:09 GMT
* expire date: 2028-07-20 21:48:09 GMT
* common name: server2.domain.com (matched)
* SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
GET /idauth/isAlive.jsp HTTP/1.1
User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8y zlib/1.2.3 libidn/0.6.5
Host: server2.domain.com:8443
Accept: */*
HTTP/1.1 200 OK
Cache-Control: private
Expires: Thu, 01 Jan 1970 01:00:00 GMT
Set-Cookie: JSESSIONID=A847EA20D06FC26A9B944B9CB5CC397E; Path=/idauth/; Secure; HttpOnly
Content-Type: text/html;charset=ISO-8859-1
Content-Length: 115
Date: Fri, 06 Feb 2015 13:17:35 GMT
Server: Anonymous
html>
head>
OpenAM
/head>
body>
h1>Server is ALIVE:
* Connection 0 to host server2.domain.com left intact
* Closing connection 0
* SSLv3, TLS alert, Client hello (1):
Has anyone seen anything like this?
27 Replies
- Michael_Jenkins
Cirrostratus
Not sure if it'll help, but since it looks like you're using a non-standard port (8443), you may try to set a Client Certificate and Client Key in your monitor definition (change the drop down from Basic to Advanced to see the options).
Also, maybe try a different send string, like
GET /idauth/isAlive.jsp HTTP/1.1\r\nHost: domain.com\r\nConnection: Close\r\n\r\n
If that doesn't work, here's how I've done some monitor debugging in the past (taken from this answer)
There's SOL12531 that talks about monitors a little bit, and then this article that adds a little extra information.
Whenever I'm having monitor issues, I try a couple different things.
I try to do a1:
from the F5 CLI to verify that the url I'm trying to monitor is working properly and the response is coming back as expected.curl
Turn on monitor debugging for the member and check the log that gets created. Those steps are as follows:2:
- Open the pool that is having the problem (let's call it MY_POOL)
- Make sure the monitor is active on the pool (let's call it MY_MONITOR)
- On the
tab in the pool settings, click on one of the members (let's call it 10.0.0.1:3389)Members
-
Enable the checkbox for
Monitor Logging
- Open an SSH session (I use Putty to connect) to the device and log in to the shell.
- Run the command
cd /var/log/monitors
- Run the command
ls -l
- Look for the file referencing your pool and member (most likely in our example it'd be something like
)_Common_MY_MONITOR__Common_10.0.0.1_3389.log
- Run the command
(where FILENAME is the log filename)tail -f FILENAME
- It may be visible immediately, but you should see where the monitor sends the request to the server (and you'll see the
string), and you're looking for thesend
string that comes back.recv
- Take that
string and update the monitor and see if the pool member(s) show up green again.recv
-
Go back in and disable the monitor logging for that member so you don't have excessive logging filling up space.
Hopefully, this will help figure it out. If your log file ever looks like it's not updating, delete the log file and remove the monitor from the pool and re-add it. I've found that will kick it back off again.
- NikhilB
Employee
LyonsG, unless I am missing something, I don't see an "Alive" string from the first server. The monitor should mark this down as expected.
Can you also try running this monitor separately on each pool member instead of at the pool level?
- LyonsG_85618
Cirrostratus
Nikhil. Thats correct 1 server is ALIVE and the other is DOWN. The problem is they are both either marked up or diown when it shoudl be only 1 or the other. - NikhilB
Employee
>>>Can you also try running this monitor separately on each pool member instead of at the pool level? What are results? - LyonsG_85618
Cirrostratus
monitor on each pool member returns same results
- NikhilB_149913Historic F5 Account
LyonsG, unless I am missing something, I don't see an "Alive" string from the first server. The monitor should mark this down as expected.
Can you also try running this monitor separately on each pool member instead of at the pool level?
- LyonsG_85618
Cirrostratus
Nikhil. Thats correct 1 server is ALIVE and the other is DOWN. The problem is they are both either marked up or diown when it shoudl be only 1 or the other. - NikhilB_149913Historic F5 Account>>>Can you also try running this monitor separately on each pool member instead of at the pool level? What are results?
- LyonsG_85618
Cirrostratus
monitor on each pool member returns same results
- Amit_Karnik
Nimbostratus
-
When you use curl -k, the server certificate is being accepted. Make sure the monitor will also accept the server certificate if the certificate is signed by an Internal CA for example.
-
Check if your server responds to specific "Host:" header or "User-Agent:" header. Try setting both these headers to what curl sets.
Best.
-
- shaggy_121467
Cumulonimbus
have you tried the curl tests with the node IP address instead of the hostname? curl will automatically set the host header to the hostname entered in curl, which technically makes the curl test different than the f5's send string.
you could also use:
to open an SSL connection from the f5 to the pool member, and then paste your send string. the more data points, the better.openssl s_client -connect member-ip:port
- LyonsG_85618
Cirrostratus
curl testing results are now more consistent - but still not getting appropriate results from monitor. tcpdump being done!
- shaggy
Nimbostratus
have you tried the curl tests with the node IP address instead of the hostname? curl will automatically set the host header to the hostname entered in curl, which technically makes the curl test different than the f5's send string.
you could also use:
to open an SSL connection from the f5 to the pool member, and then paste your send string. the more data points, the better.openssl s_client -connect member-ip:port
- LyonsG_85618
Cirrostratus
curl testing results are now more consistent - but still not getting appropriate results from monitor. tcpdump being done!
- LyonsG_85618
Cirrostratus
[admin@bigip:Active:Changes Pending] ~ curl -k https://172.31.151.143:8443/idauth/isAlive.jsp
OpenAM Server is ALIVE:
[admin@bigip:Active:Changes Pending] ~ curl -k https://172.31.251.141:8443/idauth/isAlive.jsp
OpenAM Server is DOWN
Monitor looks for ALIVE and both servers get flagged as available.
Change monitor to DOWN and both servers get flagged as unavailable.
- shaggy
Nimbostratus
please run curl again with the -vv flags (-kvv)
- LyonsG_85618
Cirrostratus
[admin@bigip:Active:Changes Pending] ~ curl -k -vv https://172.31.251.141:8443/idauth/isAlive.jsp
* About to connect() to 172.31.251.141 port 8443 (0) * Trying 172.31.251.141... connected * Connected to 172.31.251.141 (172.31.251.141) port 8443 (0) * successfully set certificate verify locations: * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * SSLv3, TLS handshake, Client hello (1): * SSLv3, TLS handshake, Server hello (2): * SSLv3, TLS handshake, CERT (11): * SSLv3, TLS handshake, Server finished (14): * SSLv3, TLS handshake, Client key exchange (16): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSL connection using AES128-SHA * Server certificate: * subject: C=GB; ST=Scotland; O=company; OU=IS; CN=slglvmfi.internal.standardlife.com * start date: 2014-11-12 11:42:01 GMT * expire date: 2028-07-21 11:42:01 GMT * common name: slglvmfi (does not match '172.31.251.141') * SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway. > GET /idauth/isAlive.jsp HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8y zlib/1.2.3 libidn/0.6.5 > Host: 172.31.251.141:8443 > Accept: */* > < HTTP/1.1 401 Unauthorized < Cache-Control: private < Expires: Thu, 01 Jan 1970 01:00:00 GMT < Set-Cookie: JSESSIONID=0785483496A1B74384AB4DA3E287A91F; Path=/idauth/; Secure; HttpOnly < Content-Type: text/html;charset=ISO-8859-1 < Content-Length: 112 < Date: Mon, 09 Feb 2015 16:05:12 GMT < Server: Anonymous < OpenAM Server is DOWN * Connection 0 to host 172.31.251.141 left intact * Closing connection 0 * SSLv3, TLS alert, Client hello (1):
- LyonsG_85618
Cirrostratus
[admin@bigip:Active:Changes Pending] ~ curl -k -vv https://172.31.151.143:8443/idauth/isAlive.jsp
* About to connect() to 172.31.151.143 port 8443 (0) * Trying 172.31.151.143... connected * Connected to 172.31.151.143 (172.31.151.143) port 8443 (0) * successfully set certificate verify locations: * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * SSLv3, TLS handshake, Client hello (1): * SSLv3, TLS handshake, Server hello (2): * SSLv3, TLS handshake, CERT (11): * SSLv3, TLS handshake, Server finished (14): * SSLv3, TLS handshake, Client key exchange (16): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSL connection using AES128-SHA * Server certificate: * subject: C=GB; ST=Scotland; O=; OU=IS; CN=slglvmfk * start date: 2014-11-11 21:48:09 GMT * expire date: 2028-07-20 21:48:09 GMT * common name: slglvmfk (does not match '172.31.151.143') * SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway. > GET /idauth/isAlive.jsp HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8y zlib/1.2.3 libidn/0.6.5 > Host: 172.31.151.143:8443 > Accept: */* > < HTTP/1.1 200 OK < Cache-Control: private < Expires: Thu, 01 Jan 1970 01:00:00 GMT < Set-Cookie: JSESSIONID=2EB50E91CD9C6DAB9298DF12C3FA6EE8; Path=/idauth/; Secure; HttpOnly < Content-Type: text/html;charset=ISO-8859-1 < Content-Length: 115 < Date: Mon, 09 Feb 2015 16:08:37 GMT < Server: Anonymous < OpenAM Server is ALIVE: * Connection 0 to host 172.31.151.143 left intact * Closing connection 0 * SSLv3, TLS alert, Client hello (1):
- LyonsG_85618
Cirrostratus
Also - I am on version 11.4.1 so cannot enable monitor debugging via GUI
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com