Forum Discussion
Multiple external health monitors do not work except one. Any alternative?
I need some help from DevCentral F5 expert regarding external monitors.
In our organization, I created a bash script file, which will be associated with 4 different external monitors. Basically the script makes SSH connection to a backend server, grasp the status string, close SSH, and reporting the status to bigd.
The issue is only one external monitor out of 4 is called correctly. All the others are not called at all.
BIG-IP version: 12.1.3.3
The setup is like below: -One external bash file (Pasted below) -There are 4 external health monitors associated with the same external bash file
Issues identified -It seems bigd(A process manages health monitors) doesn’t call all external health monitors (In a bash script, it logs its status to an external file, not STDOUT) oOne external monitor is called correctly all the time. oAll the other three are never be called (No logging to external file)
I am using an external monitor because this is only method to properly handle complex backend server health status.
Now I wonder using external monitor is bad idea. If using external monitor is not a good choice of checking complex backend server health status what would be its alternative?
Thank you.
Here is my bash code.
!/bin/bash
Modified F5 DNS External Health Monitor
Script Name: interactive_ssh_extmon.sh (Interactive SSH External Health Monitor)
About external monitor CLI parameters
interactive_ssh_extmon.sh doesn't need any CLI parameters (param1 and param2 are automatically fed by F5 LTM when used with F5 LTM monitor)
- param1: Pool member IP
- param2: Service Port Number(With this external monitor, service port will be SSH (TCP 22))
- param3: SSH Username
Test Example with shell:
./interactive_ssh_extmon.sh 1.2.3.4 22 ssh_user_name
Test Example with /config/filestore/files_d/Common_d/external_monitor_d/custom_external_monitors
/bin/sh ":Common:interactive_ssh_extmon_35393_ver" 1.2.3.4 22 f5montor
F5 External script directory: /config/filestore/files_d/Common_d/external_monitor_d
Pool member UP/Down log is sent to extmon.log
Log File location: /var/log/extmon.log
How to use this external monitor with F5 LTM (BIG-IP LTM 11.x or later)
Step0 - Prerequisite (Public-Key based auto login)
: Public key needs to be exported to remote system by using the following command
- ssh-copy-id -i /home/user/.ssh/id_rsa.pub username@remote_host_ip
Step1 - Import this external monitor script onto LTM (System->File Management->External Monitor Program File List)
: Choose the script file and assign a name
Step2 - Create a monitor (Local Traffic -> Monitor -> Create)
: General Properties
- Assign a Name
- Choose "External" with Parent Monitor
: Configuration (Advanced)
- Choose the imported external monitor name
Step3 - Associate this external monitor with a pool or a pool member accordingly
If the Node IP address includes IPv6 format, remove IPv6 format.
node_ip=`echo $1 | sed 's/::ffff://'`
echo "`date` ${node_ip}: Script started" >> /var/log/extmon.log
If the number of parameters is not 3, print the usage of the script
if [ "$" -ne 3 ]
then
echo "Usage: interactive_ssh_extmon.sh Pool_member_IP Member_port ssh_user_name"
exit 1
fi
un="$3"
create a pidfile based on calling health monitor
if [ "$DEBUG1" == "1" ];then
pidfile="/var/run/`basename $0`.$node_ip.$2.pid"
elif [ "$DEBUG2" == "1" ];then
pidfile="/var/run/`basename $0`.$node_ip.$2.pid"
elif [ "$DEBUG3" == "1" ];then
pidfile="/var/run/`basename $0`.$node_ip.$2.pid"
elif [ "$DEBUG4" == "1" ];then
pidfile="/var/run/`basename $0`.$node_ip.$2.pid"
else
pidfile="/var/run/`basename $0`.$node_ip.$2.pid"
fi
echo "`date` PIDFILE: ${pidfile} PID: $$" >> /var/log/extmon.log
If pidfile exists (-f: regular file), kill the process associated with the pidfile
if [ -f $pidfile ]
then
kill -9 `cat $pidfile` > /dev/null 2>&1
fi
Save the process ID and use it later to kill the process once everything is done.
echo "$$" > $pidfile
Run SSH command and save the output
MTSstate=`ssh ${un}@${node_ip} -t "bash -l -c 'exit'" 2>/dev/null`
Save the return status of executed command
0 - success, non-zero - failure
status=$?
Command execution should be success. Otherwise exit 1
if [ $status -ne 0 ]; then exit 1; fi
"=~" operator: Consider RHS as Regex expression.
if [[ $MTSstate =~ .*UP.* ]]
then
rm -f $pidfile
echo "`date` ${node_ip}: UP" >> /var/log/extmon.log
echo "UP"
else
rm -f $pidfile
Do Not send any data to STDOUT for the monitor failure.
Failure: No data to stdout
echo "DOWN"
echo "`date` ${node_ip}: DOWN" >> /var/log/extmon.log
fi
exit 0
The issue has been resolved and I am sharing the solution for the people who may need multiple external monitors using SSH connection within their script.
Configuration and Environment
- BIG-IP version: 12.1.3.3 - Use multiple External monitors - Script(External script file) include SSH connection to backend pool member on tcp port 22 Issues identified and its solution - Issue 1: Redirection logging data to a file for the troubleshooting purpose doesn't work well. Solution: Use logger command to send logging data to /var/log/ltm (Credit to Jie. Thanks Jie)echo "`date` ${node_ip}: Script started" | logger -p local0.debug
-Issue 2: From the second SSH connection used in any external health monitors, it doesn't use the Client SSH cipher configuration under /config/ssh/ssh_config.
Ciphers aes128-cbc,aes256-cbc,aes256-ctr,aes128-ctr,aes192-ctr
Instead it starts using only ciphers aes128-cbc,aes256-cbc where the pool members (backend servers) require aes128-ctr,aes192-ctr,aes256-ctr. Because of this, SSH connection fails and finally ssh command returns 255, which says SSH connection failure - Refer to wireshark capture above
-Solution: Specify specific SSH ciphers within ssh command
RET_STATUS=`ssh -c aes128-ctr,aes192-ctr,aes256-ctr ${un}@${node_ip} -t "bash -l -c 'exit'" 2>/dev/null`
Thank you.
- JG
Cumulonimbus
It's highly unlikely, as it is the default config, but did you by any chance change the settings of "Availability Requirement" in the pool configuration to be not "All Health Monitor(s)"?
The issue has been resolved and I am sharing the solution for the people who may need multiple external monitors using SSH connection within their script.
Configuration and Environment
- BIG-IP version: 12.1.3.3 - Use multiple External monitors - Script(External script file) include SSH connection to backend pool member on tcp port 22 Issues identified and its solution - Issue 1: Redirection logging data to a file for the troubleshooting purpose doesn't work well. Solution: Use logger command to send logging data to /var/log/ltm (Credit to Jie. Thanks Jie)echo "`date` ${node_ip}: Script started" | logger -p local0.debug
-Issue 2: From the second SSH connection used in any external health monitors, it doesn't use the Client SSH cipher configuration under /config/ssh/ssh_config.
Ciphers aes128-cbc,aes256-cbc,aes256-ctr,aes128-ctr,aes192-ctr
Instead it starts using only ciphers aes128-cbc,aes256-cbc where the pool members (backend servers) require aes128-ctr,aes192-ctr,aes256-ctr. Because of this, SSH connection fails and finally ssh command returns 255, which says SSH connection failure - Refer to wireshark capture above
-Solution: Specify specific SSH ciphers within ssh command
RET_STATUS=`ssh -c aes128-ctr,aes192-ctr,aes256-ctr ${un}@${node_ip} -t "bash -l -c 'exit'" 2>/dev/null`
Thank you.
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com