BIGIP LTM Automated Pool Monitor Flap Troubleshooting Script in Bash

Problem this snippet solves:

A bash script is mainly for collecting data when F5 BIG-IP LTM pool member monitor flaps in a period of time and help determine the Root Cause of BIGIP monitor health check failure; Script will monitor the LTM logs, if new pool member down message event occurs, script will perform following functions: 1. Turn on LTM bigd debug ; 2. Start to tcpdump capture to capture relevant traffics; 3. Turn off bigd debug and terminate tcpdump process when timer elapse (timer is configurable) 4. Generate qkview (optinal) 5. Tar ball full logs files under /var/log/ directory (optinal)

Script has been tested on v11.x

Code :

#!/usr/bin/bash
##########identify the log file that script is monitoring 
filename="/var/log/ltm"
##########identify the period of time that debug and tcpdump are running, please change it according to the needs;
timer=60
##########IP address of pool member flaps 
poolMemberIP="10.10.10.229"
##########self IP address of LTM is usd to send LTM Health Monitor traffics
ltmSelfip="10.10.10.248"
##########pool member service port number
poolMemberPort="443"
##########TMOS command to turn on bigd debug 
turnonBigdDebug="tmsh modify sys db bigd.debug value enable"
##########TMOS command to turn off bigd debug 
turnoffBigdDebug="tmsh modify sys db bigd.debug value disable"
##########BASH command to tar BIGIP log files 
tarLogs="tar -czpf /var/tmp/logfiles.tar.gz /var/log/*"



####### function file check: following code will check if /var/log/ltm exist on the system, 
####### if it exists, script will be running and perform subsequent functions 
if [ -f $filename ]
then
      echo "/var/log/ltm exists and program is running to collect data when BG-IP pool member flaps"
else
####### if it does not exist, programe will be terminated and log following message 
      echo "no /var/log/ltm file found and program is terminated"
exit 0
fi
####### function file check ends

###### write timestap to /var/log/ltm for tracking purpose
echo "$(date) monitoring the log" >> $filename

###### start to monitor the /var/log/ltm for new events 
tail -f -n 0 $filename | while read -r line
do

###### counter for pool down message appears
hit=$(echo "$line" | grep -c "$poolMemberIP:$poolMemberPort monitor status down")

#echo $hit
###### 
if [ "$hit" == "1" ];
   then
###### diplay the pool down log event in file /var/log/ltm  
   echo $line
###### show timestamp of debug is on 
   echo "$(date) Turning on system bigddebug"
###### turn on bigd debug 
   echo $($turnonBigdDebug)
###### turn on tcpdump capture 
   echo $(tcpdump -ni 0.0:nnn -s0 -w /var/tmp/Monitor.pcap port $poolMemberPort and \(host $poolMemberIP and host $ltmSelfip\)) &
###### running timer 
   sleep $timer
###### show timestamp of debug is off 
   echo "$(date) Truning off system bigddebug"
###### turn off bigd debug 
   echo $($turnoffBigdDebug)
###### terminate tcpdump process 
   echo $(killall tcpdump)
###### generate qkview, it's an optional function, enable it by remove "#" sign
   #echo $(qkview)
###### tar log files, it's an optional function, enable it by remove "#" sign
   #echo $($tarLogs)
   break

#else
    #echo "Monitor in progress"
fi
done
###### show message that programe is end 
echo "$(date) exiting from programe"

###### exit from the program 
exit 0

Tested this on version: