F5 is upgrading its customer support chat feature on My.F5.com. Chat support will be unavailable from 6am-10am PST on 1/20/26. Refer to K000159584 for details.

Forum Discussion

gprabhu_133561's avatar
gprabhu_133561
Icon for Nimbostratus rankNimbostratus
Mar 27, 2014

How can I prevent, the health monitor from disabling all pool members?

Hi,

 

How can I prevent, the health monitor from disabling all pool members? In high load when member threads are busy the http monitor marks all members down.

 

Any suggestion or guidance is much appreciated.

 

Thanks, -Gopi

 

4 Replies

  • nathe's avatar
    nathe
    Icon for Cirrocumulus rankCirrocumulus

    One quick option to try is increasing the interval/timeout on the monitor itself. If you're happy that a pool member is still marked up even if it is slow to respond then increasing these options could help.

     

    N

     

  • Hi Nathan,

     

    Thanks ... I got same suggestion from my F5 call as well. It dose not solve my problem. Now I started looking into external monitor.

     

    I appreciate your help. Thanks again, -Gopi

     

  • JG's avatar
    JG
    Icon for Cumulonimbus rankCumulonimbus

    Not sure what benefit the monitor you are after will bring. Isn't the monitor doing what it's supposed to do, i.e. marking down (disallowing new connections to) a server/service that the user cannot use anyway?

     

    Or you can simply disable the monitor altogether.

     

  • I was able to write a custom external health monitor which checks before disabling a pool member. But this monitor spikes the F5 CPU by 50%. Here is my script...any suggestions to reduce the cpu usage are welcome.

    !/bin/sh Arguments required are Debug optional, pool name, uri & return string to look for

    DEBUG=0

    Verify DEBUG value

    if [ -z "$DEBUG" ]

    then

    DEBUG=0

    fi

    if [ ! "$POOL_NAME" ] || [ ! "$URI" ] || [ ! "$STR_TO_LOOK" ]

    then

    echo "Missing value for either POOL_NAME,URI or STR_TO_LOOK" | logger -p local0.debug

    exit

    fi

    if [ $DEBUG -eq 1 ]; then echo "The script is working! " | logger -p local0.debug;fi

    IP=

    echo $1 | sed 's/::ffff://'

    PORT=$2

    POOL_NAME=$POOL_NAME

    URI=$URI

    STR_TO_LOOK=$STR_TO_LOOK

    NODENAME=

    echo y | /usr/bin/tmsh list /ltm node |grep -B 1 $IP |grep "ltm node"|awk '{print $3}'

    PIDFILE="/var/run/

    basename $0
    .$IP.$PORT.pid"

    TOTAL_MEMBERS=

    /usr/bin/tmsh show /ltm pool $POOL_NAME members all-properties | grep -c "Pool Member"

    BLACKLIST_LIMIT=$(($TOTAL_MEMBERS/2))

    FILENAME=/tmp/$POOL_NAME.log

    if [ $DEBUG -eq 1 ]; then echo "Pool:$POOL_NAME, Uri:$URI, StringToLook: $STR_TO_LOOK, NodeName:$NODENAME,Tm:$TOTAL_MEMBERS,Blist:$BLACKLIST_LIMIT,FileName:$FILENAME" | logger -p local0.error; fi

    if [ -f $PIDFILE ] then

    if [ $DEBUG -eq 1 ]; then echo "EAV

    basename $0
    ($$): pid file is present, killing process..." | logger -p local0.debug; fi

    kill -9

    cat $PIDFILE
    > /dev/null 2>&1

    echo "EAV

    basename $0
    ($$): exceeded monitor interval, needed to kill ${IP}:${PORT} with PID
    cat $PIDFILE
    " | logger -p local0.error

    fi

    curl -fNs http://${IP}:${PORT}${URI} | grep -i "${STR_TO_LOOK}" 2>&1 > /dev/null

    RCURL=$?

    if [ $DEBUG -eq 1 ]; then echo "curl -fNs http://${IP}:${PORT}${URI} | grep -i "${STR_TO_LOOK}" 2>&1 > /dev/null curl_resonse:$RCURL " | logger -p local0.debug;fi

    if [ $RCURL -eq 0 ]

    then

     /usr/bin/tmsh list ltm pool $POOL_NAME members {$NODENAME:$PORT} | grep -E '(user-disabled|down)' 2>&1 > /dev/null
    
     RD=$?
    
     if [ $DEBUG -eq 1 ]; then echo "$POOL_NAME:$NODENAME Response value from (user-disabled|down) check: $RD" | logger -p local0.debug;fi
    
     if [ $RD -eq 0 ]
    
     then        
    
         /usr/bin/tmsh modify ltm pool $POOL_NAME members modify {$NODENAME:$PORT { session user-enabled}} 2>&1 > /dev/null          
    
         (
              flock -n 9 || exit 1
    
              AVALIABLE_MEMBERS=`/usr/bin/tmsh show /ltm pool $POOL_NAME members all-properties | grep -i "State" | awk {'print $NF'} |grep -ic "enabled"` 
    
              let "AVALIABLE_MEMBERS-=1" 
    
              echo $AVALIABLE_MEMBERS > $FILENAME               
    
           ) 9> $FILENAME  
    
              AVALIABLE_MEMBERS=`cat $FILENAME | awk '{print $1}'`
    
         if [ $DEBUG -eq 1 ]; then echo "$POOL_NAME:$NODENAME Response value from (Session user-enabled): $?" | logger -p local0.debug;fi   
    
     fi
    
     rm -f $PIDFILE
    
    Any value to standardOut will enable the member
    
    echo "UP"
    
    
    

    else

    if [ $TOTAL_MEMBERS -gt 1 ] 
    
    then    
    
    if [ $DEBUG -eq 1 ]; then echo "/usr/bin/tmsh show /ltm pool $POOL_NAME members all-properties | grep -i "State" | awk {'print $NF'} |grep -ic "enabled" " | logger -p local0.debug;fi
    
       (
              flock -n 9 || exit 1
    
              AVALIABLE_MEMBERS=`/usr/bin/tmsh show /ltm pool $POOL_NAME members all-properties | grep -i "State" | awk {'print $NF'} |grep -ic "enabled"` 
    
              let "AVALIABLE_MEMBERS-=1" 
    
              echo $AVALIABLE_MEMBERS > $FILENAME   
    
             ) 9> $FILENAME   
    
        AVALIABLE_MEMBERS=`cat $FILENAME | awk '{print $1}'`
    
      if [ $DEBUG -eq 1 ]; then echo "Pool $POOL_NAME details:$NODENAME Total no.of pool members:$TOTAL_MEMBERS, Avaliable no.of members:$AVALIABLE_MEMBERS,Number of members can be black listed:$BLACKLIST_LIMIT" | logger -p local0.debug;fi
    
      if [ $DEBUG -eq 2 ]; then echo " $NODENAME - Avaliable:$AVALIABLE_MEMBERS - Blacklist:$BLACKLIST_LIMIT" | logger -p local0.debug;fi
    
       if [ $AVALIABLE_MEMBERS -gt $BLACKLIST_LIMIT ]  
    
       then
    
            /usr/bin/tmsh modify ltm pool $POOL_NAME members modify {$NODENAME:$PORT { session user-disabled }} 2>&1 > /dev/null  
    
            if [ $DEBUG -eq 1 ]; then echo "$POOL_NAME:$NODENAME Response value from (session user-disabled) check: $?" | logger -p local0.debug;fi 
    
       fi 
    
    fi
    
    rm -f $PIDFILE
    
    echo "Down" 
    
    if [ $DEBUG -eq 1 ]; then echo "End of the Script" | logger -p local0.debug;fi
    
    exit           
    

    fi

    END