HTTPMonitor_cURL_ProbeTimeout

OK, I see where it's going sideways now.

CAUCHOPORT is not defined anywhere in your monitor, and without that value, the cURL call is failing, and so no request is being sent to the pool member.

If you just change the name of the PORT var back to CAUCHOPORT in the monitor definition, you should be up & running to mark pool members UP.

However, to mark pool members DOWN, you'll need to make another change: Remove the alias destination port from the monitor definition. The script must be passed the pool member's defined port in order to mark it DOWN using the bigpipe command.

So your monitor def would look like this:

monitor curlcaucho {
  defaults from external
  run "curlcaucho"
  POOL "cwmp.stage.80"
  CAUCHOPORT "4004"
  P_TIMEOUT "5"
  RECV "w50-`echo $IP | cut -d. -f4`.c50.domain.com:6802 (ok)"
  URI "/caucho-status"
}

Give that a try.

/deb

I have always had CAUCHOPORT in the template but I took it out by mistake adding it makes no difference

 

Removing the alias destination port from the monitor definition does not make the script work either

 

b monitor curlcaucho list

 

monitor curlcaucho {

 

defaults from external

 

CAUCHOPORT "4000"

 

P_TIMEOUT "5"

 

RECV "w50-`echo $IP | cut -d. -f4`.c50.domain.com:6802 (ok)"

 

run "curlcaucho"

 

URI "/caucho-status"

 

}

 

Now you are missing the POOL variable from your monitor definition.

The basic external monitor troubleshooting process is:

1) Verify the script is functioning as expected.

From the command line, export the 5 variables and run the script with the 2 expected arguments (IP & port of a healthy pool member).

  export POOL="cwmp.stage.80"
  export CAUCHOPORT="4004"
  export P_TIMEOUT="5"
  export RECV="w50-`echo $IP | cut -d. -f4`.c50.domain.com:6802 (ok)"
  export URI="/caucho-status"
  /usr/bin/monitors/curlcaucho 10.50.1.1 5004

If the script output is UP, the script is functioning as expected.

If there is no output or some other output, run under sh -x and correct all errors until expected output results.

  sh -x /usr/bin/monitors/curlcaucho 10.50.1.1 5004

tcpdump may come in handy to verify the expected request & response on the wire. Filter on the IP and port to which the monitor traffic will be sent. (Note that in this case the port to which the monitor traffic is sent is different from the port defined for the pool member.)

tcpdump -nni internal -Xs 0 host 10.50.1.1 and port 5004

You can further filter to include both the internal self-IP and the pool member's IP to eliminate load balanced traffic:

tcpdump -nni internal -Xs 0 host 10.50.1.1 and host 10.50.0.246 and port 5004

2) Define the monitor in the LTM configuration and apply to a single healthy pool member.

monitor curlcaucho {
  defaults from external
  run "curlcaucho"
  POOL "cwmp.stage.80"
  CAUCHOPORT "4004"
  P_TIMEOUT "5"
  RECV "w50-`echo $IP | cut -d. -f4`.c50.domain.com:6802 (ok)"
  URI "/caucho-status"
}

If the pool member doesn't appear as UP within the timeout period, use the same tcpdump command you used before to see whether the expected request and/or response is seen. It may be helpful to compare output from step 1 with output from step 2 to figure out what is different.

HTH

/deb

I have added the POOL variable to the GUI template but I don't think it is necessary because it is already in the script

 

It has made no difference

 

Please see post on 01.16

 

I have already done what you have asked.

 

It works manually but no packets are recieved via tcpdump while pool monitor is applied in the GUI

 

4004 is the PORT

 

4000 is the CAUCHOPORT

 

monitors tcpdump -nni internal -Xs 0 host 10.50.1.1 and host 10.50.0.246 and port 4004

 

tcpdump: listening on internal

 

 

0 packets received by filter

 

0 packets dropped by kernel

 

 

tcpdump -nni internal -Xs 0 host 10.50.1.1 and host 10.50.0.246 and port 4000

 

tcpdump: listening on internal

 

 

0 packets received by filter

 

0 packets dropped by kernel

hmmmm....

I've tested the script & monitor definition, and it works as expected on my 9.4.3 and 9.1.2 systems. If you are not seeing traffic in a trace from the monitor, but you are when you run it at the command line, there's something wrong with your monitor definition that is preventing the script from sending the request.

Here is the body of the script I tested:

IP=`echo ${1} | sed 's/::ffff://'`
PORT=${2}
PIDFILE="/var/run/`basename ${0}`.${IP}_${PORT}.pid"
if [ -f $PIDFILE ]
then
   kill -9 `cat $PIDFILE` > /dev/null 2>&1
fi
echo "$$" > $PIDFILE
curl -fNs -m ${P_TIMEOUT} http://${IP}:${CAUCHOPORT}${URI} | grep -i "${RECV}" 2>&1 > /dev/null
if [ $? -eq 0 ]
then
    echo "UP"
else
    /bin/bigpipe pool ${POOL} member ${IP}:${PORT} down 2>&1 > /dev/null
fi
rm -f $PIDFILE
exit

and my monitor definition (slightly different from yours to accommodate my server setup):

monitor caucho {
   defaults from external
   run "caucho"
   CAUCHOPORT "8080"
   POOL "myPool"
   P_TIMEOUT "5"
   RECV "Looks Good To Me"
   URI "/test.html"
}

/deb

While it has no bearing on the issue of why no traffic is sent with your current configuration, I did see one anomaly with getting the node to go back to UP once the monitor changed the status to FORCEDOWN. I've modified the code to force a DOWN pool member back up if it was previously forced DOWN and is now ready to be marked back up.

 

 

Here's the script body refelecting that change for your version (codeshare has been updated as well):

 

IP=`echo ${1} | sed 's/::ffff://'`
PORT=${2}
PIDFILE="/var/run/`basename ${0}`.${IP}_${PORT}.pid"
if [ -f $PIDFILE ]
then
   kill -9 `cat $PIDFILE` > /dev/null 2>&1
fi
echo "$$" > $PIDFILE
curl -fNs -m ${P_TIMEOUT} http://${IP}:${CAUCHOPORT}${URI} | grep -i "${RECV}" 2>&1 > /dev/null
if [ $? -eq 0 ]
then
    /bin/bigpipe pool myPool member 10.10.10.100:8080 show | grep "forced down" 2>&1 > /dev/null
    if [ $? -eq 0 ]
    then
        /bin/bigpipe pool ${POOL} member ${IP}:${PORT} up 2>&1 > /dev/null
    fi
    echo "UP"
else
    /bin/bigpipe pool ${POOL} member ${IP}:${PORT} down 2>&1 > /dev/null
fi
rm -f $PIDFILE
exit

 

 

/deb

I would not recommend stripping the ::ffff from the IP address. This only works for route domain %0.

You are better off just using the IPv6 address directly as supplied.

Below is a version of the curl command which will work irrespective of the route domain the node is in. I have added the -g switch to prevent globbing, and put {$IP} in square brackets. You would need to remove the line from the script which removes the ::fff: from $IP

curl -fNsg -m ${P_TIMEOUT} http://[${IP}]:${CAUCHOPORT}${URI} | grep -i "${RECV}" 2>&1 > /dev/null

Note that the three bigpipe commands after curl will not work outside of route domain %0 either, but that's not so simple to fix.