HTTPS SNI Monitor

Problem this snippet solves:

Hi,

You may or may not already have encountered a webserver that requires the SNI (Server Name Indication) extension in order to know which website it needs to serve you. It comes down to "if you don't tell me what you want, I'll give you a default website or even simply reset the connection". A typical IIS8.5 will do this, even with the 'Require SNI' checkbox unchecked.

So you have your F5, with its HTTPS monitors. Those monitors do not yet support SNI, as they have no means of specifying the hostname you want to use for SNI.

In comes a litle script, that will do exactly that.

Here's a few quick steps to get you started:

Download the script from this article (it's posted on pastebin: http://pastebin.com/hQWnkbMg, listed below and added as attachment). Import it under 'System' > 'File Management' > 'External Monitor Program File List'. Create a monitor of type 'External' and select the script from the picklist under 'External Program'. Add your specific variables (explanation below). Add the monitor to a pool and you are good to go.

A quick explanation of the variables:

METHOD (GET, POST, HEAD, OPTIONS, etc. - defaults to 'GET') URI ("the part after the hostname" - defaults to '/') HTTPSTATUS (the status code you want to receive from the server - defaults to '200') HOSTNAME (the hostname to be used for SNI and the Host Header - defaults to the IP of the node being targetted) TARGETIP and TARGETPORT (same functionality as the 'alias' fields in the original monitors - defaults to the IP of the node being targetted and port 443) DEBUG (set to 0 for nothing, set to 1 for logs in /var/log/ltm - defaults to '0') RECEIVESTRING (the string that needs to be present in the server response - default is empty, so not checked) HEADERX (replace the X by a number between 1 and 50, the value for this is a valid HTTP header line, i.e. "User-Agent: Mozilla" - no defaults) EXITSTATUS (set to 0 to make the monitor always mark te pool members as up; it's fairly useless, but hey... - defaults to 1)

There is a small thing you need to know though: due to the nature of the openssl binary (more specifically the sclient), we are presented with a "stdin redirection problem". The bottom line is that your F5 cannot be "slow" and by slow I mean that if it requires more than 3 seconds to pipe a string into openssl sclient, the script will always fail. This limit is defined in the variable "monitorstdinsleeptime" and defaults to '3'. You can set it to something else by adding a variable named 'STDINSLEEPTIME' and giving it a value. From my experience, anything above 3 stalls the "F5 script executer", anything below 2 is too fast for openssl to read the request from stdin, effectively sending nothing and thus yielding 'down'. When you enable debugging (DEBUG=1), you can see what I mean for yourself: no more log entries for the script when STDINSLEEPTIME is set too high; always down when you set it too low.

Kind regards,

Thomas Schockaert

Code :

#!/bin/bash
##### Sanity checks
## Strictly IPv4 notation.
# The openssl binary doesn't allow the use of IPv6 notation in the -connect parameter of the s_client subcommand.
# F5 exports the NODE_IP variable, which always contains the IPv6-form, even when the IP address is IPv4.
# Stripping the unwanted part solves this.
monitor_targetip=$(echo "$1" | sed 's/::ffff://')
##

## Binary validation
# Finding all the binaries is paramount for this script to run successfully. These binaries must be found under one or more directories in the PATH variable.
# You can modify the PATH variable under which the monitor executes by explicitly defining it in the Variables section of the monitor definition.
required_programs="openssl logger cat grep egrep awk tr seq sleep"
missing_programs=0
missing_programs_output=""
missing_programs_counter=0
for current_program in $required_programs ; do
    program_path=$(which $current_program)
    if [ $? -eq 0 ] ; then
        eval "$current_program=$program_path"
    else
        output="$missing_programs_output$current_program,"
        let missing_programs_counter=$missing_programs_counter+1
    fi
done

if [ $missing_programs_counter -gt 0 ] ; then
    echo -e "ERROR: An external monitor script failed to locate one or more of its required programs. The script cannot continue unless you fix this." >> /var/log/ltm
echo -e "The program(s) that could not be found are: $missing_programs_output" >> /var/log/ltm
echo -e "The location of these programs needs to be under one the following directories: '$PATH'" >> /var/log/ltm
    exit 1
fi
##
#####




##### Preparations before running the checks
## Setting the default settings
# These are needed in case the creator of the monitor failed to specify all the required variables
monitor_debug=0
monitor_method="GET"
monitor_uri="/"
monitor_httpstatus="200"
monitor_hostname="$monitor_targetip"
monitor_targetport="$NODE_PORT"
monitor_receivestring=""
monitor_header=""
monitor_stdin_sleeptime="3" # this is required to make openssl s_client accept the input from stdin before it closes.
monitor_exitstatus=1
log_monitor_debug_specified=0
log_monitor_method_specified=0
log_monitor_uri_specified=0
log_monitor_httpstatus_specified=0
log_monitor_hostname_specified=0
log_monitor_targetport_specified=0
log_monitor_receivestring_specified=0
log_monitor_header_specified=0
log_monitor_receivestring_match=0
log_monitor_httpstatus_match=0
##

## Overriding the default settings if needed
# This part loops through the possible variables and checks if they have been defined.
# If one has been defined, it checks if it's not empty and adds it to the actual action-variable ($monitor_something).
monitor_variable_items="method uri httpstatus hostname targetip targetport debug receivestring header stdin_sleeptime exitstatus"
for current_monitor_variable_item in $monitor_variable_items ; do
current_monitor_variable_name_for_usage="monitor_${current_monitor_variable_item}"
current_monitor_variable_name_for_logging="log_${current_monitor_variable_name_for_usage}_specified"
if [ "$current_monitor_variable_item" == "header" ] ; then
tmp=""
for i in `$seq 1 50` ; do
current_monitor_variable_name_for_input="$(echo "$current_monitor_variable_item" | $tr 'a-z' 'A-Z')$i"
eval "current_monitor_variable_value_for_input=\$$current_monitor_variable_name_for_input"
if ! [ "$current_monitor_variable_value_for_input" == "" ] ; then
if [ $i -eq 1 ] ; then
tmp="${current_monitor_variable_value_for_input}"
else
tmp="${tmp}\r\n${current_monitor_variable_value_for_input}"
fi
fi
unset current_monitor_variable_name_for_input
done
eval "$current_monitor_variable_name_for_usage=\"$tmp\""
eval "$current_monitor_variable_name_for_logging=1"
else
current_monitor_variable_name_for_input=$(echo "$current_monitor_variable_item" | tr 'a-z' 'A-Z')
eval "current_monitor_variable_value_for_input=\$$current_monitor_variable_name_for_input"
eval "current_monitor_variable_value_for_usage=\$$current_monitor_variable_name_for_usage"
if ! [ "$current_monitor_variable_value_for_input" == "" ] ; then
eval "$current_monitor_variable_name_for_usage=$current_monitor_variable_value_for_input"
eval "$current_monitor_variable_name_for_logging=1"
fi
fi
unset tmp current_monitor_variable_name_for_usage current_monitor_variable_name_for_logging current_monitor_variable_name_for_input
done
###
#####



##### Running the checks
## Obtaining the HTTP content through openssl
http_content=`(echo -e "$monitor_method $monitor_uri HTTP/1.1\r\nHost: $monitor_hostname\r\n${monitor_header}\r"; $sleep $monitor_stdin_sleeptime) | $openssl s_client -connect $monitor_targetip:$monitor_targetport -servername $monitor_hostname`
##

## Obtaining the HTTP Status code from the returned contents
http_error_code=$(echo "$http_content" | $egrep "HTTP/1\.[0-1] " | $awk '{print $2}')
##

## Determining the 'up' or 'down' status
if [ "$http_error_code" == "$monitor_httpstatus" ] ; then
log_monitor_httpstatus_match=1
if ! [ "$monitor_receivestring" == "" ] ; then
receive_string_check=$(echo "$http_content" | $grep "$monitor_receivestring")
if ! [ "$receive_string_check" == "" ] ; then
log_monitor_receivestring_match=1
monitor_exitstatus=0
fi
else
monitor_exitstatus=0
fi
fi
##
#####




##### Supplying the debug logs if requested
## Dump each variable
if [ "$monitor_debug" == "1" ] ; then
this_run=$(date +%s)
log_prefix="HTTPS_SNI [$this_run]:"
echo "" | $logger -p local0.debug
echo "$log_prefix Monitor Description:" | $logger -p local0.debug
for current_monitor_variable_item in $monitor_variable_items ; do
current_monitor_variable_name_for_usage="monitor_${current_monitor_variable_item}"
current_monitor_variable_name_for_logging="log_${current_monitor_variable_name_for_usage}_specified"
eval "current_monitor_variable_value_for_usage=\"\$$current_monitor_variable_name_for_usage\""
eval "current_monitor_variable_value_for_logging=\$$current_monitor_variable_name_for_logging"
if [ "$current_monitor_variable_value_for_logging" == "1" ] ; then
echo "$log_prefix - $current_monitor_variable_name_for_usage => $current_monitor_variable_value_for_usage" | $logger -p local0.debug
else
echo "$log_prefix - $current_monitor_variable_name_for_usage => default ($current_monitor_variable_value_for_usage)" | $logger -p local0.debug
fi
unset current_monitor_variable_name_for_usage current_monitor_variable_name_for_logging
done

echo "$log_prefix Monitor Status:" | $logger -p local0.debug
if [ "$log_monitor_receivestring_match" == "0" ] ; then
echo "$log_prefix - Receive String: no match" | $logger -p local0.debug
else
echo "$log_prefix - Receive String: match" | $logger -p local0.debug
fi

if [ "$log_monitor_httpstatus_match" == "0" ] ; then
echo "$log_prefix - HTTP Status: no match" | $logger -p local0.debug
else
echo "$log_prefix - Receive String: match" | $logger -p local0.debug
fi

if [ "$monitor_exitstatus" == "0" ] ; then
echo "$log_prefix - Final Result: UP" | $logger -p local0.debug
else
echo "$log_prefix - Final Result: DOWN" | $logger -p local0.debug
fi
echo "" | logger -p local0.debug
echo "" | logger -p local0.debug
fi
##
#####




##### Telling F5 the monitor status should be up
## Echoing 'up' to stdout
if [ "$monitor_exitstatus" == "0" ] ; then
echo "up"
fi
##

## Exiting accordingly (0 or 1)
exit $monitor_exitstatus
##
#####
Published Mar 12, 2015
Version 1.0
monitor

Was this article helpful?

6 Comments

Sort By
  • Paolo_Di_Liber1's avatar
    Paolo_Di_Liber1
    Icon for Employee rankEmployee
    Dec 11, 2015
    Hi, this is really good, thanks! Due to a customer request, I did a little change on the script, allowing the user to use regexp to identity response codes. It is really basic, but it works. It is enough to change line 113 to: 
     if echo "$http_error_code" | egrep -q "$monitor_httpstatus"; then
    You could assign a regexp to HTTPSTATUS (es. [2,3]0[0-3]) Thanks.
  • RossVermette_14's avatar
    RossVermette_14
    Icon for Nimbostratus rankNimbostratus
    Jun 14, 2017

    FYI: I really like the script but had to make a few tweaks to make it work for me.

    In line 13: added "sed" into the required_programs.

    In line 109: Changed the "http_error_code" to use sed instead of awk, as the "http_content" was being returned into a single line, and the "$awk '{print $2}'" statement would only print the second word of the line. The line looks like this:

    http_error_code=$(echo "$http_content" | $egrep "HTTP/1\.[0-1] " | $sed 's/.*HTTP\/1.[0-1] \([0-9][0-9][0-0]\).*/\1/')
  • rst's avatar
    rst
    Icon for Nimbostratus rankNimbostratus
    Jul 12, 2017

    Great script. Thanks! Just a little thing I noticed while looking at debug output: In line 161 it should probably read "HTTP Status" not "Receive String".

     

  • RossVermette_14's avatar
    RossVermette_14
    Icon for Nimbostratus rankNimbostratus
    Jun 06, 2018

    Hey Thomas, Looking at line 105, and the "stdin redirection problem into openssl timing issue". What if you changed the command a little and used a "timeout" command with openssl and option of -ign_eof?

    $http_content=`echo -e "$monitor_method $monitor_uri HTTP/1.1\r\nHost: $monitor_hostname\r\nConnection: close\r\n${monitor_header}\r" | timeout $monitor_stdin_sleeptime $openssl s_client -connect $monitor_targetip:$monitor_targetport -servername $monitor_hostname -ign_eof`
  • Nikoolayy1's avatar
    Nikoolayy1
    Icon for MVP rankMVP
    Oct 17, 2019

    I just noticed that it is "$sleep" , shouldn't it be just "sleep" as this is a command ? Also this a linked article to this one:

     

    https://devcentral.f5.com/s/articles/https-sni-monitoring-how-to?t=1571326566783

  • norman_lee's avatar
    norman_lee
    Icon for Altostratus rankAltostratus
    May 06, 2020

    Works great on an older GTM! debug log still goes to /var/log/ltm -- which is fine, just thought I'd mention it for others.

     

    Thanks for creating this!