Sideband connection HTTP example

Problem this snippet solves:

This iRule demonstrates one way to use the sideband connection functionality added in 11.0.0. It is a proof of concept rule which should be modified to fit the application requirements.

A much-expanded implementation of this concept is available as the HTTP Super SIDEBAND Requestor. You may wish to visit that page if you just want a utility tool to make HTTP requests from your iRules.

The example sends an HTTP request to a sideband destination and parses the response headers and optionally payload to determine which pool to send the client request to. In a production scenario, responses from the sideband servers should be cached in a subtable for better performance.

This functionality replaces HTTP::retry to implement an external HTTP lookup.

The sideband destination can either be an IP:port or more ideally a local virtual server name. The sideband virtual server should point to a pool of lookup servers. There should still be a default pool configured on the main virtual server in case the sideband lookup fails.

It hasn't been tested in production, so make sure to test it yourself too!

How to use this snippet:

Code :

# Sideband connection example iRule
#
# Aaron Hooley - hooleylists at gmail dot com - v0.1 - 2011-08-24
#
# This iRule demonstrates one way to use the sideband connection functionality added in 11.0.0.
#
# The example sends an HTTP request to a sideband destination and parses the response headers and optionally payload
#   to determine which pool to send the client request to.
# This is in lieu of using HTTP::retry to implement an external HTTP lookup.
#
# The sideband destination can either be an IP:port or more ideally a local virtual server name.  
# The sideband virtual server should point to a pool of lookup servers.
#
# There should still be a default pool configured on the main virtual server in case the sideband lookup fails.
#
# See the wiki pages for details on the command options:
#https://devcentral.f5.com/s/wiki/iRules.connect.ashx
#https://devcentral.f5.com/s/wiki/iRules.send.ashx
#https://devcentral.f5.com/s/wiki/iRules.recv.ashx
#https://devcentral.f5.com/s/Tutorials/TechTips/tabid/63/articleType/ArticleView/articleId/1086484/v11-iRules-Intro-to-Sideband-Connections.aspx
#
#
when RULE_INIT {

# Log debug messages to /var/log/ltm? 1=yes, 0=no
set static::sb_debug 1

# The timeout and retries determine how long in total we wait for the sideband server to send
# the response HTTP headers.  If the server fails to respond, try increasing the timeout first
# as increasing the retries will eat up more CPU cycles unnecessarily

# Time (in ms) to wait for the sideband server response HTTP headers
set static::timeout 2

# Number of times to try getting the sideband server response HTTP headers
set static::retries 10

# Local virtual server name to connect to for lookups
# The local virtual server should contain a pool of remote lookup servers
set static::sb_vserver "sideband_dest_vs"
}
when CLIENT_ACCEPTED {

# Save the client IP:port as a log prefix
set log_prefix "[IP::client_addr]:[TCP::client_port]:\t"
if {$static::sb_debug} { log local0. "$log_prefix" }
if {$static::sb_debug} { log local0. "$log_prefix Connection to [virtual name] [IP::local_addr]:[TCP::local_port]" }

# Save the name of the VS default pool in case the sideband connection does not return a valid pool name
set default_pool [LB::server pool]

}
when HTTP_REQUEST {

append log_prefix \t

# Connect to an external host with a connection timeout of 1000ms and an idle timeout of 30 seconds
#set conn [connect -timeout 1000 -idle 30 -status conn_status $static::sb_host:$static::sb_port]
set conn [connect -timeout 1000 -idle 30 -status conn_status $static::sb_vserver]
if {$static::sb_debug} { log local0. "$log_prefix Connect returns: <$conn> and conn status: <$conn_status>" }

# Check if connection was not established to sideband server
if {$conn eq ""}{
if {$static::sb_debug} { log local0. "$log_prefix Connection could not be established " }

# Use the VS default pool
return
}

# Get the current connection status
set conn_info [connect info -idle -status $conn]
if {$static::sb_debug} { log local0. "$log_prefix Connect info: <$conn_info>" }

# Send a TCP payload containing an HTTP request
set data "GET /pool_lookup?host=[HTTP::host] HTTP/1.0\r\n\r\n"
set send_info [send -timeout 3000 -status send_status $conn $data]
if {$static::sb_debug} { log local0. "$log_prefix Sent $send_info bytes, send status: <$send_status>" }

# Continue peeking at the received data until we see the end of the HTTP response
# For a response with a payload, look for a pattern of "anything, two CRLFs which terminate the headers, anything, and then a terminating CRLF" 
#
# Set a cap of 10 loops to wait for to avoid an endless loop
# To receive the HTTP response headers, we wait for up to 10ms on each recv -peek attempt, so the total wait will be up to 100ms
# The number of loops and/or recv -timeout could be extended if the sideband destination is slower
#
# There is a separate timeout on receiving the payload if one is found.
set start [clock clicks -milliseconds]
for {set i 0} {$i <= $static::retries} {incr i}{

# See what data we have received so far on the connection with a 10ms timeout
set recv_data [recv -peek -status peek_status -timeout 10 $conn]
if {$static::sb_debug} { log local0. "$log_prefix Peek ([string length $recv_data]): $recv_data" }

# Check if we have received the response headers (terminated by two CRLFs)
if {[string match "HTTP/*\r\n\r\n*" $recv_data]}{

if {$static::sb_debug} { log local0. "$log_prefix Found the end of the HTTP headers" }

# Look for a content-length header in the data we have received so far
if {[string match -nocase "*Content-Length: *" $recv_data]}{

# Calculate the length of the response headers plus the payload by parsing the Content-Length header
# Get the index of the end of the HTTP headers
set header_length [expr {[string first "\r\n\r\n" $recv_data] + 4}]

set payload_length [findstr [string tolower $recv_data] "content-length: " 16 "\r"]
if {$static::sb_debug} { log local0. "$log_prefix \$header_length: $header_length, \$payload_length: $payload_length" }

# If the payload length is greater than 0
if {$payload_length ne "" and $payload_length > 0}{
if {$static::sb_debug} { log local0. "$log_prefix Peeking for [expr {$header_length + $payload_length}] bytes to get the payload" }
set recv_data [recv -peek -timeout 3000 -status recv_status [expr {$header_length + $payload_length}] $conn]

# Exit the while loop
break
} else {
# Content-Length header is 0 so no payload to wait for.
# Exit the while loop
break
}
} else {
# No Content-Length header so assume there is no payload to wait for.
# Exit the while loop
break
}
}
}

# Get the response
if {$static::sb_debug} { log local0. "$log_prefix Recv data ([string length $recv_data] bytes) in [expr {[clock clicks -milliseconds] - $start}] ms:\
<$recv_data>, peek status: <$peek_status>" }

# Debug: log the payload in hex to show non-printable characters like CRLFs
#binary scan $recv_data H* recv_data_hex
#if {$static::sb_debug} { log local0. "log_prefix \$recv_data_hex: $recv_data_hex" }

close $conn
if {$static::sb_debug} { log local0. "$log_prefix Closed, conn info: <[connect info -status $conn]>" }

# Parse the pool name from the received data.
# In this example we look for the literal string \$pool_name= and read up until a space
set pool_name [findstr $recv_data \$pool_name= 11 " "]

# Try to assign the pool
if {[catch {pool $pool_name} result]}{
if {$static::sb_debug} { log local0. "$log_prefix Error assigning pool \"$pool_name\", using the virtual server default pool. Error: $result" }
pool $default_pool
} else {
if {$static::sb_debug} { log local0. "$log_prefix Assigned pool $pool_name" }
}
}