ProxyPass for use with APM

Kirk_Bauer_1018 · ‎18-Mar-2015

Problem this snippet solves:

This is a specially modified version of the LTM ProxyPass iRule designed to work with a virtual server with an APM access profile applied. Please see the LTM ProxyPass iRule for documentation.

Code :

# ProxyPass APM iRule, Version 10.9
# Nov 30 2012
# THIS VERSION REQUIRES APM v10 or higher. Use ProxyPass v8.2 for TMOS 9.x.

# This version is for APM-enabled virtual servers only!
#
# APM provides ACCESS_ACL_ALLOWED event for the requests that have passed through access control checks.
# This event is semantically equivalent to HTTP_REQUEST event and is triggered for each HTTP request 
# that has been allowed to go to backend server after session/policy/ACL checks.
# All HTTP request processing commands are available in ACCESS_ACL_ALLOWED. (HTTP::header etc.)
# We use ACCESS::respond instead of HTTP::redirect/HTTP::respond.
# To port the LTM ProxyPass iRule, change HTTP_REQUEST to ACCESS_ACL_ALLOWED and replace HTTP::redirect/HTTP::respond to ACCESS::respond.
#
# Created by Kirk Bauer
# https://devcentral.f5.com/s/wiki/iRules.ProxyPass_for_use_with_APM.ashx
# (please see end of iRule for additional credits)

# Purpose:
# iRule to replace the functionality of Apache Webserver ProxyPass and
# ProxyPassReverse functions.  It allows you to perform host name and path name
# modifications as HTTP traffic passes through the LTM.  In other words, you
# can have different hostnames and directory names on the client side as you
# do on the server side and ProxyPass handles the necessary translations.

# NOTE: You should not need to modify this iRule in any way except the settings
# in the RULE_INIT event.  Just apply the iRule to the virtual server and
# define the appropriate Data Group and you are done.  If you do make any
# changes to this iRule, please send your changes and reasons to me so that
# I may understand how ProxyPass is being used and possibly incorporate your
# changes into the core release. 

# Configuration Requirements
# 1) The ProxyPass iRule needs to be applied to an HTTP virtual server or
# an HTTPS virtual server with a clientssl profile applied to it.
# 2) A data group (LTM -> iRules -> Data Groups tab) must be defined with
# the name "ProxyPassVIRTUAL" where VIRTUAL is the name of the virtual server
# (case-sensitive!).  See below for the format of this data group (class).
# For 10.0.x, you must use an EXTERNAL data group.
# 3) You must define a default pool on the virtual server unless you specify
# a pool in every entry in the data group.  
# 4) If you are using ProxyPass to select alternate pools, you must define
# a OneConnect profile in most cases!
# 5) ProxyPass does not rewrite links embedded within pages by default, just 
# headers.  If you want to change this, edit the $static::RewriteResponsePayload variable in RULE_INIT
# and apply the default stream profile to the virtual server.

# Data Group Information
# For 10.0.x, you must define an external data group (type=String, read-only) which loads 
# from a file on your BIG-IP.  For 10.1 and higher you can use an internal string data group with name=value pairings.
# The format of the file is as follows:
#    "clientside" := "serverside",
# or
#    "clientside" := "serverside poolname",
# The clientside and serverside fields must contain a URI (at least a "/") and
# may also contain a hostname.  Here are some examples:
#    "/clientdir" := "/serverdir",
#    "www.host.com/clientdir" := "internal.company.com/serverdir",
#    "www.host.com/" := "internal.company.com/serverdir/",

# Notes:
# 1) You can optionally define a ProxyPassSNATs data group to SNAT based
# on the pool selected.
# 2) You can optionally define a ProxyPassSSLProfiles data group to select
# a serverssl profile based on the pool selected.
# 3) You can also use regular expressions which is documented on DevCentral.

when RULE_INIT {
# Enable to debug ProxyPass translations via log messages in /var/log/ltm
# (2 = verbose, 1 = essential, 0 = none)
set static::ProxyPassDebug 0

# Enable to rewrite page content (try a setting of 1 first)
# (2 = attempt to rewrite host/path and just /path, 1 = attempt to rewrite host/path)
set static::RewriteResponsePayload 0
}

when CLIENT_ACCEPTED {
# Get the default pool name.  This is used later to explicitly select 
# the default pool for requests which don't have a pool specified in 
# the class.
set default_pool [LB::server pool]

# The name of the Data Group (aka class) we are going to use. 
# Parse just the virtual server name by stripping off the folders (if present)
set clname "ProxyPass[URI::basename [virtual name]]"

if { $static::ProxyPassDebug > 1 } {
log local0. "[virtual name]: [IP::client_addr]:[TCP::client_port] -> [IP::local_addr]:[TCP::local_port]"
}
}

when HTTP_REQUEST {
# "bypass" tracks whether or not we made any changes inbound so we
# can skip changes on the outbound traffic for greater efficiency.
set bypass 1
}

when ACCESS_ACL_ALLOWED {

# Initialize other local variables used in this rule
set orig_uri "[HTTP::uri]"
set orig_host "[HTTP::host]"
set log_prefix "VS=[virtual name], Host=$orig_host, URI=$orig_uri"
set clientside ""
set serverside ""
set newpool ""
set ppass ""

if {! [class exists $clname]} {
log local0. "$log_prefix: Data group $clname not found, exiting."
pool $default_pool
return
} else {
set ppass [class match -element "$orig_host$orig_uri" starts_with $clname]
if {$ppass eq ""} {
# Did not find with hostname, look for just path
set ppass [class match -element "$orig_uri" starts_with $clname]
}
if {$ppass eq ""} {
# No entries found
if { $static::ProxyPassDebug > 0 } {
log local0. "$log_prefix: No rule found, using default pool $default_pool and exiting"  
}
pool $default_pool
return
}
}

# Store each entry in the data group line into a local variable
set clientside [getfield $ppass " " 1]
set serverside [string trimleft [getfield $ppass " " 2 ] "{" ]
set newpool [string trimright [getfield $ppass " " 3 ] "}" ]

# If serverside is in the form =match=replace=, apply regex
if {$serverside starts_with "="} {
set regex [getfield $serverside "=" 2]
set rewrite [getfield $serverside "=" 3]
if {[regexp -nocase $regex "$orig_host$orig_uri" 0 1 2 3 4 5 6 7 8 9]}{
# The clientside becomes the matched string and the serverside the substitution
set clientside $0
set serverside [eval set X $rewrite]
} else {
pool $default_pool
return
}
}

if {$clientside starts_with "/"} {
# No virtual hostname specified, so use the Host header instead
set host_clientside $orig_host
set path_clientside $clientside
} else {
# Virtual host specified in entry, split the host and path
set host_clientside [getfield $clientside "/" 1]
set path_clientside [substr $clientside [string length $host_clientside]]
}
# At this point $host_clientside is the client hostname, and $path_clientside
# is the client-side path as specified in the data group

set host_serverside [getfield $serverside "/" 1]
set path_serverside [substr $serverside [string length $host_serverside]]
if {$host_serverside eq ""} {
set host_serverside $host_clientside
}
# At this point $host_serverside is the server hostname, and $path_serverside
# is the server-side path as specified in the data group

# In order for directory redirects to work properly we have to be careful with slashes
if {$path_clientside equals "/"} {
# Make sure serverside path ends with / if clientside path is "/"
if {!($path_serverside ends_with "/")} {
append path_serverside "/"
}
} else {
# Otherwise, neither can end in a / (unless serverside path is just "/")
if {!($path_serverside equals "/")} {
if {$path_serverside ends_with "/"} {
set path_serverside [string trimright $path_serverside "/"]
}
if {$path_clientside ends_with "/"} {
set path_clientside [string trimright $path_clientside "/"]
}
}
}

if { $static::ProxyPassDebug } {
log local0. "$log_prefix: Found Rule, Client Host=$host_clientside, Client Path=$path_clientside, Server Host=$host_serverside, Server Path=$path_serverside"  
}

# If you go to http://www.domain.com/dir, and /dir is a directory, the web
# server will redirect you to http://www.domain.com/dir/.  The problem is, with ProxyPass, if the client-side
# path is http://www.domain.com/dir, but the server-side path is http://www.domain.com/, the server will NOT
# redirect the client (it isn't going to redirect you to http://www.domain.com//!).  Here is the problem with
# that.  If there is an image referenced on the page, say logo.jpg, the client doesn't realize /dir is a directory
# and as such it will try to load http://www.domain.com/logo.jpg and not http://www.domain.com/dir/logo.jpg.  So
# ProxyPass has to handle the redirect in this case.  This only really matters if the server-side path is "/",
# but since we have the code here we might as well offload all of the redirects that we can (that is whenever
# the client path is exactly the client path specified in the data group but not "/").
if {$orig_uri eq $path_clientside} {
if {([string index $path_clientside end] ne "/") and not ($path_clientside contains ".") } {
set is_https 0
if {[PROFILE::exists clientssl] == 1} {
set is_https 1
}
# Assumption here is that the browser is hitting http://host/path which is a virtual path and we need to do the redirect for them
if {$is_https == 1} {
ACCESS::respond 302 Location "https://$orig_host$orig_uri/"
if { $static::ProxyPassDebug } {
log local0. "$log_prefix: Redirecting to https://$orig_host$orig_uri/"
}
} else {
ACCESS::respond 302 Location "http://$orig_host$orig_uri/"
if { $static::ProxyPassDebug } {
log local0. "$log_prefix: Redirecting to http://$orig_host$orig_uri/"
}
}
return
}
}

if {$host_clientside eq $orig_host} {
if {$orig_uri starts_with $path_clientside} {
set bypass 0
# Take care of pool selection
if {$newpool eq ""} {
pool $default_pool
if { $static::ProxyPassDebug > 1 } {
log local0. "$log_prefix: Using default pool $default_pool"
}
set newpool $default_pool
} else {
pool $newpool
if { $static::ProxyPassDebug > 0 } {
log local0. "$log_prefix: Using parsed pool $newpool (make sure you have OneConnect enabled)"
}
}
}
}

# If we did not match anything, skip the rest of this event
if {$bypass} {
return
}

# The following code will look up SNAT addresses from 
# the data group "ProxyPassSNATs" and apply them. 
# 
# The format of the entries in this list is as follows: 
# 
#   
# 
# All entries are separated by spaces, and both items 
# are required.
set class_exists_cmd "class exists ProxyPassSNATs"
if {! [eval $class_exists_cmd]} {
return
}

set snat [findclass $newpool ProxyPassSNATs " "]

if {$snat eq ""} { 
# No snat found, skip rest of this event
return 
}

if { $static::ProxyPassDebug > 0 } { 
log local0. "$log_prefix: SNAT address $snat assigned for pool $newpool"  
} 

snat $snat
}

when HTTP_REQUEST_SEND {
# If we didn't match anything, skip the rest of this event
if {$bypass} {
return
}

# The following code does the actual rewrite on its way TO 
# the backend server. It replaces the URI with the newly 
# constructed one and masks the "Host" header with the FQDN 
# the backend pool server wants to see. 
# 
# If a new pool or custom SNAT are to be applied, these are 
# done here as well. If a SNAT is used, an X-Forwarded-For 
# header is attached to send the original requesting IP 
# through to the server. 

if {$host_clientside eq $orig_host} {
if {$orig_uri starts_with $path_clientside} {
if { $static::ProxyPassDebug > 1 } {
log local0. "$log_prefix: New Host=$host_serverside, New Path=$path_serverside[substr $orig_uri [string length $path_clientside]]"
}
clientside { 
# Rewrite the URI
HTTP::uri $path_serverside[substr $orig_uri [string length $path_clientside]]
# Rewrite the Host header
HTTP::header replace Host $host_serverside
# Now alter the Referer header if necessary
if { [HTTP::header exists "Referer"] } {
 set protocol [URI::protocol [HTTP::header Referer]]
 if {$protocol ne ""} {
  set client_path [findstr [HTTP::header "Referer"] $host_clientside [string length $host_clientside]]
  if {$client_path starts_with $path_clientside} {
if { $static::ProxyPassDebug > 1 } {
 log local0. "$log_prefix: Changing Referer header: [HTTP::header Referer] to $protocol://$host_serverside$path_serverside[substr $client_path [string length $path_clientside]]"
}
HTTP::header replace "Referer" "$protocol://$host_serverside$path_serverside[substr $client_path [string length $path_clientside]]"
  }
 }
}
  }
}
}

# If we're rewriting the response content, prevent the server from using
#compression in its response by removing the Accept-Encoding header
#from the request.  LTM does not decompress response content before
#applying the stream profile.  This header is only removed if we're
#rewriting response content.
clientside {
if { $static::RewriteResponsePayload } {
if { [HTTP::header exists "Accept-Encoding"] } {
HTTP::header remove "Accept-Encoding"
if { $static::ProxyPassDebug > 1} {
log local0. "$log_prefix: Removed Accept-Encoding header"
}
}
}
HTTP::header insert "X-Forwarded-For" "[IP::remote_addr]"
}
}

when HTTP_RESPONSE {
if { $static::ProxyPassDebug > 1 } {
log local0. "$log_prefix: [HTTP::status] response from [LB::server]"
}

if {$bypass} {
# No modification is necessary if we didn't change anything inbound so disable the stream filter if it was enabled

# Check if we're rewriting the response
if {$static::RewriteResponsePayload} {
if { $static::ProxyPassDebug > 1 } {
log local0. "$log_prefix: Rewriting response content enabled, but disabled on this response."
}

# Need to explicity disable the stream filter if it's not needed for this response
# Hide the command from the iRule parser so it won't generate a validation error
#when not using a stream profile
set stream_disable_cmd "STREAM::disable"

# Execute the STREAM::disable command.  Use catch to handle any errors. Save the result to $result
if { [catch {eval $stream_disable_cmd} result] } {
# There was an error trying to disable the stream profile.
log local0. "$log_prefix: Error disabling stream filter ($result). If you enable static::RewriteResponsePayload, then you should add a stream profile to the VIP.  Else, set static::RewriteResponsePayload to 0 in this iRule."
}
}

# Exit from this event.
return
}

# Check if we're rewriting the response
if {$static::RewriteResponsePayload} {
# Configure and enable the stream filter to rewrite the response payload
# Hide the command from the iRule parser so it won't generate a validation error
#when not using a stream profile
if {$static::RewriteResponsePayload > 1} {
set stream_expression_cmd "STREAM::expression \"@$host_serverside$path_serverside@$host_clientside$path_clientside@ @$path_serverside@$path_clientside@\""
} else {
set stream_expression_cmd "STREAM::expression \"@$host_serverside$path_serverside@$host_clientside$path_clientside@\""
}
set stream_enable_cmd "STREAM::enable"
if { $static::ProxyPassDebug > 1 } {
log local0. "$log_prefix: \$stream_expression_cmd: $stream_expression_cmd, \$stream_enable_cmd: $stream_enable_cmd"
}

# Execute the STREAM::expression command. Use catch to handle any errors. Save the result to $result
if { [catch {eval $stream_expression_cmd} result] } {
# There was an error trying to set the stream expression.
log local0. "$log_prefix: Error setting stream expression ($result). If you enable static::RewriteResponsePayload, then you should add a stream profile to the VIP.  Else, set static::RewriteResponsePayload to 0 in this iRule."
} else {
# No error setting the stream expression, so try to enable the stream filter
# Execute the STREAM::enable command.  Use catch to handle any errors. Save the result to $result
if { [catch {eval $stream_enable_cmd} result] } {
# There was an error trying to enable the stream filter.
log local0. "$log_prefix: error enabling stream filter ($result)"
} else {
if { $static::ProxyPassDebug > 1 } {
log local0. "$log_prefix: Successfully configured and enabled stream filter"
}
}
}
}

# Fix Location, Content-Location, and URI headers
foreach header {"Location" "Content-Location" "URI"} {
set protocol [URI::protocol [HTTP::header $header]]
if { $static::ProxyPassDebug > 1 } {
log local0. "$log_prefix: Checking $header=[HTTP::header $header], \$protocol=$protocol"
}
if {$protocol ne ""} {
set server_path [findstr [HTTP::header $header] $host_serverside [string length $host_serverside]]
if {$server_path starts_with $path_serverside} {
if { $static::ProxyPassDebug } {
log local0. "$log_prefix: Changing response header $header: [HTTP::header $header] with $protocol://$host_clientside$path_clientside[substr $server_path [string length $path_serverside]]"
}
HTTP::header replace $header $protocol://$host_clientside$path_clientside[substr $server_path [string length $path_serverside]]
}
}
}

# Rewrite any domains/paths in Set-Cookie headers
if {[HTTP::header exists "Set-Cookie"]}{
array unset cookielist
foreach cookievalue [HTTP::header values "Set-Cookie"] {
set cookiename [getfield $cookievalue "=" 1]
set namevalue ""
set newcookievalue ""
foreach element [split $cookievalue ";"] {
set element [string trim $element]
if {$namevalue equals ""} {
set namevalue $element
} else {
if {$element contains "="} {
set elementname [getfield $element "=" 1]
set elementvalue [getfield $element "=" 2]
if {[string tolower $elementname] eq "domain"} {
set elementvalue [string trimright $elementvalue "."]
if {$host_serverside ends_with $elementvalue} {
if {$static::ProxyPassDebug > 1} {
log local0. "$log_prefix: Modifying cookie $cookiename domain from $elementvalue to $host_clientside"
}
set elementvalue $host_clientside
}
append elementvalue "."
}
if {[string tolower $elementname] eq "path"} {
if {$elementvalue starts_with $path_serverside} {
if {$static::ProxyPassDebug > 1} {
log local0. "$log_prefix: Modifying cookie $cookiename path from $elementvalue to $path_clientside[substr $elementvalue [string length $path_serverside]]"
}
set elementvalue $path_clientside[substr $elementvalue [string length $path_serverside]]
}
}
append newcookievalue "; $elementname=$elementvalue"
} else {
append newcookievalue "; $element"
}
}
} 
set cookielist($cookiename) "$namevalue$newcookievalue"
}
HTTP::header remove "Set-Cookie"
foreach cookiename [array names cookielist] {
HTTP::header insert "Set-Cookie" $cookielist($cookiename)
if {$static::ProxyPassDebug > 1} {
log local0. "$log_prefix: Inserting cookie: $cookielist($cookiename)"
}
}
}
}

# Only uncomment this event if you need extra debugging for content rewriting.
# This event can only be uncommented if the iRule is used with a stream profile.
#when STREAM_MATCHED {
#if { $static::ProxyPassDebug } {
#log local0. "$log_prefix: Rewriting match: [STREAM::match]"
#}
#}

# The following code will look up SSL profile rules from 
# the Data Group ProxyPassSSLProfiles" and apply 
# them. 
# 
# The format of the entries in this list is as follows: 
# 
#   
# 
# All entries are separated by spaces, and both items 
# are required.  The virtual server also will need to
# have any serverssl profile applied to it for this to work.
when SERVER_CONNECTED { 
if {$bypass} {
return
}

set class_exists_cmd "class exists ProxyPassSSLProfiles"
if {! [eval $class_exists_cmd]} {
return
}

set pool [LB::server pool]  
set profilename [findclass $pool ProxyPassSSLProfiles " "]

if {$profilename eq ""} { 
if { [PROFILE::exists serverssl] == 1} {
# Hide this command from the iRule parser (in case no serverssl profile is applied) 
set disable "SSL::disable serverside" 
catch {eval $disable}
}
return 
}

if { $static::ProxyPassDebug > 0 } { 
log local0. "$log_prefix: ServerSSL profile $profilename assigned for pool $pool"  
} 
if { [PROFILE::exists serverssl] == 1} {
# Hide these commands from the iRule parser (in case no serverssl profile is applied)
set profile "SSL::profile $profilename"
catch {eval $profile}
set enable "SSL::enable serverside" 
catch {eval $enable}
} else {
log local0. "$log_prefix: ServerSSL profile must be defined on virtual server to enable server-side encryption!"  
}
}

# ProxyPass Release History
#v10.9: Nov 26, 2012: Used URI::basename to get the virtual server name. Thanks to Opher Shachar for the suggestion.
#Replaced indentations with tabs intead of spaces to save on characters
#v10.8: Oct 25, 2012: Updated the class name to remove the folder(s) (if present) from the virtual server name. 
# This assumes the ProxyPass data group is in the same partition as the iRule.
#v10.7: Oct 24, 2012: Changed array set cookielist {} to array unset cookielist as the former does not clear the array.
# Thanks to rhuyerman@schubergphilis.com and Simon Kowallik for pointing out the issue and this wiki page with details: http://wiki.tcl.tk/724 
#v10.6: Oct 14, 2012: Updated how the protocol is parsed from URLs in request and response headers to fix errant matches
#v10.5: Feb 2, 2012: Removed extra stream profile $result reference for debug logging.
#v10.4: Nov 23, 2011: Removed an extra colon in sever HTTP::header replace commands to prevent duplicate headers from being inserted
#v10.3: Sep 27, 2010: Moved rewrite code to HTTP_REQUEST_SEND to work with WebAccelerator
# Fixed bug with cookie rewrites when cookie value contained an "="
#v10.2: Jun 04, 2010: Can handle individual file mappings thanks to Michael Holmes from AZDOE
# Also fixed bug with directory slash logic
#v10.1: Oct 24, 2009: Now CMP-friendly! (NOTE: use ProxyPass v8.2 for TMOS v9.x)
#v10.0: May 15, 2009: Optimized for external classes in v10 only (use v8.2 for TMOS v9.x)
# Added support for regular expressions and backreferences for the translations.
# v8.2: Jun 04, 2010: Fixed bug with directory slash logic
# v8.1: May 15, 2009: Added internal redirects back in (removing them was a mistake)
# v8.0: May 13, 2009: pulled in changes submitted by Aaron Hooley (hooleylists gmail com)
# TMOS v10 support added.  Cookie domain/path rewriting added.
# v7.0: May 6, 2008: added optional serverssl contributed by Joel Moses
# v6.0: Jan 15, 2008: Small efficiency change
# v5.0: Jul 27, 2007: Added Referer header conversions
# v4.0: Jul 27, 2007: Added optional debugging flag
# v3.0: Jul 20, 2007: Added SNAT support contributed by Adam Auerbach
# v2.0: May 28, 2007: Added internal directory redirects and optional stream profile
# v1.0: Feb 20, 2007: Initial Release

Mali47_261353 · ‎11-Apr-2018

I'm trying to get this to work in APM, how do you incorporate it with VPE or do you simply just apply on the VS?