FTPS_SSL_ Termination

Problem this snippet solves:

This iRule is meant to act as a full FTPS termination rule that behaves similarly to HTTPS termination. It forces either an implicit (port 990 by default) or explicit (port 21 with AUTH TLS) connection to be made before commands are passed to the member FTP servers. All traffic from the F5 to the pool member FTP servers is then passed in cleartext and all AUTH/SSL related commands are intercepted by the F5 so that the pool members handle the session just like any other non-SSL session (again, similar to HTTPS termination).

Please note that this is currently a work-in-progress. It is very close to complete but has some lingering issues. I am hoping that in sharing my work, some of the other gurus around here will take a look and help with optimization and correcting any remaining problems. In it's current form, you can put a pool of FTP servers behind it and connect from a variety of FTP clients (lftp, filezilla, and kftpgrabber tested). The rule intercepts pasv response strings, so no additional configuration is needed on the pool member FTP servers.

However, it seems there is some sort of race/timing problem that I have yet to pinpoint. My tests using Apache FTP server (see http://mina.apache.org/ftpserver/running-ftpserver-stand-alone-in-5-minutes.html for usage) on the pool members work in large part. However, my tests using proftpd on the pool members fail the logging indicates the pasv connection is never established to the pool member server (no 'SERVER_CONNECTED' log message for the pasv connection). I am currently working to correct this problem, but please feel free to let me know if you can spot the issue.

Another issue I am using an ad hoc method of allocating and tracking vserver pasv ports. I'd much rather use the F5's built-in TCP::unused_port in conjunction with a 'listen' directive right after a PASV request is intercepted, but I have not been able to determine the proper way to use 'listen'. It looks like 'listen' and 'relate_client' are potentially what are needed, but I cannot find much documentation on these commands,

Finally, the rule is littered with log statements. I'm definitely aware that this would slow the rule down in production. I would grep those out prior to deploying this. Otherwise, despite the length of the rule, it seems to run decently fast for what it is accomplishing. The control channel connections don't pass much data, and each can be matched via 'starts_with' as opposed to 'contains', and the pasv payloads aren't matched/altered at all. Much of the processing is done only during the initial control channel connection.

Please feel free to try this iRule. I'd be happy to receive feedback on it!

Code :

# Author:  Bob Ziuchkovski 
# Date:    Sep 27, 2010
# Version: 1.0
#
# IMPORTANT: This is a "work-in-progress".  It has some problems as noted in
#       the Codeshare description.
#
# Summary: This irule is meant to implement FTPS termination transparently in
# a manner similar to F5 HTTPS termination.  We try to intercept all commands
# related to SSL/TLS (RFC4217) and FTP Auth extensions in general (RFC2228).
#
# Usage: 
#       1. Ensure the BIG-IP version is 10.2 or higher.  Versions prior
#          to 10.2 seem have problems with the SSL::respond command (and
#          9.x is missing the SSL::* commands entirely)
#       2. Add this iRule to the F5.  Edit the configuration settings between
#          the 'BEGIN' and 'END' configuration sections below
#       3. Create a client SSL profile for terminating the FTPS sessions.
#          Make sure to disable the 'Unclean Shutdown' option in the client
#          profile. Otherwise certain FTP clients will fail connections due
#          to unexpected packet lengths (most notably clients using GnuTLS)
#       4. Create a custom TCP Full Open monitor and set it to monitor
#          the control channel port of the member FTP servers.  Set the
#          receive string to match the FTP server's welcome message (i.e.
#          220 Service ready for new user.)
#       5. Create a pool for FTP servers.  Apply the custom monitor created
#          above
#       6. Add member FTP servers to the pool.  The members must be added
#          with the '*' (All ports) setting.
#       7. Create a virtual server with the following settings:
#          - Listen on all ports ('*')
#          - Port Translation setting enabled (under 'Advanced')
#          - This iRule applied
#          - Default pool set to the pool created above
#          - Optional: Set a source address persistence profile.  If all
#            member FTP servers serve identical content from a shared mount
#            then this isn't needed.  The pasv data connections are matched
#            and sent to the same member that is serving the control channel.
#            However, if there is any delay in content synchronization between
#            the member servers, then source address persistence is recommended
#            since some FTP clients will open additional control connections to
#            retrieve files.
#          * Example: A user could see a file on one connection and instruct
#            their FTP client try to retrieve the file.  The FTP client opens
#            a second control connection (a means of keeping a GUI control
#            connection responsive during file transfer), and find the file
#            missing on the second connection do to being load-balanced to a
#            server that lacks the file (i.e. rsync or simimlar delay).
#                     
# Misc: 1. The member FTP servers can be configured to use any PASV port range
#          and send any PASV IP response they want -- it doesn't matter because
#          we intercept those responses and remap ourselves.  Make sure to set
#          the correct $static::pasv_max_port below, though!
#       2. The member FTP servers should be configured to reject active mode
#          data connections (this rule supports passive mode only).
#       3. We don't incercept the 'FEAT' command to advertise TLS/SSL. I have
#          not run into any FTP clients that take issue with this, since most
#          will already establish TLS/SSL by the time they send the FEAT
#          request.  Those that don't seem to reissue FEAT again at a later
#          time after the initial failure.  That said, if running into problems
#          with an obscure FTP client, this is one of the first things I would
#          recommend checking.
#       4. This rule intercepts ADAT, MIC, CONF, and ENC commands.  I did this
#          to prevent our member FTP servers from seeing or responding to
#          anything related to FTP Auth extensions.  However, this could
#          probably be omitted as an optimization -- the negative response sent
#          by the member server would probably suffice, although the error code
#          itself wouldn't make sense (the member server will respond that no
#          auth tls/ssl is in use).
#
# Optimizations to consider:
#
#       1. I've left all sorts of DEBUG logging statements in the iRule for
#          testing.  I have been working with two copies of this rule.  One
#          with these log statements (a 'debug' version) and one with them
#          removed (simple grep -v to remove).  I don't like the overhead of
#          additional conditional if {$debug == 1} types of checks, so these
#          are not included.
#       2. Remove checks for ADAT, MIC, CONF, and ENC mentioned above.


when RULE_INIT {
#----------------------------------
#    Begin configuration portion 
#----------------------------------

# Member server port to use for control channel
set static::ftp_member_port 2121

# Vserver port to use for FTPS explicit control channel (-1 to disable)
set static::ftps_explicit_port 21

# Verserver Port to use for FTPS implicit control channel (-1 to disable)
set static::ftps_implicit_port 990

# Vserver Ports to use for PASV data channel connections -- does not need to
# match the member server PASV ports as we automatically track and map those
set static::pasv_min_port 50000
set static::pasv_max_port 50005

# Vserver IP to advertise for incoming PASV data channel connections
# NOTE: the commas are NOT a typo.  This is the format used by FTP protocol
set static::pasv_max_port "10,0,0,46"

#----------------------------------
#     End configuration portion
#----------------------------------

# TODO: Figure out how to properly use the 'listen' directive so that we can use
# 'listen' in combination with TCP::unused_port to avoid this ad hoc port tracking

# Track in-use PASV mappings (our pasv port -> client IP/member IP/member port)
array set ::pasv_to_client {}
array set ::pasv_to_mbr {}
array set ::pasv_to_mbr_port {}

# Clear out any mappings already in-use (i.e. rule reload/update)
array unset ::pasv_to_client *
array unset ::pasv_to_mbr *
array unset ::pasv_to_mbr_port *

# Track available PASV ports
set ::pasv_avail {}

# Make sure the list is cleaned-out from any prior states and add our ports
set ::pasv_avail [ lreplace $::pasv_avail 0 end ]
foreach {set i $static::pasv_max_port} { $i >= $static::pasv_min_port } {incr i -1} {
lappend ::pasv_avail $i
}

# Debugging-only -- set a numeric auto-incrementing ID for each connection
# (helps differentiate between events fired for different connections)
set ::cid 0
}

when LB_FAILED {
  log local0. "DEBUG: ($our_cid) LB_FAILED fired"
}

when CLIENT_ACCEPTED {
  incr ::cid
  set our_cid $::cid

  log local0. "DEBUG: ($our_cid) CLIENT_ACCEPTED fired"
  # We need to handle control channel and data channel connections differently
  # 0 = unknown/unauthorized, 1 = data channel, 2 = control channel
  set conn_type 0

  # Is this a pasv data channel connection? (first since triggered most often)
  if { [TCP::local_port] >= $static::pasv_min_port && [TCP::local_port] <= $static::pasv_max_port } {
    # Port is pasv port but does it match a client control connection?
    if {[info exists ::pasv_to_client([TCP::local_port])] } { 
      set orig_client $::pasv_to_client([TCP::local_port])
      if { $orig_client == [IP::client_addr] } {
        log local0. "DEBUG: ($our_cid) PASV attempt from established client [IP::client_addr] on port [TCP::local_port]"

        # Retrieve member IP/port waiting for connection
        set orig_mbr $::pasv_to_mbr([TCP::local_port])
        set orig_mbr_port $::pasv_to_mbr_port([TCP::local_port])
        log local0. "DEBUG: ($our_cid) Member associated with PASV connnection: $orig_mbr (port $orig_mbr_port)"
        if { $orig_mbr != "" && $orig_mbr_port != "" } {
          # This is a legitimate pasv data connection and a member server is waiting for it.
          set conn_type 1

          # Connect client directly to the member server on proper port
          log local0. "DEBUG: ($our_cid) Executing node command: 'node $orig_mbr $orig_mbr_port'"
          node $orig_mbr $orig_mbr_port
          log local0. "DEBUG: ($our_cid) Node command executed"
        } else {
          log local0. "DEBUG: ($our_cid) PASV connection not mapped to member server / member port"
        }
      } else {
        log local0. "ALERT: Unauthorized connection to mapped pasv port [TCP::local_port] by [IP::client_addr] -- expected $orig_client instead!"
      }
    } else {
      log local0. "ALERT: Unauthorized connection to unmapped pasv port [TCP::local_port] by [IP::client_addr]"
    }

  # Is this a control channel connection?
  } elseif { [TCP::local_port] == $static::ftps_explicit_port || [TCP::local_port] == $static::ftps_implicit_port} {
    set conn_type 2

    # Keep a persistent vserver pasv port assignment per control channel connection
    set our_pasv_port 0
    set our_pasv_str ""

    if { [TCP::local_port] == $static::ftps_explicit_port } {
      log local0. "DEBUG: ($our_cid) new ftps explicit control connection from [IP::client_addr]"

      # We need to track the ssl state of control channels
      # 0 = disabled, 1 = enabled
      set ctl_ssl_state 0
      SSL::disable
    } elseif { [TCP::local_port] == $static::ftps_implicit_port } {
      log local0. "DEBUG: ($our_cid) new ftps implicit control connection from [IP::client_addr]"

      # We need to track the ssl state of control channels
      # 0 = disabled, 1 = enabled
      set ctl_ssl_state 1
    }

    # Here we have the LB choose our node for the control channel, but we make
    # the connection with the 'node' command since member control channel port
    # might differ from vserver control channel port
    set node_selection [getfield [LB::select] " " 4]
    node $node_selection $static::ftp_member_port

  # Invalid connection -- made to a port other than one we are using for FTP
  } else {
    log local0. "ALERT: Unauthorized connection non-ftp port [TCP::local_port] by [IP::client_addr]"
  }

  # If the connection isn't legitimate, close it
  if { $conn_type == 0 } {
    log local0. "DEBUG: ($our_cid) Rejecting unauthorized connection"
    reject
  }
}

when CLIENT_CLOSED {
  log local0. "DEBUG: ($our_cid) CLIENT_CLOSED fired"
  # Remove mappings associated with control connections
  if {$conn_type == 1 } {
    log local0. "DEBUG: ($our_cid) Closing pasv data connection (port [TCP::local_port]) from [IP::client_addr]"
  } elseif {$conn_type == 2} {
    log local0. "DEBUG: ($our_cid) Closing control channel to [IP::client_addr]"
    
    # We need to clean up after ourselves if we had a pasv port in use
    if {$our_pasv_port != 0 } {
      log local0. "DEBUG: ($our_cid) Control channel for [IP::client_addr] was using pasv port $our_pasv_port.  Unmapping."


      log local0. "DEBUG: ($our_cid) Available PASV ports before freeing used port:"
      foreach pport $::pasv_avail {
        log local0. "  $pport"
      }

      # Put our port back in avail, but only if it wasn't already there
      # (avoids duplication if rule is reloaded while control channel is open)
      if { [lsearch -exact $::pasv_avail $our_pasv_port] == -1 } {
        lappend ::pasv_avail $our_pasv_port
      } else {
        log local0. "DEBUG: ($our_cid) Not adding our port to pasv pool...already there"
      }

      log local0. "DEBUG: ($our_cid) Available PASV ports after freeing used port:"
      foreach pport $::pasv_avail {
        log local0. "  $pport"
      }

      # Same concept here...check first and unmap
      if { [info exists ::pasv_to_client($our_pasv_port)] } { 
        unset ::pasv_to_client($our_pasv_port)
      } else {
        log local0. "DEBUG: ($our_cid) Nothing to remove in ::pasv_to_client array"
      }
      if { [info exists ::pasv_to_mbr($our_pasv_port)] } { 
        unset ::pasv_to_mbr($our_pasv_port)
      } else {
        log local0. "DEBUG: ($our_cid) Nothing to remove in ::pasv_to_mbr array"
      }
      if { [info exists ::pasv_to_mbr_port($our_pasv_port)] } { 
        unset ::pasv_to_mbr_port($our_pasv_port)
      } else {
        log local0. "DEBUG: ($our_cid) Nothing to remove in ::pasv_to_mbr_port array"
      }
    }
  }   
}

when CLIENT_DATA {
  log local0. "DEBUG: ($our_cid) CLIENT_DATA fired"
  log local0. "DEBUG: ($our_cid) Client data on connection type $conn_type, ssl_state $ctl_ssl_state:"
  log local0. "  PAYLOAD: [TCP::payload]"
  # We are only interested in intercepting non-SSL control channel payloads here
  if { $conn_type == 2 && $ctl_ssl_state == 0 } {
    log local0. "DEBUG: ($our_cid) Handling intercepted client data payload"
    # Here we intercept and handle all traffic prior to auth tls establishment
    set response ""
    
    if { [TCP::payload] starts_with "AUTH " } {
      # We only allow TLS auth
      if { [TCP::payload] starts_with "AUTH TLS" } {
        set response "234 Command AUTH okay; starting TLS connection.\r\n"
        set ctl_ssl_state 1
      } else {
        set response "534 Request denied for policy reasons.\r\n"
      }
    # Some clients send the following commands prior to their auth tls request
    } elseif { [TCP::payload] starts_with "PBSZ 0" } {
      # Essentially a no-op -- TODO: handle other PBSZ request sizes?
      set response "200 Command PBSZ okay.\r\n"
    } elseif { [TCP::payload] starts_with "PROT " } {
      # Reject everything but 'PROT P' mode
      if { [TCP::payload] starts_with "PROT P" } {
        set response "200 Command PROT okay.\r\n"
      } elseif { [TCP::payload] starts_with "PROT C" } {
        set response "534 Insufficient data protection.\r\n"
      } elseif { [TCP::payload] starts_with "PROT S" } {
        set response "504 Command not implemented for that parameter.\r\n"
      } elseif { [TCP::payload] starts_with "PROT E" } {
        set response "504 Command not implemented for that parameter.\r\n"
      }
    # Don't allow any other commands until auth tls has been established
    } else {
      set response "530 Access denied.\r\n"
    }
  
    # Now we have the response we wish to send, so send it
    log local0. "DEBUG: ($our_cid) Sending response to intercepted plaintext payload: $response"
    TCP::respond $response
    TCP::payload replace 0 [TCP::payload length] {}
    TCP::release
      
    # If we just received TLS request, then we will have transitioned this to 1
    # We can now enable SSL.
    if { $ctl_ssl_state == 1 } {
      log local0. "DEBUG: ($our_cid) Enabling SSL on control channel connection"
      SSL::enable
    } else {
      TCP::collect
    }
  } else {
    log local0. "DEBUG: ($our_cid) Releasing unhandled collected client TCP payload"
    TCP::release
    #TCP::collect
  }
}

when CLIENTSSL_HANDSHAKE {
  log local0. "DEBUG: ($our_cid) CLIENTSSL_HANDSHAKE fired"
  # We only want to intercept SSL payload for control channels
  if { $conn_type == 2 } {
    log local0. "DEBUG: ($our_cid) SSL handshake on control connection.  Collecting."
    SSL::collect
  }
}

when CLIENTSSL_DATA {
  log local0. "DEBUG: ($our_cid) CLIENTSSL_DATA fired"
  log local0. "DEBUG: ($our_cid) Client SSL data on connection type $conn_type, ssl_state $ctl_ssl_state:"
  log local0. "  PAYLOAD: [SSL::payload]"
  # We only want to intercept SSL payload for control channels
  if { $conn_type == 2} {
    log local0. "DEBUG: ($our_cid) Handling intercepted client SSL payload"
    # Here we handle responses to auth-related commands after tls has been established
    set response ""
  
    if { [SSL::payload] starts_with "AUTH " } {
      # Not allowed to change auth after TLS established
      set response "534 Session already secured.\r\n"
    # Yes we handled these above as well, but here they occur within SSL payload
    } elseif { [SSL::payload] starts_with "PBSZ 0" } {
      set response "200 Command PBSZ okay.\r\n"
    } elseif { [SSL::payload] starts_with "PROT " } {
      # Reject everything but 'PROT P' mode
      if { [SSL::payload] starts_with "PROT P" } {
        set response "200 Command PROT okay.\r\n"
      } elseif { [SSL::payload] starts_with "PROT C" } {
        set response "534 Insufficient data protection.\r\n"
      } elseif { [SSL::payload] starts_with "PROT S" } {
        set response "504 Command not implemented for that parameter.\r\n"
      } elseif { [SSL::payload] starts_with "PROT E" } {
        set response "504 Command not implemented for that parameter.\r\n" 
      }
    # We don't accept any of the below commands and intercept before they reach our ftp servers
    } elseif { [SSL::payload] starts_with "CCC" } {
      # Don't allow client to turn off control channel encryption
      set response "534 Request denied for policy reasons.\r\n"
    } elseif { [SSL::payload] starts_with "ADAT " } {
      set response "500 Syntax error, command unrecognized.\r\n"
    } elseif { [SSL::payload] starts_with "MIC " } {
      set response "500 Syntax error, command unrecognized.\r\n"
    } elseif { [SSL::payload] starts_with "CONF " } {
      set response "500 Syntax error, command unrecognized.\r\n"
    } elseif { [SSL::payload] starts_with "ENC " } {
      set response "500 Syntax error, command unrecognized.\r\n"
    }
  
    # In the main TCP portion, we're intercepting everything.  Here we only intercept
    # some commands and let the backend server respond to everything else
    if { $response != "" } {
      log local0. "DEBUG: ($our_cid) Sending response to intercepted SSL payload: $response"
      SSL::respond $response
      SSL::payload replace 0 [SSL::payload length] {}
    }
  
    log local0. "DEBUG: ($our_cid) Releasing intercepted SSL payload"
    SSL::release
    log local0. "DEBUG: ($our_cid) Collecting SSL again"
    SSL::collect

  } else {
    log local0. "DEBUG: ($our_cid) Releasing unhandled collected SSL payload"
    SSL::release
    #SSL::collect
  }
}

when SERVER_CONNECTED {
  log local0. "DEBUG: ($our_cid) SERVER_CONNECTED fired"
  # We only want to intercept server payloads for control channels
  if { $conn_type == 2 } {
    log local0. "DEBUG: ($our_cid) Control connection to member established.  Collecting."
    TCP::collect
  } else {
    log local0. "DEBUG: ($our_cid) Data connection to member established."
  }
}

when SERVER_CLOSED {
  log local0. "DEBUG: ($our_cid) SERVER_CLOSED fired"
}

when SERVER_DATA {
  log local0. "DEBUG: ($our_cid) SERVER_DATA fired"
  log local0. "DEBUG: ($our_cid) Server data on connection type $conn_type, ssl_state $ctl_ssl_state:"
  log local0. "  PAYLOAD: [TCP::payload]"
  # We only want to intercept server payloads for control channels
  if { $conn_type == 2 } {
    log local0. "DEBUG: ($our_cid) Handling intercepted server data payload"
    # Intercept the PASV response sent by the pool member
    if { [TCP::payload] starts_with "227 Entering Passive Mode" } {
      log local0. "DEBUG: ($our_cid) Intercepting member PASV response."
      # Extract and save pasv port given by member server -- needed for lookup on data channel connection
      set pasvstr [findstr [TCP::payload] "227 Entering Passive Mode" 27 ")"]
      set mbr_pasv_port [ expr [getfield $pasvstr ","  5] * 256 + [getfield $pasvstr ","  6] ]
      log local0. "DEBUG: ($our_cid) member server requesting connection be made to port $mbr_pasv_port"
   
      # On the F5 side, we try to reuse the same port to avoid repeated search of port list
      if { $our_pasv_port == 0 } {
        # Make sure we have an available port to use
        if { [llength $::pasv_avail] > 0 } {

          log local0. "DEBUG: ($our_cid) Available PASV ports before assignment:"
          foreach pport $::pasv_avail {
            log local0. "  $pport"
          }

          # Pop our port from end of available ports list
          set our_pasv_port [ lindex $::pasv_avail end ]
          set ::pasv_avail [ lreplace $::pasv_avail end end ]

          log local0. "DEBUG: ($our_cid) Available PASV ports after assignment:"
          foreach pport $::pasv_avail {
            log local0. "  $pport"
          }
 
          # Format our port in PASV response string format
          set our_pasv_str "[expr $our_pasv_port / 256],[expr $our_pasv_port % 256]"
          log local0. "DEBUG: ($our_cid) Setting this connection's pasv port to $our_pasv_port"
          log local0. "DEBUG: ($our_cid) Setting this connection's pasv string to $our_pasv_str"

          # The pasv port to client IP and member IP mappings will remain constant once we assign the port
          set ::pasv_to_client($our_pasv_port) [IP::client_addr]
          set ::pasv_to_mbr($our_pasv_port) [LB::server addr]
          log local0. "DEBUG: ($our_cid) Saved mapping PASV port -> client IP == $our_pasv_port -> [IP::client_addr]"
          log local0. "DEBUG: ($our_cid) Saved mapping PASV port -> member IP == $our_pasv_port -> [LB::server addr]"
        } else {
          log local0. "ALERT: No available PASV ports -- port range exhausted!"
          TCP::payload replace 0 [TCP::payload length] "421 Max connections reached.\r\n"
        }
      }

      # Make our PASV response substitution only if we have a port for use
      if { $our_pasv_port != 0 } {

        # Save mapping of our port to member port -- We have to update this on each PASV response since the member
        # server isn't gauranteed to respond with the same port to us each time
        set ::pasv_to_mbr_port($our_pasv_port) $mbr_pasv_port
        log local0. "DEBUG: ($our_cid) Saved mapping PASV port -> member port == $our_pasv_port -> $mbr_pasv_port"
      
        # Make our PASV response substitution
        set pasv_response "227 Entering Passive Mode ($static::pasv_max_port,$our_pasv_str)\r\n"
        log local0. "DEBUG: ($our_cid) Sending updated PASV response to client:"
          log local0. "  $pasv_response"
        TCP::payload replace 0 [TCP::payload length] $pasv_response
      }
    }
  
    # Continue intercepting server responses
    log local0. "DEBUG: ($our_cid) Releasing intercepted server TCP payload"
    TCP::release
    log local0. "DEBUG: ($our_cid) Collecting server TCP again"
    TCP::collect
  
    # Depending on our SSL status, we either want to collect client-side TCP or SSL
    if { $ctl_ssl_state == 0 } {
      log local0. "DEBUG: ($our_cid) Signaling clientside TCP collect"
      clientside { TCP::collect }
    } elseif { $ctl_ssl_state == 1 } {
      log local0. "DEBUG: ($our_cid) Signaling clientside SSL collect"
      clientside { SSL::collect }
    }
  } else {
    log local0. "DEBUG: ($our_cid) Releasing unhandled collected server TCP payload"
    TCP::release
    #TCP::collect
  }
}