FTPS Offload via iRules

Question: Does BIG-IP LTM support FTPS?

Answer: You might think to yourself "LTM can load balance any IP traffic, so sure!". But if you know FTPS, you know that, like FTP, things are a lot more complicated than most protocols. And although there is an FTP profile to allow us to effortlessly support FTP, there is no FTPS profile. And since FTPS involves encryption, iRules become tough. But the answer is "Yes, we can load balance FTPS, with a little iRule help."

Question: Can BIG-IP LTM offload encryption from FTPS?

Answer: You might know that FTPS uses regular SSL, just like HTTPS, so you might think you could just use a clientssl profile. I'd like to say you are right, but at least today that won't work. However, with some iRule help, we can offload FTPS as long as we don't need to support Active transfers (see below).

Question: Does BIG-IP LTM support SFTP?

Answer: SFTP is in no way related to FTPS. SFTP uses the SSH protocol. Even though the LTM uses SSH for administrative purposes, we cannot decrypt or offload SSH traffic in the traffic path. Fortunately, since SFTP uses one simple TCP connection from each client to each server, we can load balance SFTP just like we can any other generic TCP traffic.

FTP Basics
---------------
Since FTPS is simply an encrypted version of FTP, we need to talk a little about FTP first.

Control Channel: the FTP client connects to the FTP server on port 21. This connection is called the Control Channel. The control channel is how the client logs in, changes directories, requests file listings, and requests file transfers.

Active FTP: the original way to transfer files and directory listings via FTP is called Active Mode. The client issues a PORT command and tells the server its IP address and a port to connect to. The server then opens a new TCP connection to the client and begins the transfer. The outbound data connection from the server always originates on port 20.

Passive FTP: active FTP has long plagued firewalls and NAT environments and in general it doesn't make as much sense today for the server to be initiating new connections to the client. So Passive FTP is often the default behavior in FTP clients (such as web browsers) today. With Passive FTP transfers, the client issues a PASV command and the server responds with an IP address and a port. The client then connects to that port and the transfer begins. The FTP server will generally give out its own IP address but most FTP servers can be configured to give out a specific IP address (such as the VIP address in a load balancing environment).

BIG-IP FTP Profile: so why do we need an FTP profile on the BIG-IP? It provides a number of important features:

1) When the server creates outbound TCP connections for active transfers, the profile allows these connection to be established even if they don't match a virtual server (essentially they match the virtual server with the FTP profile). In addition, if SNAT is being used, when the client sends its IP address in the PORT command, the profile changes this IP to the SNAT address to make sure the connection passes through the BIG-IP.

2) When the client sends a PASV command to initiate a passive transfer, the FTP profile will change the IP address provided by the server so it is the VIP address. The LTM will then allow this connection from the client (on the VIP address but on a port other than 21) even though there is no explicit virtual server defined on that port.

3) While a transfer is occurring, the control channel remains idle. Without the FTP profile, a long transfer may cause the control channel to be closed due to an idle timeout. When the transfer completes, most clients will indicate a failure because the control connection was lost. The FTP profile ties the control and data connections together so that as long as one of them is active neither will time out.

Overview of FTPS
----------------
FTPS is a secure implementation of the FTP protocol. It has no relation at all to SFTP. The old way of using FTPS was called "Implicit FTPS". The way this works is that the client connects to a special FTPS port (usually 990) and immediately begins an SSL handshake (the same as a web browser does when connecting to port 443). We won't cover Implicit FTPS in this discussion as it is generally considered deprecated, but both of these solutions should work for it with little or no modification.

The modern implementation of FTPS is called Explicit FTPS. The way this works is the FTPS client connects to the server on port 21 just like an FTP client. Then the client issues either an "AUTH TLS" or "AUTH SSL" command. Once this command is issued, the server acknowledges it and the client begins an SSL handshake. From that point forward, the control channel (client connection to port 21) is encrypted and one can't see the commands within that channel without decrypting it first. Many clients, however, will issue the Clear Control Channel (CCC) command after logging in so the rest of the control session will return to plain text. This is to benefit network firewalls and other devices that rely on seeing inside of FTP control channels to allow data connections to be opened and timeouts of the two separate channels to be linked together. The data channel, by default, will also be encrypted. If the server connects to the client (active FTP transfer), the client will first begin SSL negotiation. If the client connects to the server (passive FTP transfer), the client will also begin SSL negotiation.

Note that the BIG-IP FTP Profile does not currently support FTPS. If the client sends the AUTH TLS or AUTH SSL commands, the message will be ignored (not sent to the server) and the client will hang waiting for a response from the server.

For more information, Wikipedia has a great write-up here: http://en.wikipedia.org/wiki/FTPS.

Solution #1: Load balancing FTPS
--------------------------------

With a simple iRule and the proper LTM configuration, you can fully load balance both FTP and Explicit FTPS. This will support both active and passive transfers. A clear control channel is not required but it does not hurt either.

1) Servers must point their default gateway to the LTM (we can't use SNAT because we can't see or alter the client's IP in PORT commands over encrypted FTPS)

2) Servers must be configured to hand out the VIP address for any Passive transfers (we can't modify the IP address the server sends in response to PASV commands because this may be encrypted).

3) An inbound virtual server is defined on the VIP address and the FTP port (21). FTP profile is *not* enabled as this will break FTPS. Timeout needs to be long as the control channel will sit idle during a long transfer and if the control channel is closed the transfer will fail.

4) Another inbound virtual server is defined on the VIP address and All Ports. This will catch all Passive FTP transfer connections from clients.

5) A source address persistence profile is defined that matches across services and across virtuals. It is applied to inbound both virtual servers so that the passive transfer connections are sent to the same server that has the control connection from that client.

6) Since the servers are using the LTM as their default gateway, you will probably need a default Forwarding (IP) virtual server doing all addresses, all ports, all protocols.

7) You will also need a forwarding virtual server that specifically matches outbound TCP traffic (but still all IPs and all ports). This virtual server will catch outbound Active FTP transfers and any other outbound TCP connections. The following iRule needs to be applied to this virtual server to make sure that any outbound active FTP transfers (coming from port 20) are SNAT'd to the VIP address (W.X.Y.Z):

when CLIENT_ACCEPTED {
  if { [TCP::remote_port] eq "20"} {
    snat W.X.Y.Z 20
  }
}

Here is how this works: Active transfers: the client connects to the VIP and is load balanced to a server. SSL begins on this connection. The client issues a PORT command to transfer a file and includes its own IP address and a port. The server then initiates an outbound TCP connection to that client IP and port which goes through the LTM because the LTM is the default gateway of the server. This connection matches the outbound TCP forwarding virtual server defined in step #7. The source port of this connection will always be port 20 so the data connection will be SNAT'd by the iRule.

Passive transfers: the client connects to the VIP and is load balanced to a server. SSL begins on this connection. The client issues a PASV command to transfer a file. The server responds with the VIP address (since it was configured that way in step #2) on a random port. When the client connects to that new port, it matches the other inbound virtual server (#4 above). Because of the persistence profile, the new inbound connection will be load balanced to the same server that the control connection was already connected to.

Note that if you could assume that clients will always issue the Clear Control Channel (CCC) command after authentication, you could use SNAT and the server would not have to hand out the VIP address if you wrote additional iRules to do the proper modifications (i.e. basically simulate the functionality of the FTP profile in an iRule).

Solution #2: Offloading SSL for FTPS
------------------------------------
This solution will handle both load balancing and offloading of SSL for FTPS. It will NOT support Active FTPS transfers -- only Passive FTPS transfers will work (This is because of the strange way active FTPS SSL negotiations work -- the server initiates a connection to the client but the client begins the SSL handshake). This solution could support the Clear Control Channel command but currently does not. This is really a pretty experimental solution and would need to be improved to make it more robust in a production environment. The second iRule has some code commented out to replace the IP the server sends for passive transfers, but it would be easier just to configure your server to hand out the VIP address.

x.x.x.x: external VIP address
y.y.y.y: any internal IP address that is not in use

1) You must be running 9.4.x as we will be using the "virtual" command to send traffic to another Virtual Server.

2) A source address persistence profile is defined that matches across services and across virtuals. It is applied to inbound both virtual servers so that the passive transfer connections are sent to the same server that has the control connection from that client.

3) Define first virtual server (where the clients actually connect to) on VIP address x.x.x.x and port 21. This virtual server needs to have a CLIENTSSL profile associated with it with a valid SSL certificate for the server (the same way you'd do it if you were offloading SSL for HTTPS). This virtual server does not need a default pool. This virtual server also needs this iRule applied to it (with y.y.y.y replaced with the actual internal IP address):

when CLIENT_ACCEPTED {
log local0. "client accepted"
SSL::disable
TCP::respond "220 My ftp server\r\n"
TCP::collect
}
when CLIENT_DATA {
log local0. "client data"
TCP::respond "234 AUTH TLS Successful\r\n"
TCP::payload replace 0 [TCP::payload length] ""
virtual VS2FTP
SSL::enable
TCP::release
log local0. "TCP Release Completed"
}

3) Define a second standard virtual server on an internal address (where the first virtual server connects to) named "internal-y.y.y.y-999" with FTP servers (on port 21) as pool members. Apply the persistence profile. It needs the following iRule:

when CLIENT_ACCEPTED {
TCP::collect
}
when CLIENT_DATA {
if { [TCP::payload] contains "PBSZ" }  {
TCP::payload replace 0 [TCP::payload length] ""
TCP::respond "200 PBSZ 0 successful\r\n"
} elseif { [TCP::payload] contains "PROT P" } {
TCP::respond "200 Protection set to Private\r\n"
TCP::payload replace 0 [TCP::payload length] ""
} elseif { [TCP::payload] contains "FEAT" } {
TCP::payload replace 0 [TCP::payload length] ""
TCP::respond "211-Features:  MDTM  REST STREAM  SIZE  AUTH TLS  PBSZ  PROT\r\n211 End\r\n"
}  
TCP::release
TCP::collect
}
when SERVER_CONNECTED {
TCP::collect
}
when SERVER_DATA {
if { [TCP::payload] contains "220 " } {
TCP::payload replace 0 [TCP::payload length] ""
} elseif { [TCP::payload] contains "Entering Passive Mode" } {
# You need to modify this section if your servers are not
# configured to hand out the VIP address for Passive transfers.
#regsub {10,10,71,1} [TCP::payload] "172,16,59,163" tmpstr
#TCP::payload replace 0 [TCP::payload length] $tmpstr
} 
TCP::release
TCP::collect
}

4) The third virtual server catches inbound client passive transfer connections. It is defined on VIP address x.x.x.x and all ports. It must have the same CLIENTSSL profile as the first virtual server, the same pool as the first virtual server, and the same persistence profile as the first virtual server. It needs this iRule as well:

when CLIENT_ACCEPTED {
SSL::disable
TCP::collect 0 0
}
when CLIENT_DATA {
SSL::enable
}
when SERVER_CONNECTED {
SSL::enable clientside
}