STARTTLS Server SMTP with cleartext and STARTTLS client support

Problem this snippet solves:

We were looking at our O365 security score and our SMTP scores were pretty sad, so I looked at how I could create a STARTTLS connection up to O365 regardless of the client support. Originally I just supported cleartext clients, but I was able to get it to support clients running STARTTLS too. Links in the iRule to some helpful threads I used while developing this.

How to use this snippet:

Attach this iRule to a SMTP VS with a clientssl, serverssl and a SMTP profile.

Code :

# This iRule allows dynamic cleartext and STARTTLS client connection with a STARTTLS connection to the upstream server
# unfortunately, cleartext to server connections do not work with the iRule as-is
when CLIENT_ACCEPTED {
    # No SSL client side, also check no SSL running already on server side
    # Debug mode
set DEBUG 1
set SERVER_SSL 0
set CLIENT_SSL 0
set EHLO_Name "smtp.domain.com"
    if { $DEBUG } { log local0. "CLIENT_ACCEPTED" }
    SSL::disable serverside 
    # Disable All TLS so we can dynamically enable it
    SSL::disable 
}
when SERVER_CONNECTED {
    if { $DEBUG } { log local0. "SERVER_CONNECTED" }
    # Start collecting from the server because in SMTP the server responds first
    TCP::collect
}

when CLIENT_DATA {
    set lcpayload [string tolower [TCP::payload]]
    if { $DEBUG } { log local0.debug "CLIENT_DATA - PAYLOAD - $lcpayload" }
    if { $lcpayload starts_with "ehlo" } {
        if { $DEBUG } { log local0.debug "CLIENT_DATA - ehlo" }
        # https://devcentral.f5.com/s/articles/offload-smtp-encryption-via-irules
        # Spoof back STARTTLS headers to the client so they'll tell us if the support it or not
        TCP::respond "250-STARTTLS\r\n250 OK\r\n"
        TCP::payload replace 0 [TCP::payload length] ""
        TCP::release
        TCP::collect
    } elseif { $lcpayload starts_with "starttls" } {
        if { $DEBUG } { log local0.debug "CLIENT_DATA - Starttls" }
        # https://devcentral.f5.com/s/articles/smtp-start-tls
        # Spoof back the 'Ready to start TLS' header to the client so we can do client<->F5 SSL
        TCP::respond "220 Ready to start TLS\r\n"
        TCP::payload replace 0 [TCP::payload length] ""
        TCP::release
        clientside {SSL::enable}
    } elseif { $lcpayload starts_with "rset" and $SERVER_SSL == 1 } {
        if { $DEBUG } { log local0.debug "CLIENT_DATA - Client RSET" }
        # In situations where the client is not encrypting, but the server is, it appears nessecary to reset and re-EHLO to the SMTP server.
        # In this case, these are the advertisements from the O365 relay so you may have to adjust them according to your SMTP server responses.
        # This can be attained by setting DEBUG variable to '1' and watching /var/log/ltm when you send a message for the response returned from 
        # the SMTP server for 'Hello'.
        # Example: Rule /Common/SMTP_STARTTLS : server SSL payload: 250-blah.mail.protection.outlook.com Hello [removed-IP]  250-SIZE 157286400  250-PIPELINING  250-DSN  250-ENHANCEDSTATUSCODES  250-8BITMIME  250-BINARYMIME  250-CHUNKING  250 SMTPUTF8 
        TCP::respond "250-STARTTLS 250-SIZE 157286400  250-PIPELINING  250-DSN  250-ENHANCEDSTATUSCODES  250-8BITMIME  250-BINARYMIME  250-CHUNKING  250 SMTPUTF8\r\n"
        TCP::payload replace 0 [TCP::payload length] ""
        TCP::release
        TCP::collect
        # /var/log/ltm complains about this, but it appears to work
        serverside { SSL::respond "EHLO $EHLO_Name\r\n" }
    } else {
        if { $DEBUG } { log local0.debug "CLIENT_DATA - Default release" }
        TCP::release
    }
}
when SERVER_DATA {
    # Most of thise was helpfully taken from https://devcentral.f5.com/s/questions/need-an-irule-for-starttls-for-smtps-server-side-only-not-client-side
    # Read in responses from remote server into a variable and log to /var/log/ltm
    set payload [string tolower [TCP::payload]]
    if { $DEBUG } { log local0. "SERVER_DATA - PAYLOAD - $payload" }
    if {$payload starts_with "220" and $payload contains "esmtp"} {    
        # Listen for remote servers opening 220 and esmtp message 
        # NOTE the ‘if’ statement above may need to be tweaked to except what message the other 
        # side is actually sending in reply. Logs should show this.
        # Respond with a EHLO to server, most servers require a name after the EHLO as well.

        TCP::respond "EHLO $EHLO_Name\r\n" 
        TCP::payload replace 0 [TCP::payload length] ""
        TCP::release
        if { $DEBUG } { log local0. "SERVER_DATA - Responded to server with EHLO" }
        serverside {TCP::collect}
    } elseif {$payload contains "250-starttls" } {    
        # Check server responds with "250-starttls", if so, respond with a STARTTLS 
        TCP::respond "STARTTLS\r\n" 
        TCP::payload replace 0 [TCP::payload length] ""
        TCP::release
        if { $DEBUG } { log local0. "SERVER_DATA - Sent the server a STARTTLS" }
        serverside {TCP::collect}
    } elseif {$payload contains "220 ready for tls" or $payload contains "220 2.0.0 continue" or $payload contains "220 2.0.0 smtp server ready" } {    
        # if server gives a 220 response, then start server side ssl profile
        # NOTE the ‘if’ statement above may need to be tweaked to except what message the other 
        # side is actually sending in reply. Logs should show this.
        ######
        # O365 Edit - O365 returns 220 2.0.0 smtp server ready after enabling TLS - Adjust as needed
        ######
        if { $DEBUG } { log local0. "SERVER_DATA - server said he is ready for TLS, enable the SSL profile" }
        TCP::payload replace 0 [TCP::payload length] ""
        TCP::release
        serverside {SSL::enable}
        # TLS hanshake should now start, which is best seen in wireshark packet captures.
    } else {
        if { $DEBUG } { log local0.debug "SERVER_DATA - Default release" }
        TCP::release
        clientside { TCP::collect }
    }
}

when SERVERSSL_HANDSHAKE {
    # This will only trigger if that is completed successfully. 
        # ServerSSL profile will need a certificate to match the outbound IP and DNS name, 
        # and you may want to set the "Server certificate" setting to "require", 
        # and the "Trusted Certificate Authorities" set to "ca-bundle".
    if { $DEBUG } { log local0. "SERVERSSL_HANDSHAKE - SSL handshake completed." }
    set SERVER_SSL 1
    if { $CLIENT_SSL == 1 } {
        if { $DEBUG } { log local0.debug "SERVERSSL_HANDSHAKE - Client respond SSL" }
        clientside { SSL::respond "220 SMTP ESMTP Relay F5\r\n" }
    } else {
        if { $DEBUG } { log local0.debug "SERVERSSL_HANDSHAKE - Client respond TCP" }
        clientside { TCP::respond "220 SMTP ESMTP Relay F5\r\n" }
        # Give the client side a chance to STARTTLS
        clientside { TCP::collect }
    }
    if { $DEBUG } { log local0.debug "SERVERSSL_HANDSHAKE - SSL collect" }
    SSL::collect
}

when CLIENTSSL_HANDSHAKE {
    # This will only trigger if that is completed successfully. 
        # ServerSSL profile will need a certificate to match the outbound IP and DNS name, 
        # and you may want to set the "Server certificate" setting to "require", 
        # and the "Trusted Certificate Authorities" set to "ca-bundle".
    if { $DEBUG } { log local0. "SSL handshake completed." }
    set CLIENT_SSL 1
    SSL::collect
}

when SERVERSSL_DATA {
    # Log the SMTP responses to see any errors.
    if { $DEBUG } { log local0. "SERVERSSL_DATA - PAYLOAD - [SSL::payload]" }
    SSL::release
    if { $CLIENT_SSL == 0 } {
        if { $DEBUG } { log local0.debug "SERVERSSL_DATA - Client TCP Collect"}
        clientside { TCP::collect }
    }
    SSL::collect 
}

when CLIENTSSL_DATA {
    # Log the SMTP responses to see any errors.
    if { $DEBUG } { log local0. "CLIENTSSL_DATA - PAYLOAD - [SSL::payload]" }
    #log local0.debug "Clientssl_data - release"
    SSL::release
    SSL::collect
}