Securing Client-Side and Server-Side SMTP Traffic

Dear all,

I'm looking to secure my SMTP server using STARTLS, on the client-side and the server-side. The SMTP server has been configured to listen on port 25 and supports STARTTLS (using self-signed certificate on the server itself). From the F5-LTM, I can create a SMTPS profile to enable the TLS feature on port 25 on the client-side, but I can't do it on the server-side. 

Here is my SMTPS profile config:

ltm profile smtps smtp-tls-25_smtps {
    activation-mode allow
    app-service none
    defaults-from /Common/smtps
}

Here is my Client-SSL config:

ltm profile client-ssl smtp.domain.com_CS {
    app-service none
    cert-key-chain {
        smtp.domain_Sectigo_cert_chain_0 {
            cert smtp.domain.com
            chain /Common/Sectigo_cert_chain.crt
            key smtp.domain.com
        }
        smtp.domain_Sectigo_cert_chain_1 {
            cert smtp.domain.com
            chain /Common/Sectigo_cert_chain.crt
            key smtp.domain.com
            usage CA
        }
    }
    defaults-from /Common/udes-default-clientssl_profile
    inherit-ca-certkeychain false
    inherit-certkeychain false
    ssl-forward-proxy enabled
    ssl-forward-proxy-verified-handshake enabled
}

Here is my Server-SSL config:

ltm profile server-ssl smtp.domain.com-proxy-fwd_SS {
    app-service none
    defaults-from /Common/udes-default-serverssl_profile
    revoked-cert-status-response-control ignore
    ssl-forward-proxy enabled
    ssl-forward-proxy-verified-handshake enabled
}

Here's my Pool config:

ltm pool smtp-25_pool {
    description "Test de passerelles SMTP sur le port 25"
    members {
        smtpi-dev01_node:25 {
            address 10.32.160.127
            session monitor-enabled
            state up
        }
    }
    monitor /Common/gateway_icmp and smtp-25_hm
    partition INFRA-DEV
}

Here's my Virtual Server config:

ltm virtual smtp-25_vs {
    destination 1.1.1.1%1:smtp
    ip-protocol tcp
    last-modified-time 2022-11-01:17:36:56
    mask 255.255.255.255
    partition INFRA-DEV
    pool smtp-25_pool
    profiles {
        /Common/tcp { }
        smtp-tls-25_smtps { }
        smtp.domain.com-proxy-fwd_SS {
            context serverside
        }
        smtp.domain.com_CS {
            context clientside
        }
    }
    rules {
        /Common/logging_clients_tcp_v3
    }
    serverssl-use-sni disabled
    source 0.0.0.0/0
    source-address-translation {
        pool natpool-inside-vlan6
        type snat
    }
    translate-address enabled
    translate-port enabled
    vs-index 36
}

When I issue the command:

openssl s_client -showcerts -starttls smtp -connect smtp-dev.domain.com:25

I got this result:

CONNECTED(00000003)
didn't found starttls in server response, try anyway...
139865767704464:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 324 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1667338630
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

 Doing the same command directly on the backend server, I got:

ONNECTED(00000003)
depth=0 CN = smtpi-dev01.domain.com
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = smtpi-dev01.domain.com
verify return:1
---
Certificate chain
 0 s:/CN=smtpi-dev01.domain.com
   i:/CN=smtpi-dev01.domain.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFbDCCA1SgAwIBAgIUVujZu3QKSfTguvw+U67aQDl8d4swDQYJKoZIhvcNAQEL
...
-----END CERTIFICATE-----
subject=/CN=smtpi-dev01.domain.com
issuer=/CN=smtpi-dev01.domain.com
---
No client certificate CA names sent
Peer signing digest: SHA256
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2647 bytes and written 450 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: E9C2D4E06675C447B026017E368AE3148F25A4118DB2D09FC53A1A97AD22165B
    Session-ID-ctx: 
    Master-Key: E12AFB816901DAABAA5BC25F7CF144138E1F1FAF82FA62A3CB734B445FC7420616B571E6377FE2E08257D08DD2B1B651
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 1 (seconds)
    TLS session ticket:
    0000 - 90 89 7c db d6 aa b9 18-5a bf 98 35 04 c0 8f 5c   ..|.....Z..5...\
    0010 - 74 29 37 2e 30 5b 97 98-11 84 51 6e c2 57 90 e4   t)7.0[....Qn.W..
    0020 - 18 33 fc 1b 64 be 35 2f-15 0a 2c b1 f2 7b f1 5b   .3..d.5/..,..{.[
    0030 - 2b 6f 69 da 5a 58 26 42-db 74 61 7e 63 f0 4c 75   +oi.ZX&B.ta~c.Lu
    0040 - 85 d5 11 ae 0c a3 d4 69-cf 23 35 ad 58 05 40 44   .......i.#5.X.@D
    0050 - 89 32 50 af c7 36 65 35-48 3e 1c c2 31 f3 d8 84   .2P..6e5H>..1...
    0060 - 3a b6 3c 52 2f 3c 94 90-3f c6 77 e1 b4 9a 01 54   :.<R/<..?.w....T
    0070 - 90 6a 0a c3 6e e3 20 1c-71 aa 66 7e bb 07 60 fe   .j..n. .q.f~..`.
    0080 - f3 41 e2 73 94 0f 25 f9-70 92 9c ac 01 ef 26 d2   .A.s..%.p.....&.
    0090 - 42 c9 bd aa 84 41 79 21-05 09 a7 16 cd 31 7c 2c   B....Ay!.....1|,

    Start Time: 1667339374
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---
250 HELP

So I'm wondering how to have thin working. We are using port 25 for the client because we want to allow user to connect to the server in plain text (no SSL) as well as TLS (starttls). The communication to the backend servers can always be encrypted. I tried use the port 465 into the pool definition (SMTP server has been configured to listen on that port), but I got the same answer (didn't found starttls in server response, try anyway...) .

I think I will need to use some iRules to enable STARTTLS on server side, but I'm not sure how to configure it.

Thank you all in advance for your help!

Best Regards,
Yanick