SNI Routing with BIG-IP

[Eric_Chen]

In the previous article, The Three HTTP Routing Patterns, Lori MacVittie covers 3 methods of routing. Today we will look at Server Name Indication (SNI) routing as an additional method of routing HTTPS or any protocol that uses TLS and SNI.

Using SNI we can route traffic to a destination without having to terminate the SSL connection. This enables several benefits including:

• Reduced number of Public IPs
• Simplified configuration
• More intelligent routing of TLS traffic

Terminating SSL Connections

When you have a SSL certificate and key you can perform the cryptographic actions required to encrypt traffic using TLS.  This is what I refer to as “terminating the SSL connection” throughout this article.  When you want to route traffic this is a chicken and an egg problem, because for TLS traffic you want to be able to route the traffic by being able to inspect the contents, but this normally requires being able to “terminate the SSL connection”.  The goal of this article is to layer in traffic routing for TLS traffic without having to require having/knowing the original SSL certificate and key.

Server Name Indication (SNI)

SNI is a TLS extension that makes it possible to "share" certificates on a single IP address. This is possible due to a client using a TLS extension that requests a specific name before the server responds with a SSL certificate.

Prior to SNI, the other options would be a wildcard certificate or Subject Alternative Name (SAN) that allows you to specify multiple names with a single certificate.

SNI with Virtual Servers

It has been possible to use SNI on F5 BIG-IP since TMOS 11.3.0. The following KB13452 outlines how it can be configured. In this scenario (from the KB article) the BIG-IP is terminating the SSL connection.  Not all clients support SNI and you will always need to specify a “fallback” profile that will be used if a SNI name is not used or matched. The next example will look at how to use SNI without terminating the SSL connection.

SNI Routing

Occasionally you may have the need to have a hybrid configuration of terminating  SSL connections on the BIG-IP and sending connections without terminating SSL.  

One method is to create two separate virtual servers, one for SSL connections that the BIG-IP will handle (using clientssl profile) and one that it will not handle SSL (just TCP). This works OK for a small number of backends, but does not scale well if you have many backends (run out of Public IP addresses).

Using SNI Routing we can handle everything on a single virtual server / Public IP address. There are 3 methods for performing SNI Routing with BIG-IP

1. iRule with binary scan a. Article by Colin Walker code attribute to Joel Moses b. Code Share by Stanislas Piron
2. iRule with SSL::extensions
3. Local Traffic Policy

Option #1 is for folks that prefer complete control of the TLS protocol. It only requires the use of a TCP profile. Options #2 and #3 only require the use of a SSL persistence profile and TCP profile.

SNI Routing with Local Traffic Policy

We will skip option #1 and #2 in this article and look at using a Local Traffic Policy for SNI Routing. For a review of Local Traffic Policies check out the following DevCentral articles:

• LTM Policy Jan 2015
• Simplifying Local Traffic Policies in BIG-IP 12.1 June 2016

In previous articles about Local Traffic Policies the focus was on routing HTTP traffic, but today we will use it to route SSL connections using SNI.

In the following example, using a Local Traffic Policy named “sni_routing”, we are setting a condition on the SSL Extension “servername” and sending the traffic to a pool without terminating the SSL connection. The pool member could be another server or another BIG-IP device.

The next example will forward the traffic to another virtual server that is configured with a clientssl profile.  This uses VIP targeting to send traffic to another virtual server on the same device.

In both examples it is important to note that the “condition”/“action” has been changed from occurring on “request” (that maps to a HTTP L7 request) to “ssl client hello”.  By performing the action prior to any L7 functions occurring, we can forward the traffic without terminating the SSL connection.  The previous example policy, “sni_routing”, can be attached to a Virtual Server that only has a TCP profile and SSL persistence profile.  No HTTP or clientssl profile is required!

This method can also be used to solve the issue of how to consolidate multiple SSL virtual servers behind a single virtual server that have different APM and/or ASM policies.  This is similar to the architecture that is used by the Container Connector for Cloud Foundry; in creating a two-tier load balancing solution on a single device.

Routed Correctly?

TLS 1.3 has interesting proposals on how to obscure the servername (TLS in TLS?), but for now this is a useful and practical method  of handling multiple SSL certs on a single IP.  In the future this may still be possible as well with TLS 1.3.  For example the use of a HTTP Fronting service could be a tier 1 virtual server (this is just my personal speculation, I have not tried, at the time of publishing this was still a draft proposal). 

In other news it has been demonstrated that a combination of using SNI and a different host header can be used for “domain fronting”.  A method to enforce consistent policy (prevent domain fronting) would be to layer in additional conditions that match requested SNI servername  (TLS extension) with requested HOST header (L7 HTTP header).  This would help enforce that a tenant is using a certificate that is associated with their application and not “borrowing” the name and certificate that is being used by an adjacent service.

We don’t think of a TLS extension as an attribute that can be used to route application traffic, but it is useful and possible on BIG-IP.

[Stanislas_Piro2]

Nice article!

Before writing my irule, I tried with policy but I forgot to change default "request" to "ssl client hello" in action —> this policy required a http profile and didn’t work! 😞

Even if ssl persistence requirement seems like a bug at the beginning, it makes sense to manage persistence on pool assignment and enable ssl handshake inspection without clientssl profile.

I will try this and update the codeshare to add this article link!

[Eric_Chen]

Thanks!

Your iRule is very nice and useful if you want to combine with tactics like TLS Fingerprinting.

AFAIK, the requirement for SSL persistence profile enables inspection of the connection without requiring a clientssl profile. Normally it is used to extract the SSL session ID, but the TLS extensions is also available.

To prove that no clientssl profile is required, here's an example from my lab setup (using tmsh output).

ltm virtual test_vs {
    destination 192.0.2.10:443
    ip-protocol tcp
    mask 255.255.255.255
    persist {
        ssl {
            default yes
        }
    }
    policies {
        sni_routing { }
    }
    profiles {
        tcp { }
    }
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    translate-address enabled
    translate-port enabled
    vs-index 5
}

and here's the matching policy (the screenshots in the article are inverted from my diagram, I updated the following to match the diagram).

ltm policy sni_routing {
    controls { forwarding }
    last-modified 2018-05-23:00:37:54
    requires { client-ssl }
    rules {
        rule_001 {
            actions {
                0 {
                    forward
                    ssl-client-hello
                    select
                    virtual /Common/test_ssl_vs
                }
            }
            conditions {
                0 {
                    ssl-extension
                    ssl-client-hello
                    server-name
                    values { app1.example.com }
                }
            }
            description sni:app1.example.com,virtual:test_ssl_vs
            ordinal 2
        }
        rule_002 {
            actions {
                0 {
                    forward
                    ssl-client-hello
                    select
                    pool test_ssl
                }
            }
            conditions {
                0 {
                    ssl-extension
                    ssl-client-hello
                    server-name
                    values { app2.example.com }
                }
            }
            description sni:app2.example.com,pool:test_ssl
            ordinal 3
        }
    }
    status published
    strategy first-match
}

On version 13.1 (used above) it will show in the policy requires "client-ssl", prior to that you will see "ssl-persistence". In either version you can use the policy w/out a clientssl profile as you can see in the earlier output.

You can verify all is well by using openssl

[demo]$ echo | openssl s_client -showcerts -servername app1.example.com -connect 192.0.2.10:443 2>/dev/null |grep "subject="
subject=/CN=app1.example.com
[demo]$ echo | openssl s_client -showcerts -servername app2.example.com -connect 192.0.2.10:443 2>/dev/null |grep "subject="
subject=/CN=app2.example.com

Checking out the stats via the GUI or TMSH is helpful too.

(tmos) show /ltm policy sni_routing

---------------------------
Ltm::Policy: sni_routing
---------------------------
Virtual Server Name     N/A

Status
  Actions invoked   :     7
  Actions succeeded :     7

  --------------------------------------------------
  | Rule                  Action  Invoked  Succeeded
  --------------------------------------------------
  | rule_001  0 [forward select]        4          4
  | rule_002  0 [forward select]        3          3

[Eric_Chen]

adding the second virtual for completeness. In this case the SSL certificate is known by the BIG-IP (app1.example.com). The second pool "test_ssl" points directly to the backend web server and the BIG-IP does not have the certificate (app2.example.com).

The virtual listens on port 9443, but you could also use a different private address and/or restrict connections (i.e. set the source to be 192.0.2.10/32).

ltm virtual test_ssl_vs {
    destination 192.0.2.10:9443
    ip-protocol tcp
    mask 255.255.255.255
    pool test_pool
    profiles {
        app1.example.com_clientssl {
            context clientside
        }
        http { }
        tcp { }
    }
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    translate-address enabled
    translate-port enabled
    vs-index 10
}

[dragonflymr]

Hi,

As usual very good article! I wonder if this is better (at least performance wise) to use vip targeting that VS with client/server ssl profile and iRule for disabling and enabling client ssl and server ssl profiles based on some conditions.

As far as I tested doing so via policy is not possible so in this case vip targeting seems to be only option.

I wonder as well why at least ssl persistence profile is needed on VS - is this profile necessary for enabling SSL handshake related events? How having ssl persistence can influence traffic distribution if pool is selected instead of vs. I guess in case vs is selected there is none but if pool will persistence will kick in - I mean SSL sessions with same ID will be send to pool member with existing persistence record?

Sometimes it could be unwanted behavior.

Piotr

[Eric_Chen]

Re: performance

I have not done any performance comparison.

Re: iRule for enable client/server ssl profile

You can only select a server ssl profile from a Local Traffic Policy. IMHO it makes sense to use a Local Traffic Policy as "tier 1" that uses VIP targeting (via the policy) to multiple different virtual servers that have different clientssl profile attached.

Alternately, you could set a variable in the local traffic policy and later use an iRule (attached to the same virtual server as the local traffic policy) to assign the clientssl profile via SSL::profile (I have not tried this, seems plausible).

not sure if that answers your question.

Re: why ssl persistence

My best guess is that it exposes variables that are necessary by Local Traffic Policy. For example if you try to invoke an iRule that acts on the CLIENTSSL_CLIENTHELLO event you will get the following error

01071912:3: CLIENTSSL_CLIENTHELLO event in rule (/Common/test_sni) requires an associated CLIENTSSL or PERSIST profile on the virtual-server (/Common/test_sni).

Re: ssl persistence on pool

If you are sending traffic to a pool you may want to have a combination of ssl persistence and source ip persistence. otherwise you could see an issue where a client will renegotiate the ssl session. this could occur when reconnecting and getting sent to a different backend server.

agree that the behavior could be unwanted depending on the use-case. typically you want to preserve connections to the same backend for SSL connections.

Thanks for your feedback/questions!

Eric

[dragonflymr]

Hi,

Thanks a lot for answers. I was rather thinking about scenario when VS has:

• Multiple client ssl profiles attached (ClientHello SNI matching)
• No Server SSL profile (SSL offload or SSL pass through only)

not about selecting client ssl profile via Local Traffic Policy (LTP). Then logic like in Stanislas iRule used:

• For given SNIs do SSL offload (so enable client ssl after it was disabled at the beginning of CLIENT_ACCEPTED)
• For given SNIs do SSL pass through (client ssl not enabled)

but it is not possible with LTP (at least I can't see how) because Action: Disable client ssl blocks condition SSL Extension - what is quite logic.

So indeed your solution with pool or vs forwarding is only solution. In the end it's simpler and more elegant so thanks again for sharing.

What I miss regarding LTP is lack of in depth description of all elements and how they interact with each other and how choice of given condition or action relates to necessary profiles. Sure it can be figured out (sometimes with lab tests) but it takes time 😞

Piotr

[Stanislas_Piro2]

Hi,

I tried this configuration and it works like a charm!

Do you know the minimum TMOS version required to do such configuration?

I also tried to look at fingerprinting irule as you suggested but the database is not yet updated and both my chrome and firefox uses ciphers not listed at all in the son file (0x1301, 0x1302 and 0x1303)

This TLS routing feature is really useful for customers without enough available IP addresses and requiring different services behavior.

when working with APM, Rewriting irules, or more complex solutions splitting services on multiple virtual servers, this can solve several issues.

Another use case is when a customer protect all services in Azure (or other cloud). only one public IP address is assigned to the BigIP VE. In this case, this solution can make the configure simpler!

I read TLS 1.3 draft and I didn't see any reference of server name extension obscured, but this server name extension is required for 0-RTT. I still think such solution will be compatible with TLS 1.3.

[Eric_Chen]

Re: minimum TMOS version

This works at least on 12.1.x and newer, but it may also work in older versions (Local Traffic Policies was introduced in 11.4.0, I only tested the above on 12.1.x and 13.1.x).

I agree that this can be a useful strategy for keeping it simple and secure.

Thanks for taking a look at TLS Fingerprinting/TLS 1.3. I have not deep-dived into either yet.

[dragonflymr]

Hi Eric,

I noticed this when reading article again. Maybe I am wrong but for me SNI Routing drawing do not match LTP crated.

For App1 on drawing description says HTTP (SSL Offload), but description of LTP rule says:

"...we are setting a condition on the SSL Extension “servername” and sending the traffic to a pool without terminating the SSL connection. The pool member could be another server or another BIG-IP device.

So for me first rule is in fact doing SSL Pass Trough and second rule is doing kind of SSL Offload (assuming that targeted VS is in front of destination server without SSL - so VS has Client SSL profile attached, decrypts traffic and sends it to the target server using configured Pool.

Am I right or wrong here?

Piotr

[Eric_Chen]

Piotr,

Yes, I think between writing the article and creating the lab setup there is inconsistency between the text/diagrams/configurations.

Looking back at the diagram and screen shots it looks like I inverted them. "app1" (from the config screenshot) is actually doing pass through (no SSL termination) and "app2" is forwarding to a virtual server that is doing SSL offload.

I will leave the article as-is, but for the observant they should take note of this from the comments. Thank you!

[dragonflymr]

Hi Eric,

At least it means I was reading article with some understanding 🙂

I wonder if you can help with understanding LTP conditions evaluation. I am trying to setup VS on both 80 and 443 ports (as you know possible in v 14.1+). When dst port is 80 I am disabling Client SSL profile at client accepted (separate policy). Then I am checking Host header to make forwarding decision. So far so good. Then when port is 443 I am not disabling Client SSL and trying to check SNI at SSL Client Hello - now everything breaks for port 80 - immediate TCP RST. It looks like even if traffic is not matching port condition LTM is still trying to check SSL Extension - and this is not present in HTTP request, so no target is selected and there is failure. At the same time traffic to port 443 works without issue.

If only port 443 is in conditions then both HTTP and HTTPS works.

Here is LTP I tried to use:

ltm policy vh-04_ltp {
controls { forwarding server-ssl }
last-modified 2019-02-19:16:47:48
requires { http tcp client-ssl }
rules {
    nothing-matched_r {
        actions {
            0 {
                log
                write
                facility local0
                message "No rule was matched"
                priority info
            }
        }
        ordinal 3
    }
    site41-http-https_r {
        actions {
            0 {
                log
                write
                facility local0
                message "site41-http-https hit"
                priority info
            }
            1 {
                forward
                select
                pool site41-443_pl
            }
            2 {
                tcl
                set-variable
                expression site41-http-https.f5demo.com
                name host_ltp
            }
        }
        conditions {
            0 {
                http-host
                host
                values { site41-http-https.vlab.f5demo.com }
            }
            1 {
                tcp
                client-accepted
                port
                local
                values { 80 }
            }
        }
        ordinal 1
    }
    site41-http-to-http_r {
        actions {
            0 {
                forward
                select
                pool site41-80_pl
            }
            1 {
                server-ssl
                disable
            }
            2 {
                log
                write
                facility local0
                message "site41-http-http hit"
                priority info
            }
        }
        conditions {
            0 {
                http-host
                host
                values { site41-http-http.vlab.f5demo.com }
            }
            1 {
                tcp
                client-accepted
                port
                local
                values { 80 }
            }
        }
    }
    site41-https-https_r {
        actions {
            0 {
                log
                write
                facility local0
                message "site41-https-https HIT"
                priority info
            }
            1 {
                forward
                select
                pool site41-443_pl
            }
            2 {
                tcl
                set-variable
                expression site41-https-https.f5demo.com
                name host_ltp
            }
        }
        conditions {
            0 {
                tcp
                client-accepted
                port
                local
                values { 443 }
            }
            1 {
                ssl-extension
                ssl-client-hello
                server-name
                values { site41-https-https.vlab.f5demo.com }
                missing
            }
        }
        ordinal 2
    }
}
status published
strategy first-match

}

I know how to do that other way, I am just curious why SSL Extension condition is breaking HTTP part.

Piotr

[Eric_Chen]

@Piotr,

Re: why is SSL condition breaking HTTP part

The short answer is that you probably want to have a separate LTP for HTTP and HTTPS.

I suspect the issue is that with HTTP you are missing condition HTTP request for checking a host header and that is incompatible with your condition for SSL Client Hello for HTTPS. These two will keep on conflicting. With an iRule you could disable HTTP or SSL Persistence, but not sure whether it is feasible with a LTP.

[dragonflymr]

Hi Eric,

Weird part is that you can actually disable both SSL persistence and HTTP profile. Disabling HTTP profile allows HTTPS traffic to pass through to another VS or Pool. Disabling Persistence do not help with HTTP traffic 😞

Best result I was able to achieve is to forward HTTP traffic to another VS based on dst port (has to be set as first rule).

For HTTPS I am able (disabling HTTP profile) to both forward traffic to another VS and directly to pool based on SNI.

What bothers me is logic of performing Condition test. If my rule contains Condition TCP port at local side equal to 443 at client accepted then it should fail for HTTP (local port 80) and pass processing to another rule.

It seems however that even both HTTPS related rules are failing they as well somehow stoping progress of connection processing (at least for rule conditions).

Having iRule attached to the VS I can see log entries like that:

• No match (from LTP log action)
• Host is some.site.com from HTTP_REQUEST event log action

So in the end traffic processing is continued for HTTP but it seems to be too late for rule condition. I wonder if this is feature or bug?

Piotr

[stboiss]

Hi,

I have trouble implementing this solution.

Can you help me configure the first VS with the policy ?

I have :

1st VS with policy and ssl client profile "test"

the policy forward traffic to a 2nd VS

2nd VS with ssl client profile "subtest"

what is the good config for ssl client profile "test" ?

thanks in advance

[BinChen]

Hi, Eric, per your SNI Routing diagram above, it seems it is possible to use SNI for SSL pass-through as well as SSL termination. Do you have an example of how to implement SNI Routing diagram above? Thanks.