SNI Routing with BIG-IP
In the previous article, The Three HTTP Routing Patterns, Lori MacVittie covers 3 methods of routing. Today we will look at Server Name Indication (SNI) routing as an additional method of routing HTTPS or any protocol that uses TLS and SNI. Using SNI we can route traffic to a destination without having to terminate the SSL connection. This enables several benefits including: Reduced number of Public IPs Simplified configuration More intelligent routing of TLS traffic Terminating SSL Connections When you have a SSL certificate and key you can perform the cryptographic actions required to encrypt traffic using TLS. This is what I refer to as “terminating the SSL connection” throughout this article. When you want to route traffic this is a chicken and an egg problem, because for TLS traffic you want to be able to route the traffic by being able to inspect the contents, but this normally requires being able to “terminate the SSL connection”. The goal of this article is to layer in traffic routing for TLS traffic without having to require having/knowing the original SSL certificate and key. Server Name Indication (SNI) SNI is a TLS extension that makes it possible to "share" certificates on a single IP address. This is possible due to a client using a TLS extension that requests a specific name before the server responds with a SSL certificate. Prior to SNI, the other options would be a wildcard certificate or Subject Alternative Name (SAN) that allows you to specify multiple names with a single certificate. SNI with Virtual Servers It has been possible to use SNI on F5 BIG-IP since TMOS 11.3.0. The following KB13452 outlines how it can be configured. In this scenario (from the KB article) the BIG-IP is terminating the SSL connection. Not all clients support SNI and you will always need to specify a “fallback” profile that will be used if a SNI name is not used or matched. The next example will look at how to use SNI without terminating the SSL connection. SNI Routing Occasionally you may have the need to have a hybrid configuration of terminating SSL connections on the BIG-IP and sending connections without terminating SSL. One method is to create two separate virtual servers, one for SSL connections that the BIG-IP will handle (using clientssl profile) and one that it will not handle SSL (just TCP). This works OK for a small number of backends, but does not scale well if you have many backends (run out of Public IP addresses). Using SNI Routing we can handle everything on a single virtual server / Public IP address. There are 3 methods for performing SNI Routing with BIG-IP iRule with binary scan a. Article by Colin Walker code attribute to Joel Moses b. Code Share by Stanislas Piron iRule with SSL::extensions Local Traffic Policy Option #1 is for folks that prefer complete control of the TLS protocol. It only requires the use of a TCP profile. Options #2 and #3 only require the use of a SSL persistence profile and TCP profile. SNI Routing with Local Traffic Policy We will skip option #1 and #2 in this article and look at using a Local Traffic Policy for SNI Routing. For a review of Local Traffic Policies check out the following DevCentral articles: LTM Policy Jan 2015 Simplifying Local Traffic Policies in BIG-IP 12.1 June 2016 In previous articles about Local Traffic Policies the focus was on routing HTTP traffic, but today we will use it to route SSL connections using SNI. In the following example, using a Local Traffic Policy named “sni_routing”, we are setting a condition on the SSL Extension “servername” and sending the traffic to a pool without terminating the SSL connection. The pool member could be another server or another BIG-IP device. The next example will forward the traffic to another virtual server that is configured with a clientssl profile. This uses VIP targeting to send traffic to another virtual server on the same device. In both examples it is important to note that the “condition”/“action” has been changed from occurring on “request” (that maps to a HTTP L7 request) to “ssl client hello”. By performing the action prior to any L7 functions occurring, we can forward the traffic without terminating the SSL connection. The previous example policy, “sni_routing”, can be attached to a Virtual Server that only has a TCP profile and SSL persistence profile. No HTTP or clientssl profile is required! This method can also be used to solve the issue of how to consolidate multiple SSL virtual servers behind a single virtual server that have different APM and/or ASM policies. This is similar to the architecture that is used by the Container Connector for Cloud Foundry; in creating a two-tier load balancing solution on a single device. Routed Correctly? TLS 1.3 has interesting proposals on how to obscure the servername (TLS in TLS?), but for now this is a useful and practical method of handling multiple SSL certs on a single IP. In the future this may still be possible as well with TLS 1.3. For example the use of a HTTP Fronting service could be a tier 1 virtual server (this is just my personal speculation, I have not tried, at the time of publishing this was still a draft proposal). In other news it has been demonstrated that a combination of using SNI and a different host header can be used for “domain fronting”. A method to enforce consistent policy (prevent domain fronting) would be to layer in additional conditions that match requested SNI servername (TLS extension) with requested HOST header (L7 HTTP header). This would help enforce that a tenant is using a certificate that is associated with their application and not “borrowing” the name and certificate that is being used by an adjacent service. We don’t think of a TLS extension as an attribute that can be used to route application traffic, but it is useful and possible on BIG-IP.Three Ways to Specify Multiple Ports on a Virtual Server
Last month, community member Racquel Mays asked for some assistance with creating a local traffic policy to apply to a virtual server to listen only on specific ports. I knew of only two obvious ways to solve that problem until fellow F5er Simon Blakely dumped a whole bowl of awesome sauce on us. In this article, I’ll cover not one way, not two ways, but also a completely new to me third way: traffic matching criteria. Before jumping into the specifics, I want to clarify something in Racquel’s question. If a virtual server is listening on port 0, then it is listening on all ports, even if you then, after the fact, filter them down to a select interesting list. For details on packets/flows in the BIG-IP system, check out my Lightboard Lesson on that. Option 1 - Use an iRule This is almost always my first thought. Mostly because I love iRules and I’m comfortable with them. But iRules aren’t always the best option, so it’s a good idea to evaluate from an operational perspective as well as performance. If both are negligible, this solution is very simple to implement, easily understood, and because of the simplicity, fairly performant when compared with the next option. ltm rule allports_irule { when CLIENT_ACCEPTED { switch [TCP::local_port] { 80 - 8080 { pool nerdlife_pool } default { reject } } } } In this iRule, when the TCP connection is established, the local port (BIG-IP side of the client connection) is evaluated and if it handled by a switch case, the pool is selected. The same pool is not required for each service but is used in this example. Any port that is not specified in the switch statement will result in a rejected TCP connection to the client. Option 2 - Use a Local Traffic Policy For a growing set of services, iRules can be retired and polices can be used in their place. They are built-in to TMOS and (unless you call Tcl from them) do not require the Tcl interpreter and thus will likely be more performant. However, if you have to split functionality between iRules and policies on a single application service, my preference is to keep the logic in one place so operational complexity is reduced. ltm policy allports_policy { controls { forwarding } requires { tcp } rules { tcp-80 { actions { 0 { forward client-accepted select pool nerdlife_pool } } conditions { 0 { tcp client-accepted port local values { 80 } } } } tcp-8080 { actions { 0 { forward client-accepted select pool nerdlife_pool } } conditions { 0 { tcp client-accepted port local values { 8080 } } } ordinal 1 } tcp-all-else { actions { 0 { shutdown client-accepted connection } } conditions { 0 { tcp client-accepted port local not values { 80 8080 } } } ordinal 2 } } status published strategy first-match } The policy is a tad verbose, no? This is due to the hierarchical nature of policies, having the policy root details like publications status and matching strategy, and then the rules for conditions and actions. Here in the text it’s easy to spot the condition of the local TCP port, but in the GUI, you have to select that in a hidden options menu. That got me for a hot minute while trying to make the policy work. In any event, options one and two are functionally equivalent when applied to a virtual server listening on port 0 with a TCP profile applied, so either approach is a good option depending on your business and technical standards for how BIG-IP objects are used. Option 3 - Traffic Matching Criteria I wanted to write the article when Simon shared this solution so I could be cool by saying “I was today years old when I learned you could do this” but alas, priorities! In this solution, there are two differences in this approach from the other two: multiple addresses as well as multiple ports are supported, so you can scale without requiring configuring additional virtuals there is no way in the criteria to map service port -> pool, so if you need that beyond a default pool in the virtual config, you're going to need an iRule or policy anyway You can find and create address and port lists in the GUI under Main->Shared Objects. I started my work in tmsh for this article and the lists are in the tmsh /net namespace, so I had to hunt for these as I was looking for them in the GUI’s Network section. You’ll need those to create your own traffic-matching-criteria object, but you can’t do that directly in the GUI. It is done on your behalf when you specify the lists in the creation of a virtual server. This will create this traffic-matching-criteria object in tmsh: ltm traffic-matching-criteria vip3_tmc_VS_TMC_OBJ { destination-address-inline 0.0.0.0 destination-address-list dal1 destination-port-list dpl1 protocol tcp source-address-inline 0.0.0.0 } It appends _VS_TMC_OBJ to the virtual server name as the object name it creates. The GUI also specifies the destination-address-inline and source-address-inline arguments, but I didn't need those in my object for it to work. That said, if the GUI selects them, there's probably a good reason so maybe default there as well and adjust as necessary in testing. Do It Yourself You already have the iRule and policy you need to test in your own lab above, now all you need are the other the virtual servers for options one and two and then the lists, traffic-matching-criteria, and virtual for option three. You can create all those with the tmsh commands below. My client-side lab network is 192.168.102.0/24 and I have a working pool (nerdlife_pool) on the server-side. You’ll need to swap those values out for your environment. # With iRule - Address 192.168.102.61 # tmsh create ltm virtual vip1_irule destination 192.168.102.61:0 ip-protocol tcp profiles add { http { } tcp { } } rules { allports_irule } source-address-translation { type automap } # With Policy - Address 192.168.102.62 # tmsh create ltm virtual vip2_policy destination 192.168.102.62:0 ip-protocol tcp profiles add { http { } tcp { } } policies add { allports_policy { } } source-address-translation { type automap } # With Matching Criteria - Address 192.168.102.63 # # If you do this in the GUI, it creates the traffic-matching-criteria for you, but no control of naming or modification that way tmsh create /net address-list dal1 addresses add { 192.168.102.63 } tmsh create /net port-list dpl1 ports add { 80 8080 } tmsh create /ltm traffic-matching-criteria tmc1 protocol tcp destination-address-list dal1 destination-port-list dpl1 tmsh create /ltm virtual vip3_tmc ip-protocol tcp traffic-matching-criteria tmc1 source-address-translation { type automap } pool nerdlife_pool profiles add { tcp http } You should now have three virtual servers with like functionality. Test Results Now that everything is configured, let’s test it out! We’ll test the two ports we want to answer, 80 and 8080, and then a third port we don’t, 8081. We should get an “achievement: unlocked” on the first two and a connection reset on the third. # Test results - iRule rahm@FLD-ML-00029232 ~ % curl http://192.168.102.61/a achievement: unlocked rahm@FLD-ML-00029232 ~ % curl http://192.168.102.61:8080/a achievement: unlocked rahm@FLD-ML-00029232 ~ % curl http://192.168.102.61:8081/a curl: (56) Recv failure: Connection reset by peer # Test results - Policy rahm@FLD-ML-00029232 ~ % curl http://192.168.102.62/a achievement: unlocked rahm@FLD-ML-00029232 ~ % curl http://192.168.102.62:8080/a achievement: unlocked rahm@FLD-ML-00029232 ~ % curl http://192.168.102.62:8081/a curl: (56) Recv failure: Connection reset by peer Test results - TMC rahm@FLD-ML-00029232 ~ % curl http://192.168.102.63/a achievement: unlocked rahm@FLD-ML-00029232 ~ % curl http://192.168.102.63:8080/a achievement: unlocked rahm@FLD-ML-00029232 ~ % curl http://192.168.102.63:8081/a curl: (7) Failed to connect to 192.168.102.63 port 8081: Connection refused The traffic-matching-criteria virtual didn't reset like the other two explicitly configured to do so, but it did refuse the connection all the same. I couldn't walk away from that without understanding why so I fired up Wireshark and took a tcpdump capture to investigate. What’s happening here is that with the iRule and policy solutions, we are acting at the CLIENT_ACCEPTED event, which means the TCP session has already been established and the HTTP request is in flight. So you see two TCP resets for each of the curl requests to 192.168.102.61:8081 and 192.168.102.62:8081, the first to the rejected TCP port, and the second to the HTTP request, which because it was already in flight, arrived even before the first TCP reset was sent. With the traffic-matching-criteria solution, it’s all wrapped up before a connection is even established so only the one reset message. That results in greater efficiency with local resources. Conclusion I say this a lot but I’m always excited to learn new things, particularly new ways of doing something I’ve done differently for years. If you’ve ever watched a master craftsman work, they have their favorite tools, but they have specialty tools as well for situations that call for it. I liken discoveries like this to that scenario. I’m not sure how many use cases I’d find for the traffic-matching-criteria that wouldn’t also rely on a policy and/or iRule, but I’m happy to now know that it exists just in case. How many of you knew about this and didn’t tell me? Drop me a comment below!
