Multiple Certs, One VIP: TLS Server Name Indication via iRules
An age old question that we’ve seen time and time again in the iRules forums here on DevCentral is “How can I use iRules to manage multiple SSL certs on one VIP"?”. The answer has always historically been “I’m sorry, you can’t.”. The reasoning is sound. One VIP, one cert, that’s how it’s always been. You can’t do anything with the connection until the handshake is established and decryption is done on the LTM. We’d like to help, but we just really can’t. That is…until now. The TLS protocol has somewhat recently provided the ability to pass a “desired servername” as a value in the originating SSL handshake. Finally we have what we’ve been looking for, a way to add contextual server info during the handshake, thereby allowing us to say “cert x is for domain x” and “cert y is for domain y”. Known to us mortals as "Server Name Indication" or SNI (hence the title), this functionality is paramount for a device like the LTM that can regularly benefit from hosting multiple certs on a single IP. We should be able to pull out this information and choose an appropriate SSL profile now, with a cert that corresponds to the servername value that was sent. Now all we need is some logic to make this happen. Lucky for us, one of the many bright minds in the DevCentral community has whipped up an iRule to show how you can finally tackle this challenge head on. Because Joel Moses, the shrewd mind and DevCentral MVP behind this example has already done a solid write up I’ll quote liberally from his fine work and add some additional context where fitting. Now on to the geekery: First things first, you’ll need to create a mapping of which servernames correlate to which certs (client SSL profiles in LTM’s case). This could be done in any manner, really, but the most efficient both from a resource and management perspective is to use a class. Classes, also known as DataGroups, are name->value pairs that will allow you to easily retrieve the data later in the iRule. Quoting Joel: Create a string-type datagroup to be called "tls_servername". Each hostname that needs to be supported on the VIP must be input along with its matching clientssl profile. For example, for the site "testsite.site.com" with a ClientSSL profile named "clientssl_testsite", you should add the following values to the datagroup. String: testsite.site.com Value: clientssl_testsite Once you’ve finished inputting the different server->profile pairs, you’re ready to move on to pools. It’s very likely that since you’re now managing multiple domains on this VIP you'll also want to be able to handle multiple pools to match those domains. To do that you'll need a second mapping that ties each servername to the desired pool. This could again be done in any format you like, but since it's the most efficient option and we're already using it, classes make the most sense here. Quoting from Joel: If you wish to switch pool context at the time the servername is detected in TLS, then you need to create a string-type datagroup called "tls_servername_pool". You will input each hostname to be supported by the VIP and the pool to direct the traffic towards. For the site "testsite.site.com" to be directed to the pool "testsite_pool_80", add the following to the datagroup: String: testsite.site.com Value: testsite_pool_80 If you don't, that's fine, but realize all traffic from each of these hosts will be routed to the default pool, which is very likely not what you want. Now then, we have two classes set up to manage the mappings of servername->SSLprofile and servername->pool, all we need is some app logic in line to do the management and provide each inbound request with the appropriate profile & cert. This is done, of course, via iRules. Joel has written up one heck of an iRule which is available in the codeshare (here) in it's entirety along with his solid write-up, but I'll also include it here in-line, as is my habit. Effectively what's happening is the iRule is parsing through the data sent throughout the SSL handshake process and searching for the specific TLS servername extension, which are the bits that will allow us to do the profile switching magic. He's written it up to fall back to the default client SSL profile and pool, so it's very important that both of these things exist on your VIP, or you may likely find yourself with unhappy users. One last caveat before the code: Not all browsers support Server Name Indication, so be careful not to implement this unless you are very confident that most, if not all, users connecting to this VIP will support SNI. For more info on testing for SNI compatibility and a list of browsers that do and don't support it, click through to Joel's awesome CodeShare entry, I've already plagiarized enough. So finally, the code. Again, my hat is off to Joel Moses for this outstanding example of the power of iRules. Keep at it Joel, and thanks for sharing! when CLIENT_ACCEPTED { if { [PROFILE::exists clientssl] } { # We have a clientssl profile attached to this VIP but we need # to find an SNI record in the client handshake. To do so, we'll # disable SSL processing and collect the initial TCP payload. set default_tls_pool [LB::server pool] set detect_handshake 1 SSL::disable TCP::collect } else { # No clientssl profile means we're not going to work. log local0. "This iRule is applied to a VS that has no clientssl profile." set detect_handshake 0 } } when CLIENT_DATA { if { ($detect_handshake) } { # If we're in a handshake detection, look for an SSL/TLS header. binary scan [TCP::payload] cSS tls_xacttype tls_version tls_recordlen # TLS is the only thing we want to process because it's the only # version that allows the servername extension to be present. When we # find a supported TLS version, we'll check to make sure we're getting # only a Client Hello transaction -- those are the only ones we can pull # the servername from prior to connection establishment. switch $tls_version { "769" - "770" - "771" { if { ($tls_xacttype == 22) } { binary scan [TCP::payload] @5c tls_action if { not (($tls_action == 1) && ([TCP::payload length] > $tls_recordlen)) } { set detect_handshake 0 } } } default { set detect_handshake 0 } } if { ($detect_handshake) } { # If we made it this far, we're still processing a TLS client hello. # # Skip the TLS header (43 bytes in) and process the record body. For TLS/1.0 we # expect this to contain only the session ID, cipher list, and compression # list. All but the cipher list will be null since we're handling a new transaction # (client hello) here. We have to determine how far out to parse the initial record # so we can find the TLS extensions if they exist. set record_offset 43 binary scan [TCP::payload] @${record_offset}c tls_sessidlen set record_offset [expr {$record_offset + 1 + $tls_sessidlen}] binary scan [TCP::payload] @${record_offset}S tls_ciphlen set record_offset [expr {$record_offset + 2 + $tls_ciphlen}] binary scan [TCP::payload] @${record_offset}c tls_complen set record_offset [expr {$record_offset + 1 + $tls_complen}] # If we're in TLS and we've not parsed all the payload in the record # at this point, then we have TLS extensions to process. We will detect # the TLS extension package and parse each record individually. if { ([TCP::payload length] >= $record_offset) } { binary scan [TCP::payload] @${record_offset}S tls_extenlen set record_offset [expr {$record_offset + 2}] binary scan [TCP::payload] @${record_offset}a* tls_extensions # Loop through the TLS extension data looking for a type 00 extension # record. This is the IANA code for server_name in the TLS transaction. for { set x 0 } { $x < $tls_extenlen } { incr x 4 } { set start [expr {$x}] binary scan $tls_extensions @${start}SS etype elen if { ($etype == "00") } { # A servername record is present. Pull this value out of the packet data # and save it for later use. We start 9 bytes into the record to bypass # type, length, and SNI encoding header (which is itself 5 bytes long), and # capture the servername text (minus the header). set grabstart [expr {$start + 9}] set grabend [expr {$elen - 5}] binary scan $tls_extensions @${grabstart}A${grabend} tls_servername set start [expr {$start + $elen}] } else { # Bypass all other TLS extensions. set start [expr {$start + $elen}] } set x $start } # Check to see whether we got a servername indication from TLS. If so, # make the appropriate changes. if { ([info exists tls_servername] ) } { # Look for a matching servername in the Data Group and pool. set ssl_profile [class match -value [string tolower $tls_servername] equals tls_servername] set tls_pool [class match -value [string tolower $tls_servername] equals tls_servername_pool] if { $ssl_profile == "" } { # No match, so we allow this to fall through to the "default" # clientssl profile. SSL::enable } else { # A match was found in the Data Group, so we will change the SSL # profile to the one we found. Hide this activity from the iRules # parser. set ssl_profile_enable "SSL::profile $ssl_profile" catch { eval $ssl_profile_enable } if { not ($tls_pool == "") } { pool $tls_pool } else { pool $default_tls_pool } SSL::enable } } else { # No match because no SNI field was present. Fall through to the # "default" SSL profile. SSL::enable } } else { # We're not in a handshake. Keep on using the currently set SSL profile # for this transaction. SSL::enable } # Hold down any further processing and release the TCP session further # down the event loop. set detect_handshake 0 TCP::release } else { # We've not been able to match an SNI field to an SSL profile. We will # fall back to the "default" SSL profile selected (this might lead to # certificate validation errors on non SNI-capable browsers. set detect_handshake 0 SSL::enable TCP::release } } }iRule Editor - Auto Connection
In the latest release of the iRule Editor v 0.10.1, I added several new features. This tutorial will walk through the Auto Connection features I've added that allow you to supply command line arguments to have the iRule editor automatically connect to your specified BIG-IP. Usage: iRuler.exe [args] args ----------- /h hostname - the hostname (or ip) of your BIG-IP /u username - the admin username /p password - the admin password The following video will walk you through the steps to auto connect from the command line as well as how to create a desktop shortcut to each of your BIG-IP systems.BIG-IP LTM VE: Transfer your iRules in style with the iRule Editor
The new LTM VE has opened up the possibilities for writing, testing and deploying iRules in a big way. It’s easier than ever to get a test environment set up in which you can break things develop to your heart’s content. This is fantastic news for us iRulers that want to be doing the newest, coolest stuff without having to worry about breaking a production system. That’s all well and good, but what the heck do you do to get all of your current stuff onto your test system? There are several options, ranging from copy and paste (shudder) to actual config copies and the like, which all work fine. Assuming all you’re looking for though is to transfer over your iRules, like me, the easiest way I’ve found is to use the iRule editor’s export and import features. It makes it literally a few clicks and super easy to get back up and running in the new environment. First, log into your existing LTM system with your iRule editor (you are using the editor, right? Of course you are…just making sure). You’ll see a screen something like this (right) with a list of a bagillionty iRules on the left and their cool, color coded awesomeness on the right. You can go through and select iRules and start moving them manually, but there’s really no need. All you need to do is go up to the File –> Archive –> Export option and let it do its magic. All it’s doing is saving text files to your local system to archive off all of your iRuley goodness. Once that’s done, you can then spin up your new LTM VE and get logged in via the iRule editor over there. Connect via the iRule editor, and go to File –> Archive –> Import, shown below. Once you choose the import option you’ll start seeing your iRules popping up in the left-hand column, just like you’re used to. This will take a minute depending on how many iRules you have archived (okay, so I may have more than a few iRules in my collection…) but it’s generally pretty snappy. One important thing to note at his point, however, is that all of your iRules are bolded with an asterisk next to them. This means they are not saved in their current state on the LTM. If you exit at this point, you’ll still be iRuleless, and no one wants that. Luckily Joe thought of that when building the iRule editor, so all you need to do is select File –> Save All, and you’ll be most of the way home. I say most of the way because there will undoubtedly be some errors that you’ll need to clean up. These will be config based errors, like pools that used to exist on your old system and don’t now, etc. You can either go create the pools in the config or comment out those lines. I tend to try and keep my iRules as config agnostic as possible while testing things, so there aren’t a ton of these but some of them always crop up. The editor makes these easy to spot and fix though. The name of the iRule that’s having a problem will stay bolded and any errors in that particular code will be called out (assuming you have that feature turned on) so you can pretty quickly spot them and fix them. This entire process took me about 15 minutes, including cleaning up the code in certain iRules to at least save properly on the new system, and I have a bunch of iRules, so that’s a pretty generous estimate. It really is quick, easy and painless to get your code onto an LTM VE and get hacking coding. An added side benefit, but a cool one, is that you now have your iRules backed up locally. Not only does this mean you’re double plus sure that they won’t be lost, but it means the next time you want to deploy them somewhere, all you have to do is import from the editor. So if you haven’t yet, go download your BIG-IP LTM VE and get started. I can’t recommend it enough. Also make sure to check out some of the really handy DC content that shows you how to tweak it for more interfaces or Joe’s supremely helpful guide on how to use a single VM to run an entire client/LTM/server setup. Wicked cool stuff. Happy iRuling. #ColiniRules 101 - #12 - The Session Command
One of the things that makes iRules so incredibly powerful is the fact that it is a true scripting language, or at least based on one. The fact that they give you the tools that TCL brings to the table - regular expressions, string functions, even things as simple as storing, manipulating and recalling variable data - sets iRules apart from the rest of the crowd. It also makes it possible to do some pretty impressive things with connection data and massaging/directing it the way you want it. Other articles in the series: Getting Started with iRules: Intro to Programming with Tcl | DevCentral Getting Started with iRules: Control Structures & Operators | DevCentral Getting Started with iRules: Variables | DevCentral Getting Started with iRules: Directing Traffic | DevCentral Getting Started with iRules: Events & Priorities | DevCentral Intermediate iRules: catch | DevCentral Intermediate iRules: Data-Groups | DevCentral Getting Started with iRules: Logging & Comments | DevCentral Advanced iRules: Regular Expressions | DevCentral Getting Started with iRules: Events & Priorities | DevCentral iRules 101 - #12 - The Session Command | DevCentral Intermediate iRules: Nested Conditionals | DevCentral Intermediate iRules: Handling Strings | DevCentral Intermediate iRules: Handling Lists | DevCentral Advanced iRules: Scan | DevCentral Advanced iRules: Binary Scan | DevCentral Sometimes, though, a simple variable won't do. You've likely heard of global variables in one of the earlier 101 series and read the warning there, and are looking for another option. So here you are, you have some data you need to store, which needs to persist across multiple connections. You need it to be efficient and fast, and you don't want to have to do a whole lot of complex management of a data structure. One of the many ways that you can store and access information in your iRule fits all of these things perfectly, little known as it may be. For this scenario I'd recommend the usage of the session command. There are three main permutations of the session command that you'll be using when storing and referencing data within the session table. These are: session add: Stores user's data under the specified key for the specified persistence mode session lookup: Returns user data previously stored using session add session delete: Removes user data previously stored using session add A simple example of adding some information to the session table would look like: when CLIENTSSL_CLIENTCERT { set ssl_cert [SSL::cert 0] session add ssl $ssl_cert 90 } By using the session add command, you can manually place a specific piece of data into the LTM's session table. You can then look it up later, by unique key, with the session lookup command and use the data in a different section of your iRule, or in another connection all together. This can be helpful in different situations where data needs to be passed between iRules or events that it might not normally be when using a simple variable. Such as mining SSL data from the connection events, as below: when CLIENTSSL_CLIENTCERT { # Set results in the session so they are available to other events session add ssl [SSL::sessionid] [list [X509::issuer] [X509::subject] [X509::version]] 180 } when HTTP_REQUEST { # Retrieve certificate information from the session set sslList [session lookup ssl [SSL::sessionid]] set issuer [lindex sslList 0] set subject [lindex sslList 1] set version [lindex sslList 2] } Because the session table is optimized and designed to handle every connection that comes into the LTM, it's very efficient and can handle quite a large number of items. Also note that, as above, you can pass structured information such as TCL Lists into the session table and they will remain intact. Keep in mind, though, that there is currently no way to count the number of entries in the table with a certain key, so you'll have to build all of your own processing logic for now, where necessary. It's also important to note that there is more than one session table. If you look at the above example, you'll see that before we listed any key or data to be stored, we used the command session add ssl. Note the "ssl" portion of this command. This is a reference to which session table the data will be stored in. For our purposes here there are effectively two session tables: ssl, and uie. Be sure you're accessing the same one in your session lookup section as you are in your session add section, or you'll never find the data you're after. This is pretty easy to keep straight, once you see it. It looks like: session add uie ... session lookup uie Or: session add ssl ... session lookup ssl You can find complete documentation on the session command here, in the iRules, as well as some great examples that depict some more advanced iRules making use of the session command to great success. Check out Codeshare for more examples.Another FSE iRules Challenge, Even More Surprising Results
I have an awesome job. I get to play with cool technology, with good people, at an awesome company, and actually don’t get in trouble for doing so. I get to blend writing and talking and blathering on endlessly to anyone that will listen with completely geeking out and diving into the nuts and bolts of things to see what makes things work. This doesn’t suck. One of the things that doesn’t suck the most is getting to kick on the light bulb for people that haven’t quite gotten their hands around our programmability technologies just yet. F5 is laden with opportunities to get your script on. From iRules to iControl to iCall and TMSH scripting, there is no shortage of opportunities to get down and dirty with some code. That being said, not everyone is up to speed on such things yet and I take particular joy in being able to help them connect the wires, get the first flickers of that “Holy crap this stuff is cool!” halogen, and then go on their merry. Lucky as I am, what with the awesome job and all, I get many opportunities for just such interactions. More seasoned readers may be familiar with the one on which today’s post is focused, the FSE challenge. I haven’t posted one of these in a little bit so let’s have a refresher from days of yore. First off, What is an FSE? An FSE is a Field Sales Engineer. FSEs are the engineering lifeblood of the sales force here at F5. They’re the ones out in the trenches dealing with customer requirements and issues, building real world solutions, and generally doing all the cool stuff that I get to talk about theoretically, but in the real world. I’ve got mad respect for those FSEs that take their jobs seriously and learn how to build full fledged F5 solutions that leverage our crazy broad product set and, you guessed it, our out of the box tools like iControl and iRules. Those that choose to flex those muscles garner a special place in my encrypted little heart. Next, What is with this challenge business? Every time we get a new batch of FSEs in at corporate for brainwashing err, training we put them through what we lovingly refer to as a boot camp. This is, as you might expect given the name, a rapid way of getting folks up to speed on not only F5 technology but all of the surrounding whats-its and know-how that is expected of someone out in the field slinging our tech. This invariably includes a delve into iRules. There is formal training, of course, but the challenge is a different beast all together. I effectively pose as a customer in the field with a complex (at least complex by beginners’ standards) problem that needs solving. I present it to the batch of keyboard jockeys, give them time to ask questions, take notes, etc., then cut them loose. In their “free time” (See: sleepless hours well into the night) they get to hash out the solution to the problem. They are expected to write, test, and lightly document an iRules solution to eradicate the posed problem point by point. Points are awarded for effectiveness, efficiency, and exportability, meaning ease of use and hand-off. I come back in a week later, after pouring over the proffered code snippets, and announce the winners (top3) based on said criteria, who are then awarded fabulous cash and prizes! (Bold + italics means it must be true, right? Even when it’s not. Since it’s not. At all.) Lastly, What was the challenge? People are always curious to hear what the actual challenge was when looking at the submissions, so here you go: Scenario: A client has an https based application that is undergoing upgrades and large changes, and they need to create business logic in the network layer to allow for a smooth transition and consistent user experience. Desired Solution: Ensure that all requests from the client to the BIG-IP are SSL encrypted Ensure all traffic to the back end is plain-text For all canonical names of domain.com (I.E. bob.domain.com, app1.domain.com, etc.) remove the canonical name and prepend to URI (I.E. bob.domain.com/my/app becomes domain.com/bob/my/app). Standard canonicals are excluded from this re-writing (mail, smtp, www, ns, ns1) These host/uri changes must happen transparently to the clients accessing the application. Anyone accessing the application from the internal network (10.1.*) with an appropriate auth cookie (Name: X-Int-Auth. Value=True) bypasses the above logic and accesses the old structure. Log any request (IP of client and URI requested) to a canonical name x.domain.com that is non standard (mail, smtp, www, ns, ns1) so that data can be collected as to when users have fully transitioned. So now that the table is set, on with the feast! This time around I had another killer host of entries into the hopper. There were people of all experience levels from “newbie, never coded, what does this double equals thing do?” levels to one heck of a ringer, who would make himself known eventually, even though he flew under the radar at first, sly dog that he is. Out of the raft of valiant attempts and solid efforts, my arduous duty was to narrow it down to the top three and announce them to the group, and later (now) the world. Such was my task, and such was performed. I bring you this quarter’s FSE iRules Challenge winners: 3rd Place – Benn Alp Complete with ASCII art, Benn put in an heroic effort on this submission. He ended up with a heck of a lot of code, and most of it was extremely valid, which just goes to show that while he didn’t have the most efficient solution, he stuck with things until he got where he wanted. I encourage less meandering approaches to coding, but I was impressed by the thought that went into this one and the potential that is apparent in his thought process, logic and effort. Way to go Benn! ############################################################################## # _ ____ _ ____ _ _ _ # # (_) _ \ _ _| | ___ ___ / ___| |__ __ _| | | ___ _ __ __ _ ___ # # | | |_) | | | | |/ _ \/ __| | | | '_ \ / _` | | |/ _ \ '_ \ / _` |/ _ \ # # | | _ <| |_| | | __/\__ \ | |___| | | | (_| | | | __/ | | | (_| | __/ # # |_|_| \_\\__,_|_|\___||___/ \____|_| |_|\__,_|_|_|\___|_| |_|\__, |\___| # # |___/ # ############################################################################## # # Benn Alp b.alp@f5.com # when RULE_INIT { # # CONFIGURABLE ITEMS #---------------------------------------------------------------------------------------------------------- # Debug Logging. (Note: Irrispective of how this is set logs to satisfy requirement 6 will be sent - # 6 Log any request (IP of client and URI requested) to a canonical name x.domain.com that is non standard # (mail, smtp, www, ns, ns1) so that data can be collected as to when users have fully transitioned" is irrespective of this setting. # 0 - Disabled # 1 - Enabled #---------------------------------------------------------------------------------------------------------- set static::DebugLogging 1 #---------------------------------------------------------------------------------------------------------- # Behaviour when requirement 1 is violated for transformed apps (Ensure that all requests from the client to the BIG-IP are SSL encrypted) # Legacy apps continue to work as per normal. # 0 - Reject # 1 - Redirect to https #---------------------------------------------------------------------------------------------------------- set static::SSLBehaviour 1 #---------------------------------------------------------------------------------------------------------- } when HTTP_REQUEST { set Rewrite 0 if { $static::DebugLogging } { log local0. "Trigger HTTP_REQUEST" } # Requirement 5 - Anyone accessing the application from the Internal Network (10.1.x.x) and with an appropriate auth cookie (Name: X-Int-Auth Value=True) bypassess the above logic and access the old structure, or anybody using domain.com bypass/return if {[IP::client_addr] starts_with "10.1." and [HTTP::header "X-Init-Auth"] equals "True" or [string tolower [HTTP::host]] equals "domain.com" } { if { $static::DebugLogging } { log local0. "Trigger return based on Requirement 5 or domain=domain.com" } return } else { if { [HTTP::host] contains ".domain.com" } { # Requirement 3.1 Standard canonicals are excluded from this re-writing (mail, smtp, www, ns, ns1) and # Requirement 6 - Log any request (IP of Client and URI requested) to a canonical name x.domain.com that is non standard (Mail.smtp,www,ns,ns1) so that data can be collected as to when users have fully transitioned # Switch -exact was faster than data groups.. switch -exact [string tolower [HTTP::host]] { "mail.domain.com" { log local0. "Legacy Connection - USERIP [IP::client_addr] - mail.domain.com/[HTTP::uri]" } "smtp.domain.com" { log local0. "Legacy Connection - USERIP [IP::client_addr] - smtp.domain.com/[HTTP::uri]" } "www.domain.com" { log local0. "Legacy Connection - USERIP [IP::client_addr] - URI www.domain.com/[HTTP::uri]" } "ns.domain.com" { log local0. "Legacy Connection - USERIP [IP::client_addr] - URI ns.domain.com/[HTTP::uri]" } "ns1.domain.com" { log local0. "Legacy Connection - USERIP [IP::client_addr] - URI ns1.domain.com/[HTTP::uri]" } ... And that's all I'm showing of Benn's solution. It goes on for a while, and was an awesome effort but it's...rather long. ;) 2nd Place – Max Iftikhar Max set a high bar indeed with his submission which used the uber efficient stream profile, a solid cut at response re-writing, one of the pitfalls of this particular challenge, and some handy dandy string manipulation. This one was efficient, brief, and looked like it could have been the overall winner. All things being equal, in many other FSE classes this very well could have won, as it is a darn fine effort, and Max should hold his head high while coding. Unless of course he can’t see the monitor, then hold it rather normally and just know you kicked some tail, Max. when HTTP_REQUEST { if { [TCP::local_port] == 80 } { # redirect to https HTTP::redirect "https://[getfield [HTTP::host] ":" 1][HTTP::uri]" } } when HTTP_REQUEST { set rewrite 0 set canonical [getfield [HTTP::host] "." 1] set host1 [HTTP::host] set host2 [getfield [HTTP::host] "$canonical" 1] set uri1 "[HTTP::uri]" set uri2 ""/"$canonical[HTTP::uri]" if {[IP::addr [IP::client_addr] equals 10.1.x.x/16] and [HTTP::cookie exists"X-Int-Auth"} { pool http_pool } else { log local0. "Received request from [IP::client_addr] -> [HTTP::host][HTTP::uri]" } # Rewrite the Host header HTTP::header replace "Host" $::host2 # Make uri path start with /canoncial if it doesn't already if { not ([HTTP::uri] starts_with "/$canonical") } { HTTP::uri [string map -nocase {$uri1 $uri2} [HTTP::uri]] set urlRewrite 1 } } when HTTP_RESPONSE { if {$rewrite} { # Check if response is a redirect if {[HTTP::is_redirect] and [HTTP::header Location] contains $find} { # Rewrite the redirect Location header value HTTP::header replace "Host" $::host1 HTTP::header replace Location [string map -nocase "$url1 $url2" [HTTP::header Location]] } # Check if response payload type is text if {[HTTP::header value Content-Type] contains "text"} { # Set the replacement strings STREAM::expression "@$url1@$url2@" # Enable the stream filter for this response only STREAM::enable } } } Winner! – Joe Martin Last but the exact inverse of least, our winner in fact, was Joe Martin. Joe seemed like a normal, average, every day FSE challenge entrant upon first blush. He didn’t even bother to out himself at the onset as having written iRules before when I asked for experience levels. Clever ploy, Joe, very clever. As I was later to find out Joe seemed to in fact be a cyborg-robot-iRules-ninja-hacker-dinosaur sent back from the future to bust the curve for all FSE iRules Challengees everywhere. Seriously, this guy knew what he was doing. This iRule is pretty darn close to the code I would churn out to solve this particular problem and, not to self aggrandize, but that’s not such a bad thing coming from the guy judging the challenge, amirite? Upon presenting the results and having shaken the hand of the Cylon (No windows, you may not autocorrect Cylon to colon. Go away, I’m making jokes here.) in charge of iRules affairs himself, I asked Joe how many iRules he’d written before, because it was obvious that he had done so. Much to his credit he admitted to having written hundreds, which makes a whole heck of a lot of sense, and makes me able to sleep just a bit better at night without keeping a light on to watch out for those cyborg iRules ninja invaders. Big congrats to Joe for a darn fine hunk of codey bits. when HTTP_REQUEST { set request_rewrite 0 #Check to see if this in an internal developer request (internal IP and X-Auth header) if { ([HTTP::header X-Int-Auth] equals "True") && ([IP::client_addr] equals "10.1.0.0/16") } { pool http_pool } else { set orig_host [string tolower [HTTP::host]] scan $orig_host %s.%s.%s host domain tld set new_host "$domain.$tld" #Make sure "host" portion of DNS name is not in exclusion data group "class_no-rewrite" if { ![class match $host equals class_no-rewrite] } { #Flag connection as "request rewritten", rewrite host header and URI, and log request info" set request_rewrite 1 HTTP::header replace Host $new_host HTTP::uri "/$host[HTTP::uri]" log local0. "Request to $orig_host from [IP::client_addr] rewritten to [HTTP::uri]" pool http_pool } } } when HTTP_RESPONSE { #If the request was rewritten we need to rewrite Location headers and embedded URLs if {$request_rewrite} { if { [HTTP::is_redirect] } { HTTP::header replace Location [string map -nocase "$new_host $orig_host" [HTTP::header Location]] } else { STREAM::expression "@/$host/@/@ @http://$new_host@https://$orig_host@" STREAM::enable } } } All said and done it was another fine experience hosting the FSE iRules challenge. There was code, and fun, and fun code, and coding fun, and funny code, and…well you get the idea. I’m looking forward to the next crop and seeing what they’re capable of. I’ll be working on my cyborg detection methodologies in the meantime. Until then, remember kids: code hard. #Colin #iRules #iRulesChallenge #Cylons
Maintenance page - hosted on LTM or redirect with fallback host - or both?
I'm in the process of implementing an automated maintenance page that is displayed when I have a pool with no healthy members. Looking around, I see two distinct methods of doing this - utilizing the fallback host feature and redirecting to another url, or setting up a page to be hosted on the LTM and using an iRule with " [active_members [LB::server pool]] < 1" in it. Does anyone have any opinions on which one is preferred, and why? Currently, I'm using the fallback host method and I'm redirecting to a page hosted on AWS. My setup includes about 70 virtual servers on a 3600 HA cluster - some are QA, some are non-http. I will likley have the need for multiple versions of the maintenance page, depending on the site content it fronts. The one thing I do see as an advantage of the LTM hosted option is that an iRule code example shows a refresh option being used to automatically pull up the healthy site when it becomes available. Thanks!! ChrisIntermediate iRules: Nested Conditionals
Conditionals are a pretty standard tool in every programmer's toolbox. They are the functions that allow us to decided when we want certain actions to happen, based on, well, conditions that can be determined within our code. This concept is as old as compilers. Chances are, if you're writing code, you're going to be using a slew of these things, even in an Event based language like iRules. iRules is no different than any other programming/scripting language when it comes to conditionals; we have them. Sure how they're implemented and what they look like change from language to language, but most of the same basic tools are there: if, else, switch, elseif, etc. Just about any example that you might run across on DevCentral is going to contain some example of these being put to use. Learning which conditional to use in each situation is an integral part to learning how to code effectively. Once you have that under control, however, there's still plenty more to learn. Now that you're comfortable using a single conditional, what about starting to combine them? There are many times when it makes more sense to use a pair or more of conditionals in place of a single conditional along with logical operators. For example: if { [HTTP::host] eq "bob.com" and [HTTP::uri] starts_with "/uri1" } { pool pool1 } elseif { [HTTP::host] eq "bob.com" and [HTTP::uri] starts_with "/uri2" } { pool pool2 } elseif { [HTTP::host] eq "bob.com" and [HTTP::uri] starts_with "/uri3" } { pool pool3 } Can be re-written to use a pair of conditionals instead, making it far more efficient. To do this, you take the common case shared among the example strings and only perform that comparison once, and only perform the other comparisons if that result returns as desired. This is more easily described as nested conditionals, and it looks like this: if { [HTTP::host] eq "bob.com" } { if {[HTTP::uri] starts_with "/uri1" } { pool pool1 } elseif {[HTTP::uri] starts_with "/uri2" } { pool pool2 } elseif {[HTTP::uri] starts_with "/uri3" } { pool pool3 } } These two examples are logically equivalent, but the latter example is far more efficient. This is because in all the cases where the host is not equal to "bob.com", no other inspection needs to be done, whereas in the first example, you must perform the host check three times, as well as the uri check every single time, regardless of the fact that you could have stopped the process earlier. While basic, this concept is important in general when coding. It becomes exponentially more important, as do almost all optimizations, when talking about programming in iRules. A script being executed on a server firing perhaps once per minute benefits from small optimizations. An iRule being executed somewhere in the order of 100,000 times per second benefits that much more. A slightly more interesting example, perhaps, is performing the same logical nesting while using different operators. In this example we'll look at a series of if/elseif statements that are already using nesting, and take a look at how we might use the switch command to even further optimize things. I've seen multiple examples of people shying away from switch when nesting their logic because it looks odd to them or they're not quite sure how it should be structured. Hopefully this will help clear things up. First, the example using if statements: when HTTP_REQUEST { if { [HTTP::host] eq "secure.domain.com" } { HTTP::header insert "Client-IP:[IP::client_addr]" pool sslServers } elseif { [HTTP::host] eq "www.domain.com" } { HTTP::header insert "Client-IP:[IP::client_addr]" pool httpServers } elseif { [HTTP::host] ends_with "domain.com" and [HTTP::uri] starts_with "/secure"} { HTTP::header insert "Client-IP:[IP::client_addr]" pool sslServers } elseif {[HTTP::host] ends_with "domain.com" and [HTTP::uri] starts_with "/login"} { HTTP::header insert "Client-IP:[IP::client_addr]" pool httpServers } elseif { [HTTP::host] eq "intranet.myhost.com" } { HTTP::header insert "Client-IP:[IP::client_addr]" pool internal } } As you can see, this is completely functional and would do the job just fine. There are definitely some improvements that can be made, though. Let's try using a switch statement instead of several if comparisons for improved performance. To do that, we're going to have to use an if nested inside a switch comparison. While this might be new to some or look a bit odd if you're not used to it, it's completely valid and often times the most efficient you’re going to get. This is what the above code would look like cleaned up and put into a switch: when HTTP_REQUEST { HTTP::header insert "Client-IP:[IP::client_addr]" switch -glob [HTTP::host] { "secure.domain.com" { pool sslServers } "www.domain.com" { pool httpServers } "*.domain.com" { if { [HTTP::uri] starts_with "/secure" } { pool sslServers } else { pool httpServers } } "intranet.myhost.com" { pool internal } } } As you can see this is not only easier to read and maintain, but it will also prove to be more efficient. We've moved to the more efficient switch structure, we've gotten rid of the repeat host comparisons that were happening above with the /secure vs /login uris, and while I was at it I got rid of all those examples of inserting a header, since that was happening in every case anyway. Hopefully the benefit this technique can offer is clear, and these examples did the topic some justice. With any luck, you'll nest those conditionals with confidence now.Removing port from a redirect
Hi all, One of our web developers has asked me if we could strip off a port number in a redirect they are doing. I thought the following would do this, but it doesn't appear to work. when HTTP_REPSONSE { if { [HTTP::is_redirect] } { if { [HTTP::header Location] contains "www.acme.com:10040" } { log "Original Location value: [HTTP::header Location]" HTTP::header replace Location [string map -nocase {www.acme.com:10400 www.acme.com} [HTTP::header value Location]] log "Updated Location value: [HTTP::header Location]" return } } } And here is what is written to the log Original Location value: www.acme.com:10040/secure/discussion-forum Updated Location value: www.acme.com:10040/secure/discussion-forum Note: actually the log includes http but if I enter in a URL in this new forum s/w it does odd things to it. Any help appreciated. Craig
5 Years Later: OpenAJAX Who?
Five years ago the OpenAjax Alliance was founded with the intention of providing interoperability between what was quickly becoming a morass of AJAX-based libraries and APIs. Where is it today, and why has it failed to achieve more prominence? I stumbled recently over a nearly five year old article I wrote in 2006 for Network Computing on the OpenAjax initiative. Remember, AJAX and Web 2.0 were just coming of age then, and mentions of Web 2.0 or AJAX were much like that of “cloud” today. You couldn’t turn around without hearing someone promoting their solution by associating with Web 2.0 or AJAX. After reading the opening paragraph I remembered clearly writing the article and being skeptical, even then, of what impact such an alliance would have on the industry. Being a developer by trade I’m well aware of how impactful “standards” and “specifications” really are in the real world, but the problem – interoperability across a growing field of JavaScript libraries – seemed at the time real and imminent, so there was a need for someone to address it before it completely got out of hand. With the OpenAjax Alliance comes the possibility for a unified language, as well as a set of APIs, on which developers could easily implement dynamic Web applications. A unified toolkit would offer consistency in a market that has myriad Ajax-based technologies in play, providing the enterprise with a broader pool of developers able to offer long term support for applications and a stable base on which to build applications. As is the case with many fledgling technologies, one toolkit will become the standard—whether through a standards body or by de facto adoption—and Dojo is one of the favored entrants in the race to become that standard. -- AJAX-based Dojo Toolkit , Network Computing, Oct 2006 The goal was simple: interoperability. The way in which the alliance went about achieving that goal, however, may have something to do with its lackluster performance lo these past five years and its descent into obscurity. 5 YEAR ACCOMPLISHMENTS of the OPENAJAX ALLIANCE The OpenAjax Alliance members have not been idle. They have published several very complete and well-defined specifications including one “industry standard”: OpenAjax Metadata. OpenAjax Hub The OpenAjax Hub is a set of standard JavaScript functionality defined by the OpenAjax Alliance that addresses key interoperability and security issues that arise when multiple Ajax libraries and/or components are used within the same web page. (OpenAjax Hub 2.0 Specification) OpenAjax Metadata OpenAjax Metadata represents a set of industry-standard metadata defined by the OpenAjax Alliance that enhances interoperability across Ajax toolkits and Ajax products (OpenAjax Metadata 1.0 Specification) OpenAjax Metadata defines Ajax industry standards for an XML format that describes the JavaScript APIs and widgets found within Ajax toolkits. (OpenAjax Alliance Recent News) It is interesting to see the calling out of XML as the format of choice on the OpenAjax Metadata (OAM) specification given the recent rise to ascendancy of JSON as the preferred format for developers for APIs. Granted, when the alliance was formed XML was all the rage and it was believed it would be the dominant format for quite some time given the popularity of similar technological models such as SOA, but still – the reliance on XML while the plurality of developers race to JSON may provide some insight on why OpenAjax has received very little notice since its inception. Ignoring the XML factor (which undoubtedly is a fairly impactful one) there is still the matter of how the alliance chose to address run-time interoperability with OpenAjax Hub (OAH) – a hub. A publish-subscribe hub, to be more precise, in which OAH mediates for various toolkits on the same page. Don summed it up nicely during a discussion on the topic: it’s page-level integration. This is a very different approach to the problem than it first appeared the alliance would take. The article on the alliance and its intended purpose five years ago clearly indicate where I thought this was going – and where it should go: an industry standard model and/or set of APIs to which other toolkit developers would design and write such that the interface (the method calls) would be unified across all toolkits while the implementation would remain whatever the toolkit designers desired. I was clearly under the influence of SOA and its decouple everything premise. Come to think of it, I still am, because interoperability assumes such a model – always has, likely always will. Even in the network, at the IP layer, we have standardized interfaces with vendor implementation being decoupled and completely different at the code base. An Ethernet header is always in a specified format, and it is that standardized interface that makes the Net go over, under, around and through the various routers and switches and components that make up the Internets with alacrity. Routing problems today are caused by human error in configuration or failure – never incompatibility in form or function. Neither specification has really taken that direction. OAM – as previously noted – standardizes on XML and is primarily used to describe APIs and components - it isn’t an API or model itself. The Alliance wiki describes the specification: “The primary target consumers of OpenAjax Metadata 1.0 are software products, particularly Web page developer tools targeting Ajax developers.” Very few software products have implemented support for OAM. IBM, a key player in the Alliance, leverages the OpenAjax Hub for secure mashup development and also implements OAM in several of its products, including Rational Application Developer (RAD) and IBM Mashup Center. Eclipse also includes support for OAM, as does Adobe Dreamweaver CS4. The IDE working group has developed an open source set of tools based on OAM, but what appears to be missing is adoption of OAM by producers of favored toolkits such as jQuery, Prototype and MooTools. Doing so would certainly make development of AJAX-based applications within development environments much simpler and more consistent, but it does not appear to gaining widespread support or mindshare despite IBM’s efforts. The focus of the OpenAjax interoperability efforts appears to be on a hub / integration method of interoperability, one that is certainly not in line with reality. While certainly developers may at times combine JavaScript libraries to build the rich, interactive interfaces demanded by consumers of a Web 2.0 application, this is the exception and not the rule and the pub/sub basis of OpenAjax which implements a secondary event-driven framework seems overkill. Conflicts between libraries, performance issues with load-times dragged down by the inclusion of multiple files and simplicity tend to drive developers to a single library when possible (which is most of the time). It appears, simply, that the OpenAJAX Alliance – driven perhaps by active members for whom solutions providing integration and hub-based interoperability is typical (IBM, BEA (now Oracle), Microsoft and other enterprise heavyweights – has chosen a target in another field; one on which developers today are just not playing. It appears OpenAjax tried to bring an enterprise application integration (EAI) solution to a problem that didn’t – and likely won’t ever – exist. So it’s no surprise to discover that references to and activity from OpenAjax are nearly zero since 2009. Given the statistics showing the rise of JQuery – both as a percentage of site usage and developer usage – to the top of the JavaScript library heap, it appears that at least the prediction that “one toolkit will become the standard—whether through a standards body or by de facto adoption” was accurate. Of course, since that’s always the way it works in technology, it was kind of a sure bet, wasn’t it? WHY INFRASTRUCTURE SERVICE PROVIDERS and VENDORS CARE ABOUT DEVELOPER STANDARDS You might notice in the list of members of the OpenAJAX alliance several infrastructure vendors. Folks who produce application delivery controllers, switches and routers and security-focused solutions. This is not uncommon nor should it seem odd to the casual observer. All data flows, ultimately, through the network and thus, every component that might need to act in some way upon that data needs to be aware of and knowledgeable regarding the methods used by developers to perform such data exchanges. In the age of hyper-scalability and über security, it behooves infrastructure vendors – and increasingly cloud computing providers that offer infrastructure services – to be very aware of the methods and toolkits being used by developers to build applications. Applying security policies to JSON-encoded data, for example, requires very different techniques and skills than would be the case for XML-formatted data. AJAX-based applications, a.k.a. Web 2.0, requires different scalability patterns to achieve maximum performance and utilization of resources than is the case for traditional form-based, HTML applications. The type of content as well as the usage patterns for applications can dramatically impact the application delivery policies necessary to achieve operational and business objectives for that application. As developers standardize through selection and implementation of toolkits, vendors and providers can then begin to focus solutions specifically for those choices. Templates and policies geared toward optimizing and accelerating JQuery, for example, is possible and probable. Being able to provide pre-developed and tested security profiles specifically for JQuery, for example, reduces the time to deploy such applications in a production environment by eliminating the test and tweak cycle that occurs when applications are tossed over the wall to operations by developers. For example, the jQuery.ajax() documentation states: By default, Ajax requests are sent using the GET HTTP method. If the POST method is required, the method can be specified by setting a value for the type option. This option affects how the contents of the data option are sent to the server. POST data will always be transmitted to the server using UTF-8 charset, per the W3C XMLHTTPRequest standard. The data option can contain either a query string of the form key1=value1&key2=value2 , or a map of the form {key1: 'value1', key2: 'value2'} . If the latter form is used, the data is converted into a query string using jQuery.param() before it is sent. This processing can be circumvented by setting processData to false . The processing might be undesirable if you wish to send an XML object to the server; in this case, change the contentType option from application/x-www-form-urlencoded to a more appropriate MIME type. Web application firewalls that may be configured to detect exploitation of such data – attempts at SQL injection, for example – must be able to parse this data in order to make a determination regarding the legitimacy of the input. Similarly, application delivery controllers and load balancing services configured to perform application layer switching based on data values or submission URI will also need to be able to parse and act upon that data. That requires an understanding of how jQuery formats its data and what to expect, such that it can be parsed, interpreted and processed. By understanding jQuery – and other developer toolkits and standards used to exchange data – infrastructure service providers and vendors can more readily provide security and delivery policies tailored to those formats natively, which greatly reduces the impact of intermediate processing on performance while ensuring the secure, healthy delivery of applications.
