iRules Style Guide
This article (formatted here in collaboration with and from the notes of F5er Jim_Deucker) features an opinionated way to write iRules, which is an extension to the Tcl language. Tcl has its own style guide for reference, as do other languages like my personal favorite python. From the latter, Guido van Rossum quotes Ralph Waldo Emerson: "A foolish consistency is the hobgoblin of little minds..," or if you prefer, Morpheus from the Matrix: "What you must learn is that these rules are no different than the rules of a computer system. Some of them can be bent. Others can be broken." The point? This is a guide, and if there is a good reason to break a rule...by all means break it! Editor Settings Setting a standard is good for many reasons. It's easier to share code amongst colleagues, peers, or the larger community when code is consistent and professional looking. Settings for some tools are provided below, but if you're using other tools, here's the goal: indent 4 spaces (no tab characters) 100-column goal for line length (120 if you must) but avoid line continuations where possible file parameters ASCII Unix linefeeds (\n) trailing whitespace trimmed from the end of each line file ends with a linefeed Visual Studio Code If you aren't using VSCode, why the heck not? This tool is amazing, and with the F5 Networks iRules extension coupled with The F5 Extension, you get the functionality of a powerful editor along with the connectivity control of your F5 hosts. With code diagnostics and auto formatting based on this very guide, the F5 Networks iRules Extension will make your life easy. Seriously...stop reading and go set up VSCode now. EditorConfig For those with different tastes in text editing using an editor that supports EditorConfig: # 4 space indentation [*.{irule,irul}] indent_style = space indent_size = 4 end_of_line = lf insert_final_newline = true charset = ascii trim_trailing_whitespace = true Vim I'm a vi guy with sys-admin work, but I prefer a full-fledge IDE for development efforts. If you prefer the file editor, however, we've got you covered with these Vim settings: # in ~/.vimrc file set tabstop=4 set shiftwidth=4 set expandtab set fileencoding=ascii set fileformat=unix Sublime There are a couple tools for sublime, but all of them are a bit dated and might require some work to bring them up to speed. Unless you're already a Sublime apologist, I'd choose one of the other two options above. sublime-f5-irules (bitwisecook fork, billchurch origin) for editing Sublime Highlight for export to RTF/HTML Guidance Watch out for smart-quotes and non-breaking spaces inserted by applications like Microsoft Word as they can silently break code. The VSCode extension will highlight these occurrences and offer a fix automatically, so again, jump on that bandwagon! A single iRule has a 64KB limit. If you're reaching that limit it might be time to question your life choices, I mean, the wisdom of the solution. Break out your iRules into functional blocks. Try to separate (where possible) security from app functionality from stats from protocol nuances from mgmt access, etc. For example, when the DevCentral team managed the DevCentral servers and infrastructure, we had 13 iRules to handle maintenance pages, masking application error codes and data, inserting scripts for analytics, managing vanity links and other structural rewrites to name a few. With this strategy, priorities for your events are definitely your friend. Standardize on "{" placement at the end of a line and not the following line, this causes the least problems across all the BIG-IP versions. # ### THIS ### if { thing } { script } else { other_script } # ### NOT THIS ### if { thing } { script } else { other_script } 4-character indent carried into nested values as well, like in switch. # ### THIS ### switch -- ${thing} { "opt1" { command } default { command } } Comments (as image for this one to preserve line numbers) Always comment at the same indent-level as the code (lines 1, 4, 9-10) Avoid end-of-line comments (line 11) Always hash-space a comment (lines 1, 4, 9-10) Leave out the space when commenting out code (line 2) switch statements cannot have comments inline with options (line 6) Avoid multiple commands on a single line. # ### THIS ### set host [getfield [HTTP::host] 1] set port [getfield [HTTP::host] 2] # ### NOT THIS ### set host [getfield [HTTP::host] 1]; set port [getfield [HTTP::host] 2] Avoid single-line if statements, even for debug logs. # ### THIS ### if { ${debug} } { log local0. "a thing happened...." } # ### NOT THIS ### if { ${debug} } { log local0. "a thing happened..."} Even though Tcl allows a horrific number of ways to communicate truthiness, Always express or store state as 0 or 1 # ### THIS ### set f 0 set t 1 if { ${f} && ${t} } { ... } # ### NOT THIS ### # Valid false values set f_values "n no f fal fals false of off" # Valid true values set t_values "y ye yes t tr tru true on" # Set a single valid, but unpreferred, state set f [lindex ${f_values} [expr {int(rand()*[llength ${f_values}])}]] set t [lindex ${t_values} [expr {int(rand()*[llength ${t_values}])}]] if { ${f} && ${t} } { ... } Always use Tcl standard || and && boolean operators over the F5 special and and or operators in expressions, and use parentheses when you have multiple arguments to be explicitly clear on operations. # ### THIS ### if { ${state_active} && ${level_gold} } { if { (${state} == "IL") || (${state} == "MO") } { pool gold_pool } } # ### NOT THIS ### if { ${state_active} and ${level_gold} } { if { ${state} eq "IL" or ${state} eq "MO" } { pool gold_pool } } Always put a space between a closing curly bracket and an opening one. # ### THIS ### if { ${foo} } { log local0.info "something" } # ### NOT THIS ### if { ${foo} }{ log local0.info "something" } Always wrap expressions in curly brackets to avoid double expansion. (Check out a deep dive on the byte code between the two approaches shown in the picture below) # ### THIS ### set result [expr {3 * 4}] # ### NOT THIS ### set result [expr 3 * 4] Always use space separation around variables in expressions such as if statements or expr calls. Always wrap your variables in curly brackets when referencing them as well. # ### THIS ### if { ${host} } { # ### NOT THIS ### if { $host } { Terminate options on commands like switch and table with "--" to avoid argument injection if if you're 100% sure you don't need them. The VSCode iRules extension will throw diagnostics for this. SeeK15650046 for more details on the security exposure. # ### THIS ### switch -- [whereis [IP::client_addr] country] { "US" { table delete -subtable states -- ${state} } } # ### NOT THIS ### switch [whereis [IP::client_addr] country] { "US" { table delete -subtable states ${state} } } Always use a priority on an event, even if you're 100% sure you don't need them. The default is 500 so use that if you have no other starting point. Always put a timeout and/or lifetime on table contents. Make sure you really need the table space before settling on that solution, and consider abusing the static:: namespace instead. Avoid unexpected scope creep with static:: and table variables by assigning prefixes. Lacking a prefix means if multiple rules set or use the variable changing them becomes a race condition on load or rule update. when RULE_INIT priority 500 { # ### THIS ### set static::appname_confvar 1 # ### NOT THIS ### set static::confvar 1 } Avoid using static:: for things like debug configurations, it's a leading cause of unintentional log storms and performance hits. If you have to use them for a provable performance reason follow the prefix naming rule. # ### THIS ### when CLIENT_ACCEPTED priority 500 { set debug 1 } when HTTP_REQUEST priority 500 { if { ${debug} } { log local0.debug "some debug message" } } # ### NOT THIS ### when RULE_INIT priority 500 { set static::debug 1 } when HTTP_REQUEST priority 500 { if { ${static::debug} } { log local0.debug "some debug message" } } Comments are fine and encouraged, but don't leave commented-out code in the final version. Wrapping up that guidance with a final iRule putting it all into practice: when HTTP_REQUEST priority 500 { # block level comments with leading space #command commented out if { ${a} } { command } if { !${a} } { command } elseif { ${b} > 2 || ${c} < 3 } { command } else { command } switch -- ${b} { "thing1" - "thing2" { # thing1 and thing2 business reason } "thing3" { # something else } default { # default branch } } # make precedence explicit with parentheses set d [expr { (3 + ${c} ) / 4 }] foreach { f } ${e} { # always braces around the lists } foreach { g h i } { j k l m n o p q r } { # so the lists are easy to add to } for { set i 0 } { ${i} < 10 } { incr i } { # clarity of each parameter is good } } What standards do you follow for your iRules coding styles? Drop a comment below!
Demonstration of Device DoS and Per-Service DoS protection
Dear Reader, This article is intended to show what effect the different threshold modes have on the Device and Per-Service (VS/PO) context. I will be using practical examples to demonstrate those effects. You will get to review a couple of scripts which will help you to do DoS flood tests and “visualizing” results on the CLI. In my first article (https://devcentral.f5.com/s/articles/Concept-of-F5-Device-DoS-and-DoS-profiles), I talked about the concept of F5 Device DoS and Per-Service DoS protection (DoS profiles). I also covered the physical and logical data path, which explains the order of Device DoS and Per-Service DoS using the DoS profiles. In the second article (https://devcentral.f5.com/s/articles/Explanation-of-F5-DDoS-threshold-modes), I explained how the different threshold modes are working. In this third article, I would like to show you what it means when the different modes work together. But, before I start doing some tests to show the behavior, I would like to give you a quick introduction into the toolset I´m using for these tests. Toolset First of all, how to do some floods? Different tools that can be found on the Internet are available for use. Whatever tools you might prefer, just download the tool and run it against your Device Under Test (DUT). If you would like to use my script you can get it from GitHub: https://github.com/sv3n-mu3ll3r/DDoS-Scripts With this script - it uses hping - you can run different type of attacks. Simply start it with: $ ./attack_menu.sh <IP> <PORT> A menu of different attacks will be presented which you can launch against the used IP and port as a parameter. Figure 1:Attack Menu To see what L3/4 DoS events are currently ongoing on your BIG-IP, simply go to the DoS Overview page. Figure 2:DoS Overview page I personally prefer to use the CLI to get the details I´m interested in. This way I don´t need to switch between CLI to launch my attacks and GUI to see the results. For that reason, I created a script which shows me what I am most interested in. Figure 3:DoS stats via CLI You can download that script here: https://github.com/sv3n-mu3ll3r/F5_show_DoS_stats_scripts Simply run it with the “watch” command and the parameter “-c” to get a colored output (-c is only available starting with TMOS version 14.0): What is this script showing you? context_name: This is the context, either PO/VS or the Device in which the vector is running vector_name: This is the name of the DoS vector attack_detected:When it shows “1”, then an attack has been detected, which means the ‘stats_rate’ is above the ‘detection-rate'. stats_rate: Shows you the current incoming pps rate for that vector in this context drops_rate: Shows you the number dropped pps rate in software (not FPGA) for that vector in this context int_drops_rate: Shows you the number dropped pps rate in hardware (FPGA) for that vector in this context ba_stats_rate: Shows you the pps rate for bad actors ba_drops_rate: Shows you the pps rate of dropped ‘bad actors’ in HW and SW bd_stats_rate:Shows you the pps rate for attacked destination bd_drop_rate: Shows you the pps rate for dropped ‘attacked destination’ mitigation_curr: Shows the current mitigation rate (per tmm) for that vector in that context detection: Shows you the current detection rate (per tmm) for that vector in that context wl_count: Shows you the number of whitelist hits for that vector in that context hw_offload: When it shows ‘1’ it means that FPGAs are involved in the mitigation int_dropped_bytes_rate: Gives you the rate of in HW dropped bytes for that vector in that context dropped_bytes_rate: Gives you the rate of in SW dropped bytes for that vector in that context When a line is displayed in green, it means packets hitting that vector. However, no anomaly is detected or anything is mitigated (dropped) via DoS. If a line turns yellow, then an attack - anomaly – has been detected but no packets are dropped via DoS functionalities. When the color turns red, then the system is actually mitigating and dropping packets via DoS functionalities on that vector in that context. Start Testing Before we start doing some tests, let me provide you with a quick overview of my own lab setup. I´m using a DHD (DDoS Hybrid Defender) running on a i5800 box with TMOS version 15.1 My traffic generator sends around 5-6 Gbps legitimate (HTTP and DNS) traffic through the box which is connected in L2 mode (v-wire) to my network. On the “client” side, where my clean traffic generator is located, my attacking clients are located as well by use of my DoS script. On the “server” side, I run a web service and DNS service, which I´m going to attack. Ok, now let’s do some test so see the behavior of the box and double check that we really understand the DDoS mitigation concept of BIG-IP. Y-MAS flood against a protected server Let’s start with a simple Y-MAS (all TCP flags cleared) flood. You can only configure this vector on the device context and only in manual mode. Which is ok, because this TCP packet is not valid and would get drop by the operating system (TMOS) anyway. But, because I want this type of packet get dropped in hardware (FPGA) very early, when they hit the box, mostly without touching the CPU, I set the thresholds to ‘10’ on the Mitigation Threshold EPS and to ‘10’ on Detection Threshold EPS. That means as soon as a TMM sees more then 10 pps for that vector it will give me a log message and also rate-limit this type of packets per TMM to 10 packets per second. That means that everything below that threshold will get to the operating system (TMOS) and get dropped there. Figure 4:Bad TCP Flags vector As soon as I start the attack, which targets the web service (10.103.1.50, port 80) behind the DHD with randomized source IPs. $ /usr/sbin/hping3 --ymas -p 80 10.103.1.50 --flood --rand-source I do get a log messages in /var/log/ltm: Feb5 10:57:52 lon-i5800-1 err tmm3[15367]: 01010252:3: A Enforced Device DOS attack start was detected for vector Bad TCP flags (all cleared), Attack ID 546994598. And, my script shows me the details on that attack in real time (the line is in ‘red’, indicating we are mitigating): Currently 437569 pps are hitting the device. 382 pps are blocked by DDoS in SW (CPU) and 437187 are blocked in HW (FPGA). Figure 5:Mitigating Y-Flood Great, that was easy. :-) Now, let’s do another TCP flood against my web server. RST-flood against a protected server with Fully manual Threshold mode: For this test I´m using the “Fully Manual” mode, which configures the thresholds for the whole service we are protecting with the DoS profile, which is attached to my VS/PO. Figure 6:RST flood with manual configuration My Detection Threshold and my Mitigation Threshold EPS is set to ‘100’. That means as soon as we see more then 100 RST packets hitting the configured VS/PO on my BIG-IP for this web server, the system will start to rate-limit and send a log message. Figure 7:Mitigating RST flood on PO level Perfect. We see the vector in the context of my web server (/Common/PO_10.103.1.50_HTTP) is blocking (rate-limiting) as expected from the configuration. Please ignore the 'IP bad src' which is in detected mode. This is because 'hping' creates randomized IPs and not all of them are valid. RST-flood against a protected server with Fully Automatic Threshold mode: In this test I set the Threshold Mode for the RST vector on the DoS profile which is attached to my web server to ‘Fully Automatic’ and this is what you would most likely do in the real world as well. Figure 8:RST vector configuration with Fully Automatic But, what does this mean for the test now? I run the same flood against the same destination and my script shows me the anomaly on the VS/PO (and on the device context), but it does not mitigate! Why would this happen? Figure 9:RST flood with Fully Automatic configuration When we take a closer look at the screenshot we see that the ‘stats_rate’ shows 730969 pps. The detection rate shows 25. From where is this 25 coming? As we know, when ‘Fully Automatic’ is enabled then the system learns from history. In this case, the history was even lower than 25, but because I set the floor value to 100, the detection rate per TMM is 25 (floor_value / number of TMMs), which in my case is 100/4 = 25 So, we need to recognize, that the ‘stats_rate’ value represents all packets for that vector in that context and the detection value is showing the per TMM value. This value explains us why the system has detected an attack, but why is it not dropping via DoS functionalities? To understand this, we need to remember that the mitigation in ‘Fully Automatic’ mode will ONLY kick in if the incoming rate for that vector is above the detection rate (condition is here now true) AND the stress on the service is too high. But, because BIG-IP is configures as a stateful device, this randomized RST packets will never reach the web service, because they get all dropped latest by the state engine of the BIG-IP. Therefor the service will never have stress caused by this flood.This is one of the nice benefits of having a stateful DoS device. So, the vector on the web server context will not mitigate here, because the web server will not be stressed by this type of TCP attack. This does also explains the Server Stress visualization in the GUI, which didn´t change before and during the attack. Figure 10:DoS Overview in the GUI But, what happens if the attack gets stronger and stronger or the BIG-IP is too busy dealing with all this RST packets? This is when the Device DOS kicks in but only if you have configured it in ‘Fully Automatic’ mode as well. As soon as the BIG-IP receives more RST packets then usually (detection rate) AND the stress (CPU load) on the BIG-IP gets too high, it starts to mitigate on the device context. This is what you can see here: Figure 11:'massive' RST flood with Fully Automatic configuration The flood still goes against the same web server, but the mitigation is done on the device context, because the CPU utilization on the BIG-IP is too high. In the screenshot below you can see that the value for the mitigation (mitigation_curr) is set to 5000 on the device context, which is the same as the detection value. This value resultsfrom the 'floor' value as well. It is the smallest possible value, because the detection and mitigation rate will never be below the 'floor' value. The mitigation rate is calculated dynamically based on the stress of the BIG-IP. In this test I artificially increased the stress on the BIG-IP and therefor the mitigation rate was calculated to the lowest possible number, which is the same as the detection rate. I will provide an explanation of how I did that later. Figure 13:Device context configuration for RST Flood Because this is the device config, the value you enter in the GUI is per TMM and this is reflected on the script output as well. What does this mean for the false-positive rate? First of all, all RST packets not belonging to an existing flow will kicked out by the state engine. At this point we don´t have any false positives. If the attack increases and the CPU can´t handle the number of packets anymore, then the DOS protection on the device context kicks in. With the configuration we have done so far, it will do the rate-limiting on all RST packets hitting the BIG-IP. There is no differentiation anymore between good and bad RST, or if the RST has the destination of server 1 or server 2, and so on. This means that at this point we can face false positives with this configuration. Is this bad? Well, false-positives are always bad, but at this point you can say it´s better to have few false-positives then a service going down or, even more critical, when your DoS device goes down. What can you do to only have false positives on the destination which is under attack? You probably have recognized that you can also enable “Attacked Destination Detection” on the DoS vector, which makes sense on the device context and on a DoS profile which is used on protected object (VS), that covers a whole network. Figure 14:Device context configuration for RST Flood with 'Attacked Destination Detection' enabled If the attack now hits one or multiple IPs in that network, BIG-IP will identify them and will do the rate-limiting only on the destination(s) under attack. Which still means that you could face false positives, but they will be at least only on the IPs under attack. This is what we see here: Figure 15:Device context mitigation on attacked destination(s) The majority of the RST packet drops are done on the “bad destination” (bd_drops_rate), which is the IP under attack. The other option you can also set is “Bad Actor Detection”. When this is enabled the system identifies the source(s) which causes the load and will do the rate limiting for that IP address(es). This usually works very well for amplification attacks, where the attack packets coming from ‘specific’ hosts and are not randomized sources. Figure 16:Device context mitigation on attacked destination(s) and bad actor(s) Here you can see the majority of the mitigation is done on ‘bad actors’. This reduces the chance of false positives massively. Figure 17:Device context configuration for RST Flood with 'Attacked Destination Detection' and 'Bad Actor Detection' enabled You also have multiple additional options to deal with ‘attacked destination’ and ‘bad actors’, but this is something I will cover in another article. Artificial increase BIG-IPs CPU load Before I finish this article, I would like to give you an idea on how you could increase the CPU load of the BIG-IP for your own testing. Because, as we know, with “Fully Automatic” on the device context, the mitigation kicks only in if the packet rate is above the detection rate AND the CPU utilization on the BIG-IP is “too” high. This is sometimes difficult to archive in a lab because you may not be able to send enough packets to stress the CPUs of a HW BIG-IP. In this use-case I use a simple iRule, which I attach to the VS/PO that is under attack. Figure 18:Stress iRule When I start my attack, I send the traffic with a specific TTL for identification. This TTL is configured in my iRule in order to get a CPU intensive string compare function to work on every attack packet. Here is an example for 'hping': $ /usr/sbin/hping3 --rst -p 80 10.103.1.50 --ttl 5 --flood --rand-source This can easily drive the CPU very high and the DDoS rate-limiting kicks in very aggressive. Summary I hope that this article provides you with an even better understanding on what effect the different threshold modes have on the attack mitigation. Of course, keep in mind this are just the ‘static DoS’ vectors. In later articles I will explain also the 'Behavioral DoS' and the Cookie based mitigation, which helps to massively reduce the chance of a false positives. But, also keep in mind, the DoS vectors start to act in microseconds and are very effective for protecting. Thank you, sVen Mueller
Decrypting TLS traffic on BIG-IP
1 Introduction As soon as I joined F5 Support, over 5 years ago, one of the first things I had to learn quickly was to decrypt TLS traffic becausemost of our customers useL7 applications protectedby TLS layer. In this article, I will show 4 ways to decrypt traffic on BIG-IP, including thenew one just release in v15.xthat is ideal for TLS1.3 where TLS handshake is also encrypted. If that's what you want to know just skip totcpdump --f5 ssl optionsection as this new approach is just a parameter added to tcpdump. As this article is very hands-on, I will show my lab topology for the tests I performed and then every possible way I used to decrypt customer's traffic working for Engineering Services at F5. 2 Lab Topology This is the lab topology I used for the lab test where all tests were performed: Also, for every capture I issued the followingcurlcommand: Update: the virtual server's IP address is actually 10.199.3.145/32 2 The 4 ways to decrypt BIG-IP's traffic RSA private key decryption There are 3 constraints here: Full TLS handshake has to be captured CheckAppendix 2to learn how to to disable BIG-IP's cache RSA key exchange has to be used, i.e. no (EC)DHE CheckAppendix1to understand how to check what's key exchange method used in your TLS connection CheckAppendix 2to understandhow to prioritise RSA as key exchange method Private key has to be copied to Wireshark machine (ssldump command solves this problem) Roughly, to accomplish that we can setCache Sizeto 0 on SSL profile and remove (EC)DHE fromCipher Suites(seeAppendix 1for details) I first took a packet capture using :pmodifier to capture only the client and server flows specific to my Client's IP address (10.199.3.1): Note: The0.0interface will capture any forwarding plane traffic (tmm) andnnnis the highest noise to capture as much flow information as possible to be displayed on the F5 dissector header.For more details about tcpdump syntax, please have a look atK13637: Capturing internal TMM information with tcpdumpandK411: Overview of packet tracing with the tcpdump utility. Also,we need to make sure we capture the full TLS handshake. It's perfectly fine to captureresumed TLS sessionsas long as full TLS handshake has been previously captured. Initially, our capture is unencrypted as seen below: On Mac, I clicked onWireshark→Preferences: ThenProtocols→TLS→RSA keys listwhere we see a window where we can reference BIG-IP's (or server if we want to decrypt server SSL side) private key: Once we get there, we need to add any IP address of the flow we want Wireshark to decrypt, the corresponding port and the private key file (default.crtfor Client SSL profile in this particular lab test): Note:For Client SSL profile, this would be the private key onCertificate Chainfield corresponding to the end entity Certificate being served to client machines through the Virtual Server. For Server SSL profile, the private key is located on the back-end server and the key would be the one corresponding to the end entity Certificate sent in the TLSCertificatemessage sent from back-end server to BIG-IP during TLS handshake. Once we clickOK, we can see the HTTP decrypted traffic (in green): In production environment, we would normally avoid copying private keys to different machines soanother option is usessldumpcommand directly on the server we're trying to capture. Again, if we're capturing Client SSL traffic,ssldumpis already installed on BIG-IP. We would follow the same steps as before but instead of copying private key to Wireshark machine, we would simply issue this command on the BIG-IP (or back-end server if it's Server SSL traffic): Syntax:ssldump-r<capture.pcap>-k<private key.key>-M<type a name for your ssldump file here.pms>. For more details, please have a look atK10209: Overview of packet tracing with the ssldump utility. Inssldump-generated.pms, we should find enough information for Wireshark to decrypt the capture: Syntax:ssldump-r<capture.pcap>-k<private key.key>-M<type a name for your ssldump file here.pms>. For more details, please have a look atK10209: Overview of packet tracing with the ssldump utility. Inssldump-generated.pms, we should find enough information for Wireshark to decrypt the capture: After I clickedOK, we indeed see the decrypted http traffic back again: We didn't have to copy BIG-IP's private key to Wireshark machine here. iRules The only constraint here is that we should apply the iRule to the virtual server in question. Sometimes that's not desirable, especially when we're troubleshooting an issue where we want the configuration to be unchanged. Note: there is abugthat affects versions 11.6.x and 12.x that was fixed on 13.x. It records the wrong TLS Session ID to LTM logs. The workaround would be to manually copy the Session ID from tcpdump capture or to use RSA decryption as in previous example. You can also combine bothSSL::clientrandomandSSL::sessionidwhich isthe ideal: Reference: K12783074: Decrypting SSL traffic using the SSL::sessionsecret iRules command (12.x and later) Again, I took a capture usingtcpdumpcommand: After applying above iRule to our HTTPS virtual server and taking tcpdump capture, I see this on /var/log/ltm: To copy this to a *.pms file wecanuseon Wireshark we can use sed command (reference:K12783074): Note:If you don't want to overwrite completely the PMS file make sure youuse >> instead of >. The endresult would be something like this: As both resumed and full TLS sessions have client random value, I only had to copy CLIENT_RANDOM + Master secret to our PMS file because all Wireshark needs is a session reference to apply master secret. To decrypt file on Wireshark just go toWireshark→Preferences→Protocols→TLS→Pre-Master Key logfile namelike we did inssldumpsection and add file we just created: As seen on LTM logs,CLIENTSSL_HANDSHAKEevent captured master secret from our client-side connection andSERVERSSL_HANDSHAKEfrom server side. In this case, we should have both client and server sides decrypted, even though we never had access to back-end server: Notice added anhttpfilter to show you both client and and server traffic were decrypted this time. tcpdump --f5 ssl option This was introduced in 15.x and we don't need to change virtual server configuration by adding iRules. The only thing we need to do is to enabletcpdump.sslproviderdb variable which is disabled by default: After that, when we take tcpdump capture, we just need to add --f5 ssl to the command like this: Notice that we've got a warning message because Master Secret will be copied to tcpdump capture itself, so we need to be careful with who we share such capture with. I had to update my Wireshark to v3.2.+ and clicked onAnalyze→Enabled Protocols: And enable F5 TLS dissector: Once we open the capture, we can find all the information you need to create our PMS file embedded in the capture: Very cool, isn't it? We can then copy the Master Secretand Client Random values by right clicking like this: And then paste it to a blank PMS file. I first pasted the Client Random value followed by Master Secret value like this: Note: I manually typedCLIENT_RANDOMand then pasted both values for both client and server sides directly from tcpdump capture. The last step was to go toWireshark→Preferences→Protocols→TLSand add it toPre-master-Secret log filenameand clickOK: Fair enough! Capture decrypted on both client and server sides: I usedhttpfilter to display only decrypted HTTP packets just like in iRule section. Appendix 1 How do we know which Key Exchange method is being used? RSA private key can only decrypt traffic on Wireshark if RSA is the key exchange method negotiated during TLS handshake. Client side will tell the Server side which ciphers it support and server side will reply with the chosen cipher onServer Hellomessage. With that in mind, on Wireshark, we'd click onServer Helloheader underCipher Suite: Key Exchange and Authentication both come before theWITHkeyword. In above example, because there's only RSA we can say that RSA is used for both Key Exchange and Authentication. In the following example, ECDHE is used for key exchange and RSAfor authentication: Appendix 2 Disabling Session Resumption and Prioritising RSA key exchange We can set Cache Size to 0 to disableTLS session resumptionand change the Cipher Suites to anything that makes BIG-IP pick RSA for Key Exchange:
APM Configuration to Support Duo MFA using iRule
Overview BIG-IP APM has supported Duo as an MFA provider for a long time with RADIUS-based integration. Recently, Duo has added support for Universal Prompt that uses Open ID Connect (OIDC) protocol to provide two-factor authentication. To integrate APM as an OIDC client and resource server, and Duo as an Identity Provider (IdP), Duo requires the user’s logon name and custom parameters to be sent for Authentication and Token request. This guide describes the configuration required on APM to enable Duo MFA integration using an iRule. iRules addresses the custom parameter challenges by generating the needed custom values and saving them in session variables, which the OAuth Client agent then uses to perform MFA with Duo. This integration procedure is supported on BIG-IP versions 13.1, 14.1x, 15.1x, and 16.x. To integrate Duo MFA with APM, complete the following tasks: 1. Choose deployment type: Per-request or Per-session 2. Configure credentials and policies for MFA on the DUO web portal 3. Create OAuth objects on the BIG-IP system 4. Configure the iRule 5. Create the appropriate access policy/policies on the BIG-IP system 6. Apply policy/policies and iRule to the APM virtual server Choose deployment type APM supports two different types of policies for performing authentication functions. Per-session policies: Per-session policies provide authentication and authorization functions that occur only at the beginning of a user’s session. These policies are compatible with most APM use cases such as VPN, Webtop portal, Remote Desktop, federation IdP, etc. Per-request policies: Per-request policies provide dynamic authentication and authorization functionality that may occur at any time during a user’s session, such as step-up authentication or auditing functions only for certain resources. These policies are only compatible with Identity Aware Proxy and Web Access Management use cases and cannot be used with VPN or webtop portals. This guide contains information about setting up both policy types. Prerequisites Ensure the BIG-IP system has DNS and internet connectivity to contact Duo directly for validating the user's OAuth tokens. Configure credentials and policies for MFA on Duo web portal Before you can protect your F5 BIG-IP APM Web application with Duo, you will first need to sign up for a Duo account. 1. Log in to the Duo Admin Panel and navigate to Applications. 2. Click Protect an application. Figure 1: Duo Admin Panel – Protect an Application 3. Locate the entry for F5 BIG-IP APM Web in the applications list and click Protect to get the Client ID, Client secret, and API hostname. You will need this information to configure objects on APM. Figure 2: Duo Admin Panel – F5 BIG-IP APM Web 4. As DUO is used as a secondary authentication factor, the user’s logon name is sent along with the authentication request. Depending on your security policy, you may want to pre-provision users in Duo, or you may allow them to self-provision to set their preferred authentication type when they first log on. To add users to the Duo system, navigate to the Dashboard page and click the Add New...-> Add User button. A Duo username should match the user's primary authentication username. Refer to the https://duo.com/docs/enrolling-users link for the different methods of user enrollment. Refer to Duo Universal Prompt for additional information on Duo’s two-factor authentication. Create OAuth objects on the BIG-IP system Create a JSON web key When APM is configured to act as an OAuth client or resource server, it uses JSON web keys (JWKs) to validate the JSON web tokens it receives from Duo. To create a JSON web key: 1. On the Main tab, select Access > Federation > JSON Web Token > Key Configuration. The Key Configuration screen opens. 2. To add a new key configuration, click Create. 3. In the ID and Shared Secret fields, enter the Client ID and Client Secret values respectively obtained from Duo when protecting the application. 4. In the Type list, select the cryptographic algorithm used to sign the JSON web key. Figure 3: Key Configuration screen 5. Click Save. Create a JSON web token As an OAuth client or resource server, APM validates the JSON web tokens (JWT) it receives from Duo. To create a JSON web token: 1. On the Main tab, select Access > Federation > JSON Web Token > Token Configuration. The Token Configuration screen opens. 2. To add a new token configuration, click Create. 3. In the Issuer field, enter the API hostname value obtained from Duo when protecting the application. 4. In the Signing Algorithms area, select from the Available list and populate the Allowed and Blocked lists. 5. In the Keys (JWK) area, select the previously configured JSON web key in the allowed list of keys. Figure 4: Token Configuration screen 6. Click Save. Configure Duo as an OAuth provider APM uses the OAuth provider settings to get URIs on the external OAuth authorization server for JWT web tokens. To configure an OAuth provider: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > Provider. The Provider screen opens. 2. To add a provider, click Create. 3. In the Name field, type a name for the provider. 4. From the Type list, select Custom. 5. For Token Configuration (JWT), select a configuration from the list. 6. In the Authentication URI field, type the URI on the provider where APM should redirect the user for authentication. The hostname is the same as the API hostname in the Duo application. 7. In the Token URI field, type the URI on the provider where APM can get a token. The hostname is the same as the API hostname in the Duo application. Figure 5: OAuth Provider screen 8. Click Finished. Configure Duo server for APM The OAuth Server settings specify the OAuth provider and role that Access Policy Manager (APM) plays with that provider. It also sets the Client ID, Client Secret, and Client’s SSL certificates that APM uses to communicate with the provider. To configure a Duo server: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > OAuth Server. The OAuth Server screen opens. 2. To add a server, click Create. 3. In the Name field, type a name for the Duo server. 4. From the Mode list, select how you want the APM to be configured. 5. From the Type list, select Custom. 6. From the OAuth Provider list, select the Duo provider. 7. From the DNS Resolver list, select a DNS resolver (or click the plus (+) icon, create a DNS resolver, and then select it). 8. In the Token Validation Interval field, type a number. In a per-request policy subroutine configured to validate the token, the subroutine repeats at this interval or the expiry time of the access token, whichever is shorter. 9. In the Client Settings area, paste the Client ID and Client secret you obtained from Duo when protecting the application. 10. From the Client's ServerSSL Profile Name, select a server SSL profile. Figure 6: OAuth Server screen 11. Click Finished. Configure an auth-redirect-request and a token-request Requests specify the HTTP method, parameters, and headers to use for the specific type of request. An auth-redirect-request tells Duo where to redirect the end-user, and a token-request accesses the authorization server for obtaining an access token. To configure an auth-redirect-request: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > Request. The Request screen opens. 2. To add a request, click Create. 3. In the Name field, type a name for the request. 4. For the HTTP Method, select GET. 5. For the Type, select auth-redirect-request. 6. As shown in Figure 7, specify the list of GET parameters to be sent: request parameter with value depending on the type of policy For per-request policy: %{subsession.custom.jwt_duo} For per-session policy: %{session.custom.jwt_duo} client_id parameter with type client-id response_type parameter with type response-type Figure 7: Request screen with auth-redirect-request (Use “subsession.custom…” for Per-request or “session.custom…” for Per-session) 7. Click Finished. To configure a token-request: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > Request. The Request screen opens. 2. To add a request, click Create. 3. In the Name field, type a name for the request. 4. For the HTTP Method, select POST. 5. For the Type, select token-request. 6. As shown in Figure 8, specify the list of POST parameters to be sent: client_assertion parameter with value depending on the type of policy For per-request policy: %{subsession.custom.jwt_duo_token} For per-session policy: %{session.custom.jwt_duo_token} client_assertion_type parameter with value urn:ietf:params:oauth:client-assertion-type:jwt-bearer grant_type parameter with type grant-type redirect_uri parameter with type redirect-uri Figure 8: Request screen with token-request (Use “subsession.custom…” for Per-request or “session.custom…” for Per-session) 7. Click Finished. Configure the iRule iRules gives you the ability to customize and manage your network traffic. Configure an iRule that creates the required sub-session variables and usernames for Duo integration. Note: This iRule has sections for both per-request and per-session policies and can be used for either type of deployment. To configure an iRule: 1. On the Main tab, click Local Traffic > iRules. 2. To create an iRules, click Create. 3. In the Name field, type a name for the iRule. 4. Copy the sample code given below and paste it in the Definition field. Replace the following variables with values specific to the Duo application: <Duo Client ID> in the getClientId function with Duo Application ID. <Duo API Hostname> in the createJwtToken function with API Hostname. For example, https://api-duohostname.com/oauth/v1/token. <JSON Web Key> in the getJwkName function with the configured JSON web key. Note: The iRule ID here is set as JWT_CREATE. You can rename the ID as desired. You specify this ID in the iRule Event agent in Visual Policy Editor. Note: The variables used in the below example are global, which may affect your performance. Refer to the K95240202: Understanding iRule variable scope article for further information on global variables, and determine if you use a local variable for your implementation. proc randAZazStr {len} { return [subst [string repeat {[format %c [expr {int(rand() * 26) + (rand() > .5 ? 97 : 65)}]]} $len]] } proc getClientId { return <Duo Client ID> } proc getExpiryTime { set exp [clock seconds] set exp [expr $exp + 900] return $exp } proc getJwtHeader { return "{\"alg\":\"HS512\",\"typ\":\"JWT\"}" } proc getJwkName { return <JSON Web Key> #e.g. return "/Common/duo_jwk" } proc createJwt {duo_uname} { set header [call getJwtHeader] set exp [call getExpiryTime] set client_id [call getClientId] set redirect_uri "https://" set redirect [ACCESS::session data get "session.server.network.name"] append redirect_uri $redirect append redirect_uri "/oauth/client/redirect" set payload "{\"response_type\": \"code\",\"scope\":\"openid\",\"exp\":${exp},\"client_id\":\"${client_id}\",\"redirect_uri\":\"${redirect_uri}\",\"duo_uname\":\"${duo_uname}\"}" set jwt_duo [ ACCESS::oauth sign -header $header -payload $payload -alg HS512 -key [call getJwkName] ] return $jwt_duo } proc createJwtToken { set header [call getJwtHeader] set exp [call getExpiryTime] set client_id [call getClientId] set aud "<Duo API Hostname>/oauth/v1/token" #Example: set aud https://api-duohostname.com/oauth/v1/token set jti [call randAZazStr 32] set payload "{\"sub\": \"${client_id}\",\"iss\":\"${client_id}\",\"aud\":\"${aud}\",\"exp\":${exp},\"jti\":\"${jti}\"}" set jwt_duo [ ACCESS::oauth sign -header $header -payload $payload -alg HS512 -key [call getJwkName] ] return $jwt_duo } when ACCESS_POLICY_AGENT_EVENT { set irname [ACCESS::policy agent_id] if { $irname eq "JWT_CREATE" } { set ::duo_uname [ACCESS::session data get "session.logon.last.username"] ACCESS::session data set session.custom.jwt_duo [call createJwt $::duo_uname] ACCESS::session data set session.custom.jwt_duo_token [call createJwtToken] } } when ACCESS_PER_REQUEST_AGENT_EVENT { set irname [ACCESS::perflow get perflow.irule_agent_id] if { $irname eq "JWT_CREATE" } { set ::duo_uname [ACCESS::session data get "session.logon.last.username"] ACCESS::perflow set perflow.custom [call createJwt $::duo_uname] ACCESS::perflow set perflow.scratchpad [call createJwtToken] } } Figure 9: iRule screen 5. Click Finished. Create the appropriate access policy/policies on the BIG-IP system Per-request policy Skip this section for a per-session type deployment The per-request policy is used to perform secondary authentication with Duo. Configure the access policies through the access menu, using the Visual Policy Editor. The per-request access policy must have a subroutine with an iRule Event, Variable Assign, and an OAuth Client agent that requests authorization and tokens from an OAuth server. You may use other per-request policy items such as URL branching or Client Type to call Duo only for certain target URIs. Figure 10 shows a subroutine named duosubroutine in the per-request policy that handles Duo MFA authentication. Figure 10: Per-request policy in Visual Policy Editor Configuring the iRule Event agent The iRule Event agent specifies the iRule ID to be executed for Duo integration. In the ID field, type the iRule ID as configured in the iRule. Figure 11: iRule Event agent in Visual Policy Editor Configuring the Variable Assign agent The Variable Assign agent specifies the variables for token and redirect requests and assigns a value for Duo MFA in a subroutine. This is required only for per-request type deployment. Add sub-session variables as custom variables and assign their custom Tcl expressions as shown in Figure 12. subsession.custom.jwt_duo_token = return [mcget {perflow.scratchpad}] subsession.custom.jwt_duo = return [mcget {perflow.custom}] Figure 12: Variable Assign agent in Visual Policy Editor Configuring the OAuth Client agent An OAuth Client agent requests authorization and tokens from the Duo server. Specify OAuth parameters as shown in Figure 13. In the Server list, select the Duo server to which the OAuth client directs requests. In the Authentication Redirect Request list, select the auth-redirect-request configured earlier. In the Token Request list, select the token-request configured earlier. Some deployments may not need the additional information provided by OpenID Connect. You could, in that case, disable it. Figure 13: OAuth Client agent in Visual Policy Editor Per-session policy Configure the Per Session policy as appropriate for your chosen deployment type. Per-request: The per-session policy must contain at least one logon page to set the username variable in the user’s session. Preferably it should also perform some type of primary authentication. This validated username is used later in the per-request policy. Per-session: The per-session policy is used for all authentication. A per-request policy is not used. Figures 14a and 14b show a per-session policy that runs when a client initiates a session. Depending on the actions you include in the access policy, it can authenticate the user and perform actions that populate session variables with data for use throughout the session. Figure 14a: Per-session policy in Visual Policy Editor performs both primary authentication and Duo authentication (for per-session use case) Figure 14b: Per-session policy in Visual Policy Editor performs primary authentication only (for per-request use case) Apply policy/policies and iRule to the APM virtual server Finally, apply the per-request policy, per-session policy, and iRule to the APM virtual server. You assign iRules as a resource to the virtual server that users connect. Configure the virtual server’s default pool to the protected local web resource. Apply policy/policies to the virtual server Per-request policy To attach policies to the virtual server: 1. On the Main tab, click Local Traffic > Virtual Servers. 2. Select the Virtual Server. 3. In the Access Policy section, select the policy you created. 4. Click Finished. Figure 15: Access Policy section in Virtual Server (per-request policy) Per-session policy Figure 16 shows the Access Policy section in Virtual Server when the per-session policy is deployed. Figure 16: Access Policy section in Virtual Server (per-session policy) Apply iRule to the virtual server To attach the iRule to the virtual server: 1. On the Main tab, click Local Traffic > Virtual Servers. 2. Select the Virtual Server. 3. Select the Resources tab. 4. Click Manage in the iRules section. 5. Select an iRule from the Available list and add it to the Enabled list. 6. Click Finished.
iRule to set SameSite for compatible clients and remove it for incompatible clients (LTM|ASM|APM)
A bunch of us have been refining approaches to help customers handle the new browser enforcement of the HTTP cookie SameSite attribute. I think we have a pretty solid approach now to handle compatible and incompatible user-agents. The iRule: Allows the admin to set the SameSite attribute on BIG-IP and web application cookies (all cookies, explicitly named cookies or cookies that start with a string) for user-agents that handle the SameSite attribute Allows the admin to remove the SameSite attribute for user-agents which do not support SameSite=None. The behavior for an incompatible client receiving a cookie with SameSite not set should be the same as a compatible client handling SameSite=None (the incompatible client should send the cookie on third party requests) The iRule uses Simon Kowallik's updated string matching logic to handle the incompatible user-agent from Chomium's blog: https://www.chromium.org/updates/same-site/incompatible-clients Note this iRule only modifies BIG-IP and web application cookies found in Set-Cookie headers. It does not attempt to modify cookies that the BIG-IP or web application sets via Javascript or other methods. BIG-IP ASM is known to set some cookies via Javascript. If you require support for this, please open a case with F5 support (https://support.f5.com) and request your case be added to: BZ875909: Allow admin configuration of SameSite attribute on ASM system cookies set via Set-Cookie and Javascript Updates to the iRule can be found in the irules-toolbox repo on GitHub. This specific version is for v12+, but there is a pre-v12 version in the repo as well. Configuration options in the iRule: samesite_security: Set this to Strict, Lax or None. The description for these values is in the iRule quoted below: # Set BIG-IP and app cookies found in Set-Cookie headers using this iRule to: # # none: Cookies will be sent in both first-party context and cross-origin requests; #however, the value must be explicitly set to None and all browser requests must #follow the HTTPS protocol and include the Secure attribute which requires an encrypted #connection. Cookies that don't adhere to that requirement will be rejected. #Both attributes are required together. If just None is specified without Secure or #if the HTTPS protocol is not used, the third-party cookie will be rejected. # # lax: Cookies will be sent automatically only in a first-party context and with HTTP GET requests. #SameSite cookies will be withheld on cross-site sub-requests, such as calls to load images or iframes, #but will be sent when a user navigates to the URL from an external site, e.g., by following a link. # # strict: browser never sends cookies in requests to third party domains # #Above definitions from: https://docs.microsoft.com/en-us/microsoftteams/platform/resources/samesite-cookie-update # # Note: this iRule does not modify cookies set on the client using Javascript or other methods outside of Set-Cookie headers! set samesite_security "none" Uncomment the next command if you're using this iRule on an APM virtual server with an access profile: # Uncomment when using this iRule on an APM-enabled virtual server so the MRHSession cookies will be rewritten # The iRule cannot be saved on a virtual server with this option uncommented if there is no Access profile also enabled #ACCESS::restrict_irule_events disable Now define whether you want to rewrite all web application and BIG-IP cookies found in the Set-Cookie header(s). Set this to 1 to rewrite SameSite on all cookies in Set-Cookie headers. Else, if you want to define specifically named or prefixed cookies, set this option to 0, and proceed to the next two config options, #2 and #3 # 1. If you want to set SameSite on all BIG-IP and web application cookies for compliant user-agents, set this option to 1 # Else, if you want to use the next two options for rewriting explicit named cookies or cookie prefixes, set this option to 0 set set_samesite_on_all 0 If you don't want to rewrite all cookies using option #1 above, you can choose to rewrite explicitly named cookies in option #2. Set the exact cookie names in the named_cookie list. Replace MRHSession and LastMRH_Session, which are examples of the cookies APM uses. If you do not want to rewrite exact cookie names, comment out the first example and uncomment the second example "set named_cookies {}" # 2. Rewrite SameSite on specific named cookies # # To enable this, list the specific named cookies in the list command and comment out the second set command below # To disable this, set this variable to {} and comment out the first set command below set named_cookies [list {MRHSession} {LastMRH_Session}] #set named_cookies {} If you don't want to rewrite all cookies using option #1 above, you can choose to rewrite cookies using a prefix in option #3. Set the cookie name prefixes in the named_cookie list. Replace BIGipServer and TS, which are examples of the cookie prefixes LTM uses for persistence and ASM uses for session tracking, with the prefixes of the cookie names you want to rewrite. If you do not want to rewrite using cookie name prefixes, comment out the first example and uncomment the second example "set named_cookies {}" # 3. Rewrite cookies with a prefix like BIG-IP persistence cookies # To enable this, list the cookie name prefixes in the list command and comment out the second set command below # To disable this, set this variable to {} and comment out the first set command below set cookie_prefixes [list {BIGipServer} {TS}] #set cookie_prefixes {} If your application or BIG-IP configuration sets cookies in the Set-Cookie headers with SameSite=None, incompatible user-agents will either reject the cookie or treat the cookie as if it was set for SameSite=Strict (https://www.chromium.org/updates/same-site/incompatible-clients). You can set remove_samesite_for_incompatible_user_agents to 1 to have this iRule remove SameSite attributes from all cookies sent to incompatible browsers. # For incompatible user-agents, this iRule can remove the SameSite attribute from all cookies sent to the client via Set-Cookie headers # This is only necessary if BIG-IP or the web application being load balanced sets SameSite=None for all clients # set to 1 to enable, 0 to disable set remove_samesite_for_incompatible_user_agents 1 While testing, you can set samesite_debug to 1 to test and get debug written to /var/log/ltm. Make sure to disable this option when you're done testing, before putting the iRule into production! # Log debug to /var/log/ltm? 1=yes, 0=no # set to 0 after testing set samesite_debug 1 The full iRule: (Updates can be found in the irules-toolbox repo on GitHub. This specific version is for v12+, but there is a pre-v12 version in the repo as well.) # iRule: samesite_cookie_handling # author: Simon Kowallik # version: 1.3 # # History: version - author - description # 1.0 - Simon Kowallik - initial version # 1.1 - Aaron Hooley - updated to add support for setting SameSite to Strict|Lax|None for BIG-IP and app cookies in Set-Cookie headers # - Add option to remove SameSite=None cookies for incompatible browsers # 1.2 - Aaron Hooley - Added option to rewrite all cookies without naming them explicitly or with prefixes # 1.3 - Aaron Hooley - set samesite_compatible to 0 by default instead of a null string # # What the iRule does: # Sets SameSite to Strict, Lax or None (and sets Secure when SameSite=None) for compatible user-agents # Optionally removes SameSite attribute from all cookies for incompatible user-agents so they'll handle cookies as if they were SameSite=None # # The iRule should work for: # - LTM for web app cookies and persistence cookies, except those that the web app sets via Javascript # - ASM for web app cookies and all ASM cookies except those that ASM or the web app sets via Javascript # - APM for web app cookies and all APM cookies you configure in the config variable $named_cookies, except those that the web app sets via Javascript # # The iRule requires BIG-IP v12 or greater to use the HTTP::cookie attribute command # # RFC "standards" # https://tools.ietf.org/html/draft-west-cookie-incrementalism-00 # https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-05 # further reading: # https://web.dev/samesite-cookies-explained/ # https://web.dev/samesite-cookie-recipes/ # https://blog.chromium.org/2019/10/developers-get-ready-for-new.html # https://www.chromium.org/updates/same-site # https://www.chromium.org/updates/same-site/incompatible-clients proc checkSameSiteCompatible {user_agent} { # Procedure to check if a user-agent supports SameSite=None on cookies # # usage: # set isSameSiteCompatible [call checkSameSiteCompatible {User-Agent-String}] # # check for incompatible user-agents: https://www.chromium.org/updates/same-site/incompatible-clients # based on https://devcentral.f5.com/s/articles/HTTP-cookie-SameSite-test-detection-of-browsers-with-incompatible-SameSite-None-handling switch -glob -- [set user_agent [string tolower $user_agent]] { {*chrome/5[1-9].[0-9]*} - {*chrome/6[0-6].[0-9]*} - {*chromium/5[1-9].[0-9]*} - {*chromium/6[0-6].[0-9]*} - {*ip?*; cpu *os 12*applewebkit*} - {*macintosh;*mac os x 10_14*version*safari*} - {mozilla*macintosh;*mac os x 10_14*applewebkit*khtml, like gecko*} { # no samesite support return 0 } {*ucbrowser/*} { switch -glob -- $user_agent { {*ucbrowser/[1-9].*} - {*ucbrowser/1[0-1].*} - {*ucbrowser/12.[0-9].*} - {*ucbrowser/12.1[0-1].*} - {*ucbrowser/12.12.*} - {*ucbrowser/12.13.[0-2]*} { # no samesite support return 0 } } } } # If the current user-agent didn't match any known incompatible browser list, assume it can handle SameSite=None return 1 # CPU Cycles on Executing (>100k test runs) # Average 22000-42000 (fastest to slowest path) # Maximum 214263 # Minimum 13763 } # the iRule code when CLIENT_ACCEPTED priority 100 { # Set BIG-IP and app cookies found in Set-Cookie headers using this iRule to: # # none: Cookies will be sent in both first-party context and cross-origin requests; # however, the value must be explicitly set to None and all browser requests must # follow the HTTPS protocol and include the Secure attribute which requires an encrypted # connection. Cookies that don't adhere to that requirement will be rejected. # Both attributes are required together. If just None is specified without Secure or # if the HTTPS protocol is not used, the third-party cookie will be rejected. # # lax: Cookies will be sent automatically only in a first-party context and with HTTP GET requests. # SameSite cookies will be withheld on cross-site sub-requests, such as calls to load images or iframes, # but will be sent when a user navigates to the URL from an external site, e.g., by following a link. # # strict: browser never sends cookies in requests to third party domains # # Above definitions from: https://docs.microsoft.com/en-us/microsoftteams/platform/resources/samesite-cookie-update # # Note: this iRule does not modify cookies set on the client using Javascript or other methods outside of Set-Cookie headers! set samesite_security "none" # Uncomment when using this iRule on an APM-enabled virtual server so the MRHSession cookies will be rewritten # The iRule cannot be saved on a virtual server with this option uncommented if there is no Access profile also enabled #ACCESS::restrict_irule_events disable # 1. If you want to set SameSite on all BIG-IP and web application cookies for compliant user-agents, set this option to 1 # Else, if you want to use the next two options for rewriting explicit named cookies or cookie prefixes, set this option to 0 set set_samesite_on_all 0 # 2. Rewrite SameSite on specific named cookies # # To enable this, list the specific named cookies in the list command and comment out the second set command below # To disable this, set this variable to {} and comment out the first set command below set named_cookies [list {MRHSession} {LastMRH_Session}] #set named_cookies {} # 3. Rewrite cookies with a prefix like BIG-IP persistence cookies # To enable this, list the cookie name prefixes in the list command and comment out the second set command below # To disable this, set this variable to {} and comment out the first set command below set cookie_prefixes [list {BIGipServer} {TS}] #set cookie_prefixes {} # For incompatible user-agents, this iRule can remove the SameSite attribute from all cookies sent to the client via Set-Cookie headers # This is only necessary if BIG-IP or the web application being load balanced sets SameSite=None for all clients # set to 1 to enable, 0 to disable set remove_samesite_for_incompatible_user_agents 1 # Log debug to /var/log/ltm? 1=yes, 0=no # set to 0 after testing set samesite_debug 1 # You shouldn't have to make changes to configuration below here # Track the user-agent and whether it supports the SameSite cookie attribute set samesite_compatible 0 set user_agent {} if { $samesite_debug }{ set prefix "[IP::client_addr]:[TCP::client_port]:" log local0. "$prefix [string repeat "=" 40]" log local0. "$prefix \$samesite_security=$samesite_security; \$set_samesite_on_all=$set_samesite_on_all; \$named_cookies=$named_cookies; \$cookie_prefixes=$cookie_prefixes, \ \$remove_samesite_for_incompatible_user_agents=$remove_samesite_for_incompatible_user_agents" } } # Run this test event before any other iRule HTTP_REQUEST events to set the User-Agent header value # Comment out this event when done testing user-agents #when HTTP_REQUEST priority 2 { # known compatible # HTTP::header replace user-agent {my compatible user agent string} # known INcompatible # HTTP::header replace user-agent {chrome/51.10} #} # Run this iRule before any other iRule HTTP_REQUEST events when HTTP_REQUEST priority 100 { # If we're setting samesite=none, we need to check the user-agent to see if it's compatible if { not [string equal -nocase $samesite_security "none"] }{ # Not setting SameSite=None, so exit this event return } # Inspect user-agent once per TCP session for higher performance if the user-agent hasn't changed if { $samesite_compatible == 0 or $user_agent ne [HTTP::header value {User-Agent}]} { set user_agent [HTTP::header value {User-Agent}] set samesite_compatible [call checkSameSiteCompatible $user_agent] if { $samesite_debug }{ log local0. "$prefix Got \$samesite_compatible=$samesite_compatible and saved current \$user_agent: $user_agent" } } } # Run this response event with priority 900 after all other iRules to parse the final cookies from the application and BIG-IP when HTTP_RESPONSE_RELEASE priority 900 { # Log the pre-existing Set-Cookie header values if { $samesite_debug }{ log local0. "$prefix Set-Cookie value(s): [HTTP::header values {Set-Cookie}]" } if { $samesite_compatible } { # user-agent is compatible with SameSite=None, set SameSite on matching cookies if { $set_samesite_on_all }{ if { $samesite_debug }{ log local0. "$prefix Setting SameSite=$samesite_security on all cookies and exiting" } foreach cookie [HTTP::cookie names] { if { $samesite_debug }{ log local0. "$prefix Set SameSite=$samesite_security on $cookie" } # Remove any prior instances of SameSite attributes HTTP::cookie attribute $cookie remove {samesite} # Insert a new SameSite attribute HTTP::cookie attribute $cookie insert {samesite} $samesite_security # If samesite attribute is set to None, then the Secure flag must be set for browsers to accept the cookie if {[string equal -nocase $samesite_security "none"]} { HTTP::cookie secure $cookie enable } } # Exit this event in this iRule as we've already rewritten all cookies with SameSite return } # Match named cookies exactly if { $named_cookies ne {} }{ foreach cookie $named_cookies { if { [HTTP::cookie exists $cookie] } { # Remove any pre-existing SameSite attributes from this cookie as most clients use the most strict value if multiple instances are set HTTP::cookie attribute $cookie remove {SameSite} # Insert the SameSite attribute HTTP::cookie attribute $cookie insert {SameSite} $samesite_security # If samesite attribute is set to None, then the Secure flag must be set for browsers to accept the cookie if {[string equal -nocase $samesite_security "none"]} { HTTP::cookie secure $cookie enable } if { $samesite_debug }{ log local0. "$prefix Matched explicitly named cookie $cookie, set SameSite=$samesite_security" } if { $samesite_debug }{ log local0. "$prefix " } } } } # Match a cookie prefix (cookie name starts with a prefix from the $cookie_prefixes list) if { $cookie_prefixes ne {} }{ foreach cookie [HTTP::cookie names] { foreach cookie_prefix $cookie_prefixes { if { $cookie starts_with $cookie_prefix } { # Remove any pre-existing SameSite attributes from this cookie as most clients use the most strict value if multiple instances are set HTTP::cookie attribute $cookie remove {SameSite} # Insert the SameSite attribute HTTP::cookie attribute $cookie insert {SameSite} $samesite_security # If samesite attribute is set to None, then the Secure flag must be set for browsers to accept the cookie if { [string equal -nocase $samesite_security "none"] } { HTTP::cookie secure $cookie enable } if { $samesite_debug }{ log local0. "$prefix Matched prefixed cookie $cookie, with prefix $cookie_prefix, set SameSite=$samesite_security, breaking from loop" } break } } } } } else { # User-agent can't handle SameSite=None if { $remove_samesite_for_incompatible_user_agents }{ # User-agent can't handle SameSite=None, so remove SameSite attribute from all cookies if SameSite=None # This will use CPU cycles on BIG-IP so only enable it if you know BIG-IP or the web application is setting # SameSite=None for all clients including incompatible ones foreach cookie [HTTP::cookie names] { if { [string tolower [HTTP::cookie attribute $cookie value SameSite]] eq "none" }{ HTTP::cookie attribute $cookie remove SameSite if { $samesite_debug }{ log local0. "$prefix Removing SameSite for incompatible client from cookie=$cookie" } } } } } # Log the modified Set-Cookie header values if { $samesite_debug }{ log local0. "$prefix Final Set-Cookies: [HTTP::header values {Set-Cookie}]" } }APM-DHCP Access Policy Example and Detailed Instructions
Prepared with Mark Quevedo, F5 Principal Software Engineer May, 2020 Sectional Navigation links Important Version Notes || Installation Guide || What Is Going On Here? || Parameters You Set In Your APM Access Policy || Results of DHCP Request You Use in Access Policy || Compatibility Tips and Troubleshooting Introduction Ordinarily you assign an IP address to the “inside end” of an APM Network Tunnel (full VPN connection) from an address Lease Pool, from a static list, or from an LDAP or RADIUS attribute. However, you may wish to assign an IP address you get from a DHCP server. Perhaps the DHCP server manages all available client addresses. Perhaps it handles dynamic DNS for named client workstations. Or perhaps the DHCP server assigns certain users specific IP addresses (for security filtering). Your DHCP server may even assign client DNS settings as well as IP addresses. APM lacks DHCP address assignment support (though f5's old Firepass VPN had it ). We will use f5 iRules to enable DHCP with APM. We will send data from APM session variables to the DHCP server so it can issue the “right” IP address to each VPN tunnel based on user identity, client info, etc. Important Version Notes Version v4c includes important improvements and bug fixes. If you are using an older version, you should upgrade. Just import the template with “Overwrite existing templates” checked, then “reconfigure” your APM-DHCP Application Service—you can simply click “Finished” without changing any options to update the iRules in place. Installation Guide First install the APM-DHCP iApp template (file DHCP_for_APM.tmpl). Create a new Application Service as shown (choose any name you wish). Use the iApp to manage the APM-DHCP virtual servers you need. (The iApp will also install necessary iRules.) You must define at least one APM-DHCP virtual server to receive and send DHCP packets. Usually an APM-DHCP virtual server needs an IP address on the subnet on which you expect your DHCP server(s) to assign client addresses. You may define additional APM-DHCP virtual servers to request IP addresses on additional subnets from DHCP. However, if your DHCP server(s) support subnet-selection (see session.dhcp.subnet below) then you may only need a single APM-DHCP virtual server and it may use any IP that can talk to your DHCP server(s). It is best to give each APM-DHCP virtual server a unique IP address but you may use an BIG-IP Self IP as per SOL13896 . Ensure your APM and APM-DHCP virtual servers are in the same TMOS Traffic Group (if that is impossible set TMOS db key tmm.sessiondb.match_ha_unit to false). Ensure that your APM-DHCP virtual server(s) and DHCP server(s) or relay(s) are reachable via the same BIG-IP route domain. Specify in your IP addresses any non-zero route-domains you are using (e.g., “192.168.0.20%3”)—this is essential. (It is not mandatory to put your DHCP-related Access Policy Items into a Macro—but doing so makes the below screenshot less wide!) Into your APM Access Policy, following your Logon Page and AD Auth (or XYZ Auth) Items (etc.) but before any (Full/Advanced/simple) Resource Assign Item which assigns the Network Access Resource (VPN), insert both Machine Info and Windows Info Items. (The Windows Info Item will not bother non-Windows clients.) Next insert a Variable Assign Item and name it “DHCP Setup”. In your “DHCP Setup” Item, set any DHCP parameters (explained below) that you need as custom session variables. You must set session.dhcp.servers. You must also set session.dhcp.virtIP to the IP address of an APM-DHCP virtual server (either here or at some point before the “DHCP_Req” iRule Event Item). Finally, insert an iRule Event Item (name it “DHCP Req”) and set its Agent ID to DHCP_req. Give it a Branch Rule “Got IP” using the expression “expr {[mcget {session.dhcp.address}] ne ""}” as illustrated. You must attach iRule ir-apm-policy-dhcp to your APM virtual server (the virtual server to which your clients connect). Neither the Machine Info Item nor the Windows Info Item is mandatory. However, each gathers data which common DHCP servers want to see. By default DHCP_req will send that data, when available, to your DHCP servers. See below for advanced options: DHCP protocol settings, data sent to DHCP server(s), etc. Typically your requests will include a user identifier from session.dhcp.subscriber_ID and client (machine or connection) identifiers from other parameters. The client IP address assigned by DHCP will appear in session.dhcp.address. By default, the DHCP_req iRule Event handler will also copy that IP address into session.requested.clientip where the Network Access Resource will find it. You may override that behavior by setting session.dhcp.copy2var (see below). Any “vendor-specific information” supplied by the DHCP server 1 (keyed by the value of session.dhcp.vendor_class) will appear in variables session.dhcp.vinfo.N where N is a tag number (1-254). You may assign meanings to tag numbers. Any DNS parameters the DHCP server supplies 2 are in session.dhcp.dns_servers and session.dhcp.dns_suffix. If you want clients to use those DNS server(s) and/or DNS default search domain, put the name of every Network Access Resource your Access Policy may assign to the client into the session.dhcp.dns_na_list option. NB: this solution does not renew DHCP address leases automatically, but it does release IP addresses obtained from DHCP after APM access sessions terminate. 3 Please configure your DHCP server(s) for an address lease time longer than your APM Maximum Session Timeout. Do not configure APM-DHCP virtual servers in different BIG-IP route domains so they share any part of a DHCP client IP range (address lease pool). For example, do not use two different APM-DHCP virtual servers 10.1.5.2%6 and 10.1.5.2%8 with one DHCP client IP range 10.1.5.10—10.1.5.250. APM-DHCP won’t recognize when two VPN sessions in different route domains get the same client IP from a non-route-domain-aware DHCP server, so it may not release their IP’s in proper sequence. This solution releases DHCP address leases for terminated APM sessions every once in a while, when a new connection comes in to the APM virtual server (because the BIG IP only executes the relevant iRules on the “event” of each new connection). When traffic is sparse (say, in the middle of the night) there may be some delay in releasing addresses for dead sessions. If ever you think this solution isn’t working properly, be sure to check the BIG IP’s LTM log for warning and error messages. DHCP Setup (a Variable Assign Item) will look like: Put the IP of (one of) your APM-DHCP virtual server(s) in session.dhcp.virtIP. Your DHCP server list may contain addresses of DHCP servers or relays. You may list a directed broadcast address (e.g., “172.16.11.255”) instead of server addresses but that will generate extra network chatter. To log information about DHCP processing for the current APM session you may set variable session.dhcp.debug to true (don’t leave it enabled when not debugging). DHCP Req (an iRule Event Item) will look like: Note DHCP Req branch rules: If DHCP fails, you may wish to warn the user: (It is not mandatory to Deny access after DHCP failure—you may substitute another address into session.requested.clientip or let the Network Access Resource use a Lease Pool.) What is going on here? We may send out DHCP request packets easily enough using iRules’ SIDEBAND functions, but it is difficult to collect DHCP replies using SIDEBAND. 4 Instead, we must set up a distinct LTM virtual server to receive DHCP replies on UDP port 67 at a fixed address. We tell the DHCP server(s) we are a DHCP relay device so replies will come back to us directly (no broadcasting). 5 For a nice explanation of the DHCP request process see http://technet.microsoft.com/en-us/library/cc940466.aspx. At this time, we support only IPv4, though adding IPv6 would require only toil, not genius. By default, a DHCP server will assign a client IP on the subnet where the DHCP relay device (that is, your APM-DHCP virtual server) is homed. For example, if your APM-DHCP virtual server’s address were 172.30.4.2/22 the DHCP server would typically lease out a client IP on subnet 172.30.4.0. Moreover, the DHCP server will communicate directly with the relay-device IP so appropriate routes must exist and firewall rules must permit. If you expect to assign client IP’s to APM tunnel endpoints on multiple subnets you may need multiple APM-DHCP virtual servers (one per subnet). Alternatively, some but not all DHCP servers 6 support the rfc3011 “subnet selection” or rfc3527 “subnet/link-selection sub-option” so you can request a client IP on a specified subnet using a single APM-DHCP virtual server (relay device) IP which is not homed on the target subnet but which can communicate easily with the DHCP server(s): see parameter session.dhcp.subnet below. NOTE: The subnet(s) on which APM Network Access (VPN) tunnels are homed need not exist on any actual VLAN so long as routes to any such subnet(s) lead to your APM (BIG-IP) device. Suppose you wish to support 1000 simultaneous VPN connections and most of your corporate subnets are /24’s—but you don’t want to set up four subnets for VPN users. You could define a virtual subnet—say, 172.30.4.0/22—tell your DHCP server(s) to assign addresses from 172.30.4.3 thru 172.30.7.254 to clients, put an APM-DHCP virtual server on 172.30.4.2, and so long as your Layer-3 network knows that your APM BIG-IP is the gateway to 172.30.4.0/22, you’re golden. When an APM Access Policy wants an IP address from DHCP, it will first set some parameters into APM session variables (especially the IP address(es) of one or more DHCP server(s)) using a Variable Assign Item, then use an iRule Event Item to invoke iRule Agent DHCP_req in ir apm policy dhcp. DHCP_req will send DHCPDISCOVERY packets to the specified DHCP server(s). The DHCP server(s) will reply to those packets via the APM-DHCP virtual-server, to which iRule ir apm dhcp must be attached. That iRule will finish the 4-packet DHCP handshake to lease an IP address. DHCP_req handles timeouts/retransmissions and copies the client IP address assigned by the DHCP server into APM session variables for the Access Policy to use. We use the APM Session-ID as the DHCP transaction-ID XID and also (by default) in the value of chaddr to avert collisions and facilitate log tracing. Parameters You Set In Your APM Access Policy Required Parameters session.dhcp.virtIP IP address of an APM-DHCP virtual-server (on UDP port 67) with iRule ir-apm-dhcp. This IP must be reachable from your DHCP server(s). A DHCP server will usually assign a client IP on the same subnet as this IP, though you may be able to override that by setting session.dhcp.subnet. You may create APM-DHCP virtual servers on different subnets, then set session.dhcp.virtIP in your Access Policy (or branch) to any one of them as a way to request a client IP on a particular subnet. No default. Examples (“Custom Expression” format): expr {"172.16.10.245"} or expr {"192.0.2.7%15"} session.dhcp.servers A TCL list of one or more IP addresses for DHCP servers (or DHCP relays, such as a nearby IP router). When requesting a client IP address, DHCP packets will be sent to every server on this list. NB: IP broadcast addresses like 10.0.7.255 may be specified but it is better to list specific servers (or relays). Default: none. Examples (“Custom Expression” format): expr {[list "10.0.5.20" "10.0.7.20"]} or expr {[list "172.30.1.20%5"]} Optional Parameters (including some DHCP Options) NOTE: when you leave a parameter undefined or empty, a suitable value from the APM session environment may be substituted (see details below). The defaults produce good results in most cases. Unless otherwise noted, set parameters as Text values. To exclude a parameter entirely set its Text value to '' [two ASCII single-quotes] (equivalent to Custom Expression return {''} ). White-space and single-quotes are trimmed from the ends of parameter values, so '' indicates a nil value. It is best to put “Machine Info” and “Windows Info” Items into your Access Policy ahead of your iRule Event “DHCP_req” Item (Windows Info is not available for Mac clients beginning at version 15.1.5 as they are no longer considered safe). session.dhcp.debug Set to 1 or “true” to log DHCP-processing details for the current APM session. Default: false. session.dhcp.firepass Leave this undefined or empty (or set to “false”) to use APM defaults (better in nearly all cases). Set to “true” to activate “Firepass mode” which alters the default values of several other options to make DHCP messages from this Access Policy resemble messages from the old F5 Firepass product. session.dhcp.copy2var Leave this undefined or empty (the default) and the client IP address from DHCP will be copied into the Access Policy session variable session.requested.clientip, thereby setting the Network Access (VPN) tunnel’s inside IP address. To override the default, name another session variable here or set this to (Text) '' to avert copying the IP address to any variable. session.dhcp.dns_na_list To set the client's DNS server(s) and/or DNS default search domain from DHCP, put here a Custom Expression TCL list of the name(s) of the Network Access Resource(s) you may assign to the client session. Default: none. Example: expr {[list "/Common/NA" "/Common/alt-NA"]} session.dhcp.broadcast Set to “true” to set the DHCP broadcast flag (you almost certainly should not use this). session.dhcp.vendor_class Option 60 A short string (32 characters max) identifying your VPN server. Default: “f5 APM”. Based on this value the DHCP server may send data to session.dhcp.vinfo.N (see below). session.dhcp.user_class Option 77 A Custom Expression TCL list of strings by which the DHCP server may recognize the class of the client device (e.g., “kiosk”). Default: none (do not put '' here). Example: expr {[list "mobile" "tablet"]} session.dhcp.client_ID Option 61 A unique identifier for the remote client device. Microsoft Windows DHCP servers expect a representation of the MAC address of the client's primary NIC. If left undefined or empty the primary MAC address discovered by the Access Policy Machine Info Item (if any) will be used. If no value is set and no Machine Info is available then no client_ID will be sent and the DHCP server will distinguish clients by APM-assigned ephemeral addresses (in session.dhcp.hwcode). If you supply a client_ID value you may specify a special code, a MAC address, a binary string, or a text string. Set the special code “NONE” (or '') to avoid sending any client_ID, whether Machine Info is available or not. Set the special code “XIDMAC” to send a unique MAC address for each APM VPN session—that will satisfy DHCP servers desiring client_ID‘s while averting IP collisions due to conflicting Machine Info MAC’s like Apple Mac Pro’s sometimes provide. A value containing twelve hexadecimal digits, possibly separated by hyphens or colons into six groups of two or by periods into three groups of four, will be encoded as a MAC address. Values consisting only of hexadecimal digits, of any length other than twelve hexits, will be encoded as a binary string. A value which contains chars other than [0-9A-Fa-f] and doesn't seem to be a MAC address will be encoded as a text string. You may enclose a text string in ASCII single-quotes (') to avert interpretation as hex/binary (the quotes are not part of the text value). On the wire, MAC-addresses and text-strings will be prefixed by type codes 0x01 and 0x00 respectively; if you specify a binary string (in hex format) you must include any needed codes. Default: client MAC from Machine Info, otherwise none. Example (Text value): “08-00-2b-2e-d8-5e”. session.dhcp.hostname Option 12 A hostname for the client. If left undefined or empty, the short computer name discovered by the APM Access Policy Windows Info Item (if any) will be used. session.dhcp.subscriber_ID Sub-option 6 of Option 82 An identifier for the VPN user. If undefined or empty, the value of APM session variable session.logon.last.username will be used (generally the user's UID or SAMAccountName). session.dhcp.circuit_ID Sub-option 1 of Option 82 An identifier for the “circuit” or network endpoint to which client connected. If left undefined or empty, the IP address of the (current) APM virtual server will be used. session.dhcp.remote_ID Sub-option 2 of Option 82 An identifier for the client's end of the connection. If left undefined or empty, the client’s IP address + port will be used. session.dhcp.subnet Option 118 Sub-option 5 of Option 82 The address (e.g., 172.16.99.0) of the IP subnet on which you desire a client address. With this option you may home session.dhcp.virtIP on another (more convenient) subnet. MS Windows Server 2016 added support for this but some other DHCP servers still lack support. Default: none. session.dhcp.hwcode Controls content of BOOTP htype, hlen, and chaddr fields. If left undefined or empty, a per-session value optimal in most situations will be used (asserting that chaddr, a copy of XID, identifies a “serial line”). If your DHCP server will not accept the default, you may set this to “MAC” and chaddr will be a locally-administered Ethernet MAC (embedding XID). When neither of those work you may force any value you wish by concatenating hexadecimal digits setting the value of htype (2 hexits) and chaddr (a string of 0–32 hexits). E.g., a 6-octet Ethernet address resembles “01400c2925ea88”. Most useful in the last case is the MAC address of session.dhcp.virtIP (i.e., a specific BIG-IP MAC) since broken DHCP servers may send Layer 2 packets directly to that address. Results of DHCP Request For Use In Access Policy session.dhcp.address <-- client IP address assigned by DHCP! session.dhcp.message session.dhcp.server, session.dhcp.relay session.dhcp.expires, session.dhcp.issued session.dhcp.lease, session.dhcp.rebind, session.dhcp.renew session.dhcp.vinfo.N session.dhcp.dns_servers, session.dhcp.dns_suffix session.dhcp.xid, session.dhcp.hex_client_id, session.dhcp.hwx If a DHCP request succeeds the client IP address appears in session.dhcp.address. If that is empty look in session.dhcp.message for an error message. The IP address of the DHCP server which issued (or refused) the client IP is in session.dhcp.server (if session.dhcp.relay differs then DHCP messages were relayed). Lease expiration time is in session.dhcp.expires. Variables session.dhcp.{lease, rebind, renew} indicate the duration of the address lease, plus the rebind and renew times, in seconds relative to the clock value in session.dhcp.issued (issued time). See session.dhcp.vinfo.N where N is tag number for Option 43 vendor-specific information. If the DHCP server sends client DNS server(s) and/or default search domain, those appear in session.dhcp.dns_servers and/or session.dhcp.dns_suffix. To assist in log analysis and debugging, session.dhcp.xid contains the XID code used in the DHCP request. The client_ID value (if any) sent to the DHCP server(s) is in session.dhcp.hex_client_id. The DHCP request’s htype and chaddr values (in hex) are concatenated in session.dhcp.hwx. Compatibility Tips and Troubleshooting Concern Response My custom parameter seems to be ignored. You should set most custom parameters as Text values (they may morph to Custom Expressions). My users with Apple Mac Pro’s sometimes get no DHCP IP or a conflicting one. A few Apple laptops sometimes give the Machine Info Item bogus MAC addresses. Set session.dhcp.client_ID to “XIDMAC“ to use unique per-session identifiers for clients. After a VPN session ends, I expect the very next session to reuse the same DHCP IP but that doesn’t happen. Many DHCP servers cycle through all the client IP’s available for one subnet before reusing any. Also, after a session ends APM-DHCP takes a few minutes to release its DHCP IP. When I test APM-DHCP with APM VE running on VMware Workstation, none of my sessions gets an IP from DHCP. VMware Workstation’s built-in DHCP server sends bogus DHCP packets. Use another DHCP server for testing (Linux dhcpd(8) is cheap and reliable). I use BIG-IP route domains and I notice that some of my VPN clients are getting duplicate DHCP IP addresses. Decorate the IP addresses of your APM-DHCP virtual servers, both in the iApp and in session.dhcp.virtIP, with their route-domain ID’s in “percent notation” like “192.0.2.5%3”. APM-DHCP is not working. Double-check your configuration. Look for errors in the LTM log. Set session.dhcp.debug to “true” before trying to start a VPN session, then examine DHCP debugging messages in the LTM log to see if you can figure out the problem. Even after looking at debugging messages in the log I still don’t know why APM-DHCP is not working. Run “tcpdump –ne -i 0.0 -s0 port 67” to see where the DHCP handshake fails. Are DISCOVER packets sent? Do any DHCP servers reply with OFFER packets? Is a REQUEST sent to accept an OFFER? Does the DHCP server ACK that REQUEST? If you see an OFFER but no REQUEST, check for bogus multicast MAC addresses in the OFFER packet. If no OFFER follows DISCOVER, what does the DHCP server’s log show? Is there a valid zone/lease-pool for you? Check the network path for routing errors, hostile firewall rules, or DHCP relay issues. Endnotes In DHCP Option 43 (rfc2132). In DHCP Options 6 and 15 (rfc2132). Prior to version v3h, under certain circumstances with some DHCP servers, address-release delays could cause two active sessions to get the same IP address. And even more difficult using [listen], for those of you in the back of the room. A bug in some versions of VMware Workstation’s DHCP server makes this solution appear to fail. The broken DHCP server sends messages to DHCP relays in unicast IP packets encapsulated in broadcast MAC frames. Anormal BIG-IP virtual server will not receive such packets. As of Winter 2017 the ISC, Cisco, and MS Windows Server 2016 DHCP servers support the subnet/link selection options but older Windows Server and Infoblox DHCP servers do not. Supporting Files - Download attached ZIP File Here.HTTP Brute Force Mitigation Playbook: BIG-IP LTM Mitigation Options for HTTP Brute Force Attacks - Chapter 3
HTTP Brute Force Attacks can be mitigated using BIG-IP LTM features. It could be a straightforward rejection of traffic from a specific source IP, network, geolocation, HTTP request properties or monitoring the number requests from a certain source and unique characteristic and rate limiting by dropping/rejecting requests exceeding a defined threshold. Prerequisites Managing the BIGIP configuration requires Administrator access. Ensure access to Configuration Utility (Web GUI) and SSH is available. These management interfaces will be helpful in configuring, verifying and troubleshooting on the BIGIP.Having access to serial console output of the BIGIP is also helpful. Local Traffic Manager (LTM) and Application Visibility and Reporting (AVR) license are required to use the related features. Prevent traffic from a Source IP or Network As demonstrated on the Data gathering chapter for iRules, LTM Policy and of F5 AVR, it is possible that a specific IP address or a specific network may be sending suspicious and malicious traffic. One of the common way to limit access to a HTTP Virtual Server is to either define a whitelist or blacklist of IP addresses. HTTP Brute Force Attacks on a Virtual Server can be mitigated by blocking a suspicious IP address or network. These can be done thru iRules, LTM Policy or Network Packet filter. Note that when blocking source IPs or networks, it is possible that the source IP is a proxy server and proxies request from internal clients and blocking it may have unintentional blocking of legitimate clients. Monitor traffic that are getting blocked and make necessary adjustments to the related configuration. The diagram below shows the packet processing path on a BIG-IP. Notice it also shows reference to Advance Firewall Manager (AFM) packet path. https://techdocs.f5.com/content/dam/f5/kb/global/solutions/K31591013_images.html/2018-0613%20AFM%20Packet%20Flow.jpg Mitigation: LTM Packet Filter On the left side of the BIG-IP packet processing path diagram, we can see the Ingress section and if the packet information is not in the Hardware Acceleration ePVA (Packet Velocity Asic) of the BIG-IP, it will be checked against the packet filter. Thus, after determining that an IP address or a Network is suspicious and/or malicious based on gathered data from either the LTM Policy/iRules or AVR or external monitoring tools, a packet filter can be created to block these suspected malicious traffic sources. Packet Filter(ing) can be enabled in the Configuration Utility, Network››Packet Filters : General Packet filter rules can be configured at Network››Packet Filters : Rules Sample Packet Filter Configuration: Packet filter configuration to block a specific IP address with the reject action and have logging enabled Packet filter configuration to block a Network with the reject action An Existing Packet filter rule Packet Filter generated logs can be reviewed at System››Logs : Packet Filter This log shows an IP address was rejected by a packet filter rule Mitigation: LTM Policy LTM Policy can be configured to block a specified IP address or from a iRule Datagroup. Sample LTM Policy: LTM policy tmsh output: Tip: TMOS shell (tmsh) command 'tmsh load sys config from-terminal merge' can be used to quickly load the configuration. Sample: root@(sec8)(cfg-sync Standalone)(Active)(/Common)(tmos)# load sys config from-terminal merge Enter configuration. Press CTRL-D to submit or CTRL-C to cancel. LTM policy will block specified IP address: ltm policy block_source_ip { last-modified 2019-02-20:22:55:25 requires { http tcp } rules { block_source_ip { actions { 0 { shutdown connection } 1 { log write facility local0 message "tcl:IP [IP::client_addr] is blocked by LTM Policy" priority info } } conditions { 0 { tcp address matches values { 172.16.7.31 } } } } } status published strategy first-match } LTM policy will block specified IP address in defined iRule Datagroup: root@(sec8)(cfg-sync Standalone)(Active)(/Common)(tmos)# list ltm policy block_source_ip ltm policy block_source_ip { last-modified 2019-12-01:14:40:57 requires { http tcp } rules { block_source_ip { actions { 0 { shutdown connection } 1 { log write facility local0 message "tcl:IP [IP::client_addr] is blocked by LTM Policy" priority info } } conditions { 0 { tcp address matches datagroup malicious_ip_dg } } } } status published strategy first-match } malicious_ip_dg is an iRule Datagroup where the IP address is defined Apply the LTM Policy to the Virtual Server that needs to be protected. Mitigation:iRule to block an IP address Using iRule to block an IP address can be done in different stages of the BIG-IP packet processing. The sample iRule will block the matched IP address during the FLOW_INIT event. FLOW_INIT definition: This event is triggered (once for TCP and unique UDP/IP flows) after packet filters, but before any AFM and TMM work occurs. https://clouddocs.f5.com/api/irules/FLOW_INIT.html Diagram Snippet from 2.1.9. iRules HTTPS Events. FLOW_INIT event happens after packet filter events. If an IP address is identified as malicious, blocking it earlier before further processing would save CPU resource as iRules processing are resource intensive. Additionally, if the blocking of an IP address can be done using LTM packet filter, or LTM policy, use it instead of iRules approach. https://f5-agility-labs-irules.readthedocs.io/en/latest/class1/module1/iRuleEventsFlowHTTPS.html Sample iRule: when FLOW_INIT { set ipaddr [IP::client_addr] if { [class match $clientip equals malicious_ip_dg] } { log local0. "Attacker IP [IP::client_addr] blocked" #logging can be removed/commented out if not required drop } } malicious_ip_dg is an iRule Datagroup where the IP address is defined Sample iRule is from K43383890: Blocking IP addresses using the IP geolocation database and iRules. there are more sample iRules in the referenced F5 Knowledge Article. https://support.f5.com/csp/article/K43383890 Apply the iRule to the Virtual Server that needs to be protected. Mitigation: Rate Limit based on IP address using iRules Common scenario during increase of connection when a suspected brute force attack on a Virtual Server with HTTP application is looking for options to rate limit connections to it. Using iRule to rate limit connection based IP address is possible. It also offers levels of control and additional logic should it be needed. Here is a sample iRule to Rate limit IP addresses. when RULE_INIT { # Default rate to limit requests set static::maxRate 15 # Default rate to set static::warnRate 12 # During this many seconds set static::timeout 1 } when CLIENT_ACCEPTED { # Increment and Get the current request count bucket set epoch [clock seconds] set currentCount [table incr -mustexist "Count_[IP::client_addr]_${epoch}"] if { $currentCount eq "" } then { # Initialize a new request count bucket table set "Count_[IP::client_addr]_${epoch}" 1 indef $static::timeout set currentCount 1 } # Actually check for being over limit if { $currentCount >= $static::maxRate } then { log local0. "ERROR: IP:[IP::client_addr] exceeded ${static::maxRate} requests per second. Rejecting request. Current requests: ${currentCount}." event disable all drop } elseif { $currentCount > $static::warnRate } then { log local0. "WARNING: IP:[IP::client_addr] exceeded ${static::warnRate} requests per second. Will reject at ${static::maxRate}. Current requests: ${currentCount}." } log local0. "IP:[IP::client_addr]: currentCount: ${currentCount}" } Attach the iRule to Virtual Server that needs to be protected. HTTP information from sample requests In the previous chapter "Bad Actor Behavior and Gathering Statistics using BIG-IP LTM Policies and iRules and BIG-IP AVR", some HTTP information will be available via AVR statistics and some may be gathered thru LTM policy or iRules where logs were generated when HTTP requests are received on a F5 Virtual Server which has the iRule or LTM Policy or the HTTP Analytics profile is applied to. These logs are typically logged in /var/log/ltm as normally configured in the irule "log local0." statements or in LTM policy, by default. In the course of troubleshooting and investigation, a customer/incident analyst may decide on what HTTP related information they will consider as malicious or undesirable. In the following sample iRule and LTM Policy mitigation, HTTP related elements were used. Typical HTTP information from the sample request that are used are the HTTP User-Agent header or a HTTP parameter. Other HTTP information can be used as well such as other HTTP headers. Mitigation: Prevent a specific HTTP header value During HTTP Brute Force attacks, HTTP header User-Agent value is often what an incident analyst will review and prevent traffic based on its value, where, a certain user-agent value will be used by automated bots that launches the attack. Sample Rule and LTM Policy to block a specific User-Agent root@(asm6)(cfg-sync Standalone)(Active)(/Common)(tmos)# list ltm policy Malicious_User_Agent ltm policy Malicious_User_Agent { last-modified 2019-12-04:17:30:38 requires { http } rules { block_UA { actions { 0 { shutdown connection } 1 { log write facility local0 message "tcl:the user agent [HTTP::header User-Agent] from [IP::client_addr] is blocked" priority info } } conditions { 0 { http-header name User-Agent values { "Mozilla/5.0 (A-malicious-UA)" } } } } } status published strategy first-match } logs generated by LTM Policy in /var/log/ltm Jan 16 13:11:06 sec8 info tmm3[11305]: [/Common/Malicious_User_Agent/block_UA]: the user agent Mozilla/5.0 (A-malicious-UA) from 172.16.10.31 is blocked Jan 16 13:11:06 sec8 info tmm5[11305]: [/Common/Malicious_User_Agent/block_UA]: the user agent Mozilla/5.0 (A-malicious-UA) from 172.16.10.31 is blocked Jan 16 13:11:06 sec8 info tmm7[11305]: [/Common/Malicious_User_Agent/block_UA]: the user agent Mozilla/5.0 (A-malicious-UA) from 172.16.10.31 is blocked Mitigation: Rate Limit a HTTP Header with a unique value During a HTTP Brute Force Attack, there may be instances in the attack traffic that a HTTP Header may have a certain value. If the HTTP Header value is being repeatedly used and appears to be an automated request, an iRule can be used to monitor the value of the HTTP header and be rate limited. Example: when HTTP_REQUEST { if { [HTTP::header exists ApplicationSpecificHTTPHeader] } { set DEBUG 0 set REQ_TIMEOUT 60 set MAX_REQ 3 ## set ASHH_ID [HTTP::header ApplicationSpecificHTTPHeader] set requestCnt [table lookup -notouch -subtable myTable $ASHH_ID] if { $requestCnt >= $MAX_REQ } { set remtime [table timeout -subtable myTable -remaining $ASHH_ID] if { $DEBUG > 0 } { log local0. "Dropped! wait for another $remtime seconds" } reject #this could also be changed to "drop" instead of "reject" to be more stealthy } elseif { $requestCnt == "" } { table set -subtable myTable [HTTP::header ApplicationSpecificHTTPHeader] 1 $REQ_TIMEOUT if { $DEBUG > 0 } { log local0. "Hit 1: Passed!" } } elseif { $requestCnt < $MAX_REQ } { table incr -notouch -subtable myTable [HTTP::header ApplicationSpecificHTTPHeader] if { $DEBUG > 0 } { log local0. "Hit [expr {$requestCnt + 1}]: Passed!" } } } } In this example iRule, the variable MAX_REQ has a value of 3 and means will limit the request from the HTTP Header -ApplicationSpecificHTTPHeader - with specific value to 3 requests. irule logs generated in /var/log/ltm Jan 16 13:04:39 sec8 info tmm3[11305]: Rule /Common/rate-limit-specific-http-header <HTTP_REQUEST>: Hit 1: Passed! Jan 16 13:04:39 sec8 info tmm5[11305]: Rule /Common/rate-limit-specific-http-header <HTTP_REQUEST>: Hit 2: Passed! Jan 16 13:04:39 sec8 info tmm7[11305]: Rule /Common/rate-limit-specific-http-header <HTTP_REQUEST>: Hit 3: Passed! Jan 16 13:04:39 sec8 info tmm6[11305]: Rule /Common/rate-limit-specific-http-header <HTTP_REQUEST>: Dropped! wait for another 60 seconds Jan 16 13:04:39 sec8 info tmm[11305]: Rule /Common/rate-limit-specific-http-header <HTTP_REQUEST>: Dropped! wait for another 60 seconds Jan 16 13:04:39 sec8 info tmm2[11305]: Rule /Common/rate-limit-specific-http-header <HTTP_REQUEST>: Dropped! wait for another 60 seconds Sample curl command to test the iRule. Notice the value of the ApplicationSpecificHTTPHeader HTTP header. for i in {1..50}; do curl http://172.16.8.86 -H "ApplicationSpecificHTTPHeader: couldbemaliciousvalue"; done Mitigation: Rate Limit a username parameter from HTTP payload Common HTTP Brute Force attack scenario involves credentials being tried repeatedly. In this sample iRule, the username parameter from a HTTP POST request payload can be observed for a HTTP login url and if the username is used multiple times and exceed the defined maximum requests in a defined time frame, the connection will be dropped. when RULE_INIT { # The max requests served within the timing interval per the static::timeout variable set static::maxReqs 4 # Timer Interval in seconds within which only static::maxReqs Requests are allowed. # (i.e: 10 req per 2 sec == 5 req per sec) # If this timer expires, it means that the limit was not reached for this interval and # the request counting starts over. Making this timeout large increases memory usage. # Making it too small negatively affects performance. set static::timeout 2 } when HTTP_REQUEST { if { ( [string tolower [HTTP::uri]] equals "/wackopicko/users/login.php" ) and ( [HTTP::method] equals "POST" ) } { HTTP::collect [HTTP::header Content-Length] } } when HTTP_REQUEST_DATA { set username "unknown" foreach x [split [string tolower [HTTP::payload]] "&"] { if { [string tolower $x] starts_with "token=" } { log local0. "login parameters are $x" set username [lindex [split $x "="] 1] set getcount [table lookup -notouch $username] if { $getcount equals "" } { table set $username "1" $static::timeout $static::timeout # Record of this session does not exist, starting new record # Request is allowed. } elseif { $getcount < $static::maxReqs } { log local0. "Request Count for $username is $getcount" table incr -notouch $username # record of this session exists but request is allowed. } elseif { $getcount >= $static::maxReqs } { drop log local0. "User $username exceeded login limit current count:$getcount from [IP::client_addr]:[TCP::client_port]" } else { #log local0. "User $username attempted login from [IP::client_addr]:[TCP::client_port]" } } } } logs generated in /var/log/ltm Jan 16 12:34:05 sec8 info tmm7[11305]: Rule /Common/post_request_username <HTTP_REQUEST_DATA>: login parameters are username=!@%23$%25 Jan 16 12:34:05 sec8 info tmm7[11305]: Rule /Common/post_request_username <HTTP_REQUEST_DATA>: User !@%23$%25 exceeded login limit current count:5 from 172.16.10.31:57128 Jan 16 12:34:05 sec8 info tmm1[11305]: Rule /Common/post_request_username <HTTP_REQUEST_DATA>: login parameters are username=!@%23$%25 Jan 16 12:34:05 sec8 info tmm1[11305]: Rule /Common/post_request_username <HTTP_REQUEST_DATA>: User !@%23$%25 exceeded login limit current count:5 from 172.16.10.31:57130 Additional reference: lindex - Retrieve an element from a list https://www.tcl.tk/man/tcl8.4/TclCmd/lindex.htm Prevent traffic source based on Behavior Mitigation: TLS Fingerprint In the reference Devcentral Article, https://devcentral.f5.com/s/articles/tls-fingerprinting-a-method-for-identifying-a-tls-client-without-decrypting-24598, it was demonstrated that clients using certain TLS fingerprints can be identified. In a HTTP brute force attack, attacking clients may have certain TLS fingerprint that can be observed and be later on, rate limited or dropped. TLS fingerprint can be gathered and used to manually or dynamically prevent malicious and suspicious clients coming from certain source IPs from accessing the iRule protected Virtual Server. The sample TLS Fingerprint Rate Limiting and TLS Fingerprint proc iRules (see HTTP Brute Force Mitigation: Appendix for sample iRule and other related configuration) works to identify, observe and block TLS fingerprints that are considered malicious based on the amount of traffic it sent. The TLS Fingerprinting Proc iRule extracts the TLS fingerprint from the client hello packet of the incoming client traffic which is unique for certain client devices. The TLS Fingerprint Rate Limiting iRule checks a TLS fingerprint if it is an expected TLS fingerprint or is considered malicious or is suspicious. The classification of an expected or malicious TLS fingerprint is done thru LTM rule Data Group. Example: Malicious TLS Fingerprint Data Group ltm data-group internal malicious_fingerprintdb { records { 0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102 { data curl-bot } } type string } In this example Malicious TLS Fingerprint Data Group, the defined fingerprint may be included manually as decided by a customer/analyst as the TLS fingerprint may have been observed to be sending abnormal amount of traffic during a HTTP brute force event. Expected / Good TLS Fingerprint Data Group ltm data-group external fingerprint_db { external-file-name fingerprint_db type string } System ›› File Management : Data Group File List ›› fingerprint_db Properties Namefingerprint_dbPartition / PathCommonData Group Name fingerprint_db TypeStringKey / Value Pair Separator:= sample TLS signature: signatures:#"0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102" := "User-Agent: curl-bot", Taking the scenario where a TLS fingerprint is defined in the malicious fingerprint data group, it will be actioned as defined in the TLS fingerprint Rate Limiting iRule. If a TLS fingerprint is neither malicious or expected, the TLS fingerprint Rate Limiting iRule will consider it suspicious and be rate limited should certain number of request is exceeded from this particular TLS fingerprint and IP address combination. Here are example logs generated by the TLS Fingerprint Rate Limiting iRule. Monitor the number of request sent from the suspicious TLS fingerprint and IP address combination. from the generated log, review the "currentCount" Dec 16 16:36:58 sec8 info tmm1[11545]: Rule /Common/fingerprintTLS-irule <CLIENT_DATA>: fingerprint:172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102: currentCount: 14 Dec 16 16:36:58 sec8 info tmm7[11545]: Rule /Common/fingerprintTLS-irule <CLIENT_DATA>: fingerprint:172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102: currentCount: 15 The HTTP User-Agent header value is included in the log to have a record of the TLS fingerprint and the HTTP User-Agent sending the suspicious traffic. This can later be used to define the suspicious TLS fingerprint and the HTTP User-Agent as a malicious fingerprint. Dec 16 16:36:58 sec8 info tmm1[11545]: Rule /Common/fingerprintTLS-irule <HTTP_REQUEST>: WARNING: suspicious_fingerprint: 172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102: User-Agent:curl/7.47.1 exceeded 12 requests per second. Will reject at 15. Current requests: 14. Dec 16 16:36:58 sec8 info tmm7[11545]: Rule /Common/fingerprintTLS-irule <HTTP_REQUEST>: WARNING: suspicious_fingerprint: 172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102: User-Agent:curl/7.47.1 exceeded 12 requests per second. Will reject at 15. Current requests: 15. The specific TLS fingerprint and IP combination is monitored and as it exceeds the defined request per second threshold in the TLS Fingerprint Rate Limiting iRule, further attempt to initiate a TLS handshake with the protected Virtual Server will fail. The iRule action in this instance is "drop". This will cause the connection to stall on the client side as the BIG-IP will not be sending any further traffic back to the suspicious client. Dec 16 16:36:58 sec8 info tmm1[11545]: Rule /Common/fingerprintTLS-irule <CLIENT_DATA>: ERROR: fingerprint:172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102 exceeded 15 requests per second. Rejecting request. Current requests: 16. Dec 16 16:36:58 sec8 info tmm1[11545]: Rule /Common/fingerprintTLS-irule <CLIENT_DATA>: fingerprint:172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102: currentCount: 16 Dec 16 16:40:04 sec8 warning tmm7[11545]: 01260013:4: SSL Handshake failed for TCP 172.16.7.31:24814 -> 172.16.8.84:443 Dec 16 16:40:04 sec8 info tmm7[11545]: Rule /Common/fingerprintTLS-irule <CLIENT_DATA>: ERROR: fingerprint:172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102 exceeded 15 requests per second. Rejecting request. Current requests: 17. Dec 16 16:40:04 sec8 info tmm7[11545]: Rule /Common/fingerprintTLS-irule <CLIENT_DATA>: fingerprint:172.16.7.31_0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102: currentCount: 17 Dec 16 16:40:04 sec8 warning tmm7[11545]: 01260013:4: SSL Handshake failed for TCP 172.16.7.31:35509 -> 172.16.8.84:443 If a TLS fingerprint is observed to be sending abnormal amount of traffic during a HTTP brute force event, this TLS fingerprint may be included manually as decided by a customer/analyst in the Malicious TLS Fingerprint Data Group. In our example, this is the malicious_fingerprintdb Data group. from the reference observed TLSfingerprint, an entry in the data group can be added. String: 0301+0303+0076+C030C02CC028C024C014C00A00A3009F006B006A0039003800880087C032C02EC02AC026C00FC005009D003D00350084C02FC02BC027C023C013C00900A2009E0067004000330032009A009900450044C031C02DC029C025C00EC004009C003C002F00960041C012C00800160013C00DC003000A00FF+1+00+000B000A000D000F3374+00190018001600170014001500120013000F00100011+060106020603050105020503040104020403030103020303020102020203+000102 Value: malicious-client Sample Data group in edit mode to add an entry: Mitigation: Prevent based on Geolocation It is possible during a HTTP Brute Force Attack that the source of the attack traffic is from a certain Geolocation. Attack traffic can be easily dropped from unexpected Geolocation thru an irule. The FLOW_INIT event is triggered when a packet initially hits a Virtual Server. be it UDP or TCP traffic.During an attack the source IP and geolocation information can be observed using the sample iRule and manually update the reference Data Group with country code where the attack traffic is sourcing from. Example: Unexpected Geolocation (Blacklist) iRule: when FLOW_INIT { set ipaddr [IP::client_addr] set clientip [whereis $ipaddr country] #logging can be removed/commented out if not required log local0. "Source IP $ipaddr from $clientip" if { [class match $clientip equals unexpected_geolocations] } { log local0. "Attacker IP detected $ipaddr from $clientip: Drop!" #logging can be removed/commented out if not required drop } } Data Group: root@(sec8)(cfg-sync Standalone)(Active)(/Common)(tmos)# list ltm data-group internal unexpected_geolocations ltm data-group internal unexpected_geolocations { records { KZ { data Kazakhstan } } type string } Generated log in /var/log/ltm: Dec 16 21:21:03 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Source IP 5.188.153.248 from KZ Dec 16 21:21:03 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Attacker IP detected 5.188.153.248 from KZ: Drop! Dec 16 21:21:04 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Source IP 5.188.153.248 from KZ Dec 16 21:21:04 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Attacker IP detected 5.188.153.248 from KZ: Drop! Dec 16 21:21:06 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Source IP 5.188.153.248 from KZ Dec 16 21:21:06 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Attacker IP detected 5.188.153.248 from KZ: Drop! Dec 16 21:21:11 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Source IP 5.188.153.248 from KZ Dec 16 21:21:11 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Attacker IP detected 5.188.153.248 from KZ: Drop! Dec 16 21:21:15 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Source IP 5.188.153.248 from KZ Dec 16 21:21:15 sec8 info tmm7[11545]: Rule /Common/block_unexpected_geolocation <FLOW_INIT>: Attacker IP detected 5.188.153.248 from KZ: Drop! Similarly, it is sometime easier to whitelist or allow only specific Geolocation to access the protected Virtual Server. Here is a sample iRule and its Data Group as a possible option. Expected Geolocation (Whitelist) iRule: when FLOW_INIT { set ipaddr [IP::client_addr] set clientip [whereis $ipaddr country] #logging can be removed/commented out if not required log local0. "Source IP $ipaddr from $clientip" if { not [class match $clientip equals expected_geolocations] } { log local0. "Attacker IP detected $ipaddr from $clientip: Drop!" #logging can be removed/commented out if not required drop } } Data Group: root@(sec8)(cfg-sync Standalone)(Active)(/Common)(tmos)# list ltm data-group internal expected_geolocations ltm data-group internal unexpected_geolocations { records { US { data US } } type string } sample curl command which will source the specified interface IP address [root@asm6:Active:Standalone] config # ip add | grep 5.188.153.248 inet 5.188.153.248/32 brd 5.188.153.248 scope global fop-lan [root@asm6:Active:Standalone] config # curl --interface 5.188.153.248 -k https://172.16.8.84 curl: (7) Failed to connect to 172.16.8.84 port 443: Connection refused Mitigation: Prevent based on IP Reputation IP Reputation can be used along with many features in the BIG-IP. IP reputation is enabled thru an add-on license and when licensed, the BIG-IP downloads an IP reputation database and is checked against the IP traffic, usually done during connection establishment and matches the IP's category . If a condition to block a category is set, depending on the BIG-IP feature being used, the connection can be dropped or TCP reset or even, return a HTTP custom response page. It is possible that IPs with bad reputation will send the attack traffic during a HTTP Brute Force attack and blocking these categorised bad IP will help in lessening the traffic that a website needs to process. Example: Using LTM Policy Using a LTM Policy, IP reputation can be checked and be TCP Reset if the IP matches a defined category [root@sec8:Active:Standalone] config # tmsh list ltm policy IP_reputation_bad ltm policy IP_reputation_bad { draft-copy Drafts/IP_reputation_bad last-modified 2019-12-17:15:08:52 rules { IP_reputation_bad_reset { actions { 0 { shutdown client-accepted connection } } conditions { 0 { iprep client-accepted values { BotNets "Windows Exploits" "Web Attacks" Proxy } } } } } status published strategy first-match } Verifying the connection was TCP Reset after the Three Way Handshake via tcpdump tcpdump -nni 0.0:nnn host 72.52.179.174 15:15:12.905996 IP 72.52.179.174.8500 > 172.16.8.84.443: Flags [S], seq 4061893880, win 29200, options [mss 1460,sackOK,TS val 313915334 ecr 0,nop,wscale 7], length 0 in slot1/tmm0 lis= flowtype=0 flowid=0 peerid=0 conflags=0 inslot=63 inport=23 haunit=0 priority=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0 15:15:12.906077 IP 172.16.8.84.443 > 72.52.179.174.8500: Flags [S.], seq 2839531704, ack 4061893881, win 14600, options [mss 1460,nop,wscale 0,sackOK,TS val 321071554 ecr 313915334], length 0 out slot1/tmm0 lis=/Common/vs-172.16.8.84 flowtype=64 flowid=56000151BD00 peerid=0 conflags=100200004000024 inslot=63 inport=23 haunit=1 priority=3 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0 15:15:12.907573 IP 72.52.179.174.8500 > 172.16.8.84.443: Flags [.], ack 1, win 229, options [nop,nop,TS val 313915335 ecr 321071554], length 0 in slot1/tmm0 lis=/Common/vs-172.16.8.84 flowtype=64 flowid=56000151BD00 peerid=0 conflags=100200004000024 inslot=63 inport=23 haunit=0 priority=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0 15:15:12.907674 IP 172.16.8.84.443 > 72.52.179.174.8500: Flags [R.], seq 1, ack 1, win 0, length 0 out slot1/tmm0 lis=/Common/vs-172.16.8.84 flowtype=64 flowid=56000151BD00 peerid=0 conflags=100200004808024 inslot=63 inport=23 haunit=1 priority=3 rst_cause="[0x273e3e7:998] reset by policy" peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0 Using an iRule Using an iRule, IP reputation can be checked and if the client IP matches a defined category, traffic can be dropped See reference article https://clouddocs.f5.com/api/irules/IP-reputation.html In this example iRule, if a source IP address matches any of IP reputation categories, it will be dropped. #Drop the packet at initial packet received if the client has a bad reputation when FLOW_INIT { # Check if the IP reputation list for the client IP is not 0 if {[llength [IP::reputation [IP::client_addr]]] != 0}{ log local0. "[IP::client_addr]: category: \"[IP::reputation [IP::client_addr]]\"" #remove/comment log if not needed # Drop the connection drop } } Generated log for blocked IP with bad reputation Dec 17 16:22:49 sec8 info tmm6[11427]: Rule /Common/ip_reputation_block <FLOW_INIT>: 72.52.179.174: category: "Proxy {Mobile Threats}" Final Thoughts on LTM based iRule and LTM Policy Mitigations The usage of iRule and LTM policy for mitigating HTTP Brute Force Attacks are great if there is only LTM module provisioned in the BIGIP and situation requires quick mitigation. iRules are community supported and are not officially supported by F5 Support. The sample iRules here are tested in a lab environment and will work based on lab scenario which are closely modeled on actual observed attacks. iRules are best configured and implemented by F5 Professional Services which works closely with customer and scope the functionality of the iRule as per customer requirement. Some of the mitigation can be done thru LTM Policy. LTM Policy is a native feature of BIGIP and unlike iRules, does not need "on the fly compilation", and thus will be faster and is the preferred configuration over iRules. LTM Policy configuration are straightforward while iRules can be complicated but also flexible and its advantage over LTM Policies. Rate limiting requests during HTTP Brute Force attack may be away to preserve some of the legal requests and using iRules, flexible approaches can be done. There are more advanced mitigation for HTTP Brute Force attacks using the Application Security Manager (ASM) Moduleand is preferred over iRules. Example in BIGIP version 14 for ASM, TLS Fingerprinting is a functionality included in the ASM Protection Profiles. TPS based mitigation can also be configured using the ASM protection profiles - example, if request from a source IP is exceeding the defined request threshold, it can be action-ed as configured - example, blocked or challenged using CAPTCHA. Using an ASM Security Policy, attacks such as Credential Stuffing can be mitigated using the Brute Force Protection configuration. Bots can also be categorized and be allowed or challenged or blocked using Bot Defense Profile and Bot Signatures.Advanced iRules: Tables
We’ve covered quite a bit of ground in the Getting Started with iRules and Intermediate iRules series. In this series, we’ll dive even deeper down the rabbit hole, starting with the table command. This article is a primer for the power of tables, but we actually have an entire 9 part series on the table command alone, so after reading this overview, I highly recommend you dig in to the meat of what tables can really do. What is a table? A table is somewhat similar to a data-group in that it is, at its core, a memory structure to contain lists. That’s about where the similarities end, though. Tables are stored in memory alone, which means they don’t have the config object backing that a data-group does, but that also makes them inherently more flexible. A table is a read/write memory structure that is maintained across connections, making it a solid contender for storing data that needs to be persistent, as opposed to connection based like normal iRules variables. Tables also allow a few extremely powerful concepts, namely sub tables and a built in timeout/lifetime structure. Both of these things add possibilities to what tables are capable of. Tables are also extremely efficient, much like data-groups, though on a slightly smaller scale given their more flexible nature. Tables make use of the session table within LTM, which is designed to handle every connection into or out of the device, and as such is very high performance. Table queries share this performance and can be a quick way to construct simple, persistent memory structures or more complex near DB like sets of sub-tables, depending on your needs. What are the benefits of a table? Tables are a high performance, highly scalable, persistent (non-connection bound), read/write data structure. That fact alone makes them unique within iRules and extremely powerful. There simply aren’t other things that fill that role. There are only a couple of ways to have read/write variable data across connections, and tables are by far the best option in almost every case. The ability to create what amounts to a mini database in memory from within your iRule is massively useful in many scenarios also. This is easy to do via subtables. You can create not just a flat list, but named lists to segregate data however you’d like. By doing so you can create relations and a relatively complex schema for data storage and accounting. Now add in the fact that there are configurable timeout and lifetime options, which means you’ll never have to worry about memory management or programmatic cleanup of a table, as things will time out as designed, and you’ve got another layer of usability and ease of use that is unique to tables. The bottom line is that tables are one of the more powerful, flexible features to hit iRules in quite a while. What command(s) would I use to access data in a table? To access tables, you make use of the table command, much like the class command to access data-groups. This command can also get quite complex quite fast, so I won’t attempt to cover all the possible permutations here. I’ll list a couple of simple examples, and give you a link to the full documentation. # Limit each client IP address to 20 concurrent connections when CLIENT_ACCEPTED { # Check if the subtable has over 20 entries if { [table keys -subtable connlimit:[IP::client_addr] -count] >= 20 } { reject } else { # Add the client IP:port to the client IP-specific subtable # with a max lifetime of 180 seconds table set -subtable connlimit:[IP::client_addr] [TCP::client_port] "" 180 } } when CLIENT_CLOSED { # When the client connection is closed, remove the table entry table delete -subtable connlimit:[IP::client_addr] [TCP::client_port] } As you can see tables are extremely powerful, and can add a powerful tool to your quiver for your future iRule endeavors. There are plenty of examples of using these powerful commands out on DevCentral, so there is no shortage of information to be found and code to be lifted for re-use if you scour Q&A and the Codeshare. In an attempt to keep this consumable I've not gone through the full list of command permutations or anywhere near the full possibilities. I'll leave discovering those things to the more advanced and/or eager iRulers, but suffice to say the possibilities are vast.DeviceID with APM and API Server
This article covers the use case to use APM and an external API Server to store DeviceID+. As you learnt in the previous articles in this series, DeviceID+ is a way to "identify" a user. And of course, it is a great solution to combine with F5 APM (Access Policy Manager) - https://www.f5.com/products/security/access-policy-manager The F5 Shape DeviceID+ solution does not store the IDs anywhere. It is up to you to store/use/consume them on your own. To do so, I propose to use an external API Server to store the DeviceID / Username peer. Steps: Built an API server based on loopback.io stack. You can download the project (docker-compose) here : https://github.com/MattDierick/DeviceID-api-server/blob/main/README.md Build an APM Pre Request Policy with HTTP Connector to query the API server (available also on my Github) Created the Postman collection (available also on my Github) First of all, watch the demo video to understand the use case and the outcomes: https://www.youtube.com/watch?v=PVYwh76nGVE The API Server: The API server has a Swagger UI installed so you can "watch" the API (go to /explorer) Structure of the API Below is the structure of the API payload. As you can see, we store the username with the associated DeviceIDs { "username": "dierick", "deviceid": [ "Ac31A2AAAAAA12uwoekcLffhgfjgjghj", "AUq7Xti2Eowm6yVog4CYvt6nRrookqgW" ] } The REST calls used by APM 1. GET DeviceID per username GET /api/deviceids/findOne?filter={"where":{"username":"dierick"}} 2. POST a new DeviceID POST /api/deviceids HTTP/1.1 { "username": "dierick", "deviceid": [ "Ac31A2AAAAAA12uwoekcLffhgfjgjghj", "AUq7Xti2Eowm6yVog4CYvt6nRrookqgW" ] } 3. UPDATE DeviceID per username POST /api/deviceids/upsertWithWhere?where={"username": "dierick"} HTTP/1.1 { "deviceid": [ "Rtdsflkj534565465kenfter" ] } The iRule to collect the DeviceID from the Shape cookie when ACCESS_SESSION_STARTED { log local0. "Access Session Started" if [HTTP::cookie exists _imp_apg_r_] { set deviceid [URI::decode [HTTP::cookie _imp_apg_r_]] log local0. "URL Decoded cookie is $deviceid" set deviceida [lindex [regexp -inline -- (?:"diA":")(.*?)(?:") $deviceid] 1] log local0. "diA = $deviceida" set deviceidb [lindex [regexp -inline -- (?:"diB":")(.*?)(?:") $deviceid] 1] log local0. "diB = $deviceidb" log local0. "IP is [IP::client_addr]" log local0. "Path os [HTTP::path]" ACCESS::session data set session.logon.deviceid $deviceid ACCESS::session data set session.logon.deviceid.A $deviceida ACCESS::session data set session.logon.deviceid.B $deviceidb } else { log local0. "No cookie" } } The APM Per Request Policy Below an example using the APM Per Request Policy. A Per Request Policy is required instead of a Per Session Policy in order to use the HTTP Connector. HTTP Connector is a REST API client. Create the different CALLS, and assign each call in the right branch (new DeviceID, update DeviceID ...)
