Automate Let's Encrypt Certificates on BIG-IP
To quote the evil emperor Zurg: "We meet again, for the last time!" It's hard to believe it's been six years since my first rodeo with Let's Encrypt and BIG-IP, but (uncompromised) timestamps don't lie. And maybe this won't be my last look at Let's Encrypt, but it will likely be the last time I do so as a standalone effort, which I'll come back to at the end of this article. The first project was a compilation of shell scripts and python scripts and config files and well, this is no different. But it's all updated to meet the acme protocol version requirements for Let's Encrypt. Here's a quick table to connect all the dots: Description What's Out What's In acme client letsencrypt.sh dehydrated python library f5-common-python bigrest BIG-IP functionality creating the SSL profile utilizing an iRule for the HTTP challenge The f5-common-python library has not been maintained or enhanced for at least a year now, and I have an affinity for the good work Leo did with bigrest and I enjoy using it. I opted not to carry the SSL profile configuration forward because that functionality is more app-specific than the certificates themselves. And finally, whereas my initial project used the DNS challenge with the name.com API, in this proof of concept I chose to use an iRule on the BIG-IP to serve the challenge for Let's Encrypt to perform validation against. Whereas my solution is new, the way Let's Encrypt works has not changed, so I've carried forward the process from my previous article that I've now archived. I'll defer to their how it works page for details, but basically the steps are: Define a list of domains you want to secure Your client reaches out to the Let’s Encrypt servers to initiate a challenge for those domains. The servers will issue an http or dns challenge based on your request You need to place a file on your web server or a txt record in the dns zone file with that challenge information The servers will validate your challenge information and notify you You will clean up your challenge files or txt records The servers will issue the certificate and certificate chain to you You now have the key, cert, and chain, and can deploy to your web servers or in our case, to the BIG-IP Before kicking off a validation and generation event, the client registers your account based on your settings in the config file. The files in this project are as follows: /etc/dehydrated/config # Dehydrated configuration file /etc/dehydrated/domains.txt # Domains to sign and generate certs for /etc/dehydrated/dehydrated # acme client /etc/dehydrated/challenge.irule # iRule configured and deployed to BIG-IP by the hook script /etc/dehydrated/hook_script.py # Python script called by dehydrated for special steps in the cert generation process # Environment Variables export F5_HOST=x.x.x.x export F5_USER=admin export F5_PASS=admin You add your domains to the domains.txt file (more work likely if signing a lot of domains, I tested the one I have access to). The dehydrated client, of course is required, and then the hook script that dehydrated interacts with to deploy challenges and certificates. I aptly named that hook_script.py. For my hook, I'm deploying a challenge iRule to be applied only during the challenge; it is modified each time specific to the challenge supplied from the Let's Encrypt service and is cleaned up after the challenge is tested. And finally, there are a few environment variables I set so the information is not in text files. You could also move these into a credential vault. So to recap, you first register your client, then you can kick off a challenge to generate and deploy certificates. On the client side, it looks like this: ./dehydrated --register --accept-terms ./dehydrated -c Now, for testing, make sure you use the Let's Encrypt staging service instead of production. And since I want to force action every request while testing, I run the second command a little differently: ./dehydrated -c --force --force-validation Depicted graphically, here are the moving parts for the http challenge issued by Let's Encrypt at the request of the dehydrated client, deployed to the F5 BIG-IP, and validated by the Let's Encrypt servers. The Let's Encrypt servers then generate and return certs to the dehydrated client, which then, via the hook script, deploys the certs and keys to the F5 BIG-IP to complete the process. And here's the output of the dehydrated client and hook script in action from the CLI: # ./dehydrated -c --force --force-validation # INFO: Using main config file /etc/dehydrated/config Processing example.com + Checking expire date of existing cert... + Valid till Jun 20 02:03:26 2022 GMT (Longer than 30 days). Ignoring because renew was forced! + Signing domains... + Generating private key... + Generating signing request... + Requesting new certificate order from CA... + Received 1 authorizations URLs from the CA + Handling authorization for example.com + A valid authorization has been found but will be ignored + 1 pending challenge(s) + Deploying challenge tokens... + (hook) Deploying Challenge + (hook) Challenge rule added to virtual. + Responding to challenge for example.com authorization... + Challenge is valid! + Cleaning challenge tokens... + (hook) Cleaning Challenge + (hook) Challenge rule removed from virtual. + Requesting certificate... + Checking certificate... + Done! + Creating fullchain.pem... + (hook) Deploying Certs + (hook) Existing Cert/Key updated in transaction. + Done! This results in a deployed certificate/key pair on the F5 BIG-IP, and is modified in a transaction for future updates. This proof of concept is on github in the f5devcentral org if you'd like to take a look. Before closing, however, I'd like to mention a couple things: This is an update to an existing solution from years ago. It works, but probably isn't the best way to automate today if you're just getting started and have already started pursuing a more modern approach to automation. A better path would be something like Ansible. On that note, there are several solutions you can take a look at, posted below in resources. Resources https://github.com/EquateTechnologies/dehydrated-bigip-ansible https://github.com/f5devcentral/ansible-bigip-letsencrypt-http01 https://github.com/s-archer/acme-ansible-f5 https://github.com/s-archer/terraform-modular/tree/master/lets_encrypt_module (Terraform instead of Ansible) https://community.f5.com/t5/technical-forum/let-s-encrypt-with-cloudflare-dns-and-f5-rest-api/m-p/292943 (Similar solution to mine, only slightly more robust with OCSP stapling, the DNS instead of HTTP challenge, and with bash instead of python)
tcl logic in SAML Attribute value field possible?
Hi. We're running BigIP as a SAML IDP. Can I somehow issue tcl logic in a SAML attributes? I'm talking about the Access ›› Federation : SAML Identity Provider : Local IdP Services, editing an object, under SAML Attributes. Based on what's in the memberOf attribute, I need to issue as a value either empty string or "SpecificValue". I am familiar with the %{session.variable} construct, but I don't want to clutter the session with more variables if I can avoid it, as that impacts all sessions using our IDP (30 or so federated services on the same VIP and AP). I tried these two approches: %{ set result {} ; if { [mcget {session.ad.last.attr.memberOf}] contains {| CN=SpecificGroup,OU=Resource groups,OU=Groups,DC=Domain,DC=com |}} { set result {SpecificValue} } ; return $result } expr { set result {} ; if { [mcget {session.ad.last.attr.memberOf}] contains {| CN=SpecificGroup,OU=Resource groups,OU=Groups,DC=Domain,DC=com |}} { set result {SpecificValue} } ; return $result } Expected result: An issued claim with the value "" or "SpecificValue" Actual result: An issued claim with the above code as the value As I mentioned, we've set it up using one VIP that is hosting 30 or so services. We're running 16.1.3.1. They are using the same SSO configuration and there's an iRule triggerd at ACCESS_POLICY_AGENT_EVENT, which does some magic to extract issuer and suchlike, and that helps to make decisions later in the Access Policy. It also populates a few session variables under the session.custom namespace for use in the Access Policy. Additional session variables are being populated in the Access Policy, such as resolved manager and their email address. I have looked briefly at the ASSERT::saml functions, but even if it would be possible to manipulate that way, I wish to keep this set up as stream lined as possible and with as few new "special cases" in an iRule. So while I appreciate pointers along that route as well, I would first of all like to know if there is a way to do it natively in the SAML attribute value field. And if there are any options I have not yet explored here?
Is it possible to select ASM BoT profile from irule?
Hi. . Is it possible to select BoT profile from irule? . Concept is we have different set of IP which need to allow "some" BoT type. That why we can't use whitelist IP in BoT profile because it will allow all BoT type. So We want to use iRule to check if it IP A > use BoT profile which have some exception, but if all other IP > use normally BoT profile. . when HTTP_REQUEST { # Check IP and select BoT profile from that if { [IP::client_addr] eq "A" } { ASM::enable allow_some_bot_profile } else { ASM::enable normally_bot_profile } } ps. I didn't see any document about how to select BoT profile. So I'm not sure if ASM::enable can do that.Best approach to serve maintenance page
Hi, We need to put website under maintenance for about 6 hours. Traffic flow: Clients -->Akamai--->F5-->Backend servers. We have maintenance page hosted in AWS cloud Front. which approach is better? DNS Change – Temporarily point our domain (via CNAME) to CloudFront by adjusting the TTL to 2 minutes. F5 Configuration – Issue a 302 redirect from F5 to CloudFront or forward (reverse proxy) traffic from F5 to CloudFront by modifying the Host header. This keeps the browser on our domain and returns a 200 OK. Main concerns : Avoiding browser/edge caching issues (we can clear Akamai cache if needed). Ensuring a quick rollback after maintenance. Which approach would be best? Could you advise on the correct implementation?Trigger js challenge/Captcha for ip reputation/ip intelligence categories
Problem solved by this Code Snippet Because some ISP or cloud providers do not monitor their users a lot of times client ip addresses are marked as "spam sources" or "windows exploits" and as the ip addresses are dynamic and after time a legitimate user can use this ip addresses the categories are often stopped in the IP intelligence profile or under the ASM/AWAF policy. This usually happens in Public Clouds that do not monitor what their users do and the IP gets marked as bad then another good user after a day or two has this ip address and this causes the issue. For many of my clients I had to stop the ip reputation/ip intelligence category "spam sources" and in some cases "windows exploits" so having a javascript/captcha checks seems a nice compromise 😎 To still make use of this categories the users coming from those ip addresses can be forced to solve captcha checks or at least to be checked for javascript support! How to use this Code Snippet Have AWAF/ASM and ip intelligence licensed Add AWAF/ASM policy with irule support option (by default not enabled under the policy) or/and Bot profile under the Virtual server Optionally add IP intelligence profile or enable the Ip intelligence under the WAF policy without the categories that cause a lot of false positives, Add the irule and if needed modify the categories for which it triggers Do not forget to first create the data group, used in the code or delete that part of the code and to uncomment the Bot part of the code, if you plan to do js check and not captcha and maybe comment the captcha part ! Code Snippet Meta Information Version: 17.1.3 Coding Language: TCL Code You can find the code and further documentation in my GitHub repository: reputation-javascript-captcha-challlenge/ at main · Nikoolayy1/reputation-javascript-captcha-challlenge when HTTP_REQUEST { # Take the ip address for ip reputation/intelligence check from the XFF header if it comes from the whitelisted source ip addresses in data group "client_ip_class" if { [HTTP::header exists "X-Forwarded-For"] && [class match [IP::client_addr] equals "/Common/client_ip_class"] } { set trueIP [HTTP::header "X-Forwarded-For"] } else { set trueIP [IP::client_addr] } # Check if IP reputation is triggered and it is containing "Spam Sources" if { ([llength [IP::reputation $trueIP]] != 0) && ([IP::reputation $trueIP] contains "Spam Sources") }{ log local0. "The category is [IP::reputation $trueIP] from [IP::client_addr]" # Set the variable 1 or bulean true as to trigger ASM captcha or bot defense javascript set js_ch 1 } else { set js_ch 0 } # Custom response page just for testing if there is no real backend origin server for testing if {!$js_ch} { HTTP::respond 200 content { <html> <head> <title>Apology Page</title> </head> <body> We are sorry, but the site you are looking for is temporarily out of service<br> If you feel you have reached this page in error, please try again. </body> </html> } } } # when BOTDEFENSE_ACTION { # Trigger bot defense action javascript check for Spam Sources # if {$js_ch && (not ([BOTDEFENSE::reason] starts_with "passed browser challenge")) && ([BOTDEFENSE::action] eq "allow") }{ # BOTDEFENSE::action browser_challenge # } # } when ASM_REQUEST_DONE { # Trigger ASM captcha check only for users comming from Spam sources that have not already passed the captcha check (don't have the captcha cookie) if {$js_ch && [ASM::captcha_status] ne "correct"} { set res [ASM::captcha] if {$res ne "ok"} { log local0. "Cannot send captcha_challenge: \"$res\"" } } } Extra References: BOTDEFENSE::action ASM::captcha ASM::captcha_statusJson parsing with iRules
JSON is now the format of choice for most APIs. It's time we were able to parse JSON with F5 iRules too, as simple string matching is not always good enough. That's why I wrote a simple JSON parser for iRules. It is a validating single pass parser that processes the JSON string char by char until the JsonPath expression matches, no recursion or any other fancy stuff. As I do not wanted to reinvent the wheel, it is basically a rewrite of the JSON parser found in the mongoose webserver project in plain TCL. The usage is very simple: set token [call json::json_get_tok { $json $path }] $json is the json string to parse $path is a JsonPath expression, following operators are implemented: Operator Description $ The root element to query. This starts all path expressions. .<name> Dot-notated child. [<number>] Array index. Example Simple JSON: { "aud": "audience \"test\"", "iss": "https://issuer.de/issuer/", "iat": 1701422123, "roles": [ "role1", "role2" ], "obj": { "sub": "adcad2b8", }, "ver": "2.0" } JsonPath expression to parse this simple JSON: JsonPath Return value $.aud "audience \"test\"" $.iat 1701422123 $.obj.sub "adcad2b8" $.roles[0] "role1" To decode the extracted JSON string: set decoded [call json::json_decode_str { $token }] This removes the enclosing quotes from a string and decodes JSON escapes. Code You can find the code and further documentation in my GitHub repository: https://github.com/JuergenMang/f5-irules-json
