Automatic Certificate Management with ACMEv2 in F5 BIG-IP

[mendes]

One of the most anticipated features of F5 BIG-IP is integration with ACMEv2.

With the General Availability of BIG-IP 21.1.0 on May/26, this feature came into being.

In this tutorial, we are going to configure it, using Let’s Encrypt as the CA. The domain for which we are generating/renewing certificates is carlosf5lab.lat.

The official docs for this feature are located in SSL Certificate Management | BIG-IP Documentation.

Pre-requisite 1: DNS Resolver that can reach the internet (at least the CA endpoints).

In this case, we are using the native DNS Resolver that comes with BIG-IP.

Pre-requisite 2: The internal proxy that will make the connection with the CA.

Pre-requisite 3: a self signed SSL certificate that the ACMEv2 protocol uses as the identifier for a device account. You don’t have to fill the Subject Alternative Name. For the Common Name, an e-mail contact is advised.

Now, we are going to create the ACME Provider object. Give it a name, and select the internal proxy previously created. For the CA Certificate to enable the secure connection with the Directory URL, you can use the default ca-bundle.crt.

The Directory URL is the endpoint for the ACMEv2 protocol. In Let’s Encrypt case, it is https://acme-v02.api.letsencrypt.org/directory

For the Account Key, choose the previously created self-signed certificate. For the trickier part of all, the field “Contacts” is mandatory, and it must be an URL. That’s why you must use the format mailto:email_address. Check the Terms and Conditions, and the Create Account boxes.

After a while, the Account Status must read as “Valid”.

To prove you own the domain whose certificate Let’s Encrypt is going to create/renew, it must be pointing to an IP (A Record) where you must have your Virtual Server listening on Port 80 configured to respond to the ACMEv2 Challenge. (In this specific lab, the domain carlosf5lab.lat points to a Public IP mapped to an internal IP).

Now you can order your first certificate via ACMEv2 on BIG-IP:

After a while, the Key tab should read something like:

Which means your certificate was generated:

To track the ACME Provider, you can check its statistics:

That’s it, my friend! If it helped you, give a thumbs up to this post!

[markus1634]

HI Mendes,

thanks for your step by step guide!

Just one question. We would like to redirect non ACME challenge requests to HTTPS.

in the official documentation SSL Certificate Management | BIG-IP Documentation there is an example iRule.

Unfortunatelly this iRule breaks my ACME flow. Without the iRule it is working fine.

Any suggestions why?

Best,
Markus

[jcgutierrez]

Hi there Mendes:

This is great, we tried to do it using BIG-IQ, but having the option on the F5 is nice.

Stupid question, how do I assign the new cert to a VIP, can we automate the binding too?

Thanks in advance

[markus1634]

This is the regular process via the SSL client profile. Just include the cert, key and chain there. They will update automatically in the background.

[mendes]

One suggestion, yes. In ACME, if using the http-01 challenge type, the server attempts to verify your domain ownership by fetching a specific file from a well-known public URI:

• URI Path: http://<your-domain>/.well-known/acme-challenge/<TOKEN>

The following condition checks if the request URI is not exactly equal to “/.well-known/acme-challenge.”

if { [HTTP::uri] ne “/.well-known/acme-challenge” }

The ACME challenge typically includes a token or additional subpaths after the main directory (e.g., /.well-known/acme-challenge/uTF8…). If the request contains anything after the slash, the ne (not equal) operator will fail and the request will be redirected, which may cause certificate validation to fail. Changing to starts_with ensures that any subdirectories within the challenge are allowed.

if { !($[HTTP::uri] starts_with “/.well-known/acme-challenge”) }

I haven’t tested this in my lab yet, but I believe this might be the reason. If so, I’ll discuss it with the team so they can update the documentation.

[markus1634]

Thanks, I can confirm your modification.

The iRule I successfully tested is the following:

when HTTP_REQUEST {
    if { !([HTTP::uri] starts_with "/.well-known/acme-challenge") } {
        HTTP::redirect https://[getfield [HTTP::host] ":" 1][HTTP::uri]
    }
}

Would be highly appreciated if the official docs would be ajusted.

Best, Markus

[Falkenbe]

Hi Mendes,

Thanks for the great tutorial! Will a BIG-IP automatically renew certificates issued by an ACME provider, or do I have to click renew every 30 days (or more likely script the renewal process)?

[mendes]

Done! The docs have been adjusted 😄

[Chung_Yu]

Hi Mendes

How does this all tie in operationally if we have many sites deployed via AS3?  This seems ok if you are using /Common partition but AS3 is in it’s own partitions.

Regards

[omermi]

Hi,

Could anyone clarify the key rotation interval for the ACME provider?

I currently have a certificate issued with a 31 day validity period and would like to know when the automatic renewal process is triggered.

Also, how does it handle ultra short certificates valid for just a week?

[Hook]

Hi @mendes ,
I’m trying really hard to make it work, but I keep failing.
I’ve followed your instructions carefully, but at the ACME Provider object creation, when I save it I get a “Error” at Account Status. Looking at the Local Traffic Logs I have:

err keymgmtd[6558]: 01a40000:3: Failed to get ACME directory listing from https://acme-v02.api.letsencrypt.org/directory. curl_code=56,response_code=0, error=Failure when receiving data from the peer

err keymgmtd[6558]: 01a40000:3: Cannot take account action as directory listing is invalid

(Copilot was not helpful in finding a solution )

[Marvin]

When are you planning to support DNS challenge type by either perform DNS TXT update to DNS server and/or via API call for example to cloud based DNS server like cloudflare or akamai etcetera? We see that a lot at customers and also supports wildcards.

[markus1634]

Hi, same thing here? When are the going to renew themselfs? Do we need to do anything @mendes ?

[Stephen_Archer_1]

If you want to route directly out to your `ACME Provider` (not using a proxy), you still require an `Internal Proxy` be configured/referenced in the ACME Provider, however just make sure to configure a `DNS Resolver` and leave `Use Proxy Server` unchecked.  In this case, keymgmtd sends GET/POST requests via BIG-IP internal-proxy listener:39443 (without CONNECT method), which resolves the name to an IP routes the request using the TMM routing table (note: NOT the mgmt interface).

In my case, I didn’t have any TMM routes configured and received the same error as you.  I added a route via my `external` TMM interface next-hop and it resolved the problem.

One other piece of advice is that you can enable/disable debug logging to /var/log/ltm with:

set debug = `tmsh modify sys db log.keymgmtd.level value debug`

unset debug = `tmsh modify sys db log.keymgmtd.level value notice`

[Stephen_Archer_1]

Auto-renewal starts 14 days before expiry with tiered retries that never give up.

[Hook]

Hi Stephen,
In the meantime, I opened a support ticket, and interestingly, your reply arrived on the same day as the proposed solution from the Support Team: TMM needs routing information.

We initially tried adding a route only to the public DNS server, but that was not sufficient. We then added a route to the Let’s Encrypt public IP serving the production ACME URL. While this worked, such a configuration is not a viable long-term solution.

The final recommendation from Support was simply to add a 0.0.0.0/0 default route pointing to the gateway behind the F5-Self-IP. This resolved the issue.

[shane_e]

So, from what I am reading, does each 443 vip needing the ACME renewal on our F5, need to have a http vip with the same IP as the 443 vip?

[Marvin]

Exactly that is how acme letsencrypt works regardless of the native integration in version 21.1.0 or using the acme script custom installer. The validation from acme to the F5 is proof that you own the domain and is done by an http call to /.well-known… and the challenge is responded within a dynamically updated datagroup value. Normally with Acme you are also able to change the validation type from HTTP to DNS which updates (either by DNS protocol or API call for example when using public DNS in cloud CDN) the external DNS zone with a TXT record containing the challenge string which in turn is validated by ACME. So i hope F5 would support the other DNS types as well natively but for now forced to keep using the custom script which works fine with no issues for years at multiple installations regardless of the validation type.

[infrastructure_Clark]

Any Planned support for DNZ Challenge so we can do Wildcard cert?