Scaling BIG-IP With ECMP and RedHat Ansible
| A long time ago, in a business unit far, |
| far away... |
I had a customer that wanted to flow - literally - 4 million qps of DNS.. per Point of Presence (PoP)! Now, that's a lot of DNS, first of all. Second thing to note, though.. they HAD TO do it on VE. It's funny.. a couple years ago, this sort of statement may have been somewhat provocative, but today, it's becoming more and more the norm. Virtualization was supposed to be the fixer for so many of the issues we've had in our networks for decades, but getting from point A to point B introduces SO many headaches for our traditional networks.
Imagining a VE pair of DNS boxes, I'm thinking we are absolutely capped at 1M qps (4 x 250K qps licenses per VE), but, being new to the Service Provider SE team at the time, I was not aware of how to use ECMP for clustering. I came to understand that this is an architecture that has found itself in many mobility and backbone networks through the years and I truly feel that this clustering mechanism is VERY appropriate for BIG-IP in the age of Kubernetes and CI/CD pipelines.
The idea is simple: An ECMP (Equal Cost Multi-Path) enabled router - most commonly via BGP (Border Gateway Protocol) Multipath - will take a packet flow, establish a 5-tuple hash for source and destination ports and IPs and send that packet to a BIG-IP, which establishes a Persistence Profile for the flow to map the packet to a Pool Member. We can complete a TCP handshake.. we can even handle SSL. If we use Ansible to maintain our configs, this makes our BIG-IP clusters VERY resilient. We fail gracefully (for shorter-lived connections), we upgrade easily via nuke and pave methodology. We have ONE config, maintained centrally, that can be used to spit out instances, as-needed. If you're using central licensing, this is a very powerful way to autoscale app needs with F5.
You can find ECMP Clustering documentation here: https://techdocs.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/bigip-system-ecmp-mirrored-clustering-12-1-0.html
Also, who's got two thumbs and a new lightboard lesson for you on this topic? This guy!
Testing with my customer found astronomical numbers using High Performance VE at speeds of nearly 4 million Packets Per Second - PER VE ! - and scaling 4 VEs with maximum performance tesed on IXIA of 16 million PPS.. Per PoP, anycast.
In horizontal scale applications for F5 security products, the performance and ease of this architecture is a real treat. I hope it's helpful.