Restricting number of concurrent TLS handshakes using Max Active Handshakes on BIG-IP

Introduction

This is the maximum number of handshakes that BIG-IP can process concurrently per TMM.

Max Active Handshakes can be used if there's an application specific reason to limit the number of concurrent TLS handshakes. It's typically used for performance or security reasons, i.e. when we know our application would only handle a specific number of concurrent TLS handshakes in a given time.

BIG-IP allows us to restrict the number of concurrent TLS handshakes per TMM using Max Active Handshakes option on both Client SSL and Server SSL profiles:

Do not confuse this setting with number of concurrent TLS connections. It is possible to have a much higher number of concurrent TLS connections than the number specified in Max Active Handshakes. We're literally talking about ongoing concurrent TLS handshakes where TLS Finished message has not been exchanged yet.

Client SSL

I've set max-active-handshakes to 1, which means that BIG-IP can only process up to 1 TLS handshake at a time.

From my client machine (10.199.3.135), I used the following command to test this feature and trigger 3 concurrent TLS handshakes:

Note: (10.199.3.101 is the Virtual Server)

Notice that there are 3 connections here and we can distinguish them by their source TCP port. First one (37062) goes through and BIG-IP (10.199.3.101) responds correctly to Client Hello (frame #11) with Server Hello (frame #14):

However, the 2 subsequent Client Hello messages sent via connection 2 (37063) and connection 3 (37066) are terminated with a TSL Fatal Alert as the maximum number of allowed handshakes configured had been reached.

Also notice that the Info column on Wireshark shows that all 3 connections were received by the same tmm (tmm0).

Note: TMM is BIG-IP's forwarding plane daemon and may have several instances depending on how many CPU cores exist in a given box.

In a subsequent test,  I sent out 2 requests via 2 different interfaces concurrently to make sure another tmm was picked:

Handshake completes successfully now even though max-active-handshakes is still set to 1, because each tmm (tmm0 and tmm1) processed one a handshake which confirms max-active-handshakes counter is per tmm:

Server SSL

I have also set max-active-handshakes to 1 here and issued the following command from my client machine:

Note: I have changed Virtual Server to 10.199.3.145 from 10.199.3.101.

Here I am filtering server-side flow only to show that only the first flow where 1st handshake was already ongoing was allowed to proceed and the 2nd handshake was not allowed:

If we also include the Client-side, we see that the reset (on client-side) was because of server-side reset and reset cause is Policy action:

I have also confirmed that when max-active-handshakes set to 1, if a 2nd concurrent server-side handshake is triggered via a different TMM, then it goes through but I did not add the test here for brevity as it's the same test I performed on client-side.

However, this also confirms that just like client-side, server-side setting works per TMM.