F5 Distributed Cloud - Service Policy - Header Matching Logic & Processing

Introduction

Who knows what an iRule is?  iRules have been used by F5 BIG-IP customers for a quarter of a century!  One of the most common use cases for iRules are for security decisions. If you're not coming from a BIG-IP and iRules background, what if I told you that you could apply 1000s of combinations of L4-L7 match criteria in order to take action on specific traffic?  This is what a Service Policy provides, similar to iRules. The ability to match traffic and allow, deny, flag, or tune application security policy based on that match. I often am asked, "Can F5 Distributed Cloud block ____ the same way I do with iRules?", and most commonly the answer is, absolutely, with a Service Policy. 

Story Time

Recently, I had a customer come to me with a challenge for blocking a specific attack based on a combination of headers. This is a common application security practice, specifically for L7 DDoS attacks, or even Account Take Over (ATO) attempts via Credential Stuffing/Brute Force Login. While F5 Distributed Cloud's Bot Defense or Malicious Users feature sets might be more dynamic tools in the toolbox for these attacks, a Service Policy is great for taking quick action. It is critical that you clearly identify the match criteria to ensure your service policy will not block good traffic.

Service Policy Logic

As stated earlier, the attack was identified by a specific combination of headers and values of these headers. The specific headers looked something like below (taken from my test environment and curl tests):

curl -I --location --request GET 'https://host2.domain.com' \
--header 'User-Agent: GoogleMobile-9.1.76' \
--header 'Content-Type: application/json; charset=UTF-8' \
--header 'Accept-Encoding: gzip, deflate, br' \
--header 'partner-name: GOOGLE' \
--header 'Referer: https://host.domain.com/'

The combination of these headers all had to be present, meaning, we needed an "and" logic for matching the headers and their values. Seems simple, but this is where the conversation between the customer and myself came into play. When applying all the headers to match as shown below, they were not matching. Can you guess why?

Figure A: Headers - FlatFigure A: Headers - Flat

The first thought that comes to mind, is probably, case sensitivity in the values. However, if we take a closer look specifically at the 'partner-name' header configuration, I've placed a transformation on this specific header. So the 'partner-name' isn't the problem.

Figure B: A transformer is applied to the request traffic attribute values before evaluating for match.Figure B: A transformer is applied to the request traffic attribute values before evaluating for match.

Give up?  The issue in this Service Policy configuration is the 'Accept-Encoding' header. Specifically the ',' {comma} character in the value. In the F5 Distributed Cloud Service Policy feature, we treat commas as separate headers with each individual value. The reason for this, is a request can have the same header multiple times, or it can have multiple values in a single header. In order to keep things consistent when parsing headers with comma delimited values, we separate them into multiple headers before matching.

I thought I could be smart when initially testing this and added multiple values to a single header. This will not match, for one because they are not separate headers with values, but also because there are multiple values within a single header.  Having multiple values in a single header within the service policy creates an "or" logic but we are looking for an "and" logic for all headers and their exact values. 

Figure C: Multiple Values in Single HeaderFigure C: Multiple Values in Single Header  

Figure D : Multiple Values within a Single Header - "or" Logic for this headerFigure D : Multiple Values within a Single Header - "or" Logic for this header

To get the proper match with "and" logic across all headers, and the header values, we need to apply the same header name multiple times.

NOTE: The 'content-type header' has a ';' {semi-colon} which is not a delimited value in F5 Distributed Cloud service policy logic and will match just fine the way it is in the defined policy shown below.

Figure E: Multiple Headers defined, with individual values, will provide "and" logic for all headers, and their values.Figure E: Multiple Headers defined, with individual values, will provide "and" logic for all headers, and their values.

Testing

In these tests, I am going to first provide an exact match to block the traffic. When we match, we provide a 403-response code back to the client. Within the individual Load Balancers objects of F5 Distributed Cloud, you can customize the messaging that comes along with the 403-response code or any response code for that matter. For my tests, I will simply use curl and update the different headers. After this initial successful block, I will show a few examples of changing the headers sent with the curl.
* For the "and" logic, any changes to the headers should result in a 200-response code.
* For the "or" logic, the response will depend on how I change the headers.

"and" logic

In this testing section, the service policy is configured like Figure E above.

All values are an exact match, with and logic, the 403 response code identifies the block from the F5 Distributed Cloud
All values are an exact match, with "and" logic, the 403-response code identifies the block from the F5 Distributed Cloud

When removing the 'g' character from gzip, the "and" logic no longer matches, as not every value is exact. This results in a 200 response code being from the origin server and F5 Distributed Cloud.When removing the 'g' character from gzip, the "and" logic no longer matches, not every value is exact. This results in a 200-response code being from the origin server and F5 Distributed Cloud.