HTTP header name with underscore cannot be masked in AWAF logging

[tatmotiv]

Hi all,

I'm currently facing the situation that I need to mask the value of several http headers in AWAF logging. I have setup both headers in identical manner within the application security policy, both with "Mask Value in Logs" option enabled:

 

Now, when I issue an http request like that with both headers being present...

...one of those headers will get masked/obfuscated in the logging, the other will not:

I suppose this is due to the underscore in the http header name. I am aware that the use of underscores in http header names is discouraged and considered deprecated, but nevertheless these are present out there and there has to be a solution to this. 

TMOS version is 15.1.5

Has anybody experienced a similar situation and knows how to circumvent this? Any help is appreciated.

Many thanks in advance,
Martin

[Daniel_Wolf]

Hi @tatmotiv,

👍 for the nickname. Works for me in 16.1.

 

Following settings:

Just checking the obvious... Did you apply the policy after making those changes.
General hint, stay away from headers with underscores. They are uncommon and some web servers don't play well with them.

KR
Daniel

[tatmotiv]

Hi Daniel - nice to see you again!
(we met before - you gave an AWAF training at my company last year, remember?)

Of course, I applied the policy with all of those settings being active, so that's not the problem. I did some additional tests in the meantime with quotation marks after colleagues pointed me here, but things got even weirder then. Now, I have included two headers with underscores im my request (for testing purposes).

One can be masked in logging, but only when I configure the header name BOTH with AND without quotation marks (either of this alone would not work).

The other cannot be masked at all in any constellation (quoted only, unquoted only, both quoted/unquoted).

I presume a bug here and probably will open a support case for this.

[Daniel_Wolf]

Hi tatmotiv,

yes, I do remember the training. Nice to see you here too. Remember I told you on the training - if you ask an AWAF question here, chances are not too bad you will get an answer promptly. 🙂 

Odly, this works also on my 15.1.5 box.

I couldn't find a known bug either for this behaviour. 
Can you share an example of your working config with the double quotes?

KR
Daniel

[tatmotiv]

Hi Daniel,

I did some more testing and created a detailled protocol in order to file a support case with f5. I will share the protocol here so you can see it, too (does not contain any sensitive data).

Spoiler alert: it's getting really weird...

Problem description

Assume we need to mask two http headers in AWAF logging (the real-world headers are different to this, but most importantly, one of them contains an underscore):

• my_claims
• mytoken2

Both headers were added to the security policy as custom headers as non-mandatory with “Check Attack Signatures and Threat Campaigns” disabled and “Mask Value in Logs” enabled – see exemplary screenshot below:

…resulting in the following list of headers for this policy (policy was applied afterwards):

Now, when issuing a request containing both of those headers like this (from postman)…

…one of them (mytoken2) will be masked, the other (my_claims) will not, as can be seen in the following screenshot:

Troubleshooting steps taken

After being pointed to https://community.f5.com/t5/technical-forum/http-header-insert-cannot-insert-header-with-an-undersco... I tried to add the non-working header (my_claims) with quotes as follows (and applied the policy):

Now, this header is listed as BOTH, quoted AND unquoted:

After issuing the same testing request as above in postman, the header “my_claims” is now being masked in the logging as shown below:

Next step: I removed the unquoted version of this header from the list, because it obviously did not work. Now, the list of headers looks as follows (again, the policy was applied afterwards):

After re-sending the same testing request with postman, the header is now no longer masked:

Conclusion: Obviously we need both, a quoted and an unquoted version of this header configured in the policy in order to have the masking work properly (I presume this is a bug).

Further testing (now it’s getting even weirder)

I restored the unquoted version of the my_claims header (the one I removed before) and added another header (aaaaa_claims) BOTH quoted AND unquoted (because that constellation worked for my_claims), resulting in the following list of custom headers (all configured equally and again, being applied to the policy):

Then, I added the aaaaa_claims header to the request in postman and send this to the AWAF:

This time, the newly added header (aaaaa_claims) is not being masked, despite being present in the configuration both quoted and unquoted:

Next test: I added another header (bbbb_claims – notice that this time, there are fewer characters in front of the underscore) to the request and to the policy (both quoted and unquoted).

Result: This one is being masked in the logging again (same as my_claims).

Next test: I removed the quoted version of bbbb_claims from the policy, resulting in the following list of custom headers:

Test result: Surprise – bbbb_claims is still being masked in the logging (!!??):

[Daniel_Wolf]

Hi Martin,
those are really interesting results you collected there. Please, keep us posted about the resolution of the case.
KR
Daniel