Forum Discussion

DrewW's avatar
DrewW
Icon for Nimbostratus rankNimbostratus
May 12, 2020

Header name with no header value

Hi,

We had to create a new App Security policy because technical support couldn't tell us why our old one wasn't doing anything after several days of working with them.

I am seeing some events for a rule that says: 'Header name with no header value'

Basically the request looks like this:

Cache-Control: max-age=88544
Connection: keep-alive
Accept: text/css,*/*;q=0.1
From: 
User-Agent: AdsBot-Google (+http://www.google.com/adsbot.html)

I'm not entirely certain how this poses a risk and it seems like it's blocking Google from crawling our website which makes it suboptimal. Are there a list of things that F5 thinks are security issues that just break your website that you have to disable?

ASM Advanced WAF
security
  • Erik_Novak's avatar
    Erik_Novak
    Icon for Employee rankEmployee
    May 12, 2020

    In terms of RFC2616 compliance, the empty From: header in your example is probably harmless, but in some cases headers with empty values can cause errors in some parsers. That is why it triggers a violation. You can turn off the block flag for the violation "Header name with no header value" if you determine it is causing a false positive. You have control over the blocking action for every single violation on the Learning and Blocking Settings page. According to RFC, the From request-header field, if given, SHOULD contain an Internet e-mail address for the human user who controls the requesting user agent. The address SHOULD be machine-usable, as defined by "mailbox" in RFC 822 [9] as updated by RFC 1123. Again, probably not malicious but informative about the clients that are accessing your app.

    • DrewW's avatar
      DrewW
      Icon for Nimbostratus rankNimbostratus
      May 13, 2020
      So does Google's crawler actually send it with an empty From: or not? Any clue? It could just be another scraper saying that its Google.
  • Erik_Novak's avatar
    Erik_Novak
    Icon for Employee rankEmployee
    May 14, 2020

    Without some forensic data, it is hard to say based on that single example. The User-Agent string looks legit, but is easily spoofed. I am not an expert on Google's bots, but sending an empty header like that is certainly atypical from what we would consider normal browsing behavior. You could try implementing a bot defense profile, and then allow bots at your discretion. Bot defense will challenge all bots for which you don't specify an exception and prevent them from scraping your application.

    • DrewW's avatar
      DrewW
      Icon for Nimbostratus rankNimbostratus
      May 14, 2020
      Unfortunately even though we pay F5 a huge sum of money we dont have bot defense.