F5 monitor for Tomcat/Pega

Hi All -

Wanted to share a problem we had with a new install of Tomcat (v9.0.71) running Pega (not sure of the version). We are transitioning from WebSphere as middleware because it's getting incredibly expensive.

The old HTTPS monitor we were using failed on the new Tomcat instance with an HTTP Status 403, saying "The server understood the request but refuses to authorize it." What's interesting is that the exact URI would work if used in curl or a browser but it would fail in the same manner as the monitor if I used openssl s_client to make the query.

After a lot of testing and contacting both F5 and Pega (F5 wasn't aware of any issues; Pega support was, frankly, not helpful at all and kept telling us that since it's not a cloud install we're on our own) - my appdev team created an HTTP instance which made it easier to see what was happening (the HTTP version of the monitor conveniently failed in the same way as the HTTPS monitor).

Using tcpdump, I looked at the difference between the monitor and a curl request and it came down to the HTTP Fields - Host, User-Agent, and Accept. When I added these all to the monitor (with \r\n between them) the monitor worked. I then started removing them one by one to see where the monitor would break again and found that the one that was necessary in this case that wasn't needed previously is the Host: field. I set it to the FQDN of the VS (Host: pegadev.my.domain) and it works. 

Normally, I don't use Host field in my HTTP(S) monitors because that allows me to use the same monitor across different environments and it makes it easier to maintain. I believe that it's the new version of Tomcat that requires the field because the 403 indicated that the source was Tomcat and not Pega. I saw another post here that said that the User-Agent field was needed by Tomcat but that's not the case here. I am not a middleware person and have no visibility into the configuration of Tomcat so it may also be the way that Tomcat is configured on our systems (RHEL v8.x, if that matters), but in my case, I needed the Host: field for the monitor to work correctly.

I also found that I needed the HTTP/1.1 at the end of the URI (separated by a space) to make it work. While I usually have it in my monitors, I don't think it was required previously, either.

Anyway - hope this helps someone. My Monitor is "GET /prweb/PRHTTPService/HAPFWCPMHCFWInt2/Services/PingResponse HTTP/1.1\r\nHost: pegadev.hap.org\r\nConnection:Close\r\n\r\n" but I think the PingResponse may be custom code my AD folks wrote when I refused to use an ICMP or plain vanilla HTTPS "GET /" monitor, so if you have Pega your mileage may vary.