HTTPS Monitor: JSON File behind login page

kgaigl · ‎19-Jan-2021

Hello,

we've already a https monitor that reads the contents of a json file, it looks like this:

send string: GET /intern/api/v1/health/portal-status HTTP/1.1\r\nHost: host.company.com\r\n\r\n
receive strimg: \"portalStatus\":\"AVAILABLE\"

this works fine.

but now I need a similar monitor, but the json file is passwort protectet, when I add username/password to the monitor, it doesn't work.

when I do a curl:

curl -sku monitor:<pw> -H "Content-Type: application/json" -X GET https://host.company.int/monitor/wga/widgets/health.json

I get the logon-page for the username/pw

I tried some other parameter for curl, but I don't get the content of the json-file.

Any ideas?

thanks in advance

Daniel_Wolf · ‎20-Jan-2021

Hello,

curl typically (depending on how it was built) supports several authentication methods.

Basic, digest, NTLM, ...

Does the application server support the method you are using?

Maybe a tcpdump would show you why you retrieve the logon page instead of being authenticated, also the application server logs might give a clue.

KR

Daniel

jaikumar_f5 · ‎20-Jan-2021

Can you open the webserver & check the htaccess file to know what type of authentication is configured. If its a simple BASIC authorization, you can check the https monitor behavior via openssl itself.

Also you can still turn on the monitor logging & see what happens for every monitor interval. Check the monitor log file in /var/log/monitors/.

Also I have read in an article that double CR/LF could cause issues on authentication headers.

So in your send string, do not give the closing double CR/LF & see how it behaves.

Refer this article - https://support.f5.com/csp/article/K54561004 - Quoted the point.

Unlike other HTTP monitor send strings, an NTLM or Basic Authentication monitor should not include a closing double CR/LF sequence. When NTLM/NTLMv2 is enabled by adding a Username and Password to the monitor configuration, the BIG-IP system appends the Authorization header and a double CR/LF sequence after the last character in the Send string.

Please keep us posted.

kgaigl · ‎21-Jan-2021

thank's for your answers!

first: it's basic authentication.

with tcpdump I've the problem with encryption.

with monitor logging, I can't see the corresponding Log-file

the part with double CR/LF seems interesting, but I think I've got a twisted pool/monitor config (wrong infos from developer)

Do you know IBM Webseal? That's the Application.

additional Info:

with

curl -sku monitor:password -H "Content-Type: application/json" -X GET https://isam-test-company.int/monitor/wga/widgets/health.json

I get:

{
    "operation" : "login"
}

jaikumar_f5 · ‎21-Jan-2021

Haven't got a chance to work with that.

But in your original post, i see few differences. In your send string, the host header is "host.company.com", whereas in your curl you are sending "host.company.int", are you positive that its .com or .int. Because this is important, most servers have site binding or certain whitelist of domains.

Hope this wasn't a blunder mistake.

Reg the encryption, assuming this is a net new setup, can you have the server team run the service on http instead of https for time being, so we can troubleshoot, else you'll have to take the long route of bring the servers private key inside the box & then do a dump.

Basically you should be seeing a 401 coming from the server on 1st request, followed by F5 sending the credentials.

Or as twisted way, see if you can put 401 as your recv string to see if its working or not...

By any chance can you paste your https monitor config & the device version.

Daniel_Wolf · ‎21-Jan-2021

For my backend I am using nginx with SSL and Basic auth. The following curl works:

[root@awaf:Active:Standalone] config # curl -k https://10.100.153.50/api/status.json --user user1:P@$$w0rd
{
    "status": "up and running"
}

And the following monitor will mark the backend as green

[root@awaf:Active:Standalone] config # tmsh list ltm monitor https mon_api_json_https
ltm monitor https mon_api_json_https {
    adaptive disabled
    defaults-from https
    interval 5
    ip-dscp 0
    password ********
    recv "HTTP/1.(0|1) 200.*up and running"
    recv-disable "HTTP/1.(0|1) 200.*maintenance"
    send "GET /api/status.json HTTP/1.1\r\nHost: localhost\r\nConnection: Close\r\n\r\n"
    time-until-up 0
    timeout 16
    username user1
}

kgaigl · ‎21-Jan-2021

thanks, but when I try your syntax, a get a lot of html-code, which generates the login-page from the IBM Security Access Manager (the Backend)

seems like this is special to the Backend

Daniel_Wolf · ‎21-Jan-2021

In that case it would be really helpful to see the log of the backend server and compare a successful with an unsuccessful attempt. Maybe IBM does a bit "more" than standard Basic Auth.

kgaigl · ‎28-Jan-2021

I got one step beyond:

now I can see with curl the content of the json-file and the string I need:

"health":"0","label":"dev-daisy-40001.bvk.int:443"

but the monitor:

ltm monitor https VIP-DEV-HEALTH {
    adaptive disabled
    defaults-from https
    destination *.https
    interval 5
    ip-dscp 0
    password $pw
    recv "\"health\“:\“0“,\“label\“:\"dev-daisy-40001.bvk.int:443\""
    recv-disable none
    send "GET /monitor/wga/widgets/health.json HTTP/1.1\r\nHost: dev-cpmpany.bvk.int\r\nConnection: Close\r\n\r\n"
    time-until-up 0
    timeout 16
    username monitorxxx

won't work

in the receive-string I think I need the backslashes because of the qoutation-marks

kgaigl · ‎28-Jan-2021

I forgot: Version is 14.1.3.1

Daniel_Wolf · ‎30-Jan-2021

I successfully tested with this monitor, assuming that "label":"dev-daisy-40001.bvk.int:443" is not relevant for the health status but the key health is either 0 or 1.

See K5917: Using regular expressions in a health monitor receive string

ltm monitor https mon_api_json_https {
    adaptive disabled
    defaults-from https
    interval 5
    ip-dscp 0
    password ********
    recv (health.*0)
    recv-disable (health.*1)
    send "GET /api/status.json HTTP/1.1\r\nHost: localhost\r\nConnection: Close\r\n\r\n"
    time-until-up 0
    timeout 16
    username user1
}

Best of luck.

kgaigl · ‎31-Jan-2021

now I got it.

the problem was the "Host" statement.

when I use the FQDN of the VS as target for curl, I don't get an answer, so the send-string looks like this:

recv "\"health\":\"0\",\"label\":\"dev-daisy-40001.bvk.int:443\""
send "GET /monitor/wga/widgets/health.json HTTP/1.0\r\n\r\n"

and this works.

but I don't know why the the App-Server doesn't like the GET with the adress of the VS, there's no ASM or so

Daniel_Wolf · ‎01-Feb-2021

There is one other difference between your initial post

GET /intern/api/v1/health/portal-status HTTP/1.1\r\nHost: host.company.com\r\n\r\n

and your current variant

GET /monitor/wga/widgets/health.json HTTP/1.0\r\n\r\n

You changed from HTTP1.1 to HTTP1.0. The host is optional in HTTP1.0.

Does your backend support HTTP1.1?

kgaigl · ‎01-Feb-2021

I've read an article, that http1.1 needs the host-statement, and http1.0 not, so I tried http1.0 without host-statement and that's working.

I'm not sure if the backend supports 1.1, maybe I ask the devs, but with this I'm happy

the first post was from another application, sorry for the confusion, we're confused by the devs too 😉

jaikumar_f5 · ‎01-Feb-2021

Grrrhhhh 😉 I told to check long back on the host header. I don't think it's an issue with http1.0 or http1.1, but it could be the host header. Some web has site binding, so it requires proper dns name, else all requests goto 403 or some other homepage.

DevCentral

HTTPS Monitor: JSON File behind login page

Khoros Kudos Award 2022