Forum Discussion
Nikoolayy1
MVP
MVPCan the F5 APM API Protection profile use for a DNS resolver a local F5 DNS/GTM listener VS?
Hello,
Can the F5 APM API Protection profile use for a DNS resolver a local F5 DNS/GTM listener VS?
My question comes from that that I have APM and the DNS modules on the same F5 APM device and the F5 API Protection seems to not work but the logs show that that the API Protection profile per request policy is allowing my traffic so I am starting to think that the provided DNS resolver that is using an IP address that is a DNS lisener on the same box is the issue as the connection times out.
I did a tcpdump so the F5 never tries to contact the virtual servers or real servers that are used as API Protection servers attached to the paths and real servers are layer 2 connected and VS servers are on the F5 itself so it is not network or other connection issue as F5 just does not try to reach them at all so if it is not Per Request Policy or Network/firewall/connection issue then I am suspecting DNS issue as in the API Protection profile you need to specify URL/URI and can't use an IP address for the servers so this is why the DNS resolver should resolve the DNS FQDN names.
I think that the issue could be the same as like when I do a dig command from the F5 device and select the lisener as a DNS server I get the error "reply from an unexpected source 127.0.0.1" as the F5 device is basically asking itself for a DNS resolution and maybe the F5 DNS Resolver is having a similar issue. With a dig command from the F5 device I can specify 127.0.0.1 and I get the Wide-IP or zone runner DNS records but the DNS resolver does not accept 127.0.0.1 as a name server.
https://support.f5.com/csp/article/K12140128
I finally got to this (I upgraded to 16.1.3.2 from 16.1.3 so this could also be important) and I also tested with the pet store that was given as an example in https://support.f5.com/csp/article/K12744365?utm_source=f5support&utm_medium=RSS . With external DNS resolver (in my case google 8.8.8.8) it works like a charm and it seems to me that when referensing external or internal DNS resolvable API then the server mode is best used and when referensing a local pool of API servers then the pool mode is best used. My picture below is with the petstore servers without having a pool attached to the virtual server.
Now I have other issues as after intergrating with Azure AD as oauth provider and a sync of the public keys that are at https://login.microsoftonline.com/common/discovery/keys I see the F5 APM getting the keys every hour and they are correct but when I request a token with postman and get a correct token that I have checked at https://jwt.io/ the APM things that there is no valid key that signed the token but this seems like a Job for the support hah as the key is in the APM after I checked.
Nov 8 18:16:15 bigip1.com err apmd[12601]: 01490290:3: /Common/API-PETS_ap::77A02581./Common/oauth/HYQYY5JU7+XxiUmSSuLWgsiAtUEug9m2c9njSNvseiQ=:/Common/oauth_act_oauth_scope_subsession_ag: OAuth Scope: failed for jwt-provider-list '/Common/exampleJWTProvider' , error: None of the configured JWK keys match the received JWT token, JWT Header: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSJ9
I even dumped the token with a subsession variable just to see if the APM is getting the correct one with variable assign agent in the Per-Request API protection policy " subsession.logon.last.jwt = expr { "[mcget {request.header.authorization}]"} " and the logging agent " %{subsession.logon.last.jwt} " and it is the the same one in Postman and valid so enough of the support was done by me 🙂
I am closing this and as a recommendation an external DNS resolver going to an IP address not on the same F5 device needs to be configured. I may create an iRule and community Tech Article with TCP::collect and TCP::respond to see if this can be bypassed but that is in the future.
Edit:
I even checked when the web browser clients are using authorization codes and F5 exchanges them with Azure AD for tokens and validates them it works normally without an issue ( with https://support.f5.com/csp/article/K07645403 the tokens the F5 per-request policy gets under "subsession.oauth.client.last.access_token" can be decrypted but a logging agent is used as the per-request policy does not have message agent ) but when tokens are send directly to the F5 with postman for API protection well it seems a bug to me.
- Daniel_Wolf
Zdravo Niki,
I am pretty sure this won't work. API Protection is using a combination of HTTP methods and URIs to build a so called "path". Once a client is accessing this path, the APM will respond (for example with HTTP status 429 Too many requests). However, it's all HTTP.
DNS Requests are... not HTTP. Hence API Protection won't do anything to them.
DoH might be a different story....
Pozdrav
Daniel- Nikoolayy1
Hello Daniel_Wolf I agree that this is not for DNS but HTTP traffic as I am not trying to do API Protection for a DNS traffic but web traffic but I think that the DNS resolver in the API protection still needs to do the DNS resolution on the server objects that are configured and related to each URI/Path before sending the HTTP/HTTPS traffic to the servers and as I am using the same F5 box for DNS resolution where the F5 APM API protection is located I think I am hitting a wall with this rare case where the DNS server in the DNS resolver is not an external DNS server but a local DNS lisener.
I could be wrong as I just started playing with API Protection but as the DNS resolver is a requered object in the API Protection profile and for the server objects you can't add directly just the server IP address but it should something like http://x.x.x.x/ this is why I think the DNS resolution just does not work when trying to make the DNS call as F5 will reply to itself using the loopback and this be the issue.
I think I saw similar issue in a Advanced WAF Bot defense profile long ago for a DNS resolver having an IP address of a DNS virtual server/lisener as the configured DNS server on the same F5 box where the WAF was located.
- Nikoolayy1
I created a WIDE IP and now when I do a dig from the F5 device for resolution I get a DNS resolution as it seems that in this case the loopback does not reply to the DNS requests and it works for a dig command from the f5 device itself but for the DNS resolver using Wide-IP and not Bind/Zoneruner it still did not work. In the APM debug logs I match an Allow action for the per request policy and everything seems to be configured correctly but I see no HTTP/REST-API traffic send to the backend server from the F5 device and the connection is reset when I connect to the F5 VS where the API Prorection profile is attached.
I did a tcpdump on the lisener ip and port 53 to see the DNS Network resolver DNS traffic and I see that the DNS Network resolver seems to get the reply from the loopback address even for a Wide-IP FQDN, so it seems there is no workaround for DNS Network resolver to use a local DNS Lisener VS as a DNS server.
The issue could be something else but I may try in the future to use an external DNS server for the F5 DNS Network resolver that the F5 API Protection uses as to exclude it as the issue
192.168.1.55.domain > tmm0.48074: [bad udp cksum 0x4238 -> 0xbfdf!] 39544*- q: A? WIdEIP1.xxxxx. 1/0/1 WIdEIP1.xxxxxx. [30s] A 10.1.1.33 ar: . OPT UDPsize=4096 DO (61) out slot1/tmm0 lis=/Common/DNS1 port=loopback trunk=port=loopback trunk=
By the way Daniel_Wolf in your example picture you are using a default server object and this is why you do not have an attached servers under the API Protection Paths right?
- Daniel_Wolf
Hi Nikoolayy1,
in my case I am indeed not using a Default Server, I use the pool assigned to the virtual. However, the DNS Resolver would be used to resolve the names of the API server(s), if I had any.
So you are saying in your setup the API Protection profile cannot use the Virtual IP configured on the same box for DNS resolution?
KR
Daniel
- Nikoolayy1
As a note even on 16.1.3.5 the issue is still present.
This could be related to the issue that under oauth provider , when I try to do a discovery I get the error below, so for me F5 does not seem to be able to get the ssl certs from microsoft to check the JWT token but this is for F5 TAC and the developers to solve this nodejs issue .
Error in processing URL https://login.microsoftonline.com/xxxx/v2.0/.well-known/openid-configuration. The message is - java.net.UnknownHostException: login.microsoftonline.com
