How to achieve hiding internal URLs and HTTP dynamic redirection with F5 XC HTTP Load Balancer
This is a comprehensive list of the XC options to change the Host or URL values in the requests and to trigger dynamic redirections. XC HTTP LB basic rewrite options By default, XC with or without a route rewrites the Host value of the client’s http request to the origin server’s FQDN name in the origin pool (only if the origin server is FQDN, not IP address). Under the XC origin pool, you can control the SNI selection for the origin pool. Sometimes having not the correct SNI could generate 400 errors for nginx origin web servers, for example. XC HTTP LB Host 3xx Redirection Under the XC HTTP LB route, you can configure host redirection even for non standard port. If you don't configure an explicit path option, then XC will auto-redirect and keep the original URL. I found this not something well-explained. For basic HTTP to HTTPS redirection you can easily enable it under and HTTPS LB and the only issue is it assumes that HTTP is on port 80, so for custom redirection from HTTP on port other than 80 you need to use the below methods. For any F5 experts out there this is similar to the default http to https redirection in F5 BIG-IP ADC or F5 NEXT ADC with system iRule _sys_https_redirect. You can even configure the HTTP LB to listen on ports other than the default ones (80 for HTTP or 443 for HTTPS) and to redirect to also not custom ports. In some cases you will need to add the FQDN domain with the non-custom port. Some web browsers send traffic to not custom ports with the port in the host name and this could cause XC not to match the traffic. To test this with "curl" that can even be directed with the "resolve" option to HTTP LB that the public DNS still does not direct to it. This is similar to F5 BIG-IP/NEXT iRule TCL scripting where " [HTTP::uri] " or " [HTTP::path] " are used depending if you want the redirect to keep the query parameters or not. XC HTTP LB internal URL Rewrite Under the XC HTTP Route advanced options, you can find the Request URL rewrite can match on prefix or even a regex to be used. This in some cases helps for apps that don't handle very well 3xx redirects (mobile apps are one such case) as for example "/test" URL is auto-changed to "/" otherwise 404 code would be seen if the page does not exist! The rewrite option seems particularly useful for Customer Edge (CE) deployments that are inside the customer networks RE > CE traffic where the HTTP LB is on the RE and the origin servers in the origin pool are on the CE and the SSL/IPSEC tunnel between RE and CE is used to stich things. Sometimes a rewrite of the response is needed, as for example to change the HTML response image urls to the new ones. In this case, Nginx containers can be used with XC Virtual Kubernetes (v8ks) that will offer this advanced function with the "filter" option that I have described in article F5 XC vk8s workload with Open Source Nginx | DevCentral In F5 BIG-IP or NEXT with iRules or even Local Traffic Policies this functions are implemented. XC Custom Route Under Multi-Cloud App Connect, you can create custom route objects that can also do redirects or rewrite request URLs. The main benefit is that many HTTP LB can use the custom routes and can attach custom Request/Response headers or Cookies, even on Redirect routes that the Simple Routes that are directly created under the HTTP LB can't. This can be useful for some logic in mobile apps or custom non browser Agents like API clients. Summary The XC Cloud provides many options. In the future, you might see things like TCL iRule scripts that will offer advanced changes and rewrites of even HTTP request and response bodies without adding Nginx containers. So, keep checking release notes at Release Changelogs | F5 Distributed Cloud Technical Knowledge to not miss anything new that is released! Useful Links: Layer 7 Content Routing in F5 XC | DevCentral How to setup path-based routing or application load-balancing – F5 Distributed Cloud Services How to do an URL redirection? – F5 Distributed Cloud Services How do I configure a redirection with a URL that contains path and query parameters? Article DetailF5 XC Distributed Cloud DNS GSLB implementing Split-DNS
Have you ever wondered how to achieve Bind-like 'VIEW' behavior with F5 XC Load Balancer where depending on the customer’s IP, different DNS responses are returned? Well wonder no more! F5 XC DNS Load Balancers have the topology load balancing feature from the start, but now you can use source IP prefix lists or BGP ASN numbers that opens the door to Split-DNS similar to the BIND "View" feature! The XC DNS Load Balancers rules nowadays have new options that can support EDNS Client Subnet (ECS) feature load balancing or BGP ASN load balancing. EDNS is like X-Forwarded-For Header as it sees the real client subnet and not the DNS local resolver server IP Address. For more information, I suggest checking out the link below: EDNS0 Implementation and Troubleshooting recommendations For anyone that has worked with F5 DNS/GTM BIG-IP, this is similar to creating custom topology records. For more information, I suggest checking out the link below: Achieving split DNS behavior through BIG-IP DNS wide IPs Configuring a DNS zone First, you need to configure a primary DNS zone as shown in the picture below or use the option "Allow Application Load Balancer Managed Records" that is described in the links below that allows a created F5 XC TCP or HTTP LB to be auto-added to the DNS primary zone in XC. For more information, I suggest checking out the links below: Manage DNS Zone | F5 Distributed Cloud Technical Knowledge Use F5 Distributed Cloud to control Primary and Secondary DNS The 2 DNS Load Balancer rules match either 46.233.56.0/24 or 0.0.0.0/0 that is any IP address, and this is why the Score is set to 110 for the first rule to have higher priority. For testing, the Linux "dig" command supports +subnet option to change the EDNS subnet for example "dig test.niki.com @ns1.f5clouddns.com +subnet=46.233.56.0". If you don’t choose any subnet, the EDNS will be your IP address. But remember that the system could add a default mask like /24 if you don’t specify +subnet command with a mask. When you specify with @ to send the traffic directly to ns1.f5clouddns.com or ns2.f5clouddns.com you don't have to change your public DNS records and you can first test the XC DNS setup and then configure DNS delegation on your primary DNS servers. If you are behind NAT or VPN, use What Is My IP Address - See Your Public Address - IPv4 & IPv6 to see your public IP address. Example DNS request that will be sent to the specific pool: dig test.niki.com @ns1.f5clouddns.com +subnet=46.233.56.1 Example DNS request that will be sent to the default pool: dig test.niki.com @ns1.f5clouddns.com +subnet=5.5.5.5 Example DNS request with no "+subnet": That will be sent to the specific pool as my IP matches 46.233.56.0 subnet, and it is auto-added to the EDNS in DNS request. dig test.niki.com @ns1.f5clouddns.com You can also use the ASN for this task, and the ASN is being compared to your EDNS subnet. There are a lot of tools to see your ASN number based on your IP address 😉 Summary: XC has new and new features every day, and the DNS Load Balancer service is a clear example of this. We will see what comes next!F5 XC Session tracking with User Identification Policy
With F5 AWAF/ASM there is feature called session tracking that allows tracking and blocking users that do too many violations not only based on IP address but also things like the BIG-IP AWAF/ASM session cookie. What about F5 XC Distributed Cloud? Well now we will answer that question 😉 Why tracking on ip addresses some times is not enough? XC has a feature called malicious users that allows to block users if they generate too many service policy, waf , bot or other violations. By default users are tracked based on source IP addresses but what happens if there are proxies before the XC Cloud or NAT devices ? Well then all traffic for many users will come from a single ip address and when this IP address is blocked many users will get blocked, not just the one that did the violation. Now that we answered this question lets see what options we have. Reference: AI/ML detection of Malicious Users using F5 Distributed Cloud WAAP Trusted Client IP header This option is useful when the client real ip addresses are in something like a XFF header that the proxy before the F5 XC adds. By enabling this option automatically XC will use this header not the IP packet to get the client ip address and enforce Rate Limiting , Malicious Users blocking etc. Even in the XC logs now the ip address in the header will be shown as a source IP and if there is no such header the ip address in the packet will be used as backup. Reference: How to setup a Client IP as the Source IP on the HTTP Load Balancer headers? – F5 Distributed Cloud Services (zendesk.com) Overview of Trusted Client IP Headers in F5 Distributed Cloud Platform User Identification Policies The second more versatile feature is the XC user identification policies that by default is set to "Client IP" that will be the client ip from the IP packet or if "Trusted Client IP header" is configured the IP address from the configured header will be used. When customizing the feature allows the use of TLS fingerprints , HTTP headers like the "Authorization" header and more options to track the users and enforce rate limiters on them or if they make too many violations and Malicious users is enabled to block them based on the configured identifier if they make too many waf violations and so much more. The user identification will failover to the ip address in the packet if it can't identify the source user but multiple identification rules could be configured and evaluated one after another, as to only failover to the packet ip address if an identification rule can't be matched! If the backend upstream origin server application cookie is used for user identification and XC WAF App firewall is enabled and you can also use Cookie protection to protect the cookie from being send from another IP address! A nice option is that you can search the request and security logs using the "user id" as by default in XC for example you can't search the logs using the Authorization header that for example OAUTH and API traffic uses or a cookie like the session JSESSION or PHPSSEID but this way you can! The demo juice shop app at https://demo.owasp-juice.shop/ can be used for such testing! References Lab 3: Malicious Users (f5.com) Malicious Users | F5 Distributed Cloud Technical Knowledge Configuring user session tracking (f5.com) How to configure Cookie Protection – F5 Distributed Cloud Services (zendesk.com) Configure Rate Limiting per User | F5 Distributed Cloud Technical KnowledgeF5 XC CE Debug commands through GUI cloud console and API
Why this feature is important and helpful? With this capability if the IPSEC/SSL tunnels are up from the Customer Edge(CE) to the Regional Edge(RE), there is no need to log into the CE, when troubleshooting is needed. This is possible for Secure Mesh(SM) and Secure Mesh V2 (SMv2) CE deployments. As XC CE are actually SDN-based ADC/proxy devices the option to execute commands from the SDN controller that is the XC cloud seems a logical next step. Using the XC GUI to send SiteCLI debug commands. The first example is sending the "netstat" command to "master-3" of a 3-node CE cluster. This is done under Home > Multi-Cloud Network Connect > Overview > Infrastructure > Sites and finding the site, where you want to trigger the commands. In the VPM logs it is possible to see the command that was send in API format by searching for it or for logs starting with "debug", as to automate this task. If you capture and review the full log, you will even see not only the API URL endpoint but also the POST body data that needs to be added. The VPM logs that can also be seen from the web console and API, are the best place to start investigating issues. XC Commands reference: Node Serviceability Commands Reference | F5 Distributed Cloud Technical Knowledge Troubleshooting Guidelines for Customer Edge Site | F5 Distributed Cloud Technical Knowledge Troubleshooting Guide for Secure Mesh Site v2 Deployment | F5 Distributed Cloud Technical Knowledge Using the XC API to send SiteCLI debug commands. The same commands can be send using the XC API and first the commands can be tested and reviewed using the API doc and developer portals. API documentation even has examples of how to run these commands with vesctl that is the XC shell client that can be installed on any computer or curl. Postman can also be used instead of curl but the best option to test commands through the API is the developer portal. Postman can also be used by the "old school" people 😉 Link reference: F5 Distributed Cloud Services API for ves.io.schema.operate.debug | F5 Distributed Cloud Technical Knowledge F5 Distributed Cloud Dev Portal ves-io-schema-operate-debug-CustomPublicAPI-Exec | F5 Distributed Cloud Technical Knowledge Summary: The option to trigger commands though the XC GUI or even the API is really useful if for example there is a need to periodically monitor the cpu or memory jump with commands like "execcli check-mem" or "execcli top" or even automating the tcpdump with "execcli vifdump xxxx". The use cases for this functionality really are endless.F5 XC Distributed Cloud HTTP Header manipulations and matching of the client ip/user HTTP headers
1 . F5 XC distributed cloud HTTP Header manipulations In the F5 XC Distributed Cloud some client information is saved to variables that can be inserted in HTTP headers similar to how F5 Big-IP saves some data that can after that be used in a iRule or Local Traffic Policy. By default XC will insert XFF header with the client IP address but what if the end servers want an HTTP header with another name to contain the real client IP. Under the HTTP load balancer under "Other Options" under "More Options" the "Header Options" can be found. Then the the predefined variables can be used for this job like in the example below the $[client_address] is used. A list of the predefined variables for F5 XC: https://docs.cloud.f5.com/docs/how-to/advanced-security/configure-http-header-processing There is $[user] variable and maybe in the future if F5 XC does the authentication of the users this option will be insert the user in a proxy chaining scenario but for now I think that this just manipulates data in the XAU (X-Authenticated-User) HTTP header. 2. Matching of the real client ip HTTP headers You can also match a XFF header if it is inserted by a proxy device before the F5 XC nodes for security bypass/blocking or for logging in the F5 XC. For User logging from the XFF Under "Common Security Controls" create a "User Identification Policy". You can also match a regex that matches the ip address and this is in case there are multiple IP addresses in the XFF header as there could have been many Proxy devices in the data path and we want see if just one is present. For Security bypass or blocking based based on XFF Under "Common Security Controls" create a "Trusted Client Rules" or "Client Blocking Rules". Also if you have "User Identification Policy" then you can just use the "User Identifier" but it can't use regex in this case. To match a regex value in the header that is just a single IP address, even when the header has many ip addresses, use the regex (1\.1\.1\.1) as an example to mach address 1.1.1.1. To use the client IP address as a source Ip address to the backend Origin Servers in the TCP packet after going through the F5 XC (similar to removing the SNAT pool or Automap in F5 Big-IP) use the option below: The same way the XAU (X-Authenticated-User) HTTP header can be used in a proxy chaining topology, when there is a proxy before the F5 XC that has added this header. Edit: Keep in mind that in some cases in the XC Regex for example (1\.1\.1\.1) should be written without () as 1\.1\.1\.1 , so test it as this could be something new and I have seen it in service policy regex matches, when making a new custom signature that was not in WAAP WAF XC policy. I could make a seperate article for this 🙂 XC can even send the client certificate attributes to the backend server if Client Side mTLS is enabled but it is configured at the cert tab.
