F5 XC Distributed Cloud HTTP Header manipulations and matching of the client ip/user HTTP headers
1 . F5 XC distributed cloud HTTP Header manipulations In the F5 XC Distributed Cloud some client information is saved to variables that can be inserted in HTTP headers similar to how F5 Big-IP saves some data that can after that be used in a iRule or Local Traffic Policy. By default XC will insert XFF header with the client IP address but what if the end servers want an HTTP header with another name to contain the real client IP. Under the HTTP load balancer under "Other Options" under "More Options" the "Header Options" can be found. Then the the predefined variables can be used for this job like in the example below the $[client_address] is used. A list of the predefined variables for F5 XC: https://docs.cloud.f5.com/docs/how-to/advanced-security/configure-http-header-processing There is $[user] variable and maybe in the future if F5 XC does the authentication of the users this option will be insert the user in a proxy chaining scenario but for now I think that this just manipulates data in the XAU (X-Authenticated-User) HTTP header. 2. Matching of the real client ip HTTP headers You can also match a XFF header if it is inserted by a proxy device before the F5 XC nodes for security bypass/blocking or for logging in the F5 XC. For User logging from the XFF Under "Common Security Controls" create a "User Identification Policy". You can also match a regex that matches the ip address and this is in case there are multiple IP addresses in the XFF header as there could have been many Proxy devices in the data path and we want see if just one is present. For Security bypass or blocking based based on XFF Under "Common Security Controls" create a "Trusted Client Rules" or "Client Blocking Rules". Also if you have "User Identification Policy" then you can just use the "User Identifier" but it can't use regex in this case. To match a regex value in the header that is just a single IP address, even when the header has many ip addresses, use the regex (1\.1\.1\.1) as an example to mach address 1.1.1.1. To use the client IP address as a source Ip address to the backend Origin Servers in the TCP packet after going through the F5 XC (similar to removing the SNAT pool or Automap in F5 Big-IP) use the option below: The same way the XAU (X-Authenticated-User) HTTP header can be used in a proxy chaining topology, when there is a proxy before the F5 XC that has added this header. Edit: Keep in mind that in some cases in the XC Regex for example (1\.1\.1\.1) should be written without () as 1\.1\.1\.1 , so test it as this could be something new and I have seen it in service policy regex matches, when making a new custom signature that was not in WAAP WAF XC policy. I could make a seperate article for this 🙂Prevent BIG-IP Edge Client VPN Driver to roll back (or forward) during PPP/RAS errors
If you (like some of my customers) want to have the BIG-IP Edge Client packaged and distributed as a software package within your corporate infrastructure and therefore have switched off automatic component updates in your connectivity profiles, you might still get the covpn64.sys file upgraded or downgraded to the same version as the one installed on the BIG-IP APM server. Background We discovered that on some Windows clients the file covpn64.sys file got a newer/older timestamp in and started to investigate what caused this. The conclusion was that sometimes after hibernation or sleep, the Edge Client is unable to open the VPN interface and therefore tries to reinstall the driver. However, instead of using a local copy of the CAB file where the covpn64.sys file resides, it downloads it from the APM server regardless of if the version on the server and client match each other or not. In normal circumstances when you have automatic upgrades on the clients, this might not be a problem, however when you need to have full control on which version is being used on each connected client, this behavior can be a bit of a problem. Removing the Installer Component? Now you might be thinking, hey… Why don't you just remove the Component Installer module from the Edge Client and you won't have this issue. Well the simple answer to this is the fact that the Component Installer module is not only used to install/upgrade the client. In fact, it seems like it's also used when performing the Machine Check Info from the Access Policy when authenticating the user. So by removing the Component Installer module result in other issues. The Solution/workaround The Solution I came up with is to store each version of the urxvpn.cab file in an IFile and then use an iRule to deliver the correct version whenever a client tries to fetch the file for reinstallation. What's needed? In order to make this work we need to Grab a copy of urxvpn.cab from each version of the client Create an IFile for each of these versions Install iRule Attach iRule to the Virtual Server that is running the Access Policy Fetching the file from the apmclients ISOs For every version of the APM client that is available within your organization a corresponding iFile needs to be created. To create the iFiles automatically you can do the following on the APM server. Login to the CLI console with SSH Make sure you are in bash by typing bash Create temporary directories mkdir /tmp/apm-urxvpn mkdir /tmp/apm-iso Run the following (still in bash not TMSH) on the BIG-IP APM server to automatically extract the urxvpn.cab file from each installed image and save them in the folder /tmp/apm-urxvpn. for c in /shared/apm/images/apmclients-* do version="$(echo "$c" | awk -F. \ '{gsub(".*apmclients-","");printf "%04d.%04d.%04d.%04d", $1, $2, $3, $4}')" && \ (mount -o ro $c /tmp/apm-iso cp /tmp/apm-iso/sam/www/webtop/public/download/urxvpn.cab \ /tmp/apm-urxvpn/URXVPN.CAB-$version umount /tmp/apm-iso) done Check the files copied ls -al /tmp/apm-urxvpn Import each file either with tmsh or with GUI. We will cover how to import with tmsh below. If you prefer to do it with the GUI, more information abour how to do it can be found in K13423 You can use the following script to automatically import all files cd /tmp/apm-urxvpn for f in URXVPN.CAB-* do printf "create sys file ifile $f source-path file:$(pwd)/$f\ncreate ltm ifile $f file-name $f\n" | tmsh done Save the new configuration tmsh -c “save sys config” Time to create the iRule when CLIENT_ACCEPTED { ACCESS::restrict_irule_events disable } when HTTP_REQUEST { set uri [HTTP::uri] set ua [HTTP::header "User-Agent"] if {$uri starts_with "/vdesk" || $uri starts_with "/pre"} { set version "" regexp -- {EdgeClient/(\d{4}\.\d{4}\.\d{4}\.\d{4})} $ua var version if {$version != ""} { table set -subtable vpn_client_ip_to_versions [IP::client_addr] $version 86400 86400 } else { log local0.debug "Unable to parse version from: $ua for IP: [IP::client_addr] URI: $uri" } } elseif {$uri == "/public/download/urxvpn.cab"} { set version "" regexp -- {EdgeClient/(\d{4}\.\d{4}\.\d{4}\.\d{4})} $ua var version if {$version == ""} { log local0.warning "Unable to parse version from: $ua, will search session table" set version [table lookup -subtable vpn_client_ip_to_versions [IP::client_addr]] log local0.warning "Version in table: $version" } if {$version == ""} { log local0.warning "Unable to find version session table" HTTP::respond 404 content "Missing version in request" "Content-Type" "text/plain" } else { set out "" catch { set out [ifile get "/Common/URXVPN.CAB-$version"] } if {$out == ""} { log local0.error "Didn't find urxvpn.cab file for Edge Client version: $version" HTTP::respond 404 content "Unable to find requested file for version $version\n" "Content-Type" "text/plain" } else { HTTP::respond 200 content $out "Content-Type" "application/vnd.ms-cab-compressed" } } } } Add the iRule to the APM Virtual Server Known Limitations If multiple clients with different versions of the Edge Client are behind the same IP address, they might download the wrong version. This is due to the fact that the client doesn't present the version when the request for the file urxvpn.cab reaches the iRule. This is why the iRule tries to store IP addresses based on the source IP address of other requests related to the VPN. More information about this problem can be found in K000132735Ubuntu Virtual Machine for NGINX Microservices March 2022 Labs
Since I didn't have access to the lab environment in UDF, I decided to setup and run my own environment in VMware Workstation, so that I can run the Microservices March Labs at my own pace. This guide should help anyone to setup their own Ubuntu VM to run the labs in your environment.
Step-by-step guide to build a F5 AWAF lab on Google Cloud
This is a small step by step guide on how to build a F5 AWAF (Advanced Web Application Firewall) lab environment on GCP (Google Cloud Platform). The purpose of this guide is to provide an easy way for quickly spin up a lab environment which can be used for study or demo purposes. https://github.com/pedrorouremalta/f5-awaf-lab-on-gcpF5 AFM/Edge Firewall and the difference between Edge Firewalls and Next-generation Firewalls (NGFW)
Next-generation Firewalls (NGFW) have a lot of features like policies based on AD users and AD groups, dynamic user quarantine, Application/Service and Virus/Spyware/Vulnerability default or custom signatures to allow traffic only comming from specific applications that is scanned for viruses or other malware types. A long time ago I also did not know the difference between the F5 AFM and NGFW (I even asked a question on the forum https://community.f5.com/t5/technical-forum/to-make-the-f5-afm-like-a-full-ngfw-is-there-plans-the-f5-afm-to/td-p/207685 ), so after time I understood the difference and I have made this post to clear things out 😉 NGFW truly provides a lot of nice options but where they are lacking when they are deployed at the Internet Service Providers, Mobile Operators or at the Edge of big corporate networks or private scrubbing centers as they don't have good DDOS protections or CG-NAT functions. NGFW dp have NAT capabilities but in most cases dose capabilities are limited to basic source PAT, destination NAT or Static NAT. Also at the Edge of the Network the firewall device should have high throughput and there is no need for it to work with AD users/AD groups, user/group redistribution between the firewalls or specific Applications/Services, used just by a specific company as in the case with ISP or Mobile Operators it should protect many customers with the Advanced DOS/DDOS options, to be able to do NAT that is easily traceable in the logs which IP address to which source ip which public ip was allocated (great feature for mobile or Internet providers combined with F5 PEM for user monetization and tracking) Also the Edge firewall device may need to failover to a Scrubbing center if the DDOS attack becomes too big, so this function is nice to have or to have an ip intelligence feed list to block attacks even before doing any deep inspections just based on the source or destination IP address. This is where the F5 AFM comes into the picture as not an replacement of the NGFWs but as a complementary device that is at the Edge of the Network and filters the traffic and then the customer NGFWs do the more fine grade checks. Sometimes AFM is deployed as a server firewall together with F5 LTM/APM/aAWAF after the NGFWs for example to filter the a DDOS attack that the scrubbing center did not block as it was too small and directed to a specific destination and most scrubbing center block only really high volume attacks (most scrubbing centers can't look in the SSL data like the F5 Silverline) that can bring down the entire data center.AFM can now work with subscriber data at the ISP mobile operator level and from what I have seen the NGFW are limited in this field and they are made for internal Enterprise use, where AD groups and AD users are needed not subscriber data. The F5 AFM capabilities that I have not seen at most NGFW are : DOS based protections on the AFM have the option to be Fully Atomatic and to adjust their thresholds based Machne Learning (ML) learning, so there id no need for someone to constantly modify the DOS thresholds like with other DOS protection products. Also the DDOS protection has Dinamic signatures and with this feature a dynamic signature of the DDOS traffic is Automatically generated, so only the attackers to be blocked. By default the DDOS protection thresholds under "Security > DoS Protection > Device Protection " are inforced if a not more specific DOS profile is athached under the Virtual Server. The F5 AFM can be combined with the F5 Advanced WAF/ASM for full layer 3/4/7 DDOS protection and there is device named F5 DDoS Hybrid Defender that is combination between the Layer3/4 and the Layer7 protections and it is configured with a Guided Configuration Wizard. The F5 AFM has DDOS protections not only for TCP, UDP,ICMP traffic but also for HTTP, DNS and SIP protocols. There are great community articles about the DDOS features and their configuration that I will share: https://community.f5.com/t5/technical-articles/explanation-of-f5-ddos-threshold-modes/ta-p/286884 https://community.f5.com/t5/technical-articles/ddos-mitigation-with-big-ip-afm/ta-p/281234 Also this link is helpfull: https://support.f5.com/csp/article/K49869231 The AFM can redirect the traffic to a Scrubing Center if it becomes too big and this may save some money to only use a scrubbing center if the DDOS is too big. If BGP is used the AFM will use the F5 Zebos Routing module that is like a mini router inside F5. The previous F5 product Carrier Grade NAT is now migrated to the AFM which allows you to not only use source nat, destination nat or static nat but also to use NAT features like PBA,Deterministic NAT or PCP.The AFM can also respond to ARP requests for translated source IP addresses and this is called Proxy ARP or to intgrate with the ZebOS routin module that is like a mini router inside the F5 device to advertize the translated addresses. Port block allocation (PBA) mode is a translation mode option that reduces CGNAT logging, by logging only the allocation and release of each block of ports. When a subscriber first establishes a network connection, the BIG-IP® system reserves a block of ports on a single IP address for that subscriber. The system releases the block when no more connections are using it. This reduces the logging overhead because the CGNAT logs only the allocation and release of each block of ports. Deterministic mode is an option used to assign translation address, and is port-based on the client address/port and destination address/port. It uses reversible mapping to reduce logging, while maintaining the ability for translated IP address to be discovered for troubleshooting and compliance with regulations. Deterministic mode also provides an option to configure backup-members.And there is even a tool dnatutil to see the mapping of a client ip address. Port Control Protocol (PCP) is a computer networking protocol that allows hosts on IPv4 or IPv6 networks to control how the incoming IPv4 or IPv6 packets are translated and forwarded by an upstream router that performs network address translation (NAT) or packet filtering. By allowing hosts to create explicit port forwarding rules, handling of the network traffic can be easily configured to make hosts placed behind NATs or firewalls reachable from the rest of the Internet (so they can also act as network servers), which is a requirement for many applications. As logging the user NAT translations is mandatory this can generate a lot of logs for the Service Providers but with DNAT and PBA the needed log space is reduced as much as possible but still keeping the needed log info. The AFM now supports some of the options of F5 PEM for Traffic Intelligence or as in the NGFW applicaion discovery or subscriber discovery and security rules based on subscribers discovered by Radius or DHCP sniffing or iRules as the NGFW have AD users and AD groups but Service and Mobile providers work with IMEI phone codes and not with AD groups/users. https://community.f5.com/t5/technical-articles/traffic-intelligence-in-afm-through-categories/ta-p/295310 Another really wonderful feature is the IP intelligence that will protect you from bad source or destination ip addresses and with the AFM you can also feed the AFM custom list that are generated by your threat intelligence platform.The AFM and Advanced WAF/ASM can automatically place the IP addresses in a shun list that is blocked by the IP intelligence as the IP intelligence checks happen before the ASM or even the AFM in the traffic path! There is a nice community video about this feature:https://community.f5.com/t5/technical-articles/the-power-of-ip-intelligence-ipi/ta-p/300528 The AFM also has port misuse policies or Protocol Inspection profiles that are similar the NGFW Applications/services to allow only the correct protocol on the port not just port number or IPS/Antivirus signatures. The F5 AFM Protocol Inspection is based on SNORT so you can not only block attacks but allow traffic based on the payload, for example providing access to sertain server only if the Referer header is a sertain value by writing custom signatures. It by default has many signatures and protocol RFC compliance checks. The F5 AFM protocol inspection can also be used as as more fine grade way for custom application control than the Port Misuse policies, when creating a custom signature for example to block specific User-Agent HTTP header! One of the best features that the F5 Protocol Inspection IPS has compared even to NGFW products is to place new signatures in staging (for example after a new signature set is downloaded) for some time and to monitor how many times the signatures get triggered in that staging period before enforcing and that feature is really great. For more information I suggest checkingthe link below: https://support.f5.com/csp/article/K00322533 https://f5-agility-labs-firewall.readthedocs.io/en/latest/class2/module3/lab4.html https://support.f5.com/csp/article/K25265787 The F5 AFM is also a great Edge firewall for many protocols like DNS, SSH,SIP not only HTTP.The F5 AFM simiarly to the aWAF/ASM can work in a transperant bridged mode thanks to Vlan Groups, Wildcard VS and Proxy Arp, where it is invisible for the end users (https://support.f5.com/csp/article/K15099). Do not forget that tha AFM is before any other module except the IP intelligence and to decide if it will work in a firewall or ADC mode(https://support.f5.com/csp/article/K92958047). Also the order or the rules is important (Global context policies/rules > Route Domain Context > Virtual Server/Self IP > Managment) . You can even use DNS FQDN names in the security policy rules if needed and trace any issues related to Security Rules and DOS with the Packet Tester tool and with Timer policies you can allow long live connections that do not generate traffic through the firewall if needed!The Managment IP in newer versions can use AFM rules even without AFM being provisioned (https://support.f5.com/csp/article/K46122561), isn't that nice😀 ! F5 supports vWire or Vlan groups, so F5 AFM or F5 DHD (DDOS Hybrid Defender) can be placed not only like a layer 3 firewall but also in Transparent/Invisible layer 2 or in case or Virtual Wire layer 1 mode. The F5 AFM operations guide is trully a nice resource to review: https://support.f5.com/csp/article/K3820175502 - Visualization of F5 BIG-IP metrics on Grafana using Prometheus and Telemetry Streaming service
Configuration using CLI of F5 BIG-IP device Following steps for the configuration of telemetry streaming consumer target using CLI of F5 BIG-IP device are discussed below: Once you have accessed your F5 BIG-IP device CLI terminal then access either your default admin credentials or the new user you’ve recently created on the above section. Then execute the following commands on the terminal: On the username and password section, you either enter your default admin credentials or the new user you’ve recently created has the administrator privilege. curl -u username:password -k https://localhost/mgmt/shared/telemetry/declare Note: -k, --insecure to be made secure by using the CA certificate bundle installed by default. This makes all connections considered "insecure" fail unless -k, --insecure is used. ChangChange into tmp directory and create a file called ts-config.json and I am using vi editor for it. cd /tmp vi ts-config.json Paste the Telemetry Streaming declaration and then save the file and exit the vi editor. { "class": "Telemetry", "My_Poller": { "class": "Telemetry_System_Poller", "interval": 0 }, "My_System": { "class": "Telemetry_System", "enable": "true", "systemPoller": [ "My_Poller" ] }, "metrics": { "class": "Telemetry_Pull_Consumer", "type": "Prometheus", "systemPoller": "My_Poller" } } Then execute the following command on the terminal on thesame directory /tmp and change the username and password section with your F5 BIG-IP device credentialshaving the administrator privilege. curl -X POST -u username:password -khttps://localhost/mgmt/shared/telemetry/declare-d @ts-config.json -H “content-type:application/json” To verify the available metrics curl -u username:password -k https://localhost/mgmt/shared/telemetry/pullconsumer/metrics Section III: Configuration of Prometheus Once the telemetry streaming service has been successfully configured and the metrics are available on the path. We need to configure Prometheus in order to scrape the metrics data on the predefined path. The following are the steps to configure the Prometheus: Note: On this user-guide demonstration, both Grafana and Prometheus are installed on the same host with different service ports as mentioned earlier. CentOS 7 is used as the OS for this host machine and you may have different syntax to view the following status check. First, check the status of the Prometheus sudo systemctl status prometheus.service View the current working directory and change into /etc/prometheus pwd cd /etc/prometheus ls -al global: scrape_interval: 10s scrape_configs: - job_name: 'TelemetryStreaming' scrape_timeout: 30s scrape_interval: 30s scheme: https tls_config: insecure_skip_verify: true metrics_path: '/mgmt/shared/telemetry/pullconsumer/metrics' basic_auth: username: 'F5-BIG-IP-username' password: 'F5-BIG-IP-password' static_configs: - targets: ['BIGIP-managementIP:443'] Then restart the Prometheus service and check the status of the Prometheus service. sudo systemctl restart prometheus.service sudo systemctl status prometheus.service Note: If the configuration is correct, then the Prometheus service will be enabled otherwise, the status of the Prometheus service will be disabled. To further verify whether instances has been discovered on the Prometheus: -Go tohttp://prometheus-ip:service/port - Click on the Status option and select the Target option Section IV: Configuration on Grafana using Prometheus as a data source In this section, we need to connect Prometheus as a data source on Grafana Once the data source has been successfully configured on Grafna then Create a new dashboard and select Prometheus as the data source then select the relevant metrics and change the refresh interval as required. Save and apply the panel. Then,Save the dashboard and view the metrics on the Grafana dashboard. The possible issue that can arise during the configuration If you use the default TS declare from the official telemetry streaming document website then you may fail to view the available metrics on the mentioned link: https://<f5-management-ip>/mgmt/shared/telemetry/pullconsumer/metrics
