Using F5 Distributed Cloud DNS Load Balancer health checks and DNS observability

Introduction

This article is a continuation of my previous article that covers how to configure F5 Distributed Cloud (XC) DNS Load Balancer to provide geo-proximity and disaster recovery, in addition to other failover scenarios. This article builds on the previous configuration to add health checks and shows how the Distributed Cloud DNS service is performing.

DNS Load Balancer

Configuring DNS LB Health Checks

F5 XC can perform health checks on all IP members in a DNS Load Balancer Pool.

To configure health checks for a pool, go to DNS Management > DNS Load Balancer Management > DNS Load Balancer Health Checks, then click "Add DNS Load Balancer Health Check". Name the rule, for example, "europe-healthcheck", and choose an appropriate health check type.

The following health check types are supported for DNS LB:

HTTP
HTTPS
TCP
TCP (Hex payload)
UDP
ICMP

Each health check type, except ICMP, supports sending a custom string payload, and looks for a response to match. For example, choosing the HTTPS health check, F5 XC will first confirm whether it received a valid SSL certificate from the member. Passing the SSL certificate check, it then sends the configured "Send String" (an HTTP request). By default, the string is "HEAD / HTTP/1.0\r\n\r\n", although more complex strings are supported. The "Receive String", in regex (re2) format, validates the application layer response. The default receive string for HTTP(S) requests is "HTTP/1." A custom TCP or UDP port can also be configured to support services running on non-standard ports. Configuring the port with "0" uses the default port belonging to the intended protocol.

To apply the health check to a DNS LB Load Balancing rule, navigate to DNS Load Balancer Management > DNS Load Balancer Pools. Locate the pool to apply the health check to, and use the Manage Configuration action. Within the pool configuration, click Edit Configuration, scroll down to DNS Load Balancer Health Check, enable it, and then choose the health check created above. Save and Exit the Pool.

Status information about the health of the DNS LB pools and pool members can be found at the DNS Load Balancers Overview page. In the following example, one of the members in the "eu-pool" is unhealthy. Details about each specific pool member can be found by clicking on the pool.

Distributed Cloud DNS

Observability

The F5 XC DNS Performance Overview dashboards provide usage details for up to a 24-hour interval. Navigate to DNS Management > Overview > Performance for a high-level view showing how many requests a domain has received.

To see where DNS requests are coming from, the most requested services, and specific response details, click on each DNS zone.

The DNS performance dashboards provide the following views for each DNS zone:

Traffic Distribution
Top Requests
Total Queries
Query Type
Response Type (by RCODE)
DNS Query Rate (by Query Type)

The DNS Dashboards also include showing the type and frequency of each DNS request. Query logging is available and located in the Requests tab. This view provides up to a 24-hour interval of each DNS query.

The dashboard can be filtered to show requests from a particular geo location, resource record type, which record or records are being requested, in addition to the client IP and return code. The following image illustrates a filtered list. Records in the table below can be downloaded in a CSV formatted file.

Details about an individual request can be viewed by clicking on the ">" symbol, and the detailed record can be shown in either JSON or YAML format.

Logging & Analytics

The Global Log Receiver provides the logging of DNS Requests in addition to other security services in Distributed Cloud, including WAF and Bot Defense. This article explains how to configure Global Log Receiver and send logs using HTTPS to an ELK Stack (elasticsearch, logstash, and kibana). It also shows how to configure logstash and kibana to process GLR formatted logs from Distributed Cloud.

To configure the Global Log Receiver for DNS logging, go to Shared Configuration > Global Log Receiver, and create a new entry. In the Log Type field, choose DNS Request Logs.

In the following example, I've configured Global Log Receiver to send via HTTPS, but many other logging platforms are also supported. See this product documentation page for up to date information on all the features available with Global Log Receiver.

With DNS Request Logs configured, we can now see every DNS request to our Distributed Cloud tenant, processed by logstash, in the Kibana dashboard. The following output in Kibana shows DNS requests for all zones configured in the Distributed Cloud tenant.