DNS over HTTPS Resolution
Introduction
I work in the EMEA Professional Services team and in this article I will walk you through a recent solution which I provided to support the F5 Sales team for a Service Provider. You can see the finished iApp here which took a few days of work spread over a few months.
This solution is to provide DNS over HTTP resolution - F5 has this as a feature soon to be released but in the meantime we needed a working solution.
Development
I started by looking at the RFC ( https://tools.ietf.org/html/rfc8484 ) to work out what was required and realised that this has two methods - GET and POST, it gives examples of the payload and response so I put together a simple lab and proof of concept. This used curl to send the data - both methods send a DNS wire-format packet, GET base64url-encodes it and sends it as a parameter while POST sends it in the payload directly.
I decided it would be simplest to use an iRule with sideband to a local virtual server which would in turn send the traffic to the backend server. Below is the simple pseudocode I created to guide me:
when HTTP_REQUEST if URI if method == POST collect payload if method == GET decode payload send payload via sideband return response endwhen when HTTP_DATA send payload via sideband return response endwhen
From this you can see that I have 'send payload via sideband' in two places. I like to follow the DRY principle so i immediately thought about procedures in later versions.
I have used sideband a number of times before but i reviewed how it works in terms of destination and source address at https://clouddocs.f5.com/api/irules/SIDEBAND.html . For base64url, I decided to start with the built-in base64 decoding function ( https://clouddocs.f5.com/api/irules/b64decode.html ) and use string map to change the erroneous characters.
See my lab configuration below with my first version of the iRule:
[root@DoH-bigip1:Active:Standalone] config # tmsh list ltm pool ltm pool dns { members { server-1:domain { address 10.20.20.6 } } } [root@DoH-bigip1:Active:Standalone] config # tmsh list ltm virtual ltm virtual DoH { creation-time 2020-05-04:03:17:08 destination 10.10.10.10:https ip-protocol tcp last-modified-time 2020-05-04:03:33:22 mask 255.255.255.255 profiles { clientssl-insecure-compatible { context clientside } http { } tcp { } } rules { DoH } serverssl-use-sni disabled source 0.0.0.0/0 translate-address enabled translate-port enabled vlans { External } vlans-enabled vs-index 4 } ltm virtual dns-tcp { creation-time 2020-05-04:03:15:46 destination 10.10.10.10:domain ip-protocol tcp last-modified-time 2020-05-04:03:15:46 mask 255.255.255.255 pool dns profiles { dns { } tcp { } } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled vlans { External } vlans-enabled vs-index 3 } ltm virtual dns-udp { creation-time 2020-05-04:03:14:25 destination 10.10.10.10:domain ip-protocol udp last-modified-time 2020-05-04:03:14:25 mask 255.255.255.255 pool dns profiles { dns { } udp { } } rules { dns-log } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled vlans { External } vlans-enabled vs-index 2 } ltm rule DoH { when RULE_INIT { set static::sideband_virtual_server "/Common/dns-udp" } when HTTP_REQUEST { # https://tools.ietf.org/html/rfc8484 if { [HTTP::uri] == "/dns-query" } { if { (([HTTP::method] equals "POST") and (([HTTP::header "accept"] equals "application/dns-message") or ([HTTP::header "content-type"] equals "application/dns-message"))) } { if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 65535 && [HTTP::header "Content-Length"] > 0} { log local0.debug "Content-Length [HTTP::header value "Content-Length"]" HTTP::collect [HTTP::header value "Content-Length"] } else { HTTP::collect 100 } } else { HTTP::respond 415 content "Unsupported Media Type" } } else { HTTP::respond 404 } } when HTTP_REQUEST_DATA { # Use sideband set conn_id [connect -protocol UDP -timeout 100 -idle 30 -status conn_status /Common/dns-udp] log local0.debug "Connection status: $conn_status" log local0.debug "Payload B64: [b64encode [HTTP::payload]]" set send_bytes [send -timeout 100 -status send_status $conn_id [HTTP::payload]] log local0.debug "send_status: $send_status" recv -timeout 100 -peek -status recv_status $conn_id result log local0.debug "status: $recv_status Result B64: [b64encode $result]" HTTP::respond 200 content $result noserver content-type application/dns-message vary Accept-Encoding content-length [string length $result] } } ltm rule dns-log { when DNS_REQUEST { log local0.debug "[DNS::question name] [DNS::question class] [DNS::question type]" } when DNS_RESPONSE { set rrs [DNS::answer] foreach rr $rrs { log local0. "[DNS::rdata $rr]" } } }
I tested this setup using the following procedure:
- Create DNS wire request in binary file
echo "00 00 01 00 00 01 00 00 00 00 00 00 03 77 77 77 07 65 78 61 6d 70 6c 65 03 63 6f 6d 00 00 01 00 01"|xxd -r -p >example_req
- Perform curl request
curl -k -H 'content-type: application/dns-message' --data-binary @example_req -o output_file https://10.10.10.10/dns-query
- Inspect output file
root@DoH-xubuntu-desktop1:~# xxd output_file 00000000: 0000 8580 0001 0001 0001 0002 0377 7777 .............www 00000010: 0765 7861 6d70 6c65 0363 6f6d 0000 0100 .example.com.... 00000020: 01c0 0c00 0100 0100 093a 8000 0401 0203 .........:...... 00000030: 04c0 1000 0200 0100 093a 8000 0b09 6c6f .........:....lo 00000040: 6361 6c68 6f73 7400 c03d 0001 0001 0009 calhost..=...... 00000050: 3a80 0004 7f00 0001 c03d 001c 0001 0009 :........=...... 00000060: 3a80 0010 0000 0000 0000 0000 0000 0000 :............... 00000070: 0000 0001 ....
My BIG-IP logs:
May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST>: Content-Length 33 May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: Doing sideband May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: Connection status: connected May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: Payload B64: AAABAAABAAAAAAAAA3d3dwdleGFtcGxlA2NvbQAAAQAB May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: send_status: sent May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/dns-log <DNS_REQUEST>: www.example.com IN A May 4 05:49:12 DoH-bigip1 info tmm[22188]: Rule /Common/dns-log <DNS_RESPONSE>: 1.2.3.4 May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: status: received Result B64: AACFgAABAAEAAQACA3d3dwdleGFtcGxlA2NvbQAAAQABwAwAAQABAAk6gAAEAQIDBMAQAAIAAQAJOoAACwlsb2NhbGhvc3QAwD0AAQABAAk6gAAEfwAAAcA9ABwAAQAJOoAAEAAAAAAAAAAAAAAAAAAAAAE=
I then added GET resolution:
when RULE_INIT { set static::sideband_virtual_server "/Common/dns-udp" } when HTTP_REQUEST { # https://tools.ietf.org/html/rfc8484 if { [HTTP::path] == "/dns-query" } { if { (([HTTP::method] equals "POST") and (([HTTP::header "accept"] equals "application/dns-message") or ([HTTP::header "content-type"] equals "application/dns-message"))) } { if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 65535 && [HTTP::header "Content-Length"] > 0} { log local0.debug "Content-Length [HTTP::header value "Content-Length"]" HTTP::collect [HTTP::header value "Content-Length"] } else { HTTP::collect 100 } } elseif { [HTTP::method] equals "GET" && [URI::query [HTTP::uri] dns] != "" } { set queryname [b64decode [URI::query [HTTP::uri] dns] ] log local0.debug "QNAME: $queryname" set header [ binary format c12 "0 0 1 0 0 1 0 0 0 0 0 0" ] # FORMAT QNAME set name "" foreach { field } [split $queryname .] { set strlen [ string length $field ] append name [ binary format c $strlen ] append name [ binary format a* $field ] } # QNAME ends with NULL append name [ binary format c "0" ] # QTYPE: A append name [ binary format S 1 ] # QCLASS IN append name [ binary format S 1 ] binary scan "$header$name" H* p log local0.debug "Payload: $p" set conn_id [connect -protocol UDP -timeout 100 -idle 30 -status conn_status /Common/dns-udp] log local0.debug "Connection status: $conn_status" log local0.debug "Payload B64: [b64encode [HTTP::payload]]" set send_bytes [send -timeout 100 -status send_status $conn_id [binary format H* $p] ] log local0.debug "send_status: $send_status" recv -timeout 100 -peek -status recv_status $conn_id result log local0.debug "status: $recv_status Result B64: [b64encode $result]" HTTP::respond 200 content $result noserver content-type application/dns-message vary Accept-Encoding content-length [string length $result] } else { HTTP::respond 415 content "Unsupported Media Type" } } else { HTTP::respond 404 } } when HTTP_REQUEST_DATA { # Use sideband set conn_id [connect -protocol UDP -timeout 100 -idle 30 -status conn_status /Common/dns-udp] log local0.debug "Connection status: $conn_status" log local0.debug "Payload B64: [b64encode [HTTP::payload]]" set send_bytes [send -timeout 100 -status send_status $conn_id [HTTP::payload]] log local0.debug "send_status: $send_status" recv -timeout 100 -peek -status recv_status $conn_id result log local0.debug "status: $recv_status Result B64: [b64encode $result]" HTTP::respond 200 content $result noserver content-type application/dns-message vary Accept-Encoding content-length [string length $result] }
Tested below:
root@DoH-xubuntu-desktop1:~# curl -k -H 'content-type: application/dns-message' -o output_file_get https://10.10.10.10/dns-query?dns=d3d3LmV4YW1wbGUuY29t % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 116 100 116 0 0 974 0 --:--:-- --:--:-- --:--:-- 974
If you are sharp-eyed you will notice that i made a mistake on this - I only sent the encoded name, not an encoded DNS wireformat packet.
For reference, see below my notes on how to create the DNS wireformat packet for www.example.com ( which the RFC uses ):
00 00 01 00 00 01 00 00 00 00 00 00 03 77 77 77 07 65 78 61 6d 70 6c 65 03 63 6f 6d 00 00 01 00 01
DNS Header
ID = 00 00
Recursion Available = 01 00
QCOUNT = 00 01
ANSCOUT = 00 00
NSCOUNT = 00 00
ARCOUNT = 00 00
DNS Query
www
LENGTH = 03
FIELD = 77 77 77 ( www )
example
LENGTH = 07 ( 7 octets )
FIELD = 65 78 61 6d 70 6c 65 ( example )
com
LENGTH = 03
FIELD = 63 6f 6d ( com )
NULL LABEL = 00 ( end of question name )
QTYPE = 00 01 = A
QCLASS = 00 01 = IN
I modified this to create a v1.0 which worked adequately as a proof of concept. In testing, I saw the following statistics for 1000 consecutive POST requests:
[root@DoH-bigip1:Active:Standalone] config # tmsh show ltm rule DoH --------------------------------- Ltm::Rule Event: DoH:HTTP_REQUEST --------------------------------- Priority 500 Executions Total 1.0K Failures 0 Aborts 0 CPU Cycles on Executing Average 278.8K Maximum 6.1M Minimum 147.8K -------------------------------------- Ltm::Rule Event: DoH:HTTP_REQUEST_DATA -------------------------------------- Priority 500 Executions Total 1.0K Failures 0 Aborts 0 CPU Cycles on Executing Average 644.3K Maximum 6.3M Minimum 391.1K ------------------------------ Ltm::Rule Event: DoH:RULE_INIT ------------------------------ Priority 500 Executions Total 0 Failures 0 Aborts 0 CPU Cycles on Executing Average 0 Maximum 0 Minimum 0
Over the subsequent weeks and months, my colleagues gave me feedback on issues or suggestions, and i created a GUI for the iRule with an iApp.
The first issue was that it kept generating errors in the base64 decoding command. This is because base64 and base64url are subtly different. To cure this problem, I spent a few hours looking in detail at when it fails and what the difference between base64 and base64url would be, along with some detailed reading. The upshot is that I added a b64urldecode procedure to handle this:
proc b64urldecode { str } { # Procedure to perform base64url decoding set str [ string map { - + _ \/ } $str] set pad [expr {[string length $str] % 4}] if { $pad != 0} { append str [string repeat = [expr {4 - $pad}]] } if { [ catch { b64decode $str } decodedstr ] == 0 and $decodedstr != "" } { return $decodedstr } else { log local0.err "Base64 decoding error for string $str" return "" } }
One of my colleagues Brian also did some capacity testing to look at different options and what the highest performance is - through this we realised that going directly to the servers is more performant than sending to a local virtual server, and I also changed the code so that rather than sending, waiting a fixed period and checking the result, the iRule would check in a shorter loop.
From this:
set send_bytes [send -timeout 100 -status send_status $conn_id [HTTP::payload]] log local0.debug "send_status: $send_status" recv -timeout 100 -peek -status recv_status $conn_id result
to this:
set send_bytes [send -timeout 100 -status send_status $conn_id $payload ] log local0.debug "Sent $send_bytes Bytes to $member" for {set i 0} {$i < $totalLoops} {incr i} { recv -timeout $recvTimeout -status recv_status $conn_id result if { $recv_status == "received" } { break } }
which gave the best performance. You can see the configuration options around this in the iApp so you can set the total timeout and the checking period.
iApp
You can see that this iApp creates a simple GUI to allow the modification of the iRule - to add logs, change the destination, timers etc. This is common for us to create to allow simple operation of the iRule as part of the system.
Closing Thoughts
If I wanted to make this truly scalable, I would consider using Message Routing Framework (MRF) which decouples the message ( ie the DNS request ) from the underlying transport, meaning that we can send wherever and however required, duplicate messages, respond locally or anything else which takes your fancy. As i'm sure you know, the beauty of the BIG-IP platform is the ability to use these features to build any solution required in a short timescale.
Hopefully this has given you an insight into what we in F5 Professional Services regularly do, and if you have any questions or suggestions about the DNS over HTTPS iApp or more general Professional Services questions then feel free to message me.