DNS over HTTPS Resolution

Introduction

 

I work in the EMEA Professional Services team and in this article I will walk you through a recent solution which I provided to support the F5 Sales team for a Service Provider. You can see the finished iApp here which took a few days of work spread over a few months.

 

This solution is to provide DNS over HTTP resolution - F5 has this as a feature soon to be released but in the meantime we needed a working solution.

 

Development

I started by looking at the RFC ( https://tools.ietf.org/html/rfc8484 ) to work out what was required and realised that this has two methods - GET and POST, it gives examples of the payload and response so I put together a simple lab and proof of concept. This used curl to send the data - both methods send a DNS wire-format packet, GET base64url-encodes it and sends it as a parameter while POST sends it in the payload directly.

 

I decided it would be simplest to use an iRule with sideband to a local virtual server which would in turn send the traffic to the backend server. Below is the simple pseudocode I created to guide me:

 

when HTTP_REQUEST
  if URI 
    if method == POST
      collect payload
    if method == GET
      decode payload
      send payload via sideband
      return response
endwhen
when HTTP_DATA
  send payload via sideband
  return response
endwhen

 

From this you can see that I have 'send payload via sideband' in two places. I like to follow the DRY principle so i immediately thought about procedures in later versions.

 

I have used sideband a number of times before but i reviewed how it works in terms of destination and source address at https://clouddocs.f5.com/api/irules/SIDEBAND.html . For base64url, I decided to start with the built-in base64 decoding function ( https://clouddocs.f5.com/api/irules/b64decode.html ) and use string map to change the erroneous characters.

 

 

See my lab configuration below with my first version of the iRule:

 

[root@DoH-bigip1:Active:Standalone] config # tmsh list ltm pool
ltm pool dns {
    members {
        server-1:domain {
            address 10.20.20.6
        }
    }
}
[root@DoH-bigip1:Active:Standalone] config # tmsh list ltm virtual
ltm virtual DoH {
    creation-time 2020-05-04:03:17:08
    destination 10.10.10.10:https
    ip-protocol tcp
    last-modified-time 2020-05-04:03:33:22
    mask 255.255.255.255
    profiles {
        clientssl-insecure-compatible {
            context clientside
        }
        http { }
        tcp { }
    }
    rules {
        DoH
    }
    serverssl-use-sni disabled
    source 0.0.0.0/0
    translate-address enabled
    translate-port enabled
    vlans {
        External
    }
    vlans-enabled
    vs-index 4
}
ltm virtual dns-tcp {
    creation-time 2020-05-04:03:15:46
    destination 10.10.10.10:domain
    ip-protocol tcp
    last-modified-time 2020-05-04:03:15:46
    mask 255.255.255.255
    pool dns
    profiles {
        dns { }
        tcp { }
    }
    serverssl-use-sni disabled
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    translate-address enabled
    translate-port enabled
    vlans {
        External
    }
    vlans-enabled
    vs-index 3
}
ltm virtual dns-udp {
    creation-time 2020-05-04:03:14:25
    destination 10.10.10.10:domain
    ip-protocol udp
    last-modified-time 2020-05-04:03:14:25
    mask 255.255.255.255
    pool dns
    profiles {
        dns { }
        udp { }
    }
    rules {
        dns-log
    }
    serverssl-use-sni disabled
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    translate-address enabled
    translate-port enabled
    vlans {
        External
    }
    vlans-enabled
    vs-index 2
}
ltm rule DoH {
when RULE_INIT {
    set static::sideband_virtual_server "/Common/dns-udp"
}
when HTTP_REQUEST {
    # https://tools.ietf.org/html/rfc8484
    if { [HTTP::uri] == "/dns-query" } {
        if { (([HTTP::method] equals "POST") and (([HTTP::header "accept"] equals "application/dns-message") or ([HTTP::header "content-type"] equals "application/dns-message"))) } { 
            if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 65535 && [HTTP::header "Content-Length"] > 0} { 
                log local0.debug "Content-Length [HTTP::header value "Content-Length"]"
                HTTP::collect [HTTP::header value "Content-Length"] 
            } else { 
                HTTP::collect 100
            } 
        } else {
            HTTP::respond 415 content "Unsupported Media Type"
        }
    } else {
        HTTP::respond 404
    }
}

when HTTP_REQUEST_DATA {
    # Use sideband
    set conn_id [connect -protocol UDP -timeout 100 -idle 30 -status conn_status /Common/dns-udp]
    log local0.debug "Connection status: $conn_status"
    log local0.debug "Payload B64: [b64encode [HTTP::payload]]"
    set send_bytes [send -timeout 100 -status send_status $conn_id [HTTP::payload]]
    log local0.debug "send_status: $send_status"
    recv -timeout 100 -peek -status recv_status $conn_id result
    log local0.debug "status: $recv_status Result B64: [b64encode $result]"
    HTTP::respond 200 content $result noserver content-type application/dns-message vary Accept-Encoding content-length [string length $result] 
}
}

ltm rule dns-log {
when DNS_REQUEST {
log local0.debug "[DNS::question name] [DNS::question class] [DNS::question type]"
}
when DNS_RESPONSE {
set rrs [DNS::answer]
     foreach rr $rrs {
         log local0. "[DNS::rdata $rr]"
     }
}
}

 

I tested this setup using the following procedure:

 

• Create DNS wire request in binary file

echo "00 00 01 00 00 01 00 00 00 00 00 00 03 77 77 77 07 65 78 61 6d 70 6c 65 03 63 6f 6d 00 00 01 00 01"|xxd -r -p >example_req

 

• Perform curl request

curl -k -H 'content-type: application/dns-message' --data-binary @example_req -o output_file https://10.10.10.10/dns-query

 

• Inspect output file

root@DoH-xubuntu-desktop1:~# xxd output_file
00000000: 0000 8580 0001 0001 0001 0002 0377 7777 .............www
00000010: 0765 7861 6d70 6c65 0363 6f6d 0000 0100 .example.com....
00000020: 01c0 0c00 0100 0100 093a 8000 0401 0203 .........:......
00000030: 04c0 1000 0200 0100 093a 8000 0b09 6c6f .........:....lo
00000040: 6361 6c68 6f73 7400 c03d 0001 0001 0009 calhost..=......
00000050: 3a80 0004 7f00 0001 c03d 001c 0001 0009 :........=......
00000060: 3a80 0010 0000 0000 0000 0000 0000 0000 :...............
00000070: 0000 0001                               ....

 

My BIG-IP logs:

 

May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST>: Content-Length 33
May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: Doing sideband
May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: Connection status: connected
May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: Payload B64: AAABAAABAAAAAAAAA3d3dwdleGFtcGxlA2NvbQAAAQAB
May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: send_status: sent
May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/dns-log <DNS_REQUEST>: www.example.com IN A
May 4 05:49:12 DoH-bigip1 info tmm[22188]: Rule /Common/dns-log <DNS_RESPONSE>: 1.2.3.4
May 4 05:49:12 DoH-bigip1 debug tmm[22188]: Rule /Common/DoH <HTTP_REQUEST_DATA>: status: received Result B64: AACFgAABAAEAAQACA3d3dwdleGFtcGxlA2NvbQAAAQABwAwAAQABAAk6gAAEAQIDBMAQAAIAAQAJOoAACwlsb2NhbGhvc3QAwD0AAQABAAk6gAAEfwAAAcA9ABwAAQAJOoAAEAAAAAAAAAAAAAAAAAAAAAE=

 

I then added GET resolution:

 

when RULE_INIT {
   set static::sideband_virtual_server "/Common/dns-udp"
}

when HTTP_REQUEST {
   # https://tools.ietf.org/html/rfc8484
   if { [HTTP::path] == "/dns-query" } {
       if { (([HTTP::method] equals "POST") and (([HTTP::header "accept"] equals "application/dns-message") or ([HTTP::header "content-type"] equals "application/dns-message"))) } { 
           if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 65535 && [HTTP::header "Content-Length"] > 0} { 
               log local0.debug "Content-Length [HTTP::header value "Content-Length"]"
               HTTP::collect [HTTP::header value "Content-Length"] 
           } else { 
               HTTP::collect 100
           } 
       } elseif { [HTTP::method] equals "GET" && [URI::query [HTTP::uri] dns] != "" } {
           set queryname [b64decode [URI::query [HTTP::uri] dns] ]
           
           log local0.debug "QNAME: $queryname"
           set header [ binary format c12 "0 0 1 0 0 1 0 0 0 0 0 0" ]
           # FORMAT QNAME
           set name ""
           foreach { field } [split $queryname .] {
               set strlen [ string length $field ]
               append name [ binary format c $strlen ]
               append name [ binary format a* $field ]
           }
           # QNAME ends with NULL
           append name [ binary format c "0" ]
           
           # QTYPE: A
           append name [ binary format S 1 ]
           # QCLASS IN
           append name [ binary format S 1 ]
           
           binary scan "$header$name" H* p
           log local0.debug "Payload: $p"
           set conn_id [connect -protocol UDP -timeout 100 -idle 30 -status conn_status /Common/dns-udp]
           log local0.debug "Connection status: $conn_status"
           log local0.debug "Payload B64: [b64encode [HTTP::payload]]"
           set send_bytes [send -timeout 100 -status send_status $conn_id [binary format H* $p] ]
           log local0.debug "send_status: $send_status"
           recv -timeout 100 -peek -status recv_status $conn_id result
           log local0.debug "status: $recv_status Result B64: [b64encode $result]"
           HTTP::respond 200 content $result noserver content-type application/dns-message vary Accept-Encoding content-length [string length $result] 
       
       } else {
           HTTP::respond 415 content "Unsupported Media Type" 
       }
   } else {
       HTTP::respond 404
   }
}
when HTTP_REQUEST_DATA {
   # Use sideband
   set conn_id [connect -protocol UDP -timeout 100 -idle 30 -status conn_status /Common/dns-udp]
   log local0.debug "Connection status: $conn_status"
   log local0.debug "Payload B64: [b64encode [HTTP::payload]]"
   set send_bytes [send -timeout 100 -status send_status $conn_id [HTTP::payload]]
   log local0.debug "send_status: $send_status"
   recv -timeout 100 -peek -status recv_status $conn_id result
   log local0.debug "status: $recv_status Result B64: [b64encode $result]"
   HTTP::respond 200 content $result noserver content-type application/dns-message vary Accept-Encoding content-length [string length $result] 
}

 

Tested below:

 

root@DoH-xubuntu-desktop1:~# curl -k -H 'content-type: application/dns-message' -o output_file_get https://10.10.10.10/dns-query?dns=d3d3LmV4YW1wbGUuY29t
 % Total   % Received % Xferd Average Speed  Time   Time    Time Current
                                Dload Upload  Total  Spent   Left Speed
100  116 100  116   0    0   974     0 --:--:-- --:--:-- --:--:--  974

 

If you are sharp-eyed you will notice that i made a mistake on this - I only sent the encoded name, not an encoded DNS wireformat packet.

 

For reference, see below my notes on how to create the DNS wireformat packet for www.example.com ( which the RFC uses ):

 

00 00 01 00 00 01 00 00 00 00 00 00 03 77 77 77 07 65 78 61 6d 70 6c 65 03 63 6f 6d 00 00 01 00 01

DNS Header

ID = 00 00

Recursion Available = 01 00

QCOUNT = 00 01

ANSCOUT = 00 00

NSCOUNT = 00 00

ARCOUNT = 00 00

 

DNS Query

www

LENGTH = 03

FIELD = 77 77 77 ( www )

 

example

LENGTH = 07 ( 7 octets )

FIELD = 65 78 61 6d 70 6c 65 ( example )

 

com

LENGTH = 03

FIELD = 63 6f 6d ( com )

 

NULL LABEL = 00 ( end of question name )

 

QTYPE = 00 01 = A

QCLASS = 00 01 = IN

 

 

I modified this to create a v1.0 which worked adequately as a proof of concept. In testing, I saw the following statistics for 1000 consecutive POST requests:

 

[root@DoH-bigip1:Active:Standalone] config # tmsh show ltm rule DoH

---------------------------------
Ltm::Rule Event: DoH:HTTP_REQUEST
---------------------------------
Priority                   500
Executions
 Total                   1.0K
 Failures                   0
 Aborts                     0
CPU Cycles on Executing
 Average               278.8K
 Maximum                 6.1M
 Minimum               147.8K

--------------------------------------
Ltm::Rule Event: DoH:HTTP_REQUEST_DATA
--------------------------------------
Priority                   500
Executions
 Total                   1.0K
 Failures                   0
 Aborts                     0
CPU Cycles on Executing
 Average               644.3K
 Maximum                 6.3M
 Minimum               391.1K

------------------------------
Ltm::Rule Event: DoH:RULE_INIT
------------------------------
Priority                500
Executions
 Total                   0
 Failures                0
 Aborts                  0
CPU Cycles on Executing
 Average                 0
 Maximum                 0
 Minimum                 0

 

 

Over the subsequent weeks and months, my colleagues gave me feedback on issues or suggestions, and i created a GUI for the iRule with an iApp.

 

The first issue was that it kept generating errors in the base64 decoding command. This is because base64 and base64url are subtly different. To cure this problem, I spent a few hours looking in detail at when it fails and what the difference between base64 and base64url would be, along with some detailed reading. The upshot is that I added a b64urldecode procedure to handle this:

 

proc b64urldecode { str } {
    # Procedure to perform base64url decoding
    set str [ string map { - + _ \/ } $str]
    set pad [expr {[string length $str] % 4}]
    if { $pad != 0} {
        append str [string repeat = [expr {4 - $pad}]]
    }
    
    if { [ catch { b64decode $str } decodedstr ] == 0 and $decodedstr != "" } {
        return $decodedstr
    } else {
        log local0.err "Base64 decoding error for string $str"
        return ""
    }
}

 

 

One of my colleagues Brian also did some capacity testing to look at different options and what the highest performance is - through this we realised that going directly to the servers is more performant than sending to a local virtual server, and I also changed the code so that rather than sending, waiting a fixed period and checking the result, the iRule would check in a shorter loop.

 

From this:

 

set send_bytes [send -timeout 100 -status send_status $conn_id [HTTP::payload]]
log local0.debug "send_status: $send_status"
recv -timeout 100 -peek -status recv_status $conn_id result

 

to this:

 

set send_bytes [send -timeout 100 -status send_status $conn_id $payload ]
log local0.debug "Sent $send_bytes Bytes to $member"
for {set i 0} {$i < $totalLoops} {incr i} {
    recv -timeout $recvTimeout -status recv_status $conn_id result
    if { $recv_status == "received" } {
        break
    }
}

 

which gave the best performance. You can see the configuration options around this in the iApp so you can set the total timeout and the checking period.

 

iApp

 

You can see that this iApp creates a simple GUI to allow the modification of the iRule - to add logs, change the destination, timers etc. This is common for us to create to allow simple operation of the iRule as part of the system.

 

 

Closing Thoughts

 

If I wanted to make this truly scalable, I would consider using Message Routing Framework (MRF) which decouples the message ( ie the DNS request ) from the underlying transport, meaning that we can send wherever and however required, duplicate messages, respond locally or anything else which takes your fancy. As i'm sure you know, the beauty of the BIG-IP platform is the ability to use these features to build any solution required in a short timescale.

 

Hopefully this has given you an insight into what we in F5 Professional Services regularly do, and if you have any questions or suggestions about the DNS over HTTPS iApp or more general Professional Services questions then feel free to message me.