cancel
Showing results for 
Search instead for 
Did you mean: 
Login & Join the DevCentral Connects Group to watch the Recorded LiveStream (May 12) on Basic iControl Security - show notes included.
Eric_Chen
F5 Employee
F5 Employee

When CloudFlare took over 1.1.1.1 for their DNS service; this got me thinking about a couple of issues:

    • A. What do you do if you’ve been using 1.1.1.1 on your network, how do you unbreak the Internet?
    • B. How can you enable use of DNS over TLS for clients that don’t have native support?
    • C. How can you enable use of DNS over HTTPS for clients that don’t have native support?

      A. Unbreaking the Internet

      RFC1918 lays out “private” addresses that you should use for internal use, i.e. my home network is 192.168.1.0/24. But, we all have issues where sometimes we “forget” about the RFC and you are using routable IP space on your internal network. 1.1.1.1 is a good example of this, it’s just so nice looking and easy to type.

      In my home network I created a contrived situation to simulate breaking the Internet; creating a network route on my home router that sends all traffic for 1.1.1.1/32 to a Linux host on my internal network. I imagined a situation where I wanted to continue to send HTTP traffic to my own server, but send DNS traffic to the “real” 1.1.1.1.


      0151T000003d7AyQAI.png

      In the picture above, I have broken the Internet. It is now impossible for me to route traffic to 1.1.1.1/32.  To “fix” the Internet (image below), I used a F5 BIG-IP to send HTTP traffic to my internal server and DNS traffic to 1.1.1.1.

      0151T000003d7AzQAI.png


      The BIG-IP configuration is straight-forward. 1.1.1.1:80 points to the local server and 1.1.1.1:53 points to 1.0.0.1.  Note that I have used 1.0.0.1 since I broke routing to 1.1.1.1 (1.0.0.1 works the same as 1.1.1.1).

      0151T000003d7B0QAI.png

      B. DNS over TLS

      After fixing the Internet, I started to think about the challenge of how to use DNS over TLS. RFC7858 is a proposed standard for wrapping DNS traffic in TLS. Many clients do not support DNS over TLS unless you install additional software. I started to wonder, is it possible to have a BIG-IP “upgrade” a standard DNS request to DNS over TLS? There are two issues.

      1. DNS over TLS uses TLS (encryption)
      2. DNS over TLS uses TCP (most clients default to UDP unless handling large DNS requests)

      For the first issue, the BIG-IP can already wrap a TCP connection with TLS (often used in providing SSL visibility to security devices that cannot inspect SSL traffic, BIG-IP terminates SSL connection, passes traffic to security device, re-encrypts traffic to final destination).

      The second issue can be solved with configuring BIG-IP DNS as a caching DNS resolver that accepts UDP and TCP DNS requests and only forwards TCP DNS requests. This results in an architecture that looks like the following.

      0151T000003d7B1QAI.png

      The virtual server is configured with a TCP profile and serverssl profile.

      0151T000003d7B2QAI.png

      The serverssl profile is configure to verify the authenticity of the server certificate.

      0151T000003d7B3QAI.png

      The DNS cache is configured to use the virtual server that performs the UDP to TCP over TLS connection.

      0151T000003d7B4QAI.png

      .wlemoticon { behavior: url(#default#.WLEMOTICON_WRITER_BEHAVIOR) } img { behavior: url(#default#IMG_WRITER_BEHAVIOR) } .wlwritereditablesmartcontent { behavior: url(#default#.WLWRITEREDITABLESMARTCONTENT_WRITER_BEHAVIOR) } .wlwritersmartcontent, .wlwriterpreserve { behavior: url(#default#.WLWRITERSMARTCONTENT,_.WLWRITERPRESERVE_WRITER_BEHAVIOR) } .wlwritereditablesmartcontent > .wleditfield { behavior: url(#default#.WLWRITEREDITABLESMARTCONTENT_>_.WLEDITFIELD_WRITER_BEHAVIOR) } blockquote { behavior: url(#default#BLOCKQUOTE_WRITER_BEHAVIOR) } #extendedentrybreak { behavior: url(#default##EXTENDEDENTRYBREAK_WRITER_BEHAVIOR) } .postbody table { behavior: url(#default#.POSTBODY_TABLE_WRITER_BEHAVIOR) } .postbody td { behavior: url(#default#.POSTBODY_TD_WRITER_BEHAVIOR) } .postbody th { behavior: url(#default#.POSTBODY_TH_WRITER_BEHAVIOR) } .posttitle {margin: 0px 0px 10px 0px; padding: 0px; border: 0px;} .postbody {margin: 0px; padding: 0px; border: 0px; min-height: 400px;}

      A bonus of using the DNS cache is that subsequent DNS queries are extremely fast!

      C. DNS over HTTPS

      I felt pretty good after solving these two issues (A and B), but there was a third nagging issue (C) of how to support DNS over HTTPS. If you have ever looked at DNS traffic it is a binary format over UDP or TCP that is visible on the network. HTTPS traffic is very different in comparison; TCP traffic that is encrypted.

      The draft RFC (draft-ietf-doh-dns-over-https-05) has a handy specification for “DNS Wire Format”. Take a DNS packet, shove it into an HTTPS message, get back a DNS packet in a HTTPS response. Hmm, I bet an iRule could solve this problem of converting DNS to HTTPS.

      Using iRulesLX I created a process where a TCL iRule takes a UDP payload off the network from a DNS client, sends it to a Node.JS iRuleLX extension that invokes the HTTPS API, returns the payload to the TCL iRule that sends the result back to the DNS client on the network.

      The solution looks like the following.

      0151T000003d7B5QAI.png

      This works on my home network, but I did notice occasional 405 errors from the HTTPS services while testing.

      The TCL iRule is simple, grab a UDP payload.

      when CLIENT_DATA {
      
        set payload [UDP::payload]
        set encoded_payload [b64encode $payload]
      
        set RPC_HANDLE [ILX::init dns_over_https_plugin dns_over_https]
        set rpc_response [ILX::call $RPC_HANDLE query_dns $encoded_payload]
        UDP::respond [b64decode $rpc_response]
      
      }
      

      The Node.JS iRuleLX Extension makes the API call to the HTTPS service.

      ...
      ilx.addMethod('query_dns', function(req, res) {
         var dns_query = atob(req.params()[0]);
      var options = {
        host: '192.168.1.254',
        port: 80,
        path: '/dns-query?ct=application/dns-udpwireformat&dns=' + req.params()[0],
        method: 'GET',
        headers: {
         'Host':'cloudflare-dns.com'  
        }
      };
      var cfreq = http.request(options, function(cfres) {
        var output = "";
        cfres.setEncoding('binary');
        cfres.on('data', function (chunk) {
          output += chunk;
        });
        cfres.on('end', () => {  
         res.reply(btoa(output));
        });
      });
      cfreq.on('error', function(e) {
        console.log('problem with request: ' + e.message);
      });
      cfreq.write(dns_query);
      cfreq.end(); 
      });
      ...
      

      After deploying the iRule; you can update your DNS cache to point to the Virtual Server hosting the iRule.

      Note that the Node.JS code connects back to another Virtual Server on the BIG-IP that does a similar upgrade from HTTP to HTTPS and uses a F5 OneConnect profile.  You could also have it connect directly to the HTTPS server and recreate a new connection on each call.

      All done?

      Before I started I wasn’t sure whether I could unbreak the Internet or upgrade old DNS protocols into more secure methods, but F5 BIG-IP made it possible. Until the next time that the Internet breaks.

      Comments

      Hi,

       

      Very good article. But I have to admit it's quite a challenge for me to figure out everything described. For example resolving first issue. My assumptions for setup involving BIG-IP are:

       

      • Internet router is still configured with static route 1.1.1.1/32 gw 192.168.1.66 (probably 1.1.1.1 is configured on Linux loopback here?)
      • You actually cheating a bit 🙂 as with above config you are really not able to send DNS request to 1.1.1.1 on the Internet, sure if 1.0.0.1 is as well DNS it's working but what if not? With such router setup it's not possible to really send traffic destined to 1.1.1.1 to the Internet - or I am missing something here?

      DNS over TLS

       

      Sorry for lame question but I am by far no DNS expert. As far as I understand DNS caches on BIG-IP for your setup two VS are required:

       

      • VS (DNS_vs - my name here) to accept actual client DNS request (over UDP?) with DNS profile attached. Profile has DNS cache enabled with your resolver cache selected (CF)
      • VS (dns_over_tls) doing TLS encryption (one configured in resolver cache Forwarding Zones tab)

      Now how UDP is converted to TCP? dns_over_tls VS is TCP protocol, so it will not accept UDP packets. Will DNS_vs respond to clients requesting resending DNS queries over TCP? Some other way?

       

      DNS over HTTPS

       

      This is hard because I am not familiar with both RFC and JavaScript (iRuleLX - know the basic but never used).

       

      I am not sure how previously configured (192.168.1.254) is used by iRuleLX - as proxy to send HTTPS request to cloudflare-dns.com?

       

      So rule is just generating HTTP request with DNS binary (base64 encoded) inserted as HTTP payload and VS is just encrypting it and sending to cloudflare? Then response from cloudflare is decrypted by VS returned back to rule, HTTP payload extracted (being binary DNS response) and returned to iRule and then to requesting client - is that more or less how it works?

       

      Is the same configuration as for DNS over TLS used by 192.168.1.254 VS or it has to be updated? What actually is in the VS Pool - list of DNS servers accepting TLS (or HTTPS) encrypted DNS requests?

       

      Piotr

       

      Eric_Chen
      F5 Employee
      F5 Employee

      Piotr,

       

      Great observations!

       

      A. Unbreaking the Internet:

       

      Re: do I use 1.1.1.1/32 on loopback on the server?

       

      Yes. Not mentioned in the article I also use DSR on the BIG-IP to reduce one hop of traffic.

       

      Traffic flow: client -> home router -> BIG-IP -> server -> client

      Re: cheating

       

      I am COMPLETELY cheating. In my home setup; once I changed my home router to point to the BIG-IP there's no way for the BIG-IP to connect back out (creates a routing loop). My only option was to point to 1.0.0.1. In an environment where the BIG-IP had a direct interface to the Internet you could route directly to 1.1.1.1.

       

      Traffic flow: Client -> Internal Switch -> BIG-IP Internal Interface -> BIG-IP External Interface -> Internet

      B. DNS over TLS

       

      Re: How does UDP become TCP

       

      The trick is that when you have a DNS profile that references a cache, you are turning the BIG-IP into another DNS client (initiates a completely new connection). So the flow is:

       

      Client DNS Request (UDP request) -> [BIG-IP DNS] DNS_vs -> BIG-IP Cache DNS Request (TCP request) -> [BIG-IP DNS] dns_over_tls -> BIG-IP dns_over_tls (TCP/TLS request) -> [DNS over TLS (TCP port 853)]

      C. DNS over HTTPS

       

      This is bit convoluted, I did use the same IP address/port that adds to the confusion.

       

      Your description is spot-on. There is a separate virtual server from the previous example. This is a case where you have:

       

      192.168.1.254:53 TCP -> DNS over TLS (just performing TLS upgrade) 192.168.1.254:53 UDP -> DNS over HTTPS (performing conversion from UDP to HTTPS requests)

      Somewhat like how Google uses UDP 443 for QUIC vs. TCP 443 for HTTPS; I am re-using the same port for two DIFFERENT services.

       

      I meant for the article to be a progression of challenges. You did very well!

       

      Eric

       

      Eric_Chen
      F5 Employee
      F5 Employee

      One more VS that I didn't mention, here's the complete list (separated for clarity):

       

      1.1.1.1:53 UDP -> To 1.0.0.1 1.1.1.1:80 HTTP -> To server 192.168.1.113:53 UDP -> DNS Cache (DNS over TLS) 192.168.1.114:53 UDP -> DNS Cache (DNS over HTTPS) 192.168.1.254:53 TCP -> DNS over TLS (just performing TLS upgrade) 192.168.1.254:53 UDP -> DNS over HTTPS (performing conversion from UDP to HTTPS requests) 192.168.1.253:80 TCP -> HTTP to HTTPS (validate HTTPS server certificate, upgrade from HTTP to HTTPS)

      In the article I only reference a single DNS cache, but the above example assumes that you use two separate caches (one configured for only clientside UDP and the other only for clientside TCP)

       

      In my article the corresponding pools

       

      1.1.1.1:53 UDP -> pool: 1.0.0.1 1.1.1.1:80 HTTP -> pool: 192.168.1.66 192.168.1.113:53 UDP -> DNS cache forward . -> 192.168.1.254:53 TCP 192.168.1.114:53 UDP -> DNS cache forward . -> 192.168.1.254:53 UDP 192.168.1.254:53 TCP -> 1.0.0.1:853 (w/ serverssl profile) 192.168.1.254:53 UDP -> iRule + iRuleLX that calls 192.168.1.253:80 192.168.1.253:80 TCP -> FQDN pool to cloudflare-dns.com

      Hi,

       

      Thanks a lot for explanations. I have to dig in to be sure if I got it now. Keep going with articles like that. Plenty of nice tricks I would never figure out by myself.

       

      Piotr

       

      Karim
      Nimbostratus
      Nimbostratus

      After reading this very good article, I wanted to implement DNS over TLS using the BIG-IP in my lab environment. I decided to write this comment in case anyone is having the same trouble that I had and to make sure that I got everything correct. 

       

      So, The idea is the following: 

      • configure a simple virtual server listenning on port UDP 53 (dns_listener) . Assign to that virtual server a DNS Profile that in turn references a Cache Resolver profile
      • configure the Cache Resolver profile to send DNS queries in TCP only
      • configure a simple virtual server (kabe_vs_DNSoTLS) that listens on TCP port 853 and assign a Pool and SSL server profile to it. 
      • configure the Cache Resolver profile to forward the zone "." (everything) to the IP address of the TCP virtual server (kabe_vs_DNSoTLS)

       

      0691T000005nqpxQAA.jpg

       

      please find below my configuration. Feel free to ask question or to give me suggestions.

      ltm dns cache resolver /Common/kabe_cache_resolver { forward-zones { . { nameservers { 10.10.1.51:53 { } } } } route-domain /Common/0 use-udp no }   ltm profile dns /Common/kabe_dns_profile { app-service none cache /Common/kabe_cache_resolver defaults-from /Common/dns enable-cache yes }     ltm virtual /Common/dns_listener { creation-time 2019-11-25:21:06:30 destination /Common/10.10.1.50:53 ip-protocol udp last-modified-time 2019-11-25:22:42:12 mask 255.255.255.255 profiles { /Common/kabe_dns_profile { } /Common/udp_gtm_dns { } /* this is a standard UDP layer 4 profile */ } source 0.0.0.0/0 translate-address disabled translate-port disabled }   ltm pool /Common/kabe_DNSoTLS { members { /Common/1.1.1.1:853 { address 1.1.1.1 } } monitor /Common/gateway_icmp }     ltm virtual /Common/kabe_vs_DNSoTLS { creation-time 2019-11-25:21:14:12 destination /Common/10.10.1.51:53 ip-protocol tcp last-modified-time 2019-11-25:23:44:16 mask 255.255.255.255 pool /Common/kabe_DNSoTLS profiles { /Common/f5-tcp-lan { context clientside } /Common/f5-tcp-wan { context serverside } /Common/serverssl { /*I used the default Server SSL profile */ context serverside } } source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled }

      hope it helps,

      Many thanks

      Version history
      Last update:
      ‎09-Apr-2018 04:00
      Updated by:
      Contributors