cancel
Showing results for 
Search instead for 
Did you mean: 

Introduction

If you've ever used the old Linux Squid proxy or F5's Secure Gateway solution, you might be familiar with the existence of HTTP Explicit Proxy. If all you're looking for is to configure it ASAP, there's an iApp available to set things up in no time. This technology is also used by F5's Secure Web Gateway Services (SWG). This article will walk you through the mechanics behind the scenes, i.e. how to manually configure it and understand how it works through a lab test.

1. Lab Scenario

Expected result is that client (.135) sends HTTP GET request to server2.rodrigo.example using 10.199.3.100/32 as http proxy and BIG-IP queries our DNS server (172.16.199.31) for server2.rodrigo.example's IP address, retrieves web page and passes on to client.


0151T000002dipZQAQ.png

2. How Explicit Proxy works

In HTTP Explicit proxy, we configure our client (application) to point to BIG-IP's virtual server which will act as an HTTP proxy for external websites. Once client issues a request, BIG-IP will then open a separate connection with external server the client is trying to connect to and issue the requests on behalf of client passing requests/responses back and forth between client and external server.

In order to test HTTP Explicit proxy functionality, I configured my client's browser pointing to BIG-IP's HTTP Explicit proxy virtual server. BIG-IP is supposed to issue the requests on behalf of client and pass them back to client. The client is explicitly configured to use VIP's IP address as a proxy. Here's my config on Firefox (Preferences → Advanced → Network):

0151T000002dipeQAA.png

When using any form of unencrypted traffic Firefox forwards request straight to BIG-IP without issuing an HTTP CONNECT request:

0151T000002dipjQAA.png

We can see that BIG-IP just forwards request mostly as it is. The reason for GET / instead of FQDN on server-side is because BIG-IP performed DNS resolution before that and I filtered it.

If request is HTTPS instead of HTTP, things are slightly different. First, Firefox (my browser) tries to establish an HTTP tunnel (using CONNECT HTTP method) to https://server2.rodrigo.example, BIG-IP then immediately establishes TCP connection with remote-destination (after DNS lookup of course!). Lastly, once TCP connection is established with remote-destination, BIG-IP responds (to client) with 200 Connected signalling tunnel is successfully established and BIG-IP is ready to forward any requests through recently established tunnel:

0151T000002dipoQAA.png

Now that tunnel is properly established, BIG-IP just forwards whatever Firefox sends. In this case, the SSL handshake Client Hello was the first packet which was captured:

0151T000002diptQAA.png

It is interesting to note that there is still an HTTP header in the messages between client and BIG-IP with proxy information sent by firefox (frame 47) that encapsulates SSL message:

0151T000002dipyQAA.png

Now that we understand how HTTP Explicit proxy works under the hood, we're ready to learn how to set up BIG-IP as Explicit proxy.

3. Setting Up Explicit proxy on BIG-IP

3.1 Create Virtual Server with no pool (you can leave http profile for later):

0151T000002diq3QAA.png

3.2 Create DNS resolver:

0151T000002diq8QAA.png

In the GUI, it would be on Network → DNS Resolvers → Create. Then, we'd click on Forward Zones and add a dot '.' if we want all queries to be sent to this name server. For the purposes of this specific test, I could also have named this zone as rodrigo.example. instead.

0151T000002dipzQAA.png

Note: In my lab test I worked out that forward zone's name is really important! When I named it 'testing' or 'google' BIG-IP did not forward DNS query to 172.16.199.31. However, when I named it either . or rodrigo.example. then DNS resolution worked.

3.3 Create tunnel interface

BIG-IP already has a default http-tunnel interface. As long as encapsulation type is tcp-forward, we can either stick to default or create a new one. I decided to create a new one for the purpose of this lab test:

0151T000002diqDQAQ.png

3.4. Create http profile using http-explicit as parent and assign it to VIP created previously:

Note: When configuring it from GUI just set proxy type to explicit and http-explicit will be set to parent profile.

0151T000002dippQAA.png

The equivalent in the GUI:

0151T000002diqIQAQ.png

3.5. Testing Connection

0151T000002diq0QAA.png

3.6 Enabling encrypted requests

Initially I did not understand why HTTP traffic worked when default-connect-handling was set to deny but HTTPS did not. Below is the summary of my tests:

0151T000002diqJQAQ.png

Looking at the packet capture, I noticed that BIG-IP was explicitly denying Firefox's request to establish a tunnel as shown below:

0151T000002diqNQAQ.png

Note: For this test I created another Virtual Server called using a different address 10.199.3.101 instead of the one I originally tested just to test default-connect-handling.

After reading K40243113: Overview of the HTTP profile, I kind of understood the point:

"indicates that outbound requests are delivered only if another virtual server is listening on the tunnel for the requested outbound connection. With this setting, virtual servers are required, and the system processes the outbound traffic before it leaves the device."

I then created another VIP with same destination IP address and port as back-end server client was trying to connect to just to confirm if this time traffic would go through:

0151T000002diqEQAQ.png

And indeed it worked:

0151T000002diq1QAA.png

I also tested with a wildcard virtual server listening on *:443 instead of 172.16.199.32/32 and it worked fine as well.

What we have observed so far is that when default-connect-handling is set to denyin order for encrypted client traffic to be accepted by BIG-IP and proxy tunnel established, the destination address that matches the one on client request has to have a listener on BIG-IP for return traffic. Therefore, we can conclude that default-connect-handling setting governs what traffic is accepted by the tunnel interface, but unencrypted requests are not affected.

The below picture sums up the explanation:

0151T000002diqFQAQ.png

default-connect-handling just adds the extra step to validate whether there is a listener on BIG-IP for destination address:port or not. If not, we deny request. If yes, request goes through.

Comments

Hi,

Nice recap, glad you reminded me about default-connect-handling. Just wonder if you can pass any ideas what this additional VS can be used for. In config like yours it's actually not doing anything useful (or I am wrong?) so adding it is just introducing (even if minimal) some latency.

My first guess is that it allows you to control what port can be used by client, so if someone will try 8443 it will be blocked - can't recall if I tested it but I Am pretty sure it will work like that. Or maybe it requires adding Reject type VS to the same tunnel?

Anyway, just to make it clear, if you do not care about stuff like that (or any other traffic mods) there is really no point in using deny setting for default-connect-handling - or there is?

Piotr

Hey Piotr,

 

The way I see it is that it's an embedded option for simple external filtering. The advantage is that it doesn't even let traffic leave client-side and immediately blocks the request. Depending on your needs, adding the extra VIP might be beneficial for your network. You don't have to use it and I believe most people might not even bother using it but it's there if you need it. It allows you to control whether or not the host the client is trying to connect to will be allowed by the proxy or not when it's set to deny. If you set it to deny, you need to create a corresponding listener for return traffic, otherwise traffic is blocked by default in the beginning.

rob_carr
MVP
MVP

Hi Rodrigo,

The default-connect-handling setting appears to govern what traffic is accepted by the tunnel interface, but unencrypted requests don't use the tunnel (which is why HTTP worked while HTTPS did not).

 

If that's the case isn't it more accurate to say 'in order for an encrypted proxy request to be accepted...'?

 

Great article.

 

Hi Rob, you're absolutely right and you came up with a much better explanation/conclusion. I've updated the article based on what you pointed out.

 

Many thanks!! Cheers.

Ajit
Altostratus
Altostratus

Hi  

 

Great article and very well explained!

 

I have successfully tested this in my setup & works flawlessly. Just one questions for you:

 

When I set this up first, I unknowingly set the forward name to "DNSTest" and it didn't work at all. Later I realized my mistake, deleted that forward zone & setup a new forward zone with (dot) and the correct server IP's. However, post that correction the DNS resolvers just wouldn't do any DNS resolution whatsoever, not sure why that behavior. Ultimately, I had to re-spin the virtual F5 & follow the correct steps to make this work.

 

Any idea how to fix it when you have accidentally given the wrong forward zone name & corrected it later without having to re-spin the F5?

 

Regards,

Ajit

Hey Ajit, Glad you liked the article. I don't see how you can do that via GUI. You can (unofficially) edit bigip.conf on CLI, manually change from "DNSTest" to "." and then issue tmsh load sys config followed by tmsh save sys config. Cheers.

Hmm, you can just go to DNS resolver, delete forward name (not sure if this is right name for feature), enter new and save DNS resolver. Sure load sys config from-terminal merge can solve most but no reason here to use it - at least in my opinion, and only if I did understand the issue 😉

Thanawoot_SONGT
F5 Employee
F5 Employee

Thanawoot_SONGT_0-1652253758620.png

Seem typo on https_vs? Port should be :443 instead of :3128

If not typo, quite interesting how traffic match the listener.

Version history
Last update:
‎11-May-2020 03:49
Updated by:
Contributors