HTTP Explicit Proxy Explained in Plain English
Introduction If you've ever used the old Linux Squid proxy or F5's Secure Gateway solution, you might be familiar with the existence of HTTP Explicit Proxy. If all you're looking for is to configure it ASAP, there's an iApp available to set things up in no time. This technology is also used by F5's Secure Web Gateway Services (SWG). This article will walk you through the mechanics behind the scenes, i.e. how to manually configure it and understand how it works through a lab test. 1. Lab Scenario Expected result is thatclient (.135) sends HTTP GET request to server2.rodrigo.example using 10.199.3.100/32 as http proxy andBIG-IP queries our DNS server (172.16.199.31) for server2.rodrigo.example's IP address, retrieves web page and passes on to client. 2. How Explicit Proxy works In HTTP Explicit proxy, we configure our client (application) to point to BIG-IP's virtual server which will act as an HTTP proxy for external websites. Once client issues a request, BIG-IP will then open a separate connection with external server the client is trying to connect to and issue the requests on behalf of client passing requests/responses back and forth between client and external server. In order to test HTTP Explicit proxy functionality, I configured my client's browser pointing to BIG-IP's HTTP Explicit proxy virtual server. BIG-IP is supposed to issue the requests on behalf of client and pass them back to client. The client is explicitly configured to use VIP's IP address as a proxy. Here's my config on Firefox (Preferences→ Advanced→ Network): When using any form of unencrypted traffic Firefox forwards request straight to BIG-IP without issuing an HTTP CONNECT request: We can see that BIG-IP just forwards request mostly as it is. The reason forGET /instead ofFQDNon server-side is because BIG-IP performed DNS resolution before that and I filtered it. If request is HTTPS instead of HTTP, things are slightly different. First, Firefox (my browser) tries to establish an HTTP tunnel (using CONNECT HTTP method) tohttps://server2.rodrigo.example, BIG-IP then immediately establishes TCP connection with remote-destination (after DNS lookup of course!). Lastly, once TCP connection is established with remote-destination, BIG-IP responds (to client) with 200 Connected signalling tunnel is successfully established and BIG-IP is ready to forward any requests through recently established tunnel: Now that tunnel is properly established, BIG-IP just forwards whatever Firefox sends. In this case, the SSL handshake Client Hello was the first packet which was captured: It is interesting to note that there is still an HTTP header in the messages between client and BIG-IPwith proxy information sent by firefox (frame 47) that encapsulates SSL message: Now that we understand how HTTP Explicit proxy works under the hood, we're ready to learn how to set up BIG-IP as Explicit proxy. 3. Setting Up Explicit proxy on BIG-IP 3.1 Create Virtual Server with no pool (you can leave http profile for later): 3.2 Create DNS resolver: In the GUI, it would be on Network → DNS Resolvers → Create. Then, we'd click on Forward Zones and add a dot '.' if we want all queries to be sent to this name server. For the purposes of this specific test, I could also have named this zone asrodrigo.example.instead. Note: In my lab test I worked out that forward zone's name is really important! When I named it 'testing' or 'google' BIG-IP did not forward DNS query to 172.16.199.31. However, when I named it either . or rodrigo.example. then DNS resolution worked. 3.3 Create tunnel interface BIG-IP already has a default http-tunnel interface. As long as encapsulation type istcp-forward, we can either stick to default or create a new one. I decided to create a new one for the purpose of this lab test: 3.4. Create http profile using http-explicit as parent and assign it to VIP created previously: Note: When configuring it from GUI just set proxy type to explicit and http-explicit will be set to parent profile. The equivalent in the GUI: 3.5. Testing Connection 3.6 Enabling encrypted requests Initially I did not understand why HTTP traffic worked whendefault-connect-handling was set to deny but HTTPS did not. Below is the summary of my tests: Looking at the packet capture, I noticed that BIG-IP was explicitly denying Firefox's request to establish a tunnel as shown below: Note: For this test I created another Virtual Server called using a different address 10.199.3.101 instead of the one I originally tested just to test default-connect-handling. After reading K40243113: Overview of the HTTP profile, I kind of understood the point: "indicates that outbound requests are delivered only if another virtual server is listening on the tunnel for the requested outbound connection. With this setting, virtual servers are required, and the system processes the outbound traffic before it leaves the device." I then created another VIP withsamedestination IP address and port as back-end server client was trying to connect to just to confirm if this time traffic would go through: And indeed it worked: I also tested with a wildcard virtual server listening on *:443 instead of 172.16.199.32/32 and it worked fine as well. What we have observed so far is that when default-connect-handling is set to deny,in order for encrypted client traffic to be accepted by BIG-IP and proxy tunnel established, the destination address that matches the one on client request has to have a listener on BIG-IP for return traffic. Therefore, we can conclude thatdefault-connect-handlingsetting governs what traffic is accepted by the tunnel interface, but unencrypted requests are not affected. The below picture sums up the explanation: default-connect-handling just adds the extra step to validate whether there is a listener on BIG-IP for destination address:port or not. If not, we deny request. If yes, request goes through.SSL Orchestrator Use Case: SWGaaS
Introduction BIG-IP 16.0 with SSL Orchestrator 9.0 has support for running Secure Web Gateway (SWG) “as a Service” inside the Service Chain.This allows you to take an existing F5 SWG solution and migrate or move it to the same BIG-IP as SSL Orchestrator. Typical SWG features include: User authentication (not covered here) Enforcement of an Acceptable Use Policy (AUP) Website category database (google.com = Search Engines) Logging and Reporting (not covered here) A typical SWG deployment will have a Per-Session Policy that handles authentication.Then a Per-Request Policy that enforces the AUP. User authentication (not covered here) Refer to this Dev/Central Article for more information on this topic. Enforcement of an Acceptable Use Policy (AUP) A Per-Request Policy is used to enforce the AUP.You can find this from the Configuration Utility under Access > Profiles / Policies > Per-Request Policies.Click Edit for the Per-Session Policy and a new window like this should open: This policy does a Protocol Lookup to determine if the content is HTTP, then performs a Category Lookup based on the host header in the URI.Response Analytics will check for malicious content and pass that information on to the URL Lookup Agent. The Category is compared to the URL Filter which maps URL categories to Allow/Deny Actions.As a final result the request is either Allowed or Denied (Reject). Note: In a per-request SWG policy you would typically have a Protocol Lookup for HTTP and HTTPS.But in this case the SSL Orchestrator will perform SSL decryption so the SWG Service will receive plain-text, HTTP content.Therefore, this SWG policy is ready to be used with SSL Orchestrator. Website category database (google.com = Search Engines) The URL Filter is configured from Access > Secure Web Gateway > URL Filters. Select CorporateURLFilter in this example. This opens the Category editor.Different Categories and sub-categories are available to make Allow or Deny decisions.In this example the Games and Shopping categories have been set to Deny. Logging and Reporting (not covered here)Refer to the AskF5 Knowledge Center for more information. Configuration Export / Import the SWG Per-Request Policy The SWG Per-Request Policy is easy to export from one BIG-IP to another.From the Configuration Utility select Access > Profiles / Policy > Per-Request Policies. Click Export then OK to save the policy. The policy file can be directly imported into another BIG-IP device.On the Per-Request Policies screen click Import. Give the Policy a name, click Browse to select the policy file then Import. This policy is ready for SSL Orchestrator to use with SWGaaS.You can click Edit to verify the policy is correct. Configure the F5 SWGaaS From the SSL Orchestrator Configuration page select Services then click Add. F5 Secure Web Gateway is available on the F5 tab.Double-click the icon to configure. Give it a name.Set the Access Profile Scope to Profile.Set the Per Request Policy to the policy imported previously.Click Save and Next. Add the newly created SWGaaS to an existing Service Chain or create a new one. Select the F5_SWG Service on the left and click the right arrow to move it to the Selected column.Click Save. Save & Next. Then Deploy. Test SWG Functionality Note: be sure that a Security Policy has the Service Chain applied.Go to a client computer and test access to various web sites.News sites are allowed but Shopping is set to Block so sites like amazon.com and walmart.com should be blocked. Details from espn.com.The padlock indicates the connection is encrypted.The Issued By field indicates that this was intercepted & signed by SSL Orchestrator. Any attempts to visit a site categorized as Shopping or Games will be blocked. The configuration is now complete.SSL Forward Proxy Explained using Wireshark
Related articles:SSL Legacy Renegotiation vs Secure Renegotiation Explained using Wireshark Summary The idea behind this feature is to allow BIG-IP to sniff into SSL connections to any Internet destination that goes through it whilst preserving client's trust of the destination. When this feature is enabled, BIG-IP uses a pre-installed CA set underproxy-ca-cert(CA Certificatein the GUI) that is also trusted by client's browser and BIG-IP pretends to be the Internet destination by re-signing SSL certificates on the fly with the aforementioned CA and using public key from CA configured undercerton BIG-IP. The client trusts the connection and is unaware of BIG-IP's presence. The most common use case isSecure Web Gateway(SWG), although neither APM nor SWG are needed to deploy SSL Forward Proxy functionality.LTM is enough. Lab Scenario Lab test results: Client opens connection with BIG-IP and sendsClient Hello. BIG-IP immediately opens connection with web server and completes SSL handshake. BIG-IP creates unique certificate with following information and sends it over viaServer Hello: Serial number: random Issuer: copied fromdefault.crt Validity: notBefore (copied fromserver1.crt), notAfter: 30 days counting from time it was generated (configurable viacert-lifespanorCertificate Lifespanin the GUI) Subject: copied fromserver1.crt Public Key: copied fromltm2.CA.crt All X.509extensionsallowed incert-extensions-includeare also copied fromserver1.crtto unique custom certificate Client-side handshake completes successfully using above dynamically generated custom certificate just created by BIG-IPand data exchange works as expected. How SSL Forward Proxy works on Wireshark Actual capture used for below explanation is attached to this article below (and heressl-forward-sample-3.pcap ) Upon Client Hello sent by Client BIG-IP rushes to open new connection and completes server-side SSL handshake: Server-side handshake completes on frames 17-18 and and BIG-IP immediately resumes and completes Client-side handshake.Note:The reason why BIG-IP waits for server-side handshake to complete first is because BIG-IP uses server-side certificate information to generate acustomcertificate that will be sent to Client on client-side. The most interesting bit here is BIG-IP'sCertificatemessage sent on frame 20 to Client as it contains the customCertificate generated by BIG-IP: We can see there are 2 Certificates: CA Certificate(proxy-ca-cert) is the Certificate Authority (CA) that signs the newly created custom certificate and is supposed to be trusted by Client to confirm BIG-IP's custom (dynamically generated) certificate can be trusted too. We can immediately work out that the 2nd certificate isdefault.crtjust by matching its serial number which is unique for each issuer: However, the first certificate looked a mystery to me until I found where all the pieces came from. The serial number looks randomly generated to me as it doesn't match (not even close to any of the certificates above): The Issuer field, which shows information about who signed the certificate, shows thatdefault.crtwas the one who signed BIG-IP's custom certificate: Certificate generation date (notBefore) is copied fromserver1.crtbut expiry date (notAfter) is calculated based oncert-lifespan(Certificate Lifespanin the GUI) which by default is 30 days added to time custom certificate was generated: Subject fields, which shows information about the current certificate, is copied and identical to information onserver1.crt: And finally, the public key is copied fromcertfield (ltm2CA.crt). Notice they are the same: That's enough information for now.
