29-Oct-2018 06:00 - edited 08-Dec-2022 14:30
In a previous article, I provided a guide on using F5's Access Policy Manager (APM) and Secure Web Gateway (SWG) to provide forward web proxy services. While that guide was for organizations that are looking to provide secure internet access for their internal users, URL filtering as well as securing against both inbound and outbound malware, this guide will use only F5's Local Traffic Manager to allow internal clients external internet access.
This week I was working with F5's very talented professional services team and we were presented with a requirement to allow workstation agents internet access to known secure sites to provide logs and analytics. Of course, this capability can be used to meet a number of other use cases, this was a real-world use case I wanted to share. So with that, let's get to it!
In this use case, we will be forwarding all requests to this DNS resolver.
Note: Please use the correct DNS server for your use case.
Note: This must be an IP address the internal clients can reach.
Note: This use case was for TCP traffic directed at known hosts on the internet. If you require other protocols or all, select the correct option for your use case from the drop-down menu.
In order to catch and forward all traffic to the BIG-IP's default gateway, we will create a virtual server to accept traffic from our explicit proxy virtual server created in the previous steps.
You have now successfully configured your F5 BIG-IP to act as an explicit forward web proxy using LTM only. As stated above, this use case is not meant to fulfill all forward proxy use cases. If URL filtering and malware protection are required, APM and SWG integration should be considered. Until next time!
Why do you create a l4 virtual server instead of configuring default-connect-handling allow in http profile?
Did you see this codeshare with configuration command lines for http and ssl forward proxy?
Daemon tunnel apps for android
We have a explicit F5 forward proxy server where I am able to access all external sites and it works fine however I can't reach any my internal sites , It said "DNS lookup failed " .
from Big IP I can resolve these sites but not working in browser., In DNS resolver list I have called the internal DNS server which can resolve all the DNS entries internal and external.
So please suggest where am I missing .
@Sergi0 : the Explicit Proxy Virtual Server convert proxy connection with CONNECT method to TCP connection. CONNECT method is usually for HTTPS connection when the proxy must not inspect the content.
This connection is injected into an internal network to be forwarded to the destination.
Then the HTTPS virtual server with destination 0.0.0.0/0 listen on this internal network to enabled SSL Forward Proxy.
As Stanislas mentioned, the network tunnel will maintain the HTTP CONNECT tunnel for SSL traffic using the tcp-forward profile. I just searched for documentation around this and I honestly don't see a ton. I will keep searching and share if I find anything.
Its not working when I configured in Partition. I do not see traffic on Wild CARD VS configured in Partition. I am not sure if above solution supports in F5 Partition.
@k20 : Do you try to access FQDN or short names?
@Ajit, do you have an external self IP configured that allows access to the external internet or whatever you are using as a DNS resolver? You can also run ip route get <server ip address> to determine which IP address is being used to communicate with the DNS resolver.
Honestly this is the first time I am seeing any of these comments so if it is related to internal websites, you should probably be bypassing any type of proxy for internal addresses. If not, let me know and we can figure out how to resolve it.
No, these are Amazon VPC endpoints that I am trying to resolve. If I set the same DNS server that I use in the DNS resolver in nslookup command then it resolves without any issues. However, the same DNS server is unable to resolve via the proxy solution. Am I missing something.
, can you validate that when you do a tcpdump, you see queries sent and received on the IP you have configured in your DNS resolver? I too was getting DNS failures in my browser when I just set this up again in my own environment which let me to believe I did not have a route configured for my queries and external connections to use. I have a very basic configuration and when I did an "ip get route 220.127.116.11" it was attempting to use my mgmt IP. That of course is not going to work so I configured a default route for my BIG-IP to use a gateway that had access to the outside world. Using ip route get and tcpdump, can you validate your connections are being attempted using your external self IP? If you do not have an external self IP configured, that needs to be done first. I will be updating this article to reflect these troubleshooting steps as well. Let me know.
[root@ip-10-1-1-4:Active:Standalone] log # ip route get 18.104.22.168
22.214.171.124 via 10.1.10.1 dev External src 10.1.10.240
[root@ip-10-1-1-4:Active:Standalone] log #
[root@ip-10-1-1-4:Active:Standalone] log # tcpdump -ni 0.0 host 126.96.36.199
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on 0.0, link-type EN10MB (Ethernet), capture size 65535 bytes
17:15:32.240515 IP 10.1.10.240.31374 > 188.8.131.52.domain: 55472+ [1au] A? e13678.DSPb.akAmAIEDgE.neT. (55) out slot1/tmm1 lis=
17:15:32.266805 IP 184.108.40.206.domain > 10.1.10.240.31374: 55472 1/0/1 A 220.127.116.11 (71) in slot1/tmm1 lis=
All the configuration & routing looks perfect however, the dns resolvers are not resolving the hostnames.
I think the issue is that I had first set the forward zone name as "TestDNS" initially, later realized that it is incorrect. I then changed the forward zone name as "dot" however in-spite of the correction made the DNS resolver refuses to resolve the FQDN's whatsoever. I think it has some bug / misbehavior after the correction. When I built the same setup on another LB with the exact steps (no mistakes in any step) then it worked perfectly.
, I cannot be sure about your configuration without seeing it but I can tell you I have deployed this using v13, v14, and v15. I have customers currently running this on v13 and v14. The biggest issue my customers faced was understanding how and what self IP was being used to perform the resolution. They each experienced the same issue you did regarding the inability to resolve but after validating the external self IP being used, it began functioning as expected. Some created default routes to use the external self IP. I am sorry you are unable to get this functioning. I would definitely recommend opening a ticket with F5 support to determine why resolution is not occurring.
This also entails https traffic. There is no additional configuration for this specific use case. With that, if we are terminating SSL for inspection or authentication purposes then yes there would be additional configuration items.
Very interesting, thank you for the walkthrough. I tested this and it works quite well. I was wondering though what happens if you want to do URL Filtering. There is a guide for deploying an explicit forward proxy using APM and you can define URL Categories / URL Filtering there. Is it possible on this setup as well? I tried to apply certain iRules to restrict the explicit proxy to be able to reach e.g. *.microsoftupdates.com but it looks like the F5 doesn't really perceive this is an HTTP_REQUEST. I also tried to apply iRules on the wildcard VS, however no luck on that one as well.
I have the problem as soon as I have https traffic no more data packets are forwarded.
I can still see that there is a DNS request and it is answered. But then comes to no connection. The F5 does not send packets to the destination.
If I make the same call with http I see the packets leaving the F5.
Just deleted the Wild Card Virtual Server again and set it up again. And now it works
When defining the wildcard VS, please ensure you define a /0 mask for the destination. Just worked with another customer and this is all they were missing and everything began to function as expected.
Since we move to the cloud we need a solution to nat applications to different ip's. As the clients are in the Azure env. i thought about setting up multiple of these proxy listeners. But as we set the nat on the wildcard VS i guess i can only nat to 1 ip address per proxy. Would it work to assign every proxy listener to a different routing domain?
To get this to work I needed to enable port translation in the wildcard IP forwarding virtual server. Without that it was sending traffic out to the web server on port 8080 instead of port 443 or 80. This was on BIG-IP version 15.1.
@Erik_Roeckers Hi I have some questions.
I try to configure forward proxy on LTM following this article.
But I Can't reach outside Internet but Can on F5 explicit_proxy_vs
At virtual Server Statistics, I can see wildcard_vs packet 'zero'.
I assume that my packet can't pass through F5 LTM.
It's enabled port translation on expicit_proxy_vs but there is not ' port translation' options on wildcard ip forwarding vs...:(
@Steve_LyonsBased on this article it seems like this forward proxy configuration is suppose to work for HTTP requests to the forward proxy but does this work for HTTPS requests to the forward proxy? Currently when I attempt to utilize the forward proxy for HTTPS communication I receive a "HTTP/1.0 503 Service Unavailable" in the browser and in the tcpdump I see "Connect failed[!http]" which makes me believe this type of forward proxy does not support HTTPS communication. I am seeing "CONNECT <redactid FQDN>:443 HTTP/1.1" and "Host: <redactid FQDN>:443" in the initial request so the F5 is definitely seeing the appropriate pieces of information but it continues to fail.
I had to struggle with this one for hours
Finally it succeeded to work, but it took an extra setting: only after I added the 'route domain' directive in HTTP profile attached to the explicit proxy VIRT.
that is, I suppse, due to the multi route domain design in my env.
now everything just works, also for HTTPS traaffic.
I'm wondering how such a newly created wildcard VS with settings listed below will affect our existing Virtual Servers that were created with BigIP as a Reverse-Proxy with All VLANs and Tunnuels for "VLAN and Tunnel Traffic"?
@Sam_D_ based on the following article I believe they would technically conflict with each other so I think the index ID would most likely cause traffic to go to the previously existing VS first rather than a new one that listens on the specific interface/vlan/tunnel.
@Paulius thanks for your nice tips! That tech article really explains what confused me 😀
BTW, it'll be really appreciated if you could shed light on why we need a wildcard VS in addition to the explicit proxy VS. How do these two VSs work together to serve the forward proxying purpose?
@Sam_D_the reason the wildcard VS is necessary is because that is what creates the forward proxy tunnel between the BIG-IP and the destination that you are trying to reach through the forward proxy. If you didn't have this you wouldn't be able to dynamically configure the destination the way the wildcard VS does. I'm also glad that the article did help sort out the confussion you had on priority of VS.