Hi, we are using F5 load balancers for our Internet environment. There is now a requirement during customer login to download a trojan detection script at an external company. But for the client point of view this script should be hosted from our own domain otherwise personal firewalls will block this script. The plan is to create a pool an add this external website as a poolmember and have the F5 based on a specific URI to rewrite the URL to the external website and using SNAT to get the external script. The problem is we are now doing a POC in our DTA environment but here the F5 does not have a direct Internet connection. The only option is to go via our browser proxies. So I have created a pool wiht the proxy as a member and are using TCP 8080, redirecting normal HTTP traffic via the proxy is working, but the website hosting the external script is running on HTTPS. The problem is that the proxy for HTTPS traffic is expecting a HTTP CONNECT request instead of a GET which is standard used for HTTP. Is there anyway to change the HTTP method on the F5 from GET to CONNECT? I know you can read the method via HTTP::method but I have not found an option to actually change the method to CONNECT. Regards, Remco

Hmm... You're wanting the LTM to be an HTTP client to a proxy... OOTB this isn't going to happen. Hwoever you should be able to get something up & running with an iRule. But at first glance, it's not going to be terribly simple one as the iRule is going to have to negotiate a TLS/SSL connection on the fly after the CONNECT (The SMTPTLS iRule shows how this can be done on the client side, it should be possible to manipulate the iRule to doit on the server side instead). Do your proxies have a transparent option? H

Hi Hamish, I think setting up the F5 to be a SSL client shouldn't be a problem, this should be accomplished by using a standard serverssl profile without any key or certificate specified. The F5 should than act as an client in the SSL handshake. Since we already use an iRule to select the pool based on the URI, I can disable ssl for all other URI's in the irule. The proxies have been setup in transparent mode for SSL traffic, so they should not be involved in the SSL process. I think my biggest problem is getting the F5 to use a HTTP CONNECT method instead of the standard GET.

The proxies have been setup in transparent mode for SSL traffic, so they should not be involved in the SSL process. I think my biggest problem is getting the F5 to use a HTTP CONNECT method instead of the standard GET. If the proxies really do work in transparent mode for SSL you won't need the CONNECT method. CONNECT is only used by HTTP clients when they know about the proxy. In transparent mode the clients don't know there is a proxy and they will just try to connect directly to the target, by issuing a tcp connection to server:443. So, your ssl server profile should work out of the box. Let's assume the SSL connection works, then you still need to do this: 1.) Rewrite the URI to the location of the script "[HTTP::uri]" 2.) Rewrite the HOST header "[HTTP::header Host] replace", otherwise the target server might not accept the request. Regards Kurt Knochner

not sure if i understand correctly. this is my testing. [root@iris:Active] config b virtual bar list virtual bar { snat automap pool squid destination 172.28.17.33:https ip protocol tcp rules myrule } [root@iris:Active] config b pool squid list pool squid { members 192.168.12.105:squid {} } [root@iris:Active] config b rule myrule list rule myrule { when SERVER_CONNECTED { set bypass 0 TCP::respond "CONNECT www.google.com:443 HTTP/1.0\r\n\r\n" TCP::collect } when SERVER_DATA { if { $bypass eq 1 } { TCP::release return } if { [TCP::payload] starts_with "HTTP/1.0 200" } { TCP::payload replace 0 [TCP::payload length] "" TCP::release set bypass 1 } else { TCP::close } } } [root@iris:Active] config curl -Ik https://172.28.17.33/ HTTP/1.1 200 OK Date: Fri, 04 Nov 2011 09:18:10 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Server: gws X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Transfer-Encoding: chunked

sorry i forgot to put the reference. Virtual_to_enable_proxy-unaware_applications_to_make_outbound_TCP_connections_via_a_HTTP_CONNECT_method by kozlowc - kozlowc at ms.com http://devcentral.f5.com/wiki/iRules.Virtual_to_enable_proxy-unaware_applications_to_make_outbound_TCP_connections_via_a_HTTP_CONNECT_method.ashx

Redirecting http request to external url via proxy

nitass

Employee

Nov 08, 2011

when browsing https://172.28.65.152/

New TCP connection 3: 192.168.206.102(56162) <-> 172.28.65.152(443)
3 1  1320730108.5769 (0.0018)  C>SV3.1(139)  Handshake
      ClientHello
        Version 3.1
        random[32]=
          4e b8 bf 18 c1 41 57 a1 b6 fb 79 25 ba b0 e3 20
          17 76 59 49 ec 4d 6f 15 d5 57 12 6d 11 ac e2 75
        cipher suites
        Unknown value 0xff
        Unknown value 0xc00a
        Unknown value 0xc014
        Unknown value 0x88
        Unknown value 0x87
        TLS_DHE_DSS_WITH_AES_256_CBC_SHA
        Unknown value 0xc00f
        Unknown value 0xc005
        Unknown value 0x84
        TLS_RSA_WITH_AES_256_CBC_SHA
        TLS_DHE_RSA_WITH_AES_256_CBC_SHA
        Unknown value 0xc007
        Unknown value 0xc009
        Unknown value 0xc011
        Unknown value 0xc013
        Unknown value 0x45
        Unknown value 0x44
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA
        Unknown value 0xc00c
        Unknown value 0xc00e
        Unknown value 0xc002
        Unknown value 0xc004
        Unknown value 0x96
        Unknown value 0x41
        TLS_RSA_WITH_RC4_128_MD5
        TLS_RSA_WITH_RC4_128_SHA
        TLS_RSA_WITH_AES_128_CBC_SHA
        Unknown value 0xc008
        Unknown value 0xc012
        TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
        Unknown value 0xc00d
        Unknown value 0xc003
        Unknown value 0xfeff
        TLS_RSA_WITH_3DES_EDE_CBC_SHA
        compression methods
                  NULL
3 2  1320730108.5770 (0.0000)  S>CV3.1(81)  Handshake
      ServerHello
        Version 3.1
        random[32]=
          89 cd 97 a7 2e 9e bb d1 41 d6 69 e0 6e e6 c2 a6
          b5 a4 20 98 09 77 72 6a d4 5d 85 a9 00 b9 d6 27
        session_id[32]=
          2c c8 08 e2 44 02 e7 31 cc 99 4a 19 79 ff e8 7e
          f3 6f 94 34 b3 03 1a bb 69 92 a4 a6 e0 4b 04 0d
        cipherSuite         TLS_RSA_WITH_RC4_128_SHA
        compressionMethod                   NULL
3 3  1320730108.5770 (0.0000)  S>CV3.1(692)  Handshake
      Certificate
3 4  1320730108.5770 (0.0000)  S>CV3.1(4)  Handshake
      ServerHelloDone
3 5  1320730108.5800 (0.0030)  C>SV3.1(134)  Handshake
      ClientKeyExchange
        EncryptedPreMasterSecret[128]=
          2f 64 7c 19 00 a8 44 c9 4e 77 a0 29 a0 35 d4 1e
          b4 a6 44 a7 38 6b 80 16 ae 8c 57 d1 72 df 4f b7
          ef 1d bb 8a 7d 96 96 d0 d3 c0 ce 6c 97 5e 89 fd
          3f ec e2 51 aa 45 6f 3a 90 a7 99 c5 d3 15 d1 cd
          d7 27 eb 12 e3 c1 1c a2 6c bf 48 f4 3f 2e 1c 42
          89 1a 01 66 00 74 10 cd 1f 5a d4 4f 21 2a d8 dc
          4b e2 57 d8 55 61 4f 6d 9d 00 cd 8f 6d ca 94 21
          42 0f fd 58 13 1f 28 6e aa e9 ee 5e 15 b7 f2 2b
3 6  1320730108.5800 (0.0000)  C>SV3.1(1)  ChangeCipherSpec
3 7  1320730108.5800 (0.0000)  C>SV3.1(36)  Handshake
      Finished
        verify_data[12]=
          9e f7 0a bb 6f d4 b0 82 48 b2 67 44

3 8  1320730108.5830 (0.0030)  S>CV3.1(1)  ChangeCipherSpec
3 9  1320730108.5830 (0.0000)  S>CV3.1(36)  Handshake
      Finished
        verify_data[12]=
          f7 0f f3 da 64 40 42 19 6e 0f a0 6f

3 10 1320730108.5882 (0.0051)  C>SV3.1(389)  application_data
    ---------------------------------------------------------------
    GET / HTTP/1.1
    Host: 172.28.65.152
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.23) Gecko/20110920 Firefox/3.6.23
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-us,en;q=0.5
    Accept-Encoding: gzip,deflate
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    Keep-Alive: 115
    Connection: keep-alive

    ---------------------------------------------------------------
New TCP connection 4: 200.200.200.1(56162) <-> 200.200.200.101(80)
1320730108.5890 (0.0007)  C>S
---------------------------------------------------------------
GET / HTTP/1.1
Host: 172.28.65.152
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.23) Gecko/20110920 Firefox/3.6.23
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive

---------------------------------------------------------------

1320730108.5922 (0.0032)  S>C
---------------------------------------------------------------
HTTP/1.1 200 OK
Date: Tue, 08 Nov 2011 05:33:11 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Mon, 07 Nov 2011 11:52:31 GMT
ETag: "4183e3-2a-b08f5dc0"
Accept-Ranges: bytes
Content-Length: 42
Connection: close
Content-Type: text/html; charset=UTF-8



Hello world


---------------------------------------------------------------

3 11 1320730108.5923 (0.0040)  S>CV3.1(324)  application_data
    ---------------------------------------------------------------
    HTTP/1.1 200 OK
    Date: Tue, 08 Nov 2011 05:33:11 GMT
    Server: Apache/2.2.3 (CentOS)
    Last-Modified: Mon, 07 Nov 2011 11:52:31 GMT
    ETag: "4183e3-2a-b08f5dc0"
    Accept-Ranges: bytes
    Content-Length: 42
    Connection: close
    Content-Type: text/html; charset=UTF-8

    
    
    Hello world
    
    
    ---------------------------------------------------------------
4    1320730108.5923 (0.0000)  S>C  TCP FIN
3    1320730108.5923 (0.0000)  S>C  TCP FIN

nitass

Employee

Nov 08, 2011

when browsing https://172.28.65.152/test

New TCP connection 3: 192.168.206.102(56177) <-> 172.28.65.152(443)
3 1  1320730238.7900 (0.0027)  C>SV3.1(139)  Handshake
      ClientHello
        Version 3.1
        random[32]=
          4e b8 bf 9a 92 1c ae 2c 84 3a 1e 39 fa cb 95 03
          a2 6b e6 86 85 02 aa b1 7c e6 d3 90 51 3b d2 e6
        cipher suites
        Unknown value 0xff
        Unknown value 0xc00a
        Unknown value 0xc014
        Unknown value 0x88
        Unknown value 0x87
        TLS_DHE_DSS_WITH_AES_256_CBC_SHA
        Unknown value 0xc00f
        Unknown value 0xc005
        Unknown value 0x84
        TLS_RSA_WITH_AES_256_CBC_SHA
        TLS_DHE_RSA_WITH_AES_256_CBC_SHA
        Unknown value 0xc007
        Unknown value 0xc009
        Unknown value 0xc011
        Unknown value 0xc013
        Unknown value 0x45
        Unknown value 0x44
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA
        Unknown value 0xc00c
        Unknown value 0xc00e
        Unknown value 0xc002
        Unknown value 0xc004
        Unknown value 0x96
        Unknown value 0x41
        TLS_RSA_WITH_RC4_128_MD5
        TLS_RSA_WITH_RC4_128_SHA
        TLS_RSA_WITH_AES_128_CBC_SHA
        Unknown value 0xc008
        Unknown value 0xc012
        TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
        Unknown value 0xc00d
        Unknown value 0xc003
        Unknown value 0xfeff
        TLS_RSA_WITH_3DES_EDE_CBC_SHA
        compression methods
                  NULL
3 2  1320730238.7900 (0.0000)  S>CV3.1(81)  Handshake
      ServerHello
        Version 3.1
        random[32]=
          65 ff cf 87 45 a5 76 4e 12 cf eb b8 aa 18 2f 3e
          68 3a a9 ed 6d cd 0f 17 8e aa 76 ed e3 43 48 b9
        session_id[32]=
          2c c8 08 e2 44 02 e7 3c cc 99 4a 19 79 ff e8 7e
          f3 6f 94 34 b3 03 1a 84 69 92 a4 a6 e0 4b 04 8f
        cipherSuite         TLS_RSA_WITH_RC4_128_SHA
        compressionMethod                   NULL
3 3  1320730238.7900 (0.0000)  S>CV3.1(692)  Handshake
      Certificate
3 4  1320730238.7900 (0.0000)  S>CV3.1(4)  Handshake
      ServerHelloDone
3 5  1320730238.7980 (0.0079)  C>SV3.1(134)  Handshake
      ClientKeyExchange
        EncryptedPreMasterSecret[128]=
          ad d2 4a 08 a6 ca 81 9b 1e f2 7f 55 3c f3 81 36
          2e 0e bf cc 34 a2 ec 57 95 c2 27 02 c3 26 1e 7d
          52 80 f7 d8 7a 8d ee d4 f7 b3 aa 83 89 f3 ca c4
          95 70 7c 2b 64 b9 3e 1d 8b 5e 62 ed a1 d6 e0 31
          d4 b2 32 67 9c 11 f4 a9 5e 4f be 1a eb 49 30 33
          00 3f 2f dc 0f 4e 6e 15 69 d7 73 4f 3e fa 47 cc
          93 4b c2 39 c1 b1 c3 66 6d c6 e8 0c 5d bf 96 a1
          77 84 e1 c1 17 41 05 eb 44 19 a4 70 d0 53 83 f1
3 6  1320730238.7980 (0.0000)  C>SV3.1(1)  ChangeCipherSpec
3 7  1320730238.7980 (0.0000)  C>SV3.1(36)  Handshake
      Finished
        verify_data[12]=
          6e 42 a7 5d 2f 30 64 1c 7e 7e 29 74

3 8  1320730238.8015 (0.0035)  S>CV3.1(1)  ChangeCipherSpec
3 9  1320730238.8015 (0.0000)  S>CV3.1(36)  Handshake
      Finished
        verify_data[12]=
          44 7f 47 d5 14 0b 14 ec 59 8d df 91

3 10 1320730238.8074 (0.0058)  C>SV3.1(393)  application_data
    ---------------------------------------------------------------
    GET /test HTTP/1.1
    Host: 172.28.65.152
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.23) Gecko/20110920 Firefox/3.6.23
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-us,en;q=0.5
    Accept-Encoding: gzip,deflate
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    Keep-Alive: 115
    Connection: keep-alive

    ---------------------------------------------------------------
New TCP connection 4: 192.168.206.102(56177) <-> 1.1.1.1(443)
4 1  1320730238.8075 (0.0000)  C>SV3.1(89)  Handshake
      ClientHello
        Version 3.1
        random[32]=
          fd ac 4d 96 0f bb 8f 4b 0e 7e 8d 34 49 5e 0d 8d
          41 8a 83 61 49 f1 1a 26 d4 9d 9b e3 43 14 ac 96
        resume [32]=
          7b e2 52 da 33 4a 7a 73 73 31 52 9f 08 1c 61 66
          6a a0 90 84 5e cb f1 d4 5e c4 c6 17 33 7f 11 4e
        cipher suites
        TLS_RSA_WITH_RC4_128_SHA
        TLS_RSA_WITH_AES_128_CBC_SHA
        TLS_RSA_WITH_AES_256_CBC_SHA
        TLS_RSA_WITH_3DES_EDE_CBC_SHA
        Unknown value 0x3c
        Unknown value 0x3d
        Unknown value 0xff
        compression methods
                  NULL
4 2  1320730239.3370 (0.5294)  S>CV3.1(81)  Handshake
      ServerHello
        Version 3.1
        random[32]=
          4e b8 bf 9b 46 e4 73 2b c9 5d 5e c4 b3 0f f6 96
          36 40 c4 97 05 55 9a 12 ba b6 ee 9e 11 eb 93 11
        session_id[32]=
          7b e2 52 da 33 4a 7a 73 73 31 52 9f 08 1c 61 66
          6a a0 90 84 5e cb f1 d4 5e c4 c6 17 33 7f 11 4e
        cipherSuite         TLS_RSA_WITH_RC4_128_SHA
        compressionMethod                   NULL
4 3  1320730239.3370 (0.0000)  S>CV3.1(1)  ChangeCipherSpec
4 4  1320730239.3370 (0.0000)  S>CV3.1(36)  Handshake
4 5  1320730239.3371 (0.0001)  C>SV3.1(1)  ChangeCipherSpec
4 6  1320730239.3371 (0.0000)  C>SV3.1(36)  Handshake
4 7  1320730239.3371 (0.0000)  C>SV3.1(389)  application_data
4 8  1320730239.7781 (0.4409)  S>CV3.1(286)  application_data
4 9  1320730239.7781 (0.0000)  S>CV3.1(1054)  application_data
3 11 1320730239.7782 (0.9708)  S>CV3.1(1320)  application_data
...

Remco
Nimbostratus
Nov 08, 2011
Hi Nitass,

thanks for your suggestions. I just made the changes and it all looks like to be working as we planned.

The extra VIP for setting up the connection via the proxy does the trick in seperating the required CONNECT message and the HTTP modification needed on the client request.

Thanks all for your help.

Regards,

Remco
nitass
Employee
Nov 08, 2011
you're welcome. :-)
hooleylist
Cirrostratus
Nov 08, 2011
Great work on an iRule solution Nitass!

Remco we're seeing more and more requests from customers for this type of web proxy functionality. If you'd like to see F5 add this functionality to the product, you could get in touch with F5 Support or your F5 SE/partner to request it.

Aaron
Kurt_Knochner_5
Cirrus
Nov 08, 2011
Great work on an iRule solution Nitass!

I agree with that. Great solution!

BTW: Can someone please add the whole solution somewhere in the devcentral iRule wiki?

Regards

Kurt Knochner
David_18981
Nimbostratus
May 07, 2014
Hi DevCentral,

i'm currently trying to use this brilliant solution to send http to froward proxy server, but i'm encountering an issue :

I have configured the same setup (with 2 VS)
But when the second VS sends the "HTTP CONNECT" to the proxy, it doesn't wait for the reply "200 OK" and just sends the "CLIENT HELLO" received from the first VS.
Result => when receiving the 200 OK from the proxy, the SSL session doesn't get established as the client waits for the "SERVER HELLO" which never arrives.

Here is the diagram observed between F5 and proxy :

(F5 -> Proxy) CONNECT toto.com:443 HTTP/1.1
(F5 -> Proxy) CLIENT HELLO
(Proxy -> F5) HTTP/1.1 200 Connection established ...

Here is the expected diagram :

(F5 -> Proxy) CONNECT toto.com:443 HTTP/1.1
(Proxy -> F5) HTTP/1.1 200 Connection established
(F5 -> Proxy) CLIENT HELLO
(Proxy -> F5) SERVER HELLO, CERTIFICATE, SERVER HELLO DONE, ...
(F5 -> Proxy) Application Data

Someone knows what to do to make it works ?

Thanks...
Drew
Nimbostratus
Jan 14, 2015
Hi We are trying something similar. We want to publish a web page with a blog on that page pulled from a third party. I can capture the content by looking for a uri starting with /news-and-blog/ I send this to a proxy and rewrite the host header. This sends the blog content to a virtual server (with the proxy server as a pool member). This works OK for HTTP traffic but if we try to use HTTPS it fails. The proxy always sees HTTP for some reason. I have also tried substituting the pool for the VIP in this iRule without success.

when HTTP_REQUEST { if {[HTTP::uri] starts_with "/news-and-blog/" } { HTTP::header replace Host "myblog-uat.third_party.com" virtual /S1SIL/VS_S2S_PROXYHTTP_INTERNAL_LIVE log local0. "Sending Blog content slblog-uat.third_party.com[HTTP::uri]to 3rd pary" } else { pool pool_iab3 } }

The site at the Third party can only listen on HTTP or HTTPS, not both so we have to get them to switch when we are testing.

Any help appreciated.

Cheers

Drew
Arie
Altostratus
Jan 14, 2015
@Drew: are you saying that you're creating a composite site by using content from multiple origins?

If so, you could also accomplish that by creating pools for each of the environments. You can then select a pool based on the URI. The VIP for port 443 can be configured to pull that information from pools for any port (using port translation).

The iRule would look like this:

when HTTP_REQUEST { if { [string tolower [HTTP::path]] starts_with "/news-and-blog/" } { pool cloud_news-and-blog_80_pool persist none } else { pool pool_iab3 persist none } }
Drew
Nimbostratus
Jan 15, 2015
Hi Arie Yes we have one pool we use to serve the main content from. The blog content is somehow embedded in the webpage. If we get the 3rd party to set their site up listening on HTTP it works. On HTTPS it doesn't.

I'm trying to work out what the two VIPs above are actually doing and how they would pertain to my issue. Thanks