cancel
Showing results for 
Search instead for 
Did you mean: 
PKI
F5 Employee
F5 Employee

Introduction 

A previous advanced use case described the use of Internal Layered Architecture for reducing complexity (ref: https://devcentral.f5.com/s/articles/SSL-Orchestrator-Advanced-Use-Cases-Reducing-Complexity-with-Internal-Layered-Architecture) 

Some organizations standardize their policies and/or operational procedures based on a platform/tool, to manage security and access policies for outbound access. One interesting design that is gaining popularity is leveraging a 3rd party tool, via sideband API calls, to obtain a traffic steering/management decision. Organizations with deep investments in a specific tool/technology for policy management, cannot transition to a net-new solution without significant friction. If this tool/system has a REST API interface or a way to create a service to respond to HTTP queries, the BIG-IP can initiate a sideband API call, requesting such a decision via an HTTP request, and leverage that decision in the SSL Orchestrator flow.  Here’s what that looks like: 


0EM1T000002cmOo.pngw


Here’s a quick rundown of the flow by step numbers:  

  1. Client attempts to browse to www.xyz.com. Client points to the Explicit Proxy VIP address (usually achieved by PAC file entry), which is  the first of the multi-layered SSL Orchestrator architecture. 
  2. The BIG-IP holds the client request, extracts headers(eg: username, source IP, Dest URL) from the HTTP Connect, and initiates a sideband API call to the 3rd Party policy manager (can be anything), including the required headers that the policy manager needs to provide a decision.  
  3. Policy manager replies with a custom value, usually a custom HTTP header, that provides a decision/action that SSL Orchestrator can use to make a traffic steering/management decision. Example, if the decision header is named “X_Decrypt_Decision”: 
  4. X_Decrypt_Decision == Intercept - Send to Intercept topology 
  5. X_Decrypt_Decision == Bypass - Send to Bypass topology 
  6. X_Decrypt_Decision == Block - Send to Block Topology   
  7. Based on the policy decision, SSLo steers the traffic to the appropriate topology via VIP targeting (example iRule below) 
  8. Depending on which topology was targeted, traffic passes through the respective service chain for typical security inspection steps 
  9. Finally, once the service chain has been traversed, BIG-IP sends the traffic out the egress path as defined (can be a single egress path, or topology-specific paths)  

 

This method can provide significant flexibility, there are some minor drawbacks/trade-offs, that should be considered. A few have been listed here: 

Pros: 

1) Flexibility to define/manage policies on a 3rd party policy manager/engine 

2) Ability to create conditional logic with multiple decisions from policy manager, before a bypass/intercept/allow decision is made  

3)  Limit or remove need for URL database management and update on BIG-IP devices 

 

Cons: 

1) Additional latency when performing sideband calls to 3rd party tool 

2) Dependency on 3rd party tool for a traffic steering decision 

3) Multiple logging sources, resulting in some troubleshooting complexity 

 

We’ll walk through the steps of configuring this behavior, using a McAfee Web Gateway as our policy manager. The same principle applies to other products (eg: Broadcom ProxySG). Here, we use an Explicit Forward proxy VIP, that is a manually created VIP (i.e. not created by an SSL Orchestrator topology), to perform additional functions at the traffic steering point. A manually created virtual server allows us to insert the conditional sideband calls and traffic steering decision, early into the traffic flow, as the traffic would be too far down the road, if it already made it to the Service Chain. While the configuration here uses an Explicit forward proxy, it doesn’t have to be so, and can be a transparent proxy as well.    

 

Let’s start with the MWG: 

McAfee Web Gateway API Configuration 

  1. LTM explicit proxy VIP trigger and iRule that generates an API call to McAfee Web Gateway GTI interface on client request and MWG returns policy actions (intercept/bypass/block). MWG thus maintains sole control of policy engine (single source of truth) – no need for URLDB subscription on SSL Orchestrator. 

Initially this requires configuration on the MWG to support this mechanism, and the following presents basic settings to enable this. In this case, the MWG will be configured to listen on an alternate (proxy) port, and a corresponding blocking policy will respond directly to HTTP headers sent to it. 

Proxy 

In the MWG UI, under the Configuration section -> Appliance -> Proxies, configure a new HTTP Proxy listener on a new port (ex. 2082). A policy will listen on this port and respond to API calls. 

  

Policy 

In the MWG UI, under the Policy section, create a new Rule Set. 

  • Enable: checked 
  • Applies to: check all 
  • Apply this rule set: if the following criteria is matched: 
  • Proxy.Port equals 2082 

The MWG policy should look something like this: 

0EM1T000002cmOp.png 

Rules 

Get URL and CLIENTIP from Parameter 

## This rule extracts a URL from a “url” query string and client IP from a “clientip” query string, and stores in “URL” and “CLIENTIP” variables respectively 

  • Rule Criteria: URL.HasParameter(“url”) equals true 
  • Action: Continue 
  • Events: Set URL = URL.GetParameter(“url”)  
  • Rule Criteria: URL.HasParameter(“clientip”) equals true 
  • Action: Continue 
  • Events: Set clientip = URL.GetParameter(“clientip”)  
  • Action: Continue 

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.  

 0EM1T000002cmOq.png 

 

0EM1T000002cmOr.png 

 

0EM1T000002cmOs.png 

Note: “transid”, shown in the screenshot above, is an additional field that can be configured to assign a user-defined transaction ID that be used to correlate traffic components between SSL Orchestrator and McAfee Web Gateway. This is not a hard requirement for this setup  

0EM1T000002cmOt.png 

0EM1T000002cmOu.png 

Initialize Headers 

## This rule sets a default “Intercept” value on the “X-Decrypt-Decision” header, in the event that other conditions don’t reset the value. 

  • Rule Criteria: Always 
  • Action: Continue 
  • Events: Header.Block.Add(“X-Auth-Decision”, “auth”) 
  • Events: Header.Block.Add(“X-Allow-Decision”, “allow”) 
  • Events: Header.Block.Add(“X-Decrypt-Decision”, “Intercept”) 

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.  

0EM1T000002cmOv.png 

0EM1T000002cmOw.png 

0EM1T000002cmOx.png 

0EM1T000002cmOy.png 

0EM1T000002cmOz.png 

 

Get Categories 

## This rule creates an “X-Categories” header that contains the list of matched URL categories, and will be returned to the caller. 

  • Rule Criteria: Always 
  • Action: Continue 
  • Events:Header.Block.Add(“X-Categories”, ListOfCategory.ToString(URL.Categories<Default>)) 

 The sequence of screenshots below illustrates the process of setting this up in the MWG UI.  

0EM1T000002cmP0.png 

 

0EM1T000002cmP1.png 

0EM1T000002cmP2.png 

 

0EM1T000002cmP3.png 

0EM1T000002cmP4.png 

Set Bypass Header 

## This rule resets the “X-Decrypt-Decision” header to “Bypass” on a URL category match for Finance/Banking URLs. 

  • Rule Criteria: URL.Categories<Default> contains Finance / Banking 
  • Action: Continue 
  • Events: Header.Block.Add(“X-Decrypt-Decision”, “bypass”) 

The sequence of screenshots below illustrates the process of setting this up in the MWG UI. 

0EM1T000002cmP5.png 

0EM1T000002cmP6.png 

0EM1T000002cmP7.png 

0EM1T000002cmP8.png 

0EM1T000002cmP9.png 

Set NoAuth Header 

## This rule sets the “X-Auth-Decision” to “noauth” on a url match for “httpbin.org”. This rule set can be used to add URLs, for which you don’t need users to be authenticated 

  • Rule Criteria: URL equals “httpbin.org: 
  • Action: Continue 
  • Events: Header.Block.Add(“X-Auth-Decision”, “noauth”) 

The sequence of screenshots below illustrates the process of setting this up in the MWG UI. 

 

0EM1T000002cmPA.png 

0EM1T000002cmPB.png 

0EM1T000002cmPC.png 

 

0EM1T000002cmPD.png 

 

0EM1T000002cmPE.png 

 

Set Block Header 

## This rule sets the “X-Allow-Decision” to “block” for a URL match on “www.bing.com”. Use this rule set to explicitly define blocked URLs, that may not be in the URL categorization  

  • Rule Criteria: URL equals “www.bing.com” 
  • Action: Continue 
  • Events: Header.Block.Add(“X-Allow-Decision”, “block”) 

The sequence of screenshots below illustrates the process of setting this up in the MWG UI. 

0EM1T000002cmPF.png 

  

0EM1T000002cmPG.png 

 

0EM1T000002cmPH.png 

0EM1T000002cmPI.png 

 

0EM1T000002cmPJ.png 

 

Final Block 

## This rule is a catch-all to set a static response to API calls in case a previous rule has not matched 

  • Rule Criteria: Always 
  • Action: Block 
  • Events: None 

0EM1T000002cmPK.png 

 

0EM1T000002cmPL.png 

0EM1T000002cmPM.png 

0EM1T000002cmPN.png 

0EM1T000002cmPO.png 

 

The final ruleset looks as follows: 

0EM1T000002cmPP.png 



F5 BIG-IP Configuration 

  • LTM TCP Tunnel - A custom TCP tunnel is created to provide a channel for non-TLS traffic, to be directed through. This is to ensure that our topology selection logic, catches ALL of the traffic, not just the HTTPS (TLS) Traffic.  

In the BIG-IP UI, under Network -> Tunnels, click the Create button. The tunnel is used by the explicit proxy configuration. 

  

  • Profile: tcp-forward 

  

  • LTM HTTP Explicit Profile - A custom HTTP Explicit Profile allows a VIP to accept and act upon an HTTP CONNECT request  

In the BIG-IP UI, under Local Traffic -> Profiles -> Services -> HTTP, click the Create button. This will create an HTTP explicit proxy profile. 

  

  • Parent Profile: http-explicit 
  • Explicit Proxy 
  • Tunnel Name: <LTM TCP tunnel> 

  

  • LTM Explicit Proxy VIP (frontend) 

In the BIG-IP UI, under Local Traffic -> Virtual Servers, click the Create button. An HTTP explicit proxy configuration is made up of an explicit proxy listener (what the client is configured to talk to), and a separate TCP tunnel virtual server. Client proxy CONNECT traffic is received by the explicit proxy virtual server, and the TCP proxy tunnel is established between the client and the TCP tunnel virtual server (through the explicit proxy virtual server. The following creates the explicit proxy listener virtual server. 

  

  • Type: Standard 
  • Source: 0.0.0.0/0 
  • Destination: client-facing explicit proxy IP (the IP the client is configured to talk to) 
  • Service Port: client-facing explicit proxy port 
  • HTTP Profile (Client): <LTM HTTP explicit profile> 
  • VLANs and Tunnels: client facing VLAN 
  • Address Translation: enabled 
  • Port Translation: enabled 
  • Access Profile: <see access profiles below> 
  • iRule: <see iRules below> 

  

  • LTM TCP Tunnel VIP (frontend) 

In the BIG-IP UI, under Local Traffic -> Virtual Servers, click the Create button. The following creates the TCP tunnel virtual server. 

  

  • Type: Standard 
  • Source: 0.0.0.0/0 
  • Destination: 0.0.0.0/0 
  • Service Port: 0 (any) 
  • VLANs and Tunnels: <LTM TCP tunnel> 
  • Address Translation: disabled 
  • Port Translation: disabled 
  • Access Profile: <see access profiles below> 
  • Per-Request Policy: <see per-request policy below> 
  • iRule: <see iRules below> 

  

  • SSL Orchestrator 

Create separate topologies (as functions). These will essentially be “internal” (not listening on a client facing VLAN) and will receive traffic directly from the frontend LTM virtual servers. Topologies as functions can be simple, including a small finite set of actions (ex. allow/block, intercept/bypass, service chain), and may include different egress, SNAT and logging settings. The primary benefits on this design are that a) topology configuration can remain relatively static, reducing management overhead, b) based on traffic rules the number of topologies may typically be reduced to a common set of functions, versus potentially multiple topologies to cover multiple traffic types, and c) topologies can be created purely for testing different traffic flows, without affecting production traffic. This last benefit is extremely useful when enabled to test and analyze misbehaving applications/protocols, and/or to validate new security products. The following is an example configuration of two separate topology functions, one for TLS intercept, and one for TLS bypass. As traffic is passing to them from the frontend explicit proxy, these topologies are configured as L3 outbound (transparent proxy). Refer to official F5 documentation for detailed SSL Orchestrator configuration steps (as needed). 

  

Topology 1 (Bypass function) 

  • Topology 
  • L3 Outbound 
  • Services 
  • Create security services as needed 
  • Service chain(s) 
  • Define at least one service chain to incorporate the set of services 
  • Security Policy 
  • Remove the Pinners rule 
  • All Traffic rule 
  • Action: Allow 
  • SSL Forward Proxy Action: Bypass 
  • Service Chain: send to a service chain as required 
  • Interception Rule 
  • VLANs: create and assign a “dummy” VLAN (no interfaces assigned) 
  • Egress Setting 
  • Configure as required for Internet next hop 

  

Topology 2 (Intercept function) 

  • Topology 
  • L3 Outbound 
  • Services 
  • Create security services as needed (if not already created) 
  • Service chain(s) 
  • Define at least one service chain to incorporate the set of services (if not already defined) 
  • Security Policy 
  • All Traffic rule 
  • Action: Allow 
  • SSL Forward Proxy Action: Intercept 
  • Service Chain: send to service chain as required 
  • Interception Rule 
  • VLANs: create and assign a “dummy” VLAN (no interfaces assigned) 
  • Egress Setting 
  • Configure as required for Internet next hop 

 Note – Create additional topologies for Block/Test/etc actions, like the 2 topologies created above 

  

  • Per-Request Profiles 

While Access per-session policies are typically initiated once per authenticated user session, per-request access profiles are defined to act on “transactional” data, as in per-HTTP request. 

  

  • TCP tunnel per-request policy (attached to TCP tunnel VIP) 
  • Type: All 
  • Incomplete Action: Allow 
  • Customization Type: Standard 
  • VPE:  
  • Start 
  • iRule Event (ID arbitrary, Expect Data: Client Accepted) 
  • Allow 

  

  • iRules 

In the BIG-IP UI, under Local Traffic -> iRules, click the Create button. The frontend explicit proxy configuration will utilize two iRules, one on each virtual server, to facilitate traffic flow to the SSL Orchestrator topologies.  

  

  • Explicit proxy listener iRule – attached to the explicit proxy listener virtual server, this rule programmatically forwards all incoming traffic to the TCP tunnel virtual server. In the ‘virtual’ command below, replace the pathname of the TCP tunnel virtual server with the correct local name. 


when HTTP_PROXY_REQUEST { 

    ## get proxy request URL 

    sharedvar THIS_HOST 

   set THIS_HOST [HTTP::uri] 

when HTTP_REQUEST { 

   ## force HTTP traffic to tunnel VIP 

   virtual "/Common/tcp-tunnel-vip" [HTTP::proxy addr] [HTTP::proxy port] 


TCP tunnel iRule – attached to the TCP tunnel virtual server, this rule programmatically forwards traffic to one of the SSL Orchestrator topology listeners based on some criteria. The iRule is triggered by the iRule event agent in the per-request access profile applied to the TCP tunnel virtual server. The below is an example of a fairly simple set of conditions. In the section that follows the “## evaluate …” line, define a set of conditions that decide which SSL Orchestrator topology listener to forward the traffic flow to. The normalized host value is provided in the “this_url” variable (ex. www.example.com), but the iRule also has access to source IP [IP::client_addr], destination IP [IP::local_addr], ports, and an array of other flows values. For each “virtual” command, replace the pathname with an SSL Orchestrator topology listener virtual server. The virtual servers may end with: 

  • -in-t-4 (representing an IPv4 TCP listener) 
  • -in-t-6 (representing an IPv6 TCP listener) 

 

Topology Selector iRule 

In the BIG-IP UI, under Local Traffic -> iRules, click the Create button. The rule below is applied to the frontend TCP tunnel virtual server. In the example below, the sideband call is direct to the MWG IP:port, but can be configured to point to an LTM virtual server that load balances multiple MWGs. Simply replace the IP:port in the “connect” command with the pathname of the virtual server. The sideband call makes a HEAD request, returning only the response headers. The X-Decrypt-Decision and X-Category headers are parsed out of the reponse, and the X-Decryption-Decision value is used to decide which SSL Orchestrator topology to send the traffic to. In the example rule, the X-Category header is superfluous, but could be used for logging actions. 

  

when ACCESS_PER_REQUEST_AGENT_EVENT { 

    sharedvar THIS_USER 

   if { [info exists THIS_USER] } { log local0. "THIS_USER = $THIS_USER" } 

  

    sharedvar THIS_HOST 

   if { [info exists THIS_HOST] } { 

  

        ## parse incoming request URL 

       if { ${THIS_HOST} starts_with "http://" } {  

           set this_url [string trimright [string trimleft ${THIS_HOST} "http://"] "/"]  

       } else { 

           set this_url [findstr ${THIS_HOST} "" 0 ":"]     

       } 

  

       ## issue MWG API sideband call 

       set conn_id [connect -timeout 100 -idle 30 -status conn_status 198.19.96.61:2082]  ## Use MWG IP and appropriate port

       set data "HEAD /?url=${this_url} HTTP/1.1\r\nHost: 198.19.96.61\r\n\r\n" 

       set send_bytes [send -timeout 500 -status send_status $conn_id $data] 

       set recv_data [recv -peek -timeout 500 -status recv_status $conn_id] 

  

       ## parse sideband results 

       set X_CATEGORY [findstr [lsearch -inline [split $recv_data "\r\n"] X-Categories*] ": " 2] 

        set X_DECRYPT_DECISION [findstr [lsearch -inline [split $recv_data "\r\n"] X-Decrypt-Decision*] ": " 2] 

  

       ## evaluate the provided URL to decide which SSL Orchestrator listener to engage 

       if { ${X_DECRYPT_DECISION} eq "Bypass" } { 

           virtual "/Common/sslo_Bypass.app/sslo_Bypass-in-t-4" 

       } else { 

           virtual "/Common/sslo_Intercept.app/sslo_Intercept-in-t-4" 

       } 

   } else { 

    ## default action if THIS_HOST is not provided (in this case decrypt) 

       virtual "/Common/sslo_Intercept.app/sslo_Intercept-in-t-4" 

   } 

 

At this stage, client explicit proxy traffic passing to the F5 explicit proxy listener is forwarded to the TCP tunnel virtual server. The Access per-request profile attached to that virtual server triggers the iRule event, which makes an API call to the MWG API listener with the captured request URL. The MWG API returns an X-Decrypt-Decision header that the rule then uses to send the traffic to one of the internal SSL Orchestrator topologies. You can customize this step to include/exclude decisions from other headers (eg: X-Allow-Decision), to refine the topology selection, as needed.   

 

Testing 

To test this configuration, SSH to the BIG-IP and issue a set of cURL commands, to the VIP/MWG IP address on the appropriate port. In this simple set of policy rules, MWG is looking for a “url” query string: 

  

# curl -I http://198.19.96.61:2082?url=www.example.com 

HTTP/1.1 403 URLBlocked 

Via: 1.1 198.19.96.61 (McAfee Web Gateway 9.1.0.31562) 

Connection: Keep-Alive 

Content-Type: text/html 

X-Categories: Technical Information 

Cache-Control: no-cache 

X-Frame-Options: deny 

X-Decrypt-Decision: Intercept 

  

# curl -I http://198.19.96.61:2082?url=www.bankofamerica.com   

HTTP/1.1 403 URLBlocked 

Via: 1.1 198.19.96.61 (McAfee Web Gateway 9.1.0.31562) 

Connection: Keep-Alive 

Content-Type: text/html 

X-Categories: Finance/Banking 

Cache-Control: no-cache 

X-Frame-Options: deny 

X-Decrypt-Decision: Bypass 


Summary

Sideband calls, embedded into iRules can provide a powerful, flexible way to integrate with 3rd party tools to obtain policy decisions to incorporate into an SSL orchestrator flow. With some configuration, customization and tuning, this solution allows relatively seamless integration between tools, while leveraging layered architecture on the SSL orchestrator. This article illustrates the use of Explicit proxy config, but the same outcome can be accomplished using a transparent proxy configuration 


Comments
PKI
F5 Employee
F5 Employee

Traffic Steering with 3rd party Policy Manager (Layered Architecture with Explicit Proxy)

 

Version history
Last update:
‎01-Jun-2021 07:54
Updated by:
F5 Employee
Contributors