Traffic Steering with 3rd party Policy Manager (Layered Architecture with Explicit Proxy)

Introduction

A previous advanced use case described the use of Internal Layered Architecture for reducing complexity (ref: https://devcentral.f5.com/s/articles/SSL-Orchestrator-Advanced-Use-Cases-Reducing-Complexity-with-Internal-Layered-Architecture)

Some organizations standardize their policies and/or operational procedures based on a platform/tool, to manage security and access policies for outbound access. One interesting design that is gaining popularity is leveraging a 3rd party tool, via sideband API calls, to obtain a traffic steering/management decision. Organizations with deep investments in a specific tool/technology for policy management, cannot transition to a net-new solution without significant friction. If this tool/system has a REST API interface or a way to create a service to respond to HTTP queries, the BIG-IP can initiate a sideband API call, requesting such a decision via an HTTP request, and leverage that decision in the SSL Orchestrator flow. Here’s what that looks like:

Here’s a quick rundown of the flow by step numbers:

Client attempts to browse to www.xyz.com. Client points to the Explicit Proxy VIP address (usually achieved by PAC file entry), which is the first of the multi-layered SSL Orchestrator architecture.
The BIG-IP holds the client request, extracts headers(eg: username, source IP, Dest URL) from the HTTP Connect, and initiates a sideband API call to the 3rd Party policy manager (can be anything), including the required headers that the policy manager needs to provide a decision.
Policy manager replies with a custom value, usually a custom HTTP header, that provides a decision/action that SSL Orchestrator can use to make a traffic steering/management decision. Example, if the decision header is named “X_Decrypt_Decision”:
X_Decrypt_Decision == Intercept - Send to Intercept topology
X_Decrypt_Decision == Bypass - Send to Bypass topology
X_Decrypt_Decision == Block - Send to Block Topology
Based on the policy decision, SSLo steers the traffic to the appropriate topology via VIP targeting (example iRule below)
Depending on which topology was targeted, traffic passes through the respective service chain for typical security inspection steps
Finally, once the service chain has been traversed, BIG-IP sends the traffic out the egress path as defined (can be a single egress path, or topology-specific paths)

This method can provide significant flexibility, there are some minor drawbacks/trade-offs, that should be considered. A few have been listed here:

Pros:

1) Flexibility to define/manage policies on a 3rd party policy manager/engine

2) Ability to create conditional logic with multiple decisions from policy manager, before a bypass/intercept/allow decision is made

3) Limit or remove need for URL database management and update on BIG-IP devices

Cons:

1) Additional latency when performing sideband calls to 3rd party tool

2) Dependency on 3rd party tool for a traffic steering decision

3) Multiple logging sources, resulting in some troubleshooting complexity

We’ll walk through the steps of configuring this behavior, using a McAfee Web Gateway as our policy manager. The same principle applies to other products (eg: Broadcom ProxySG). Here, we use an Explicit Forward proxy VIP, that is a manually created VIP (i.e. not created by an SSL Orchestrator topology), to perform additional functions at the traffic steering point. A manually created virtual server allows us to insert the conditional sideband calls and traffic steering decision, early into the traffic flow, as the traffic would be too far down the road, if it already made it to the Service Chain. While the configuration here uses an Explicit forward proxy, it doesn’t have to be so, and can be a transparent proxy as well.

Let’s start with the MWG:

McAfee Web Gateway API Configuration

LTM explicit proxy VIP trigger and iRule that generates an API call to McAfee Web Gateway GTI interface on client request and MWG returns policy actions (intercept/bypass/block). MWG thus maintains sole control of policy engine (single source of truth) – no need for URLDB subscription on SSL Orchestrator.

Initially this requires configuration on the MWG to support this mechanism, and the following presents basic settings to enable this. In this case, the MWG will be configured to listen on an alternate (proxy) port, and a corresponding blocking policy will respond directly to HTTP headers sent to it.

Proxy

In the MWG UI, under the Configuration section -> Appliance -> Proxies, configure a new HTTP Proxy listener on a new port (ex. 2082). A policy will listen on this port and respond to API calls.

Policy

In the MWG UI, under the Policy section, create a new Rule Set.

Enable: checked
Applies to: check all
Apply this rule set: if the following criteria is matched:
Proxy.Port equals 2082

The MWG policy should look something like this:

Rules

Get URL and CLIENTIP from Parameter

## This rule extracts a URL from a “url” query string and client IP from a “clientip” query string, and stores in “URL” and “CLIENTIP” variables respectively

Rule Criteria: URL.HasParameter(“url”) equals true
Action: Continue
Events: Set URL = URL.GetParameter(“url”)
Rule Criteria: URL.HasParameter(“clientip”) equals true
Action: Continue
Events: Set clientip = URL.GetParameter(“clientip”)
Action: Continue

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.

Note: “transid”, shown in the screenshot above, is an additional field that can be configured to assign a user-defined transaction ID that be used to correlate traffic components between SSL Orchestrator and McAfee Web Gateway. This is not a hard requirement for this setup

Initialize Headers

## This rule sets a default “Intercept” value on the “X-Decrypt-Decision” header, in the event that other conditions don’t reset the value.

Rule Criteria: Always
Action: Continue
Events: Header.Block.Add(“X-Auth-Decision”, “auth”)
Events: Header.Block.Add(“X-Allow-Decision”, “allow”)
Events: Header.Block.Add(“X-Decrypt-Decision”, “Intercept”)

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.

Get Categories

## This rule creates an “X-Categories” header that contains the list of matched URL categories, and will be returned to the caller.

Rule Criteria: Always
Action: Continue
Events:Header.Block.Add(“X-Categories”, ListOfCategory.ToString(URL.Categories<Default>))

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.

Set Bypass Header

## This rule resets the “X-Decrypt-Decision” header to “Bypass” on a URL category match for Finance/Banking URLs.

Rule Criteria: URL.Categories<Default> contains Finance / Banking
Action: Continue
Events: Header.Block.Add(“X-Decrypt-Decision”, “bypass”)

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.

Set NoAuth Header

## This rule sets the “X-Auth-Decision” to “noauth” on a url match for “httpbin.org”. This rule set can be used to add URLs, for which you don’t need users to be authenticated

Rule Criteria: URL equals “httpbin.org:
Action: Continue
Events: Header.Block.Add(“X-Auth-Decision”, “noauth”)

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.

Set Block Header

## This rule sets the “X-Allow-Decision” to “block” for a URL match on “www.bing.com”. Use this rule set to explicitly define blocked URLs, that may not be in the URL categorization

Rule Criteria: URL equals “www.bing.com”
Action: Continue
Events: Header.Block.Add(“X-Allow-Decision”, “block”)

The sequence of screenshots below illustrates the process of setting this up in the MWG UI.

Final Block

## This rule is a catch-all to set a static response to API calls in case a previous rule has not matched

Rule Criteria: Always
Action: Block
Events: None

The final ruleset looks as follows:

F5 BIG-IP Configuration

LTM TCP Tunnel - A custom TCP tunnel is created to provide a channel for non-TLS traffic, to be directed through. This is to ensure that our topology selection logic, catches ALL of the traffic, not just the HTTPS (TLS) Traffic.

In the BIG-IP UI, under Network -> Tunnels, click the Create button. The tunnel is used by the explicit proxy configuration.

Profile: tcp-forward

LTM HTTP Explicit Profile - A custom HTTP Explicit Profile allows a VIP to accept and act upon an HTTP CONNECT request

In the BIG-IP UI, under Local Traffic -> Profiles -> Services -> HTTP, click the Create button. This will create an HTTP explicit proxy profile.

Parent Profile: http-explicit
Explicit Proxy
Tunnel Name: <LTM TCP tunnel>

LTM Explicit Proxy VIP (frontend)

In the BIG-IP UI, under Local Traffic -> Virtual Servers, click the Create button. An HTTP explicit proxy configuration is made up of an explicit proxy listener (what the client is configured to talk to), and a separate TCP tunnel virtual server. Client proxy CONNECT traffic is received by the explicit proxy virtual server, and the TCP proxy tunnel is established between the client and the TCP tunnel virtual server (through the explicit proxy virtual server. The following creates the explicit proxy listener virtual server.

Type: Standard
Source: 0.0.0.0/0
Destination: client-facing explicit proxy IP (the IP the client is configured to talk to)
Service Port: client-facing explicit proxy port
HTTP Profile (Client): <LTM HTTP explicit profile>
VLANs and Tunnels: client facing VLAN
Address Translation: enabled
Port Translation: enabled
Access Profile: <see access profiles below>
iRule: <see iRules below>

LTM TCP Tunnel VIP (frontend)

In the BIG-IP UI, under Local Traffic -> Virtual Servers, click the Create button. The following creates the TCP tunnel virtual server.

Type: Standard
Source: 0.0.0.0/0
Destination: 0.0.0.0/0
Service Port: 0 (any)
VLANs and Tunnels: <LTM TCP tunnel>
Address Translation: disabled
Port Translation: disabled
Access Profile: <see access profiles below>
Per-Request Policy: <see per-request policy below>
iRule: <see iRules below>

SSL Orchestrator

Create separate topologies (as functions). These will essentially be “internal” (not listening on a client facing VLAN) and will receive traffic directly from the frontend LTM virtual servers. Topologies as functions can be simple, including a small finite set of actions (ex. allow/block, intercept/bypass, service chain), and may include different egress, SNAT and logging settings. The primary benefits on this design are that a) topology configuration can remain relatively static, reducing management overhead, b) based on traffic rules the number of topologies may typically be reduced to a common set of functions, versus potentially multiple topologies to cover multiple traffic types, and c) topologies can be created purely for testing different traffic flows, without affecting production traffic. This last benefit is extremely useful when enabled to test and analyze misbehaving applications/protocols, and/or to validate new security products. The following is an example configuration of two separate topology functions, one for TLS intercept, and one for TLS bypass. As traffic is passing to them from the frontend explicit proxy, these topologies are configured as L3 outbound (transparent proxy). Refer to official F5 documentation for detailed SSL Orchestrator configuration steps (as needed).

Topology 1 (Bypass function)

Topology
L3 Outbound
Services
Create security services as needed
Service chain(s)
Define at least one service chain to incorporate the set of services
Security Policy
Remove the Pinners rule
All Traffic rule
Action: Allow
SSL Forward Proxy Action: Bypass
Service Chain: send to a service chain as required
Interception Rule
VLANs: create and assign a “dummy” VLAN (no interfaces assigned)
Egress Setting
Configure as required for Internet next hop

Topology 2 (Intercept function)

Topology
L3 Outbound
Services
Create security services as needed (if not already created)
Service chain(s)
Define at least one service chain to incorporate the set of services (if not already defined)
Security Policy
All Traffic rule
Action: Allow
SSL Forward Proxy Action: Intercept
Service Chain: send to service chain as required
Interception Rule
VLANs: create and assign a “dummy” VLAN (no interfaces assigned)
Egress Setting
Configure as required for Internet next hop

Note – Create additional topologies for Block/Test/etc actions, like the 2 topologies created above

Per-Request Profiles

While Access per-session policies are typically initiated once per authenticated user session, per-request access profiles are defined to act on “transactional” data, as in per-HTTP request.

TCP tunnel per-request policy (attached to TCP tunnel VIP)
Type: All
Incomplete Action: Allow
Customization Type: Standard
VPE:
Start
iRule Event (ID arbitrary, Expect Data: Client Accepted)
Allow

iRules

In the BIG-IP UI, under Local Traffic -> iRules, click the Create button. The frontend explicit proxy configuration will utilize two iRules, one on each virtual server, to facilitate traffic flow to the SSL Orchestrator topologies.

Explicit proxy listener iRule – attached to the explicit proxy listener virtual server, this rule programmatically forwards all incoming traffic to the TCP tunnel virtual server. In the ‘virtual’ command below, replace the pathname of the TCP tunnel virtual server with the correct local name.

when HTTP_PROXY_REQUEST {

## get proxy request URL

sharedvar THIS_HOST

set THIS_HOST [HTTP::uri]

}

when HTTP_REQUEST {

## force HTTP traffic to tunnel VIP

virtual "/Common/tcp-tunnel-vip" [HTTP::proxy addr] [HTTP::proxy port]

}

TCP tunnel iRule – attached to the TCP tunnel virtual server, this rule programmatically forwards traffic to one of the SSL Orchestrator topology listeners based on some criteria. The iRule is triggered by the iRule event agent in the per-request access profile applied to the TCP tunnel virtual server. The below is an example of a fairly simple set of conditions. In the section that follows the “## evaluate …” line, define a set of conditions that decide which SSL Orchestrator topology listener to forward the traffic flow to. The normalized host value is provided in the “this_url” variable (ex. www.example.com), but the iRule also has access to source IP [IP::client_addr], destination IP [IP::local_addr], ports, and an array of other flows values. For each “virtual” command, replace the pathname with an SSL Orchestrator topology listener virtual server. The virtual servers may end with:

-in-t-4 (representing an IPv4 TCP listener)
-in-t-6 (representing an IPv6 TCP listener)

Topology Selector iRule

In the BIG-IP UI, under Local Traffic -> iRules, click the Create button. The rule below is applied to the frontend TCP tunnel virtual server. In the example below, the sideband call is direct to the MWG IP:port, but can be configured to point to an LTM virtual server that load balances multiple MWGs. Simply replace the IP:port in the “connect” command with the pathname of the virtual server. The sideband call makes a HEAD request, returning only the response headers. The X-Decrypt-Decision and X-Category headers are parsed out of the reponse, and the X-Decryption-Decision value is used to decide which SSL Orchestrator topology to send the traffic to. In the example rule, the X-Category header is superfluous, but could be used for logging actions.

when ACCESS_PER_REQUEST_AGENT_EVENT {

sharedvar THIS_USER

if { [info exists THIS_USER] } { log local0. "THIS_USER = $THIS_USER" }

sharedvar THIS_HOST

if { [info exists THIS_HOST] } {

## parse incoming request URL

if { ${THIS_HOST} starts_with "http://" } {

set this_url [string trimright [string trimleft ${THIS_HOST} "http://"] "/"]

} else {

set this_url [findstr ${THIS_HOST} "" 0 ":"]

}

## issue MWG API sideband call

set conn_id [connect -timeout 100 -idle 30 -status conn_status 198.19.96.61:2082] ## Use MWG IP and appropriate port

set data "HEAD /?url=${this_url} HTTP/1.1\r\nHost: 198.19.96.61\r\n\r\n"

set send_bytes [send -timeout 500 -status send_status $conn_id $data]

set recv_data [recv -peek -timeout 500 -status recv_status $conn_id]

## parse sideband results

set X_CATEGORY [findstr [lsearch -inline [split $recv_data "\r\n"] X-Categories*] ": " 2]

set X_DECRYPT_DECISION [findstr [lsearch -inline [split $recv_data "\r\n"] X-Decrypt-Decision*] ": " 2]

## evaluate the provided URL to decide which SSL Orchestrator listener to engage

if { ${X_DECRYPT_DECISION} eq "Bypass" } {

virtual "/Common/sslo_Bypass.app/sslo_Bypass-in-t-4"

} else {

virtual "/Common/sslo_Intercept.app/sslo_Intercept-in-t-4"

}

} else {

## default action if THIS_HOST is not provided (in this case decrypt)

virtual "/Common/sslo_Intercept.app/sslo_Intercept-in-t-4"

}

At this stage, client explicit proxy traffic passing to the F5 explicit proxy listener is forwarded to the TCP tunnel virtual server. The Access per-request profile attached to that virtual server triggers the iRule event, which makes an API call to the MWG API listener with the captured request URL. The MWG API returns an X-Decrypt-Decision header that the rule then uses to send the traffic to one of the internal SSL Orchestrator topologies. You can customize this step to include/exclude decisions from other headers (eg: X-Allow-Decision), to refine the topology selection, as needed.

Testing

To test this configuration, SSH to the BIG-IP and issue a set of cURL commands, to the VIP/MWG IP address on the appropriate port. In this simple set of policy rules, MWG is looking for a “url” query string:

# curl -I http://198.19.96.61:2082?url=www.example.com

HTTP/1.1 403 URLBlocked

Via: 1.1 198.19.96.61 (McAfee Web Gateway 9.1.0.31562)

Connection: Keep-Alive

Content-Type: text/html

X-Categories: Technical Information

Cache-Control: no-cache

X-Frame-Options: deny

X-Decrypt-Decision: Intercept

# curl -I http://198.19.96.61:2082?url=www.bankofamerica.com

HTTP/1.1 403 URLBlocked

Via: 1.1 198.19.96.61 (McAfee Web Gateway 9.1.0.31562)

Connection: Keep-Alive

Content-Type: text/html

X-Categories: Finance/Banking

Cache-Control: no-cache

X-Frame-Options: deny

X-Decrypt-Decision: Bypass

Summary

Sideband calls, embedded into iRules can provide a powerful, flexible way to integrate with 3rd party tools to obtain policy decisions to incorporate into an SSL orchestrator flow. With some configuration, customization and tuning, this solution allows relatively seamless integration between tools, while leveraging layered architecture on the SSL orchestrator. This article illustrates the use of Explicit proxy config, but the same outcome can be accomplished using a transparent proxy configuration

Published Jun 01, 2021

Version 1.0

BIG-IP

security

series-ssl-orchestrator-advanced-use-cases

SSL Orchestrator