on 01-Jun-2021 07:54
A previous advanced use case described the use of Internal Layered Architecture for reducing complexity (ref: https://devcentral.f5.com/s/articles/SSL-Orchestrator-Advanced-Use-Cases-Reducing-Complexity-with-Internal-Layered-Architecture)
Some organizations standardize their policies and/or operational procedures based on a platform/tool, to manage security and access policies for outbound access. One interesting design that is gaining popularity is leveraging a 3rd party tool, via sideband API calls, to obtain a traffic steering/management decision. Organizations with deep investments in a specific tool/technology for policy management, cannot transition to a net-new solution without significant friction. If this tool/system has a REST API interface or a way to create a service to respond to HTTP queries, the BIG-IP can initiate a sideband API call, requesting such a decision via an HTTP request, and leverage that decision in the SSL Orchestrator flow. Here’s what that looks like:
w
Here’s a quick rundown of the flow by step numbers:
This method can provide significant flexibility, there are some minor drawbacks/trade-offs, that should be considered. A few have been listed here:
Pros:
1) Flexibility to define/manage policies on a 3rd party policy manager/engine
2) Ability to create conditional logic with multiple decisions from policy manager, before a bypass/intercept/allow decision is made
3) Limit or remove need for URL database management and update on BIG-IP devices
Cons:
1) Additional latency when performing sideband calls to 3rd party tool
2) Dependency on 3rd party tool for a traffic steering decision
3) Multiple logging sources, resulting in some troubleshooting complexity
We’ll walk through the steps of configuring this behavior, using a McAfee Web Gateway as our policy manager. The same principle applies to other products (eg: Broadcom ProxySG). Here, we use an Explicit Forward proxy VIP, that is a manually created VIP (i.e. not created by an SSL Orchestrator topology), to perform additional functions at the traffic steering point. A manually created virtual server allows us to insert the conditional sideband calls and traffic steering decision, early into the traffic flow, as the traffic would be too far down the road, if it already made it to the Service Chain. While the configuration here uses an Explicit forward proxy, it doesn’t have to be so, and can be a transparent proxy as well.
Let’s start with the MWG:
Initially this requires configuration on the MWG to support this mechanism, and the following presents basic settings to enable this. In this case, the MWG will be configured to listen on an alternate (proxy) port, and a corresponding blocking policy will respond directly to HTTP headers sent to it.
Proxy
In the MWG UI, under the Configuration section -> Appliance -> Proxies, configure a new HTTP Proxy listener on a new port (ex. 2082). A policy will listen on this port and respond to API calls.
Policy
In the MWG UI, under the Policy section, create a new Rule Set.
The MWG policy should look something like this:
Rules
Get URL and CLIENTIP from Parameter
## This rule extracts a URL from a “url” query string and client IP from a “clientip” query string, and stores in “URL” and “CLIENTIP” variables respectively
The sequence of screenshots below illustrates the process of setting this up in the MWG UI.
Note: “transid”, shown in the screenshot above, is an additional field that can be configured to assign a user-defined transaction ID that be used to correlate traffic components between SSL Orchestrator and McAfee Web Gateway. This is not a hard requirement for this setup
Initialize Headers
## This rule sets a default “Intercept” value on the “X-Decrypt-Decision” header, in the event that other conditions don’t reset the value.
The sequence of screenshots below illustrates the process of setting this up in the MWG UI.
Get Categories
## This rule creates an “X-Categories” header that contains the list of matched URL categories, and will be returned to the caller.
The sequence of screenshots below illustrates the process of setting this up in the MWG UI.
Set Bypass Header
## This rule resets the “X-Decrypt-Decision” header to “Bypass” on a URL category match for Finance/Banking URLs.
The sequence of screenshots below illustrates the process of setting this up in the MWG UI.
Set NoAuth Header
## This rule sets the “X-Auth-Decision” to “noauth” on a url match for “httpbin.org”. This rule set can be used to add URLs, for which you don’t need users to be authenticated
The sequence of screenshots below illustrates the process of setting this up in the MWG UI.
Set Block Header
## This rule sets the “X-Allow-Decision” to “block” for a URL match on “www.bing.com”. Use this rule set to explicitly define blocked URLs, that may not be in the URL categorization
The sequence of screenshots below illustrates the process of setting this up in the MWG UI.
Final Block
## This rule is a catch-all to set a static response to API calls in case a previous rule has not matched
The final ruleset looks as follows:
In the BIG-IP UI, under Network -> Tunnels, click the Create button. The tunnel is used by the explicit proxy configuration.
In the BIG-IP UI, under Local Traffic -> Profiles -> Services -> HTTP, click the Create button. This will create an HTTP explicit proxy profile.
In the BIG-IP UI, under Local Traffic -> Virtual Servers, click the Create button. An HTTP explicit proxy configuration is made up of an explicit proxy listener (what the client is configured to talk to), and a separate TCP tunnel virtual server. Client proxy CONNECT traffic is received by the explicit proxy virtual server, and the TCP proxy tunnel is established between the client and the TCP tunnel virtual server (through the explicit proxy virtual server. The following creates the explicit proxy listener virtual server.
In the BIG-IP UI, under Local Traffic -> Virtual Servers, click the Create button. The following creates the TCP tunnel virtual server.
Create separate topologies (as functions). These will essentially be “internal” (not listening on a client facing VLAN) and will receive traffic directly from the frontend LTM virtual servers. Topologies as functions can be simple, including a small finite set of actions (ex. allow/block, intercept/bypass, service chain), and may include different egress, SNAT and logging settings. The primary benefits on this design are that a) topology configuration can remain relatively static, reducing management overhead, b) based on traffic rules the number of topologies may typically be reduced to a common set of functions, versus potentially multiple topologies to cover multiple traffic types, and c) topologies can be created purely for testing different traffic flows, without affecting production traffic. This last benefit is extremely useful when enabled to test and analyze misbehaving applications/protocols, and/or to validate new security products. The following is an example configuration of two separate topology functions, one for TLS intercept, and one for TLS bypass. As traffic is passing to them from the frontend explicit proxy, these topologies are configured as L3 outbound (transparent proxy). Refer to official F5 documentation for detailed SSL Orchestrator configuration steps (as needed).
Topology 1 (Bypass function)
Topology 2 (Intercept function)
Note – Create additional topologies for Block/Test/etc actions, like the 2 topologies created above
While Access per-session policies are typically initiated once per authenticated user session, per-request access profiles are defined to act on “transactional” data, as in per-HTTP request.
In the BIG-IP UI, under Local Traffic -> iRules, click the Create button. The frontend explicit proxy configuration will utilize two iRules, one on each virtual server, to facilitate traffic flow to the SSL Orchestrator topologies.
when HTTP_PROXY_REQUEST {
## get proxy request URL
sharedvar THIS_HOST
set THIS_HOST [HTTP::uri]
}
when HTTP_REQUEST {
## force HTTP traffic to tunnel VIP
virtual "/Common/tcp-tunnel-vip" [HTTP::proxy addr] [HTTP::proxy port]
}
TCP tunnel iRule – attached to the TCP tunnel virtual server, this rule programmatically forwards traffic to one of the SSL Orchestrator topology listeners based on some criteria. The iRule is triggered by the iRule event agent in the per-request access profile applied to the TCP tunnel virtual server. The below is an example of a fairly simple set of conditions. In the section that follows the “## evaluate …” line, define a set of conditions that decide which SSL Orchestrator topology listener to forward the traffic flow to. The normalized host value is provided in the “this_url” variable (ex. www.example.com), but the iRule also has access to source IP [IP::client_addr], destination IP [IP::local_addr], ports, and an array of other flows values. For each “virtual” command, replace the pathname with an SSL Orchestrator topology listener virtual server. The virtual servers may end with:
Topology Selector iRule
In the BIG-IP UI, under Local Traffic -> iRules, click the Create button. The rule below is applied to the frontend TCP tunnel virtual server. In the example below, the sideband call is direct to the MWG IP:port, but can be configured to point to an LTM virtual server that load balances multiple MWGs. Simply replace the IP:port in the “connect” command with the pathname of the virtual server. The sideband call makes a HEAD request, returning only the response headers. The X-Decrypt-Decision and X-Category headers are parsed out of the reponse, and the X-Decryption-Decision value is used to decide which SSL Orchestrator topology to send the traffic to. In the example rule, the X-Category header is superfluous, but could be used for logging actions.
when ACCESS_PER_REQUEST_AGENT_EVENT {
sharedvar THIS_USER
if { [info exists THIS_USER] } { log local0. "THIS_USER = $THIS_USER" }
sharedvar THIS_HOST
if { [info exists THIS_HOST] } {
## parse incoming request URL
if { ${THIS_HOST} starts_with "http://" } {
set this_url [string trimright [string trimleft ${THIS_HOST} "http://"] "/"]
} else {
set this_url [findstr ${THIS_HOST} "" 0 ":"]
}
## issue MWG API sideband call
set conn_id [connect -timeout 100 -idle 30 -status conn_status 198.19.96.61:2082] ## Use MWG IP and appropriate port
set data "HEAD /?url=${this_url} HTTP/1.1\r\nHost: 198.19.96.61\r\n\r\n"
set send_bytes [send -timeout 500 -status send_status $conn_id $data]
set recv_data [recv -peek -timeout 500 -status recv_status $conn_id]
## parse sideband results
set X_CATEGORY [findstr [lsearch -inline [split $recv_data "\r\n"] X-Categories*] ": " 2]
set X_DECRYPT_DECISION [findstr [lsearch -inline [split $recv_data "\r\n"] X-Decrypt-Decision*] ": " 2]
## evaluate the provided URL to decide which SSL Orchestrator listener to engage
if { ${X_DECRYPT_DECISION} eq "Bypass" } {
virtual "/Common/sslo_Bypass.app/sslo_Bypass-in-t-4"
} else {
virtual "/Common/sslo_Intercept.app/sslo_Intercept-in-t-4"
}
} else {
## default action if THIS_HOST is not provided (in this case decrypt)
virtual "/Common/sslo_Intercept.app/sslo_Intercept-in-t-4"
}
}
At this stage, client explicit proxy traffic passing to the F5 explicit proxy listener is forwarded to the TCP tunnel virtual server. The Access per-request profile attached to that virtual server triggers the iRule event, which makes an API call to the MWG API listener with the captured request URL. The MWG API returns an X-Decrypt-Decision header that the rule then uses to send the traffic to one of the internal SSL Orchestrator topologies. You can customize this step to include/exclude decisions from other headers (eg: X-Allow-Decision), to refine the topology selection, as needed.
Testing
To test this configuration, SSH to the BIG-IP and issue a set of cURL commands, to the VIP/MWG IP address on the appropriate port. In this simple set of policy rules, MWG is looking for a “url” query string:
# curl -I http://198.19.96.61:2082?url=www.example.com
HTTP/1.1 403 URLBlocked
Via: 1.1 198.19.96.61 (McAfee Web Gateway 9.1.0.31562)
Connection: Keep-Alive
Content-Type: text/html
X-Categories: Technical Information
Cache-Control: no-cache
X-Frame-Options: deny
X-Decrypt-Decision: Intercept
# curl -I http://198.19.96.61:2082?url=www.bankofamerica.com
HTTP/1.1 403 URLBlocked
Via: 1.1 198.19.96.61 (McAfee Web Gateway 9.1.0.31562)
Connection: Keep-Alive
Content-Type: text/html
X-Categories: Finance/Banking
Cache-Control: no-cache
X-Frame-Options: deny
X-Decrypt-Decision: Bypass
Sideband calls, embedded into iRules can provide a powerful, flexible way to integrate with 3rd party tools to obtain policy decisions to incorporate into an SSL orchestrator flow. With some configuration, customization and tuning, this solution allows relatively seamless integration between tools, while leveraging layered architecture on the SSL orchestrator. This article illustrates the use of Explicit proxy config, but the same outcome can be accomplished using a transparent proxy configuration