Python SDK Cookbook: Encoding URLs and Using Proxies

Every now and then I get an inquiry if the Python SDK can do something I haven't personally used or seen a need for. In this article, I'll cover two such use cases: encoding URLs and using proxies to connect to BIG-IP. Both of these features aren’t technically supported by the f5-common-python SDK, but rather the underlying f5-icontrol-rest-python module that builds the requests and communicates with BIG-IP.

Encoding URLs

 In this issue, a user was trying to submit a get request to BIG-IP with a node name of 172.16.1.5%0. The actual request with a partition of test included was (in this example using the underlying iControlRESTSession):

from f5.bigip import ManagementRoot
b = ManagementRoot('localhost', 'admin', 'admin', port=13443)
name = '172.16.1.5%0'
b.tm.ltm.nodes.node.load(name=name, partition='test')
Traceback (most recent call last):
 ......
    raise iControlUnexpectedHTTPError(error_message, response=response)
iControlUnexpectedHTTPError: 400 Unexpected Error: Bad Request for uri: https://localhost:13443/mgmt/tm/ltm/node/~test~172.16.1.5%0

This resulted in a 400 error from BIG-IP because of the percent sign. The easy end-user fix (read: workaround) for this is to simply encode the node name with the urllib module before making the request:

from f5.bigip import ManagementRoot
import urllib
name = '172.16.1.5%0'
encoded_name = urllib.quote(name)
b.tm.ltm.nodes.node.load(name=encoded_name, partition='test')

b.tm.ltm.nodes.node.load(name=encoded_name, partition='test').raw
{u'kind': u'tm:ltm:node:nodestate', u'logging': u'disabled', u'name': u'172.16.1.5%0', u'state': u'unchecked', u'generation': 1443, u'partition': u'test', ...removed for clarity..., u'monitor': u'default'}

We could also fix the problem in the iControlREST module, but if this turns out to be a rare one-off where users name their objects with special characters I'll focus on bigger issues that need attention.

Using a Proxy

With this use case, my first response was “yeah, this is possible but it’s likely going to take some significant development time.” But when I had a little more time to investigate, I suspected that the iControlREST library already supported this with the native python requests module. But in order to confirm my suspicions, I needed a proxy on my development workstation. Enter Burp Suite Community Edition. I use this on my Mac a lot for various intercept and packet crafting and analysis purposes, but had yet to have a reason to install on my windows box where I have migrated most of my BIG-IP python-related development. To activate the proxy, I just start the executable and intercept is on by default. To use a proxy from the SDK, you just need to supply the proxies kwarg with a json blob specifying your proxy properties, for example: (#hattip Stack)

pxy = {
  "http": "http://10.10.1.10:3128",
  "https": "https://10.10.1.10:1080",
}

Burp Suite defaults to loopback address 127.0.0.1 and port 8080 for the proxy. So with that in mind, we can build the same query as show above with url encoding from the SDK, but this time with a proxy configuration like so:

proxy = {
    "https":"https://127.0.0.1:8080"
}
b.tm.ltm.nodes.node.load(name=encoded_name, partition='test', proxies=proxy).raw

Here is the request Burp Suite intercepted:

GET /mgmt/tm/ltm/node/~test~172.16.1.5%250 HTTP/1.1
Host: localhost:13443
Connection: close
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.21.0 f5-icontrol-rest-python/1.3.12
Content-Type: application/json
Cookie: BIGIPAuthUsernameCookie=admin; BIGIPAuthCookie=C74B069117329FE004C7840047A1DA3707965613
Authorization: Basic YWRtaW46YWRtaW4=

Immediately after I click forward in Burp Suite, I get my returned json blob from the BIG-IP:

{u'kind': u'tm:ltm:node:nodestate', u'logging': u'disabled', u'name': u'172.16.1.5%0', u'state': u'unchecked', u'generation': 1443, u'partition': u'test', ...removed for clarity..., u'monitor': u'default'}

That's all well and good, but the keen observers among you might notice that the proxies kwarg in my example above was applied to an existing session, not at instantiation with ManagementRoot. So whereas it's helpful to know that proxy support is native in the requests module, it's not supported in f5-icontrol-rest versions through 1.3.12 and f5-sdk versions through 3.0.20. But wait! Effective today, you can upgrade your SDK to the latest version, 3.0.21, for proxy support. Assuming as I've shown above I have a local BIG-IP Virtual Edition at localhost:13443 and a proxy at 127.0.0.1:8080, here's how you activate the proxy:

>>>from f5.bigip import ManagementRoot
>>>proxy = {"https": "https://127.0.0.1:8080}
>>>b = ManagementRoot('localhost', 'admin', 'admin', proxies=proxy)

I made sure to include the proxy configuration in the session object metadata, so you can reference as needed:

>>> b._meta_data['proxies']
{'https': 'https://127.0.0.1:8080'}

I love learning new things! What issues have you stymied with the SDK that I can investigate or help fill in the gaps? Drop a line in the comments below.