Getting started with the python SDK part 4: working with request parameters

In the previous article in this series, we looked at how to work with statistics. In this article, we’ll mirror some of the naked query parameters covered in the Demystifying iControl REST part 3 article, but applied through the python sdk.

The super secret magic sauce is all in the formatting:

requests_params = {‘params’: ‘insert parameters here’}

You can read all about the different query parameters in teh demystifying article I linked above. Let's just go through a few examples here that might be helpful. First, consider a pool collection. With no parameters, you get a list of all your pools and their (set) attributes like this (only one pool shown for brevity):

>>> from sprint import print as pp
>>> pools = b.tm.ltm.pools.get_collection()
>>> for pool in pools:
...    pp(pool.raw)

{u'allowNat': u'yes',
 u'allowSnat': u'yes',
 u'fullPath': u'/bc/tp1',
 u'generation': 94,
 u'ignorePersistedWeight': u'disabled',
 u'ipTosToClient': u'pass-through',
 u'ipTosToServer': u'pass-through',
 u'kind': u'tm:ltm:pool:poolstate',
 u'linkQosToClient': u'pass-through',
 u'linkQosToServer': u'pass-through',
 u'loadBalancingMode': u'least-connections-node',
 u'membersReference': {u'isSubcollection': True,
                       u'link': u'https://localhost/mgmt/tm/ltm/pool/~bc~tp1/members?ver=13.1.0.5'},
 u'minActiveMembers': 0,
 u'minUpMembers': 0,
 u'minUpMembersAction': u'failover',
 u'minUpMembersChecking': u'disabled',
 u'name': u'tp1',
 u'partition': u'bc',
 u'queueDepthLimit': 0,
 u'queueOnConnectionLimit': u'disabled',
 u'queueTimeLimit': 0,
 u'reselectTries': 0,
 u'selfLink': u'https://localhost/mgmt/tm/ltm/pool/~bc~tp1?ver=13.1.0.5',
 u'serviceDownAction': u'none',
 u'slowRampTime': 10}

But wait...there's more! You can get even more data by using the first parameter we'll cover: expandSubcollections. We get the request often on how to get all the pools and pool members in one request, which can be accomplished with this method:

pools_w_members = b.tm.ltm.pools.get_collection(requests_params={'params': 'expandSubcollections=true'})

That's great if you are going to use all that data, but what if you only really want to know the pool name and the load balancing method? It seems a tad overkill to pull all that back, doesn't it? That's where the select parameter comes in handy. We pass our

requests_params

in the

get_collection()

method and we now get a significantly reduced payload, with only the data we want.

>>> pools = b.tm.ltm.pools.get_collection(requests_params={'params': '$select=name,loadBalancingMode'})
>>> pp(pools)
[{u'loadBalancingMode': u'round-robin', u'name': u'testpool'},
 {u'loadBalancingMode': u'least-connections-node', u'name': u'tp1'},
 {u'loadBalancingMode': u'ratio-member', u'name': u'tp2'},
 {u'loadBalancingMode': u'ratio-node', u'name': u'tp1'}]

This is great, but you'll notice that I have two pools named tp1 there. You'll likely also want the partition to make sure you are manipulating the right pool. This can be done by including that in the select, but a better option might be just to use the filter parameter and return only the pools in the partition of choice.

pools = b.tm.ltm.pools.get_collection(requests_params={'params': '$select=name,loadBalancingMode&$filter=partition+eq+bc'})
pp(pools)
[{u'loadBalancingMode': u'least-connections-node', u'name': u'tp1'},
 {u'loadBalancingMode': u'ratio-member', u'name': u'tp2'}]

Like in the browser, you can also just use spaces instead of the + sign on the filter value and it will work just fine. If you had many pools, you could further reduce the data set by using the top parameter. I only have two pools in my bc partition, so for this example I'll take only the top one, but that's ridiculous in a live setting!

pools = b.tm.ltm.pools.get_collection(requests_params={'params': '$select=name,loadBalancingMode&$filter=partition eq bc&$top=1'})
pp(pools)
[{u'loadBalancingMode': u'least-connections-node', u'name': u'tp1'}]

Using top/skip can be very helpful when doing analysis of the active connection table. You can do VERY BAD THINGS to your system by trying to pull the entire table in a single query. The key to the connection table (and other tmsh commands via REST) is to use the options parameter, which for some reason does not have a leading $ like the other parameters. In this case, we are reducing the dataset returned to the server side address matching that IP. You could add a port as well if you had multiple pool members sharing an IP. When combined with top, the query would be built like this:

>>> ssconns = b.tm.sys.connection.load(requests_params={'params': '$top=50&options=ss-server-addr+192.168.103.31'})

I say would be built because as I was preparing this article, I noticed that tm/sys/connection is not yet an endpoint in the SDK, so I created issue 1461 for this and I'll try to get it added for the next release. What else would you like to see regarding request parameters via the SDK (or native for that matter)? Drop a comment below. Happy coding!

Updated Jun 06, 2023

Version 2.0

Admin

Joined January 20, 2005

View Profile

10 Comments

StephanManthey
Nacreous
Jun 15, 2018
Hi Jason,

thanks for this one as well (5*).

Along to
$select
and
filter
I´m using the
options
query parameter as well in other contexts.

Removing configuration objects

remove all management routes by using a globbing wildcard:
method:
DELETE
path:
/mgmt/tm/sys/management-route

query:
options='*'
or
options=all

remove a set of unused pools by using more specific globbing, i.e.:

method:
DELETE

path:
/mgmt/tm/ltm/pool

query:
options='pool_test_*'

Select partitions to save

save i.e. "my_partition" (using encoded spaces and curly braces):

method:
POST

data:
{"command":"save"}

path:
/mgmt/tm/sys/config

query:
options=partitions%20%7B%20my_partition%20%7D

Lookup detailled statistics

retrieve detailled statistics for a pool (i.e. my_pool) including node information:

method:
GET

path:
/mgmt/tm/ltm/pool/~Common~mypool/stats

query:
options=detail

Thanks to you and the other folks for putting up all this stuff.

It helps me a lot to get my job done!

Cheers, Stephan
JRahm
Admin
Jun 15, 2018
Nice ones!
JRahm
Admin
Jun 15, 2018
Given the nature of the way the sdk was built, I’m not sure delete functions via params will work as well because we require an object to be loaded before the delete method can be used. I’ll dig in on Monday, we might have to refactor to support those.
StephanManthey
Nacreous
Jun 15, 2018
Hi Jason, at least the examples above worked exactly this way on v13.1.0.6. Enjoy the weekend! Cheers, Stephan
JRahm
Admin
Jun 21, 2018
Hi @Stephan, changes to sdk have been merged to development branch and will be released as 3.0.17 today or Monday. I wrote up a response article to each of the conditions you mentioned in the comments above so others have the "recipe" to use them. Thanks again!
¬†
StephanManthey
Nacreous
Jun 22, 2018
Hi Jason, I will check it. Thanks! You can confirm my approach using 'options=all' works for you as well? Cheers, Stephan
EarsLikeRocketF
Nimbostratus
Oct 29, 2020
Hi Jason,

is it possible to include additional parameters if subcollections have been expanded?

I'd like to run a single query that returns all VLAN names and tags, as well as the interfaces assigned to each VLAN.
What I have so far.

The following single line query for VLAN name and tag works as expected.
VLANS = lb01.tm.net.vlans.get_collection(requests_params={'params':'$select=name,tag'})

The expand query works and I can loop through the results to get the data I want.
VLANS = chlb001.tm.net.vlans.get_collection(requests_params={'params': {'expandSubcollections':'true'}})
for vlan in VLANS:
...       print(vlan.name)
...       for interface in vlan.interfacesReference['items']:
...            print(interface['name'])
...
VLAN_ First
1.3
VLAN_Second
VLAN_Third
1.2
VLAN_Fourth
1.3

What I would like is a way to combine the expandSubcollections parameter with the $Select to provide the interfaces numbers as well. Below is an example of one of the failed iterations I've tried. In bold are the specific parameters I'd like to retrieve.

VLANS = lb01.tm.net.vlans.get_collection(requests_params={'params': {'expandSubcollections':'true'}}& {'params': '$select=name,tag,interfaces_s'}})

Thanks for any thoughts and ideas!

aj1

Nimbostratus

Dec 03, 2020

Hi Jason,

The articles have been super helpful. Thank you.

I'm trying to retrieve the connection table using the final example but it fails. It does appear to work with load() but the returned data is unstructured. I was hoping the raw output would be a dict, similar to other get_collection() results.

f5-icontrol-rest==1.3.13
f5-sdk==3.0.21
 
In [26]: ssconns = b.tm.sys.connection.get_collection(requests_params={'params': '$top=5&options=ss-server-addr+5.5.5.5'})
---------------------------------------------------------------------------
LazyAttributesRequired          Traceback (most recent call last)
<ipython-input-26-24ce13bb2114> in <module>
----> 1 ssconns = b.tm.sys.connection.get_collection(requests_params={'params': '$top=5&options=ss-server-addr+5.5.5.5'})
 
~/.virtualenvs/venv/lib/python3.8/site-packages/f5/bigip/mixins.py in __getattr__(container, name)
   93      error_message = ('"allowed_lazy_attributes" not in',
   94               'container._meta_data for class %s' % cls_name)
---> 95      raise LazyAttributesRequired(error_message)
   96
   97    # ensure the requested attr is present
 
LazyAttributesRequired: ('"allowed_lazy_attributes" not in', 'container._meta_data for class Connection')

Getting rid of request_params altogether doesn't help either.

In [27]: ssconns = b.tm.sys.connection.get_collection()
---------------------------------------------------------------------------
LazyAttributesRequired                    Traceback (most recent call last)
<ipython-input-27-5e104dde24cc> in <module>
----> 1 ssconns = b.tm.sys.connection.get_collection()
 
~/.virtualenvs/venv/lib/python3.8/site-packages/f5/bigip/mixins.py in __getattr__(container, name)
     93             error_message = ('"allowed_lazy_attributes" not in',
     94                              'container._meta_data for class %s' % cls_name)
---> 95             raise LazyAttributesRequired(error_message)
     96
     97         # ensure the requested attr is present
 
LazyAttributesRequired: ('"allowed_lazy_attributes" not in', 'container._meta_data for class Connection')

Please let me know if I'm missing something in the query or if this is the expected behavior.

Thank you.

JRahm
Admin
Dec 28, 2020
Hi I fixed that last example, it should be sys.connection.load(), sorry about that!

JRahm

Admin

Dec 29, 2020

Hi , sorry for the delayed response, this one fell through the cracks. You can definitely get payload back for a combined selection and expansion, but not exactly as you desire. Because you are doing a selection on the top level object, you need to select name, tag, and then the entire interfacesReference. This is still beneficial to reduce how much useless data you pull back, but also still requires a little post-processing. In postman, the query looks like this:

https://{{host}}/mgmt/tm/net/vlan?expandSubcollections=true&$select=name,tag,interfacesReference

From the sdk, you need to format it as a single request params object:

VLANS = b.tm.net.vlans.get_collection(requests_params={'params': {'expandSubcollections':'true', '$select': 'name,tag,interfacesReference'}})

And the data returned looks like this:

{'name': 'v.10', 'tag': 10, 'interfacesReference': {'link': 'https://localhost/mgmt/tm/net/vlan/~Common~v.10/interfaces?ver=15.1.0.5', 'isSubcollection': True, 'items': [{'kind': 'tm:net:vlan:interfaces:interfacesstate', 'name': '1.3', 'fullPath': '1.3', 'generation': 1, 'selfLink': 'https://localhost/mgmt/tm/net/vlan/~Common~v.10/interfaces/1.3?ver=15.1.0.5', 'tagMode': 'none', 'untagged': True, 'nameReference': {'link': 'https://localhost/mgmt/tm/net/interface/1.3?ver=15.1.0.5'}}]}}

Or in picture format with pretty printing:

>>> from sprint import print as pp\n>>> pools = b.tm.ltm.pools.get_collection()\n>>> for pool in pools:\n... pp(pool.raw)\n\n{u'allowNat': u'yes',\n u'allowSnat': u'yes',\n u'fullPath': u'/bc/tp1',\n u'generation': 94,\n u'ignorePersistedWeight': u'disabled',\n u'ipTosToClient': u'pass-through',\n u'ipTosToServer': u'pass-through',\n u'kind': u'tm:ltm:pool:poolstate',\n u'linkQosToClient': u'pass-through',\n u'linkQosToServer': u'pass-through',\n u'loadBalancingMode': u'least-connections-node',\n u'membersReference': {u'isSubcollection': True,\n u'link': u'https://localhost/mgmt/tm/ltm/pool/~bc~tp1/members?ver=13.1.0.5'},\n u'minActiveMembers': 0,\n u'minUpMembers': 0,\n u'minUpMembersAction': u'failover',\n u'minUpMembersChecking': u'disabled',\n u'name': u'tp1',\n u'partition': u'bc',\n u'queueDepthLimit': 0,\n u'queueOnConnectionLimit': u'disabled',\n u'queueTimeLimit': 0,\n u'reselectTries': 0,\n u'selfLink': u'https://localhost/mgmt/tm/ltm/pool/~bc~tp1?ver=13.1.0.5',\n u'serviceDownAction': u'none',\n u'slowRampTime': 10}\n

>>> pools = b.tm.ltm.pools.get_collection(requests_params={'params': '$select=name,loadBalancingMode'})\n>>> pp(pools)\n[{u'loadBalancingMode': u'round-robin', u'name': u'testpool'},\n {u'loadBalancingMode': u'least-connections-node', u'name': u'tp1'},\n {u'loadBalancingMode': u'ratio-member', u'name': u'tp2'},\n {u'loadBalancingMode': u'ratio-node', u'name': u'tp1'}]\n

pools = b.tm.ltm.pools.get_collection(requests_params={'params': '$select=name,loadBalancingMode&$filter=partition+eq+bc'})\npp(pools)\n[{u'loadBalancingMode': u'least-connections-node', u'name': u'tp1'},\n {u'loadBalancingMode': u'ratio-member', u'name': u'tp2'}]\n

pools = b.tm.ltm.pools.get_collection(requests_params={'params': '$select=name,loadBalancingMode&$filter=partition eq bc&$top=1'})\npp(pools)\n[{u'loadBalancingMode': u'least-connections-node', u'name': u'tp1'}]\n

f5-icontrol-rest==1.3.13\nf5-sdk==3.0.21\n \nIn [26]: ssconns = b.tm.sys.connection.get_collection(requests_params={'params': '$top=5&options=ss-server-addr+5.5.5.5'})\n---------------------------------------------------------------------------\nLazyAttributesRequired Traceback (most recent call last)\n<ipython-input-26-24ce13bb2114> in <module>\n----> 1 ssconns = b.tm.sys.connection.get_collection(requests_params={'params': '$top=5&options=ss-server-addr+5.5.5.5'})\n \n~/.virtualenvs/venv/lib/python3.8/site-packages/f5/bigip/mixins.py in __getattr__(container, name)\n 93 error_message = ('\"allowed_lazy_attributes\" not in',\n 94 'container._meta_data for class %s' % cls_name)\n---> 95 raise LazyAttributesRequired(error_message)\n 96\n 97 # ensure the requested attr is present\n \nLazyAttributesRequired: ('\"allowed_lazy_attributes\" not in', 'container._meta_data for class Connection')

In [27]: ssconns = b.tm.sys.connection.get_collection()\n---------------------------------------------------------------------------\nLazyAttributesRequired Traceback (most recent call last)\n<ipython-input-27-5e104dde24cc> in <module>\n----> 1 ssconns = b.tm.sys.connection.get_collection()\n \n~/.virtualenvs/venv/lib/python3.8/site-packages/f5/bigip/mixins.py in __getattr__(container, name)\n 93 error_message = ('\"allowed_lazy_attributes\" not in',\n 94 'container._meta_data for class %s' % cls_name)\n---> 95 raise LazyAttributesRequired(error_message)\n 96\n 97 # ensure the requested attr is present\n \nLazyAttributesRequired: ('\"allowed_lazy_attributes\" not in', 'container._meta_data for class Connection')

{'name': 'v.10', 'tag': 10, 'interfacesReference': {'link': 'https://localhost/mgmt/tm/net/vlan/~Common~v.10/interfaces?ver=15.1.0.5', 'isSubcollection': True, 'items': [{'kind': 'tm:net:vlan:interfaces:interfacesstate', 'name': '1.3', 'fullPath': '1.3', 'generation': 1, 'selfLink': 'https://localhost/mgmt/tm/net/vlan/~Common~v.10/interfaces/1.3?ver=15.1.0.5', 'tagMode': 'none', 'untagged': True, 'nameReference': {'link': 'https://localhost/mgmt/tm/net/interface/1.3?ver=15.1.0.5'}}]}}

Getting started with the python SDK part 4: working with request parameters

10 Comments

ABOUT DEVCENTRAL

RESOURCES

SUPPORT

PARTNERS

ABOUT DEVCENTRAL

RESOURCES

SUPPORT

PARTNERS