Demystifying iControl REST Part 1 - Understanding the request URI
iControl REST. It’s iControl SOAP’s baby brother, introduced back in TMOS version 11.4 as an early access feature but was released fully in version 11.5. Several articles on basic usage have been written on iControl REST (see the resources at the bottom of this article) so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API. This article is the first of a four part series and will cover how the URI path plays a role in how the API functions. Examining the REST interface URI With iControl SOAP, all the interfaces and methods are defined in the WSDLs. With REST, the bulk of the interface and method is part of the URI. Take for example a request to get the list of iRules on a BIG-IP. https://x.x.x.x/mgmt/tm/ltm/rule With a simple get request with the appropriate headers and credentials (see below for curl and python examples,) this breaks down to the Base URI, the Module URI, and the Sub-Module URI. #python example >>> import requests >>> import json >>> b = requests.session() >>> b.auth = ('admin', 'admin') >>> b.verify = False >>> b.headers.update({'Content-Type':'application/json'}) >>> b.get('https://172.16.44.128/mgmt/tm/ltm/rule') #curl example curl -k -u admin:admin -H “Content-Type: application/json” -X GET https://172.16.44.128/mgmt/tm/ltm/rule Beyond the Sub-Module URI is the component URI. https://x.x.x.x/mgmt/tm/ltm/rule/testlog The component URI is the spot that snags most beginners. When creating a new configuration object on an F5, it is fairly obvious which URI to use for the REST call. If you were creating a new virtual server, it would be /mgmt/tm/ltm/virtual, while a new pool would be /mgmt/tm/ltm/pool. Thus a REST call to create a new pool on an LTM with the IP address 172.16.44.128 would look like the following: #python example >>> import requests >>> import json >>> b = requests.session() >>> b.auth = ('admin', 'admin') >>> b.verify = False >>> b.headers.update({'Content-Type':'application/json'}) >>> pm = ['192.168.25.32:80', '192.168.25.33:80'] >>> payload = { } >>> payload['kind'] = 'tm:ltm:pool:poolstate' >>> payload['name'] = 'tcb-pool' >>> payload['members'] = [ {'kind': 'ltm:pool:members', 'name': member } for member in pm] >>> b.post('https://172.16.44.128/mgmt/tm/ltm/pool', data=json.dumps(payload)) #curl example curl -k -u admin:admin -H "Content-Type: \ application/json" -X POST -d \ '{"name":"tcb-pool","members":[ \ {"name":"192.168.25.32:80","description":"first member”}, \ {"name":"192.168.25.33:80","description":"second member”} ] }' \ https://172.16.44.128/mgmt/tm/ltm/pool The call would create the pool “tcb-pool” with members at 192.168.25.32:80 and 192.168.25.33:80. All other aspects of the pool would be the defaults. Thus this pool would be created in the Common partition, have the Round Robin load balancing method, and no monitor. At first glance, some programmers would then modify the pool “tcb-pool” with a command like the following (same payload in python, added the .text attribute to see the error response on the requests object): #python example >>> b.put('https://172.16.44.128/mgmt/tm/ltm/pool', data=json.dumps(payload)).text #return data u'{"code":403,"message":"Operation is not supported on component /ltm/pool.","errorStack":[]}' #curl example curl -k -u admin:admin -H "Content-Type: \ application/json" -X PUT -d \ '{"name":"tcb-pool","members":[ \ {"name":"192.168.25.32:80","description":"first member"} {"name":"192.168.25.33:80","description":"second member"} ] }' \ https://172.16.44.128/mgmt/tm/ltm/pool #return data {"code":403,"message":"Operation is not supported on component /ltm/pool.","errorStack":[]} You can see because they use the sub module URI used to create the pool this returns a 403 error. The command fails because one is trying to modify a specific pool at the generic pool level. Now that the pool exists, one must use the URI that specifies the pool. Thus, the correct command would be: #python example >>> b.put('https://172.16.44.128/mgmt/tm/ltm/pool/~Common~tcb-pool', data=json.dumps(payload)) #curl example curl -k -u admin:admin -H "Content-Type: \ application/json" -X PUT -d \ '{"members":[ \ {"name":"192.168.25.32:80","description":"first member"} {"name":"192.168.25.33:80","description":"second member"} ] }' \ https://172.16.44.128/mgmt/tm/ltm/pool/~Common~tcb-pool We add the ~Common~ in front of the pool name because it is in the Common partition. However, this would also work with https://172.16.44.128/mgmt/tm/ltm/pool/tcb-pool. It is just good practice to explicitly insert the partition name since not all configuration objects will be in the default Common partition. Because we specify the pool in the URI, it is no longer necessary to have the “name” key value pair. In practice, programmers usually correctly modify items such as virtual servers and pools. However, we encounter this confusion much more often in configuration items that are ifiles. This may be because the creation of configuration items that are ifiles is a 3-step process. For instance, in order to create an external data group, one would first scp the file to 172.16.44.128/config/filestore/data_mda_1, then issue 2 Rest commands: curl -sk -u admin:admin -H "Content-Type: application/json" -X POST -d '{"name":"data_mda_1","type":"string","source-path":"file:///config/filestore/data_mda_1"}' https://172.16.44.128/mgmt/tm/sys/file/data-group curl -sk -u admin:admin -H "Content-Type: application/json" -X POST -d '{"name":"dg_mda","external-file-name":"/Common/data_mda_1"}' https://172.16.44.128/mgmt/tm/ltm/data-group/external/ To update the external data group, many programmers first try something like the following: curl -sk -u admin:admin -H "Content-Type: application/json" -X POST -d '{"name":"data_mda_2","type":"string","source-path":"file:///config/filestore/data_mda_2”}’ https://172.16.44.128/mgmt/tm/sys/file/data-group curl -sk -u admin:admin -H "Content-Type: application/json" -X PUT -d '{"name":"dg_mda","external-file-name":"/Common/data_mda_2"}' https://172.16.44.128/mgmt/tm/ltm/data-group/external/ The first command works because we are creating a new ifile object. However, the second command fails because we are trying to modify a specific external data group at the generic external data group level. The proper command is: curl -sk -u admin:admin -H "Content-Type: application/json" -X PUT -d '{"external-file-name":"/Common/data_mda_2"}' https://172.16.44.128/mgmt/tm/ltm/data-group/external/dg_mda The python code gets a little more complex with the data-group examples, so I've uploaded it to the codeshare here. Much thanks to Pat Chang for the bulk of the content in this article. Stay tuned for part 2, where we'll cover sub collections and how to use them.Demystifying iControl REST Part 2 - Understanding sub collections and how to use them
iControl REST. It’s iControl SOAP’s baby brother, introduced back in TMOS version 11.4 as an early access feature but was released fully in version 11.5. Several articles on basic usage have been written on iControl REST (see the resources at the bottom of this article) so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API. The first article of this series covered the URI’s role in the API. This second article will cover how the URI path plays a role in how the API functions. Working with Subcollections When manipulating F5 configuration items with iControl Rest, subcollections are a powerful tool. They allow one to manipulate specific items in the subcollection instead of having to manipulate the entire sub collection. Take the pool object for example. If we just query the pool (in this case testpool,) you’ll notice the returned data does not list the pool members #query (via the Chrome advanced REST client using Authorization & Content-Type headers:) https://172.16.44.128/mgmt/tm/ltm/pool/~Common~testpool #query (via curl:) curl -k -u admin:admin https://172.16.44.128/mgmt/tm/ltm/pool/~Common~testpool #response: { kind: "tm:ltm:pool:poolstate" name: "testpool" fullPath: "testpool" generation: 1 selfLink: "https://localhost/mgmt/tm/ltm/pool/testpool?ver=11.6.0" allowNat: "yes" allowSnat: "yes" ignorePersistedWeight: "disabled" ipTosToClient: "pass-through" ipTosToServer: "pass-through" linkQosToClient: "pass-through" linkQosToServer: "pass-through" loadBalancingMode: "round-robin" minActiveMembers: 0 minUpMembers: 0 minUpMembersAction: "failover" minUpMembersChecking: "disabled" queueDepthLimit: 0 queueOnConnectionLimit: "disabled" queueTimeLimit: 0 reselectTries: 0 serviceDownAction: "none" slowRampTime: 10 membersReference: { link: "https://localhost/mgmt/tm/ltm/pool/~Common~testpool/members?ver=11.6.0" isSubcollection: true }- } Notice the isSubcollection: true for the membersReference? This is an indicator that there is a subcollection for the members keyword. If you then query the members for that pool, you will get the subcollection. #query (via the Chrome advanced REST client using Authorization & Content-Type headers:) https://172.16.44.128/mgmt/tm/ltm/pool/~Common~testpool/members #query (via curl:) curl -k -u admin:admin https://172.16.44.128/mgmt/tm/ltm/pool/~Common~testpool/members #response: { kind: "tm:ltm:pool:members:memberscollectionstate" selfLink: "https://localhost/mgmt/tm/ltm/pool/testpool/members?ver=11.6.0" items: [4] 0: { kind: "tm:ltm:pool:members:membersstate" name: "192.168.103.10:80" partition: "Common" fullPath: "/Common/192.168.103.10:80" generation: 1 selfLink: "https://localhost/mgmt/tm/ltm/pool/testpool/members/~Common~192.168.103.10:80?ver=11.6.0" address: "192.168.103.10" connectionLimit: 0 dynamicRatio: 1 ephemeral: "false" fqdn: { autopopulate: "disabled" }- inheritProfile: "enabled" logging: "disabled" monitor: "default" priorityGroup: 0 rateLimit: "disabled" ratio: 1 session: "user-enabled" state: "unchecked" } } You can see that there are four pool members as the item count is four, but I’m only showing one of them here for brevity. The pool members can be added, modified, or deleted at this level. Add a pool member URI: https://172.16.44.128/mgmt/tm/ltm/pool/~Common~testpool/members Method: POST JSON: {“name”:”192.168.103.12:80”} Modify a pool member URI: https://172.16.44.128/mgmt/tm/ltm/pool/~Common~testpool/members/~Common~192.168.103.12:80 Method: PUT JSON: {“name”:”192.168.103.12:80”,”connectionLimit”:”50”} Delete a pool member URI: https://172.16.44.128/mgmt/tm/ltm/pool/~Common~testpool/members/~Common~192.168.103.12:80 Method: DELETE JSON: none (an error will trigger if you send any data) So for subcollections like pool members, adding and deleting is pretty straight forward. Unfortunately, not all lists of configuration items are treated as subcollections. For example, take the data group. You can see for the data group testdb below, the records are not a subcollection. { kind: "tm:ltm:data-group:internal:internalstate" name: "testdb" fullPath: "testdb" generation: 1 selfLink: "https://localhost/mgmt/tm/ltm/data-group/internal/testdb?ver=11.6.0" type: "string" records: [3] 0: { name: "a" data: "one" }- 1: { name: "b" data: "two" }- 2: { name: "c" }- - } Because this is not a subcollection, any modifications to the records of this object are treated as complete replacements. Thus, this request to add a record (d) to the list: URL: https://172.16.44.128/mgmt/tm/ltm/data-group/internal/testdb Method: PUT JSON: {“records” : [ { “name”: “d” } ] } will result in a data group with only 1 entry (d)! If one wanted to add (d) to the data group, one would have to issue the same request above, but with complete JSON data representing the original records PLUS the new record. The same goes if you want to delete a record. You need to submit all the records in JSON format sans the one you wish to delete via the PUT request above. Note: Whereas it's true an update to the records attribute requires a full replacement, it IS possible to update individual records by using the options query parameter instead of updating the records attribute. For details, see the update section of this article. Thus, to modify items that are not subcollections, one would have to issue a get and parse the existing items into a list. Then one would need to modify the list as desired (adding and/or deleting items), and then issue a PUT to the object URI with the modified list as the json data. A python example of doing just that is shown below. This script grabs the records for the MyNetworks data group, adds three new networks to it, then removes those three networks to return it to its original state. __author__ = 'rahm' def get_dg(rq, url, dg_details): dg = rq.get('%s/ltm/data-group/%s/%s' % (url, dg_details[0], dg_details[1])).json() return dg def extend_dg(rq, url, dg_details, additional_records): dg = rq.get('%s/ltm/data-group/%s/%s' % (url, dg_details[0], dg_details[1])).json() current_records = dg['records'] new_records = [] for record in current_records: nr = [ {'name': record['name']}] new_records.extend(nr) for record in additional_records: nr = [ {'name': record}] new_records.extend(nr) payload = {} payload['records'] = new_records rq.put('%s/ltm/data-group/%s/%s' % (url, dg_details[0], dg_details[1]), json.dumps(payload)) def contract_dg(rq, url, dg_details, removal_records): dg = rq.get('%s/ltm/data-group/%s/%s' % (url, dg_details[0], dg_details[1])).json() new_records = [] for record in removal_records: nr = [ {'name': record}] new_records.extend(nr) current_records = dg['records'] new_records = [x for x in current_records if x not in new_records] payload = {} payload['records'] = new_records rq.put('%s/ltm/data-group/%s/%s' % (url, dg_details[0], dg_details[1]), json.dumps(payload)) if __name__ == "__main__": import requests, json b = requests.session() b.auth = ('admin', 'admin') b.verify = False b.headers.update({'Content-Type' : 'application/json'}) b_url_base = 'https://172.16.44.128/mgmt/tm' dg_details = ['internal', 'myNetworks'] net_changes = ['3.0.0.0/8', '4.0.0.0/8'] print "\nExisting Records for %s Data-Group:\n\t%s" % (dg_details[1], get_dg(b, b_url_base, dg_details)['records']) extend_dg(b, b_url_base, dg_details, net_changes) print "\nUpdated Records for %s Data-Group:\n\t%s" % (dg_details[1], get_dg(b, b_url_base, dg_details)['records']) contract_dg(b, b_url_base, dg_details, net_changes) print "\nUpdated Records for %s Data-Group:\n\t%s" % (dg_details[1], get_dg(b, b_url_base, dg_details)['records']) When running this against my lab BIG-IP, I get this output on my console Existing Records for myNetworks Data-Group: [{u'name': u'1.0.0.0/8'}, {u'name': u'2.0.0.0/8'}] Updated Records for myNetworks Data-Group: [{u'name': u'1.0.0.0/8'}, {u'name': u'2.0.0.0/8'}, {u'name': u'3.0.0.0/8'}, {u'name': u'4.0.0.0/8'}] Updated Records for myNetworks Data-Group: [{u'name': u'1.0.0.0/8'}, {u'name': u'2.0.0.0/8'}] Process finished with exit code 0 Hopefully this has been helpful in showing the power of subcollections and the necessary steps to update objects like data-groups that are not. Much thanks again to Pat Chang for the bulk of the content in this article Next up: Query Parameters and Options.
