Demystifying iControl REST Part 7 - Understanding Transactions
iControl REST. It’s iControl SOAP’s baby, brother, introduced back in TMOS version 11.4 as an early access feature but released fully in version 11.5. Several articles on basic usage have been written about the rest interface so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API.
A few months ago, a question in Q&A from community member spirrello asking how to update a tcp profile on a virtual. He was using bigsuds, the python wrapper for the soap interface. For the rest interface on this particular object, this is easy; just use the put method and supply the payload mapping the updated profile. But for soap, this requires a transaction. There are some changes to BIG-IP via the rest interface, however, like updating an ssl cert or key, that likewise will require a transaction to accomplish. In this article, I’ll show you how to use transactions with the rest interface.
The Fine Print
From the iControl REST user guide, the life cycle of a transaction progresses through three phases:
- Creation - This phase occurs when the transaction is created using a POST command.
- Modification - This phase occurs when commands are added to the transaction, or changes are made to the sequence of commands in the transaction.
- Commit - This phase occurs when iControl REST runs the transaction.
To create a transaction, post to /tm/transaction
POST https://192.168.25.42/mgmt/tm/transaction {} Response: { "transId":1389812351, "state":"STARTED", "timeoutSeconds":30, "kind":"tm:transactionstate", "selfLink":"https://localhost/mgmt/tm/transaction/1389812351?ver=11.5.0" }
Note the transId, the state, and the timeoutSeconds. You'll need the transId to add or re-sequence commands within the transaction, and the transaction will expire after 30 seconds if no commands are added. You can list all transactions, or the details of a specific transaction with a get request.
GET https://192.168.25.42/mgmt/tm/transaction GET https://192.168.25.42/mgmt/tm/transaction/transId
To add a command to the transaction, you use the normal method uris, but include the
X-F5-REST-Coordination-Id
header. This example creates a pool with a single member.
POST https://192.168.25.42/mgmt/tm/ltm/pool X-F5-REST-Coordination-Id:1389812351 { "name":"tcb-xact-pool", "members": [ {"name":"192.168.25.32:80","description":"First pool for transactions"} ] }
Not a great example because there is no need for a transaction here, but we'll roll with it! There are several other option methods for interrogating the transaction itself, see the user guide for details. Now we can commit the transaction. To do that, you reference the transaction id in the URI, remove the
X-F5-REST-Coordination-Id
header and use the patch method with payload key/value state: VALIDATING
.
PATCH https://localhost/mgmt/tm/transaction/1389812351 { "state":"VALIDATING" }
That's all there is to it! Now that you've seen the nitty gritty details, let's take a look at some code samples.
Roll Your Own
In this example, I am needing to update and ssl key and certificate. If you try to update the cert or the key, it will complain that they do not match, so you need to update both at the same time. Assuming you are writing all your code from scratch, this is all it takes in python. Note on line 21 I post with an empty payload, and then on line 23, I add the header with the transaction id. I make my modifications and then in line 31, I remove the header, and finally on line 32, I patch to the transaction id with the appropriate payload.
import json import requests btx = requests.session() btx.auth = (f5_user, f5_password) btx.verify = False btx.headers.update({'Content-Type':'application/json'}) urlb = 'https://{0}/mgmt/tm'.format(f5_host) domain = 'mydomain.local_sslobj' chain = 'mychain_sslobj try: key = btx.get('{0}/sys/file/ssl-key/~Common~{1}'.format(urlb, domain)) cert = btx.get('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, domain)) chain = btx.get('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, 'chain')) if (key.status_code == 200) and (cert.status_code == 200) and (chain.status_code == 200): # use a transaction txid = btx.post('{0}/transaction'.format(urlb), json.dumps({})).json()['transId'] # set the X-F5-REST-Coordination-Id header with the transaction id btx.headers.update({'X-F5-REST-Coordination-Id': txid}) # make modifications modkey = btx.put('{0}/sys/file/ssl-key/~Common~{1}'.format(urlb, domain), json.dumps(keyparams)) modcert = btx.put('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, domain), json.dumps(certparams)) modchain = btx.put('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, 'le-chain'), json.dumps(chainparams)) # remove header and patch to commit the transaction del btx.headers['X-F5-REST-Coordination-Id'] cresult = btx.patch('{0}/transaction/{1}'.format(urlb, txid), json.dumps({'state':'VALIDATING'})).json()
A Little Help from a Friend
The f5-common-python library was released a few months ago to relieve you of a lot of the busy work with building requests. This is great, especially for transactions. To simplify the above code just to the transaction steps, consider:
# use a transaction txid = btx.post('{0}/transaction'.format(urlb), json.dumps({})).json()['transId'] # set the X-F5-REST-Coordination-Id header with the transaction id btx.headers.update({'X-F5-REST-Coordination-Id': txid}) # do stuff here # remove header and patch to commit the transaction del btx.headers['X-F5-REST-Coordination-Id'] cresult = btx.patch('{0}/transaction/{1}'.format(urlb, txid), json.dumps({'state':'VALIDATING'})).json()
With the library, it's simplified to:
tx = b.tm.transactions.transaction with TransactionContextManager(tx) as api: # do stuff here api.do_stuff
Yep, it's that simple. So if you haven't checked out the f5-common-python library, I highly suggest you do! I'll be writing about how to get started using it next week, and perhaps a follow up on how to contribute to it as well, so stay tuned!
- MMarco_77Cirrus
I confirm, it works!
It will be interesting to understand now how to enrich the attribute by categorizing the subscriber-id
Thanks a lot for your help - Taut_SRISOMCHAIEmployee
Hi all,
I have question on the PEM rest API query.
[root@ip-10-1-1-8:Active:Standalone] config # tmsh show pem sessiondb all Pem::Sessiondb Blade number 0 TMM number 2 ------------------------------------- Subscriber Information ------------------------------------- Subscriber Id demouser1 Subscriber Id Type NAI Subscriber Type Dynamic ------------------------------------- Session Information ------------------------------------- IP Address 10.1.20.11 Policy Server Session Id Quota Server Session Id Session State provisioned Session Origin radius User-Name 3GPP-IMSI 3GPP-IMEISV 3GPP-User-Location-Info Called-Station-Id Calling-Station-Id NAS-IP-Address NAS-IPv6-Address Device Name Device OS Bytes Uplink 404726 Bytes Downlink 3060546 Flows Total 126 Flows Current 1 Flows Max 61 Transactions 36 ------------------------------------- Policy Name Policy Type -------------------------------------
.......
or query filer with "subscriber-id"[root@ip-10-1-1-8:Active:Standalone] config # tmsh show pem sessiondb Subscriber-Id demouser1 Pem::Sessiondb Blade number 0 TMM number 2 ------------------------------------- Subscriber Information ------------------------------------- Subscriber Id demouser1 Subscriber Id Type NAI Subscriber Type Dynamic ------------------------------------- Session Information ------------------------------------- IP Address 10.1.20.11 Policy Server Session Id Quota Server Session Id Session State provisioned Session Origin radius User-Name 3GPP-IMSI 3GPP-IMEISV 3GPP-User-Location-Info Called-Station-Id Calling-Station-Id NAS-IP-Address NAS-IPv6-Address Device Name Device OS Bytes Uplink 404726 Bytes Downlink 3060546 Flows Total 126 Flows Current 1 Flows Max 61 Transactions 36 ------------------------------------- Policy Name Policy Type ------------------------------------- Student_block Predefined Student Predefined ------------------------------------- Total sessions found: 1
But i don't know how can i request in API uri. I had tried some but still failed.
Can you help to educate me how to convert this as rest API request?root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb?Subscriber-Id=demosuer1 {"code":400,"message":"Query parameter Subscriber-Id is invalid.","errorStack":[],"apiError":1}[root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin c^C [root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb {"code":400,"message":"At least one filtering argument or option 'all' must be specified","errorStack":[],"apiError":26214401}[root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb/all {"code":400,"message":"Found unexpected URI tmapi_mapper/pem/sessiondb/all.","errorStack":[],"apiError":1}[root@ip-10-1-1-8:Active:Standalone] config # [root@ip-10-1-1-8:Active:Standalone] config #
- MMarco_77Cirrus
Hi,
I also have the same behavior, did you solve it?
- Taut_SRISOMCHAIEmployee
Hi Marco_77
Yes,it solved. I got the suggestion from JRahm and it works!
"Try this format:/mgmt/tm/pem/sessiondb?options=subscriber-id+demouser1"
Here is the example.
>>>>>>>>>>>>>>
[root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb?options=subscriber-id+demouser1 | jq .
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1912 100 1912 0 0 9149 0 --:--:-- --:--:-- --:--:-- 9192
{
"kind": "tm:pem:sessiondb:sessiondbstats",
"selfLink": https://localhost/mgmt/tm/pem/sessiondb?options=subscriber-id+demouser1&ver=13.1.0.6,
"entries": {
https://localhost/mgmt/tm/pem/sessiondb/0: {
"nestedStats": {
"entries": {
"blade_number": {
"description": "0"
},
"bytes_from_client": {
"description": "2524741"
},
"bytes_to_client": {
"description": "16433442"
},
"custom_attr_count": {
"description": "0"
},
"err_code": {
"description": "0"
},
"flows_current": {
"description": "1"
},
"flows_max": {
"description": "125"
},
"flows_total": {
"description": "1100"
},
"operation": {
"description": "0"
},
"provisioned": {
"description": "yes"
},
"rt_domain": {
"description": "0"
},
"session_id": {
"description": "500890"
},
"session_ip": {
"description": "10.1.20.11"
},
"session_origin": {
"description": "1"
},
"session_origin_str": {
"description": "radius"
},
"session_state": {
"description": "1"
},
"session_state_str": {
"description": "provisioned"
},
"subscriber_id": {
"description": "demouser1"
},
"subscriber_id_type": {
"description": "3"
},
"subscriber_id_type_str": {
"description": "NAI"
},
"subscriber_type": {
"description": "2"
},
"tmm_number": {
"description": "2"
},
"transactions": {
"description": "150"
},
"version": {
"description": "2535959859"
},
https://localhost/mgmt/tm/pem/sessiondb/0/ip_info: {
"nestedStats": {
"entries": {
https://localhost/mgmt/tm/pem/sessiondb/0/ip_info/0: {
"nestedStats": {
"entries": {
"ip_address": {
"description": "10.1.20.11"
},
"ip_prefix_len": {
"description": "0"
},
"rt_domain": {
"description": "0"
}
}
}
}
}
}
},
https://localhost/mgmt/tm/pem/sessiondb/0/policy_info: {
"nestedStats": {
"entries": {
https://localhost/mgmt/tm/pem/sessiondb/0/policy_info/0: {
"nestedStats": {
"entries": {
"deleted": {
"description": "no"
},
"policy_name": {
"description": "Student_block"
},
"policy_type": {
"description": "2"
}
}
}
},
https://localhost/mgmt/tm/pem/sessiondb/0/policy_info/1: {
"nestedStats": {
"entries": {
"deleted": {
"description": "no"
},
"policy_name": {
"description": "Student"
},
"policy_type": {
"description": "2"
}
}
}
}
}
}
}
}
}
}
}
}
>>>>>>>>>>>>>>
Compare to the tmsh command
root@(ip-10-1-1-8)(cfg-sync Standalone)(Active)(/Common)(tmos)# show pem sessiondb subscriber-id demouser1
Pem::Sessiondb
Blade number 0
TMM number 2
-------------------------------------
Subscriber Information
-------------------------------------
Subscriber Id demouser1
Subscriber Id Type NAI
Subscriber Type Dynamic
-------------------------------------
Session Information
-------------------------------------
IP Address 10.1.20.11
Policy Server Session Id
Quota Server Session Id
Session State provisioned
Session Origin radius
User-Name
3GPP-IMSI
3GPP-IMEISV
3GPP-User-Location-Info
Called-Station-Id
Calling-Station-Id
NAS-IP-Address
NAS-IPv6-Address
Device Name
Device OS
Bytes Uplink 4387176
Bytes Downlink 20547896
Flows Total 1998
Flows Current 1
Flows Max 125
Transactions 200
-------------------------------------
Policy Name Policy Type
-------------------------------------
Student_block Predefined
Student Predefined
-------------------------------------
Total sessions found: 1
>>>>>>>>
Hope this helps!
- J-H_JohansenNimbostratus
I'm working on a python script which needs to create a LTM policy with 1 condition and 2 corresponding actions. Creating the policy and empty rule works like a charm but when I try adding the condition and actions in a transaction it fails.
def updatePolicy(partition, policyName, serverName, virtualServerName, env): pol = '' cCondition = { u'name': u'0', u'fullPath': u'0', u'index': 0, u'all': True, u'caseInsensitive': True, u'equals': True, u'external': True, u'httpHost': True, u'present': True, u'remote': True, u'request': True, u'values': [serverName] } cAction1 = { u'name': u'0', u'fullPath': u'0', u'forwards': True, u'request': True, u'select': True, u'virtual': u'/{0}/{1}'.format(partition, virtualServerName), } cAction2 = { u'name': u'1', u'fullPath': u'1', u'disable': True, u'request': True, u'serverSsl': True, } try: pol = mgmt.tm.ltm.policys.policy.load(name=policyName, partition=partition) pol.draft() except Exception as e: try: pol = mgmt.tm.ltm.policys.policy.load(name=policyName, partition=partition, subPath='Drafts') print("...loaded policy draft") except Exception as ee: try: pol = mgmt.tm.ltm.policys.policy.create( name = policyName, subPath = 'Drafts', partition = partition, ordinal = 0, strategy = 'first-match', controls = ["forwarding","server-ssl"], requires = ["http"] ) print("...created policy") except Exception as eee: print(eee) sys.exit(1) print("...adding rule to policy {0}".format(pol.name)) rules = pol.rules_s.get_collection() rule = pol.rules_s.rules.create( name = "rule-{0}".format(serverName), subPath = 'Drafts', ordinal = 0, description = 'Redirect to /{0}/{1}'.format(partition, virtualServerName) ) # Incorrect URI path must be corrected else setting condition won't work rule._meta_data['uri'] = pol._meta_data['uri'] + 'rules/rule-{0}/'.format(serverName) tx = mgmt.tm.transactions.transaction with TransactionContextManager(tx) as api: print("...add condition") rule.conditions_s.conditions.create(**cCondition) print("...add actions") rule.actions_s.actions.create(**cAction1) rule.actions_s.actions.create(**cAction2) print("...updating rule") rule.update() pol.publish()
The issue I'm facing is maybe connected to the actions being added to the rule. When I run the script I receive the following output (rule is deleted manually before each run):
...loaded policy draft ...adding rule to policy policy-test-001 ...create rule ...add condition ...add actions Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/f5/bigip/contexts.py", line 96, in __exit__ self.transaction.modify(state="VALIDATING", File "/usr/lib/python3.9/site-packages/f5/bigip/resource.py", line 423, in modify self._modify(**patch) File "/usr/lib/python3.9/site-packages/f5/bigip/resource.py", line 408, in _modify response = session.patch(patch_uri, json=patch, **requests_params) File "/usr/lib/python3.9/site-packages/icontrol/session.py", line 295, in wrapper raise iControlUnexpectedHTTPError(error_message, response=response) icontrol.exceptions.iControlUnexpectedHTTPError: 400 Unexpected Error: Bad Request for uri: https://10.0.0.10:443/mgmt/tm/transaction/1683888226128082/ Text: '{"code":400,"message":"transaction failed:0107186c:3: Policy \'/Common/Drafts/policy-test-001\', rule \'rule-test.local\'; missing or invalid target.","errorStack":[],"apiError":2}' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/var/www/apps/f5-python/./cert.py", line 239, in main(sys.argv[1:]) File "/var/www/apps/f5-python/./cert.py", line 236, in main updatePolicy('Common','policy-test-001', serverName, virtualServerName, environment) File "/var/www/apps/f5-python/./cert.py", line 184, in updatePolicy rule.actions_s.actions.create(**cAction2) File "/usr/lib/python3.9/site-packages/f5/bigip/contexts.py", line 100, in __exit__ raise TransactionSubmitException(e) f5.sdk_exception.TransactionSubmitException: 400 Unexpected Error: Bad Request for uri: https://10.0.0.10:443/mgmt/tm/transaction/1683888226128082/ Text: '{"code":400,"message":"transaction failed:0107186c:3: Policy \'/Common/Drafts/policy-test-001\', rule \'rule-test.local\'; missing or invalid target.","errorStack":[],"apiError":2}'
If I comment out the second action additition rule.actions_s.actions.create(**cAction2) I receive the same error but this time rule.actions_s.actions.create(**cAction1).
If both action lines are removed from the code the policy is updated but with only the condition.
Are there any specific things I need to keep in mind when adding actions to a rule and how can I change the code so that it works?
Nice article. 5*
Ideally the transaction ID would be returned as a string. Due to missing quotes for the value it might be interpreted as an integer. (Actually we had this issue in VMware vRO and had to apply an extra string conversion as workaround.) To list the commands in an open transaction it is required to add a trailing
to the path. (Tested in v12.1.2)/commands
- Robert_Teller_7Historic F5 Account
Not sure how I missed this article until now, this is a powerful tool when crafting a custom API integration.
- JRahmAdmin
hi Josh, yes, you can add this property to the payload of the patch:
"validateOnly": true
Is it possible to do something like a "verify" to know whether a transaction would succeed before applying it?