Demystifying iControl REST Part 7 - Understanding Transactions

iControl REST. It’s iControl SOAP’s baby, brother, introduced back in TMOS version 11.4 as an early access feature but released fully in version 11.5. Several articles on basic usage have been written about the rest interface so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API.

A few months ago, a question in Q&A from community member spirrello asking how to update a tcp profile on a virtual. He was using bigsuds, the python wrapper for the soap interface. For the rest interface on this particular object, this is easy; just use the put method and supply the payload mapping the updated profile. But for soap, this requires a transaction. There are some changes to BIG-IP via the rest interface, however, like updating an ssl cert or key, that likewise will require a transaction to accomplish. In this article, I’ll show you how to use transactions with the rest interface.

The Fine Print

From the iControl REST user guide, the life cycle of a transaction progresses through three phases:

  1. Creation - This phase occurs when the transaction is created using a POST command.
  2. Modification - This phase occurs when commands are added to the transaction, or changes are made to the sequence of commands in the transaction.
  3. Commit - This phase occurs when iControl REST runs the transaction.

To create a transaction, post to /tm/transaction

 POST https://192.168.25.42/mgmt/tm/transaction

 {}

 Response:

 {
   "transId":1389812351,
   "state":"STARTED",
   "timeoutSeconds":30,
   "kind":"tm:transactionstate",
   "selfLink":"https://localhost/mgmt/tm/transaction/1389812351?ver=11.5.0"
 }

Note the transId, the state, and the timeoutSeconds. You'll need the transId to add or re-sequence commands within the transaction, and the transaction will expire after 30 seconds if no commands are added. You can list all transactions, or the details of a specific transaction with a get request.

 GET https://192.168.25.42/mgmt/tm/transaction
 GET https://192.168.25.42/mgmt/tm/transaction/transId

To add a command to the transaction, you use the normal method uris, but include the

X-F5-REST-Coordination-Id
header. This example creates a pool with a single member.

 POST https://192.168.25.42/mgmt/tm/ltm/pool
 X-F5-REST-Coordination-Id:1389812351
 {
   "name":"tcb-xact-pool",
   "members": [ {"name":"192.168.25.32:80","description":"First pool for transactions"} ]
 }

Not a great example because there is no need for a transaction here, but we'll roll with it! There are several other option methods for interrogating the transaction itself, see the user guide for details. Now we can commit the transaction. To do that, you reference the transaction id in the URI, remove the

X-F5-REST-Coordination-Id
header and use the patch method with payload key/value
state: VALIDATING
.

 PATCH https://localhost/mgmt/tm/transaction/1389812351
 { "state":"VALIDATING" }

That's all there is to it! Now that you've seen the nitty gritty details, let's take a look at some code samples.

Roll Your Own

In this example, I am needing to update and ssl key and certificate. If you try to update the cert or the key, it will complain that they do not match, so you need to update both at the same time. Assuming you are writing all your code from scratch, this is all it takes in python. Note on line 21 I post with an empty payload, and then on line 23, I add the header with the transaction id. I make my modifications and then in line 31, I remove the header, and finally on line 32, I patch to the transaction id with the appropriate payload.

    import json
    import requests

    btx = requests.session()
    btx.auth = (f5_user, f5_password)
    btx.verify = False
    btx.headers.update({'Content-Type':'application/json'})
    urlb = 'https://{0}/mgmt/tm'.format(f5_host)

    domain = 'mydomain.local_sslobj'
    chain = 'mychain_sslobj

    try:
        key = btx.get('{0}/sys/file/ssl-key/~Common~{1}'.format(urlb, domain))
        cert = btx.get('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, domain))
        chain = btx.get('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, 'chain'))

        if (key.status_code == 200) and (cert.status_code == 200) and (chain.status_code == 200):

            # use a transaction
            txid = btx.post('{0}/transaction'.format(urlb), json.dumps({})).json()['transId']
            # set the X-F5-REST-Coordination-Id header with the transaction id
            btx.headers.update({'X-F5-REST-Coordination-Id': txid})

            # make modifications
            modkey = btx.put('{0}/sys/file/ssl-key/~Common~{1}'.format(urlb, domain), json.dumps(keyparams))
            modcert = btx.put('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, domain), json.dumps(certparams))
            modchain = btx.put('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, 'le-chain'), json.dumps(chainparams))

            # remove header and patch to commit the transaction
            del btx.headers['X-F5-REST-Coordination-Id']
            cresult = btx.patch('{0}/transaction/{1}'.format(urlb, txid), json.dumps({'state':'VALIDATING'})).json()

A Little Help from a Friend

The f5-common-python library was released a few months ago to relieve you of a lot of the busy work with building requests. This is great, especially for transactions. To simplify the above code just to the transaction steps, consider:

            # use a transaction
            txid = btx.post('{0}/transaction'.format(urlb), json.dumps({})).json()['transId']
            # set the X-F5-REST-Coordination-Id header with the transaction id
            btx.headers.update({'X-F5-REST-Coordination-Id': txid})

# do stuff here

            # remove header and patch to commit the transaction
            del btx.headers['X-F5-REST-Coordination-Id']
            cresult = btx.patch('{0}/transaction/{1}'.format(urlb, txid), json.dumps({'state':'VALIDATING'})).json()

With the library, it's simplified to:

 tx = b.tm.transactions.transaction
 with TransactionContextManager(tx) as api:
     # do stuff here
     api.do_stuff

Yep, it's that simple. So if you haven't checked out the f5-common-python library, I highly suggest you do! I'll be writing about how to get started using it next week, and perhaps a follow up on how to contribute to it as well, so stay tuned!

Updated Jun 06, 2023
Version 2.0
  • I confirm, it works!
    It will be interesting to understand now how to enrich the attribute by categorizing the subscriber-id
    Thanks a lot for your help

  • Hi all,

    I have question on the PEM rest API query.

    [root@ip-10-1-1-8:Active:Standalone] config # tmsh show pem sessiondb all
    Pem::Sessiondb
    Blade number              0
    TMM number                2
    -------------------------------------
    Subscriber Information  
    -------------------------------------
    Subscriber Id             demouser1
    Subscriber Id Type        NAI
    Subscriber Type           Dynamic
    -------------------------------------
    Session Information     
    -------------------------------------
    IP Address                10.1.20.11
    Policy Server Session Id  
    Quota Server Session Id   
    Session State             provisioned
    Session Origin            radius
    User-Name                 
    3GPP-IMSI                 
    3GPP-IMEISV               
    3GPP-User-Location-Info   
    Called-Station-Id         
    Calling-Station-Id        
    NAS-IP-Address            
    NAS-IPv6-Address          
    Device Name               
    Device OS                 
    Bytes Uplink              404726
    Bytes Downlink            3060546
    Flows Total               126
    Flows Current             1
    Flows Max                 61
    Transactions              36
    -------------------------------------
    Policy Name               Policy Type
    -------------------------------------

    .......

    or query filer with "subscriber-id"

    [root@ip-10-1-1-8:Active:Standalone] config # tmsh show pem sessiondb Subscriber-Id demouser1
    Pem::Sessiondb
    Blade number              0
    TMM number                2
    -------------------------------------
    Subscriber Information  
    -------------------------------------
    Subscriber Id             demouser1
    Subscriber Id Type        NAI
    Subscriber Type           Dynamic
    -------------------------------------
    Session Information     
    -------------------------------------
    IP Address                10.1.20.11
    Policy Server Session Id  
    Quota Server Session Id   
    Session State             provisioned
    Session Origin            radius
    User-Name                 
    3GPP-IMSI                 
    3GPP-IMEISV               
    3GPP-User-Location-Info   
    Called-Station-Id         
    Calling-Station-Id        
    NAS-IP-Address            
    NAS-IPv6-Address          
    Device Name               
    Device OS                 
    Bytes Uplink              404726
    Bytes Downlink            3060546
    Flows Total               126
    Flows Current             1
    Flows Max                 61
    Transactions              36
    -------------------------------------
    Policy Name               Policy Type
    -------------------------------------
    Student_block             Predefined
    Student                   Predefined
    -------------------------------------
    
    Total sessions found: 1

    But i don't know how can i request in API uri. I had tried some but still failed.
    Can you help to educate me how to convert this as rest API request?

    root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb?Subscriber-Id=demosuer1
    {"code":400,"message":"Query parameter Subscriber-Id is invalid.","errorStack":[],"apiError":1}[root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin c^C
    [root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb
    {"code":400,"message":"At least one filtering argument or option 'all' must be specified","errorStack":[],"apiError":26214401}[root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb/all
    {"code":400,"message":"Found unexpected URI tmapi_mapper/pem/sessiondb/all.","errorStack":[],"apiError":1}[root@ip-10-1-1-8:Active:Standalone] config # 
    [root@ip-10-1-1-8:Active:Standalone] config # 



    • MMarco_77's avatar
      MMarco_77
      Icon for Cirrus rankCirrus

      Hi,

      I also have the same behavior, did you solve it?

       

      • Taut_SRISOMCHAI's avatar
        Taut_SRISOMCHAI
        Icon for Employee rankEmployee

        Hi Marco_77

        Yes,it solved.  I got the suggestion from JRahm and it works!
        "Try this format:

        /mgmt/tm/pem/sessiondb?options=subscriber-id+demouser1"

        Here is the example.

        >>>>>>>>>>>>>>

        [root@ip-10-1-1-8:Active:Standalone] config # curl -k -u admin:admin -H "Content-Type: application/json" -X GET https://10.1.1.8/mgmt/tm/pem/sessiondb?options=subscriber-id+demouser1 | jq .

          % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                         Dload  Upload   Total   Spent    Left  Speed

        100  1912  100  1912    0     0   9149      0 --:--:-- --:--:-- --:--:--  9192

        {

          "kind": "tm:pem:sessiondb:sessiondbstats",

          "selfLink": https://localhost/mgmt/tm/pem/sessiondb?options=subscriber-id+demouser1&ver=13.1.0.6,

          "entries": {

            https://localhost/mgmt/tm/pem/sessiondb/0: {

              "nestedStats": {

                "entries": {

                  "blade_number": {

                    "description": "0"

                  },

                  "bytes_from_client": {

                    "description": "2524741"

                  },

                  "bytes_to_client": {

                    "description": "16433442"

                  },

                  "custom_attr_count": {

                    "description": "0"

                  },

                  "err_code": {

                    "description": "0"

                  },

                  "flows_current": {

                    "description": "1"

                  },

                  "flows_max": {

                    "description": "125"

                  },

                  "flows_total": {

                    "description": "1100"

                  },

                  "operation": {

                    "description": "0"

                  },

                  "provisioned": {

                    "description": "yes"

                  },

                  "rt_domain": {

                    "description": "0"

                  },

                  "session_id": {

                    "description": "500890"

                  },

                  "session_ip": {

                    "description": "10.1.20.11"

                  },

                  "session_origin": {

                    "description": "1"

                  },

                  "session_origin_str": {

                    "description": "radius"

                  },

                  "session_state": {

                    "description": "1"

                  },

                  "session_state_str": {

                    "description": "provisioned"

                  },

                  "subscriber_id": {

                    "description": "demouser1"

                  },

                  "subscriber_id_type": {

                    "description": "3"

                  },

                  "subscriber_id_type_str": {

                    "description": "NAI"

                  },

                  "subscriber_type": {

                    "description": "2"

                  },

                  "tmm_number": {

                    "description": "2"

                  },

                  "transactions": {

                    "description": "150"

                  },

                  "version": {

                    "description": "2535959859"

                  },

                  https://localhost/mgmt/tm/pem/sessiondb/0/ip_info: {

                    "nestedStats": {

                      "entries": {

                        https://localhost/mgmt/tm/pem/sessiondb/0/ip_info/0: {

                          "nestedStats": {

                            "entries": {

                              "ip_address": {

                                "description": "10.1.20.11"

                              },

                              "ip_prefix_len": {

                                "description": "0"

                              },

                              "rt_domain": {

                                "description": "0"

                              }

                            }

                          }

                        }

                      }

                    }

                  },

                  https://localhost/mgmt/tm/pem/sessiondb/0/policy_info: {

                    "nestedStats": {

                      "entries": {

                        https://localhost/mgmt/tm/pem/sessiondb/0/policy_info/0: {

                          "nestedStats": {

                            "entries": {

                              "deleted": {

                                "description": "no"

                              },

                              "policy_name": {

                                "description": "Student_block"

                              },

                              "policy_type": {

                                "description": "2"

                              }

                            }

                          }

                        },

                        https://localhost/mgmt/tm/pem/sessiondb/0/policy_info/1: {

                          "nestedStats": {

                            "entries": {

                              "deleted": {

                                "description": "no"

                              },

                              "policy_name": {

                                "description": "Student"

                              },

                              "policy_type": {

                                "description": "2"

                              }

                            }

                          }

                        }

                      }

                    }

                  }

                }

              }

            }

          }

        }

        >>>>>>>>>>>>>>

        Compare to the tmsh command

        root@(ip-10-1-1-8)(cfg-sync Standalone)(Active)(/Common)(tmos)# show pem sessiondb subscriber-id demouser1

        Pem::Sessiondb

        Blade number              0

        TMM number                2

        -------------------------------------

        Subscriber Information 

        -------------------------------------

        Subscriber Id             demouser1

        Subscriber Id Type        NAI

        Subscriber Type           Dynamic

        -------------------------------------

        Session Information    

        -------------------------------------

        IP Address                10.1.20.11

        Policy Server Session Id 

        Quota Server Session Id  

        Session State             provisioned

        Session Origin            radius

        User-Name                

        3GPP-IMSI                

        3GPP-IMEISV              

        3GPP-User-Location-Info  

        Called-Station-Id        

        Calling-Station-Id       

        NAS-IP-Address           

        NAS-IPv6-Address         

        Device Name              

        Device OS                

        Bytes Uplink              4387176

        Bytes Downlink            20547896

        Flows Total               1998

        Flows Current             1

        Flows Max                 125

        Transactions              200

        -------------------------------------

        Policy Name               Policy Type

        -------------------------------------

        Student_block             Predefined

        Student                   Predefined

        -------------------------------------

         

        Total sessions found: 1

        >>>>>>>>

        Hope this helps!

  • I'm working on a python script which needs to create a LTM policy with 1 condition and 2 corresponding actions. Creating the policy and empty rule works like a charm but when I try adding the condition and actions in a transaction it fails.

     

    def updatePolicy(partition, policyName, serverName, virtualServerName, env):
        pol = ''
        cCondition = {
            u'name': u'0',
            u'fullPath': u'0',
            u'index': 0,
            u'all': True,
            u'caseInsensitive': True,
            u'equals': True,
            u'external': True,
            u'httpHost': True,
            u'present': True,
            u'remote': True,
            u'request': True,
            u'values': [serverName]
        }
        cAction1 = {
            u'name': u'0',
            u'fullPath': u'0',
            u'forwards': True,
            u'request': True,
            u'select': True,
            u'virtual': u'/{0}/{1}'.format(partition, virtualServerName),
        }
        cAction2 = {
            u'name': u'1',
            u'fullPath': u'1',
            u'disable': True,
            u'request': True,
            u'serverSsl': True,
        }
        try:
            pol = mgmt.tm.ltm.policys.policy.load(name=policyName, partition=partition)
            pol.draft()
        except Exception as e:
            try:
                pol = mgmt.tm.ltm.policys.policy.load(name=policyName, partition=partition, subPath='Drafts')
                print("...loaded policy draft")
            except Exception as ee:
                try:
                    pol = mgmt.tm.ltm.policys.policy.create(
                        name = policyName,
                        subPath = 'Drafts',
                        partition = partition,
                        ordinal = 0,
                        strategy = 'first-match',
                        controls = ["forwarding","server-ssl"],
                        requires = ["http"]
                    )
                    print("...created policy")
                except Exception as eee:
                    print(eee)
                    sys.exit(1)
        
        print("...adding rule to policy {0}".format(pol.name))
        rules = pol.rules_s.get_collection()
        
        rule = pol.rules_s.rules.create(
            name = "rule-{0}".format(serverName),
            subPath = 'Drafts',
            ordinal = 0,
            description = 'Redirect to /{0}/{1}'.format(partition, virtualServerName)
        )
        # Incorrect URI path must be corrected else setting condition won't work
        rule._meta_data['uri'] = pol._meta_data['uri'] + 'rules/rule-{0}/'.format(serverName)
        tx = mgmt.tm.transactions.transaction
        with TransactionContextManager(tx) as api:
            print("...add condition")
            rule.conditions_s.conditions.create(**cCondition)
            print("...add actions")
            rule.actions_s.actions.create(**cAction1)
            rule.actions_s.actions.create(**cAction2)
        print("...updating rule")
        rule.update()
        pol.publish()

     

     

    The issue I'm facing is maybe connected to the actions being added to the rule. When I run the script I receive the following output (rule is deleted manually before each run):

     

    ...loaded policy draft
    ...adding rule to policy policy-test-001
    ...create rule
    ...add condition
    ...add actions
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/f5/bigip/contexts.py", line 96, in __exit__
        self.transaction.modify(state="VALIDATING",
      File "/usr/lib/python3.9/site-packages/f5/bigip/resource.py", line 423, in modify
        self._modify(**patch)
      File "/usr/lib/python3.9/site-packages/f5/bigip/resource.py", line 408, in _modify
        response = session.patch(patch_uri, json=patch, **requests_params)
      File "/usr/lib/python3.9/site-packages/icontrol/session.py", line 295, in wrapper
        raise iControlUnexpectedHTTPError(error_message, response=response)
    icontrol.exceptions.iControlUnexpectedHTTPError: 400 Unexpected Error: Bad Request for uri: https://10.0.0.10:443/mgmt/tm/transaction/1683888226128082/
    Text: '{"code":400,"message":"transaction failed:0107186c:3: Policy \'/Common/Drafts/policy-test-001\', rule \'rule-test.local\'; missing or invalid target.","errorStack":[],"apiError":2}'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/var/www/apps/f5-python/./cert.py", line 239, in 
        main(sys.argv[1:])
      File "/var/www/apps/f5-python/./cert.py", line 236, in main
        updatePolicy('Common','policy-test-001', serverName, virtualServerName, environment)
      File "/var/www/apps/f5-python/./cert.py", line 184, in updatePolicy
        rule.actions_s.actions.create(**cAction2)
      File "/usr/lib/python3.9/site-packages/f5/bigip/contexts.py", line 100, in __exit__
        raise TransactionSubmitException(e)
    f5.sdk_exception.TransactionSubmitException: 400 Unexpected Error: Bad Request for uri: https://10.0.0.10:443/mgmt/tm/transaction/1683888226128082/
    Text: '{"code":400,"message":"transaction failed:0107186c:3: Policy \'/Common/Drafts/policy-test-001\', rule \'rule-test.local\'; missing or invalid target.","errorStack":[],"apiError":2}'

    If I comment out the second action additition rule.actions_s.actions.create(**cAction2) I receive the same error but this time rule.actions_s.actions.create(**cAction1).

    If both action lines are removed from the code the policy is updated but with only the condition.

    Are there any specific things I need to keep in mind when adding actions to a rule and how can I change the code so that it works?

  • Nice article. 5*

    Ideally the transaction ID would be returned as a string. Due to missing quotes for the value it might be interpreted as an integer. (Actually we had this issue in VMware vRO and had to apply an extra string conversion as workaround.) To list the commands in an open transaction it is required to add a trailing
    /commands
    to the path. (Tested in v12.1.2)
  • Robert_Teller_7's avatar
    Robert_Teller_7
    Historic F5 Account

    Not sure how I missed this article until now, this is a powerful tool when crafting a custom API integration.

     

  • hi Josh, yes, you can add this property to the payload of the patch:

    "validateOnly": true

  • Is it possible to do something like a "verify" to know whether a transaction would succeed before applying it?