Forum Discussion
Use F5 REST API Python SDK to update an external datagroup
I have working code using the F5 Python REST SDK that can upload a txt file and create an external datagroup from it.
But I cannot figure out how to use the API to update that datagroup file without deleting the datagroup and recreating it.
All of the examples I can find (e.g. https://devcentral.f5.com/s/question/0D51T00006i7j1ySAA/upload-programatically-external-data-group-file) do the following:
- create a datagroup file object from a sourcepath
- create a datagroup object from the datagroup file object
- display the created object's attributes
Then they delete the datagroup object and datagroup file object so that they can run again.
I cannot find a way to reload the data from an updated data file into an existing datagroup object.
I have also seen code that can walk a datagroup and update items in it one by one, but I'd really like to use a file update model.
Any ideas? Code sample follows:
from f5.bigip import ManagementRoot
# Connect to the BigIP
mgmt = ManagementRoot("IP-FIXME", "USER-FIXME", "PASSWD-FIXME")
def showDatagroups() :
print("\n\n**** Showing Datagroups")
dgs = mgmt.tm.ltm.data_group.externals.get_collection()
for idx, dg in enumerate(dgs):
print("\n{}: {}".format(idx, dg.raw))
print("\n{}: {}".format(idx, dg.name))
if hasattr(dg, 'records'):
print("\n{}: {}".format(idx, dg.records))
for record in dg.records:
print("\nrec: {}".format(record))
else:
print("\nObject {} has no records".format(dg.name))
def showDatagroupFiles():
print("\n\n**** Showing DatagroupFiles")
dgFiles = mgmt.tm.sys.file.data_groups.get_collection()
for idx, f in enumerate(dgFiles):
print('\n{}: {}'.format(idx, f.raw))
def uploadFile(f):
# This upload works - it places the file in the uploads folder.
# But I cannot seem to access the file uploaded file with the datagroup create method
print("\n\n**** Uploading datagroup file {}".format(f))
mgmt.shared.file_transfer.uploads.upload_file(f)
def createDatagroupFile(source_path, dgname, dataType):
print("\n\n**** Creating datagroup file {}, name {}, type {}".format(fname, dgname, dataType))
# check first if its there
dgFiles = mgmt.tm.sys.file.data_groups.get_collection()
found = 0
for idx, f in enumerate(dgFiles):
if f.name == dgname:
found = 1
break
if found:
print("File {} already exists".format(dgname))
# Here's where I would just update the file if I could
else:
print("Creating DG File {} from fname {}".format(dgname, sourcePath))
dgFile = mgmt.tm.sys.file.data_groups.data_group.create(sourcePath=source_path, name=dgname, type=dataType)
def createDatagroupFromFile(name, file):
print("\n\n**** Creating datagroup {} from file {}".format(name, file))
# check first if its there
dgs = mgmt.tm.sys.file.data_groups.get_collection()
found = 0
for idx, dg in enumerate(dgs):
if dg.name == name:
found = 1
break
if found:
print("Datagroup {} already exists".format(name))
# Here's where I would just update the datagroup if I could
else:
dgObject = mgmt.tm.ltm.data_group.externals.external.create(name=name, externalFileName=file)
if __name__ == "__main__":
fname = './dg_test5.txt'
# using HTTP server for raw file - this is the only way I could get the upload to work
sourcePath = 'http://127.0.0.1:8000/dg_test5.txt'
dgfilename = 'dg_test5.txt'
dgname = 'dg_test5'
uploadFile(fname)
createDatagroupFile(sourcePath, dgfilename, 'string')
createDatagroupFromFile(dgname, dgfilename)
showDatagroups()
showDatagroupFiles()
- Satoshi_Toyosa1Ret. Employee
Assuming you have both data-group file object (tmsh list sys file data-group <object>) and external data-group ltm object (tmsh list ltm data-group external <object>), then sending a PATCH request to the file object would override the file contents. The ltm object is just a reference to the file, hence the new data becomes automatically available from iRule.
For example,
1) Before the change, the external data-group file contained the following data (see the bottom for the test iRule code)
# Edited for readability 16:02:19 /Common/ExternalDg <HTTP_REQUEST>: {Shota} Short 16:02:19 /Common/ExternalDg <HTTP_REQUEST>: {Ochako} Uravity 16:02:19 /Common/ExternalDg <HTTP_REQUEST>: {Izuku} Deku 16:02:19 /Common/ExternalDg <HTTP_REQUEST>: {Tenya} Ingenium
2) Update the file by calling PATCH /mgmt/tm/sys/file/data-group/<object>.
curl -sku $PASS \ https://$HOST/mgmt/tm/sys/file/data-group/dg-testMHA.txt \ -X PATCH -H "Content-type: application/json" \ -d '{"sourcePath":"http://172.16.10.10/dg-testMHA.txt"}'
where dg-testHMA.txt is the existing file object. Note that the file contains all the records. When the call successfully completes, you will see the following messages in the ltm log:
# Edited. The same messages repeat because I have two TMMs. 16:04:24 tmm1 External Datagroup (/Common/dg-testMHA) queued for update. 16:04:24 tmm External Datagroup (/Common/dg-testMHA) queued for update. 16:04:25 tmm1 External Datagroup (/Common/dg-testMHA) update finished. 16:04:25 tmm External Datagroup (/Common/dg-testMHA) update finished.
3) Check the data contents again. New data is added.
# Edited. I do not know why the order is different. 16:04:35 /Common/ExternalDg <HTTP_REQUEST>: {Tsuyu} Floppy 16:04:35 /Common/ExternalDg <HTTP_REQUEST>: {Tenya} Ingenium 16:04:35 /Common/ExternalDg <HTTP_REQUEST>: {Ochako} Uravity 16:04:35 /Common/ExternalDg <HTTP_REQUEST>: {Shota} Short 16:04:35 /Common/ExternalDg <HTTP_REQUEST>: {Izuku} Deku
Appendix: Test iRule code
when HTTP_REQUEST { foreach {record} [class get dg-testMHA] { log local0. "$record" } }
- Bill_Homan
Nimbostratus
Thanks Satoshi! I was able to employ patch as you suggested using this Python code:
def update_edg_file(rq, url, fileserver, newfilename, dgfname): path = fileserver + "/" + newfilename payload = {} payload['sourcePath'] = path req = '{}/sys/file/data-group/{}'.format(url, dgfname) resp = rq.patch(req, json.dumps(payload)) print("update_edg_file, name: {}, PATCH response: {}".format(dgfname, resp)) # in main ... url_base = 'https://%s/mgmt/tm' % hostname rs = requests.session() rs.auth = (username, password) rs.verify = False rs.headers.update({'Content-Type':'application/json'}) .... update_edg_file(rs, url_base, external_file_server, filename, edg_file_name)
The update is pretty quick.
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com