Forum Discussion
Brad_Baker
Cirrus
CirrusModifying multiple entries in a datagroup via api?
We have a datagroup with entries like this:
domain1.com := virtual /Common/www.domain1.com_vs_443
domain2.com := virtual /Common/www.domain2.com_vs_443
domain3.com := virtual /Common/www.domain3.com_vs_443
And so forth. This datagroup gets used by an iRule for re-routing traffic based on SNI.
I need to be able to make an HTTP call to the F5 to remap these to put up a mainteance page. So in other words I want to modify the above to:
domain1.com := virtual /Common/maintenance.domain1.com_vs_443
domain2.com := virtual /Common/maintenance.domain2.com_vs_443
domain3.com := virtual /Common/maintenance.domain3.com_vs_443
I figured out that I can make a curl request as such to delete entries from the datagroup:
curl -ku "admin:superS3cret" -X PATCH -H 'Content-type: application/json' -d '{ "name":"dummy" }' https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options=records%20delete%20%7B%20www.domain1.com,www.domain2.com,www.domain3.com%20%7D |jq .
And via this article https://community.f5.com/t5/technical-forum/add-new-key-into-data-group-without-updating-entire-list-using/td-p/272699 I was able to determine we could add a key using this:
curl -ku "admin:superS3cret" -X PATCH -H 'Content-type: application/json' -d '{ "name":"dummy" }' 'https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options=records%20add%20%7b%20www.domain1.com%20%7b%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain1.com_vs_443%22%20%7d%20%7d' |jq .
However then I have to have one HTTP Request per domain. If I need to repoint 20 sites for example I have to make 20 different requests instead of one request with all domains.
Is there a way to add multiple records at once? I tried something like this:
curl -ku "admin:superS3cret" -X PATCH -H 'Content-type: application/json' -d '{ "name":"dummy" }' 'https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options%3Drecords%20add%20%7B%20www.domain1.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain1.com_vs_443%22%20%7D%20domain1.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain1.com_vs_443%22%20%7D%20www.domain2.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain2.com_vs_443%22%20%7D%20www.domain3.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain3.com_vs_443%22%20%7D%20www.domain4.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain4.com_vs_443%22%20%7D%7D' |jq .
Here's the URL HTML decoded (so its easier to read):
https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options=records add { www.domain1.com { data "virtual /Common/maintenance.domain1.com_vs_443" } domain1.com { data "virtual /Common/maintenance.domain1.com_vs_443" } www.domain2.com { data "virtual /Common/maintenance.domain2.com_vs_443" } www.domain3.com { data "virtual /Common/maintenance.domain3.com_vs_443" } www.domain4.com { data "virtual /Common/maintenance.domain4.com_vs_443" }}
However I get this error:
{
"code": 400,
"message": "one or more properties must be specified",
"errorStack": [],
"apiError": 26214401
}
I haven't been able to figure out if adding multiple records isn't supported or if I have the syntax wrong and frankly the documentation here: https://clouddocs.f5.com/api/icontrol-rest/APIRef_tm_ltm_data-group_internal.html and here: https://clouddocs.f5.com/cli/tmsh-reference/v14/modules/ltm/ltm_data-group_internal.html isn't super clear.
Frankly the fact that patch doesn't just modify entries sent seems like a bug to me and using ?options record add seems like a hack.
Alternatively is there some way to run a modify instead of running delete and then add? That would be even more efficient (less HTTP calls) and less opportunity for a call to fail.
I know I can just get all records, modify them and then post all records back. The trouble with that approach is that then I'm touching records I really don't want to touch and having to do some sort of regex replace. This seems like it could have the potential to accidentally modify records I don't want to modify if my regex isn't very explicit. I would prefer a more targeted approach which only modifies the records that need modifying and that touches nothing else.
Thanks
Hi Brad_Baker
You can additionally edit the content for a specific register
curl -ku "admin:Logi-123" -X PATCH -H 'Content-type: application/json' -d '{ "name":"test_domain" }' https://localhost/mgmt/tm/ltm/data-group/internal/test_domain?options=records%20modify%20%7B%20key3%20%7B%20data%20ke3new%20%7D%20%7D|jq .as tmos you can use all of this option to interact with the data-groups
add delete modify none replace-all-withto avoid mistakes use a tool to encode and decode the URL.
Hi Brad_Baker,
The easiest way is to delete all the content of the datagroup and then add the new records, for delete all the records use this curl:
curl -ku "admin:admin" -X PATCH 'https://192.168.30.10/mgmt/tm/ltm/data-group/internal/test_domain' --header 'Content-Type: application/json' --data '{ "records": []}'
To add the new records in only one call you can use this curl:
curl -ku "admin:admin" -X PATCH 'https://192.168.30.1/mgmt/tm/ltm/data-group/internal/test_domain' --header 'Content-Type: application/json' --data '{"records": [{"name": "domain1.com","data": "virtual /Common/maintenance.domain1.com_vs_443"},{"name": "domain2.com","data": "virtual /Common/maintenance.domain2.com_vs_443"},{"name": "domain3.com","data": "virtual /Common/maintenance.domain3.com_vs_443"}]}'
Hope it works.
Brad_BakerHey Sebastiansierra thanks for the response and suggestions.
Unfortunately, the recommendation above won't accomplish what I am trying to do.
I have approximately 100-150 entries in my datagroup. When I run the command recommended above:
curl -ku "admin:admin" -X PATCH 'https://192.168.30.1/mgmt/tm/ltm/data-group/internal/test_domain' --header 'Content-Type: application/json' --data '{"records": [{"name": "domain1.com","data": "virtual /Common/maintenance.domain1.com_vs_443"},{"name": "domain2.com","data": "virtual /Common/maintenance.domain2.com_vs_443"},{"name": "domain3.com","data": "virtual /Common/maintenance.domain3.com_vs_443"}]}'It replaces all 100-150 entries with the 4 entries in the data section.
I don't want to delete existing entries; I only want to edit specific records. So I'm looking to do a targeted modification to only touch the records that need changing and leave the rest alone.
Also, I understand I could do a GET to get all records, use a regex to replace just what I want to replace, and then do a PUT/PATCH to send the modified version back.
But that hinges on getting a regex exactly right. If my regex matches wrong (i.e. too much) I could accidentally edit the wrong record. It just seems a bit risky to do it using regex/pattern matching. Thus wanting a targeted approach that edits specific records.
Is that possible or does the F5 api just not support that?
Hi Brad_Baker,
to avoid mistakes you can use this curl call format to have more clear the new records that you want to create:
curl -ku "admin:admin" -X PATCH 'https://192.168.30.1/mgmt/tm/ltm/data-group/internal/test_domain' \ --header 'Content-Type: application/json' \ --data '{ "records": [ { "name": "domain1.com", "data": "virtual /Common/maintenance.domain1.com_vs_443" }, { "name": "domain2.com", "data": "virtual /Common/maintenance.domain2.com_vs_443" }, { "name": "domain3.com", "data": "virtual /Common/maintenance.domain3.com_vs_443" } ] }'- Brad_Baker
Sebastiansierra manually add domain4.com, domain5.com and domain6.com into the datagroup from the UI. Then run the curl request above. You'll see domain4/5/6 disappear. That's what I want to prevent. I want the query to only touch domain1-3 and leave domain4-6 alone.
Hi Brad_Baker,
In this post, someone found the method for making the changes in the datagroup:
Hope it works.
Hi Brad_Baker,
I finally found the solution to modify more than one register, it uses the method option to call tmos, so you have to adapt the URL to tmos code and it works, in my example, I have added two records, key3, and key4 in the same call:
curl -ku "admin:Logi-123" -X PATCH -H 'Content-type: application/json' -d '{ "name":"test_domain" }' https://localhost/mgmt/tm/ltm/data-group/internal/test_domain?options=records%20add%20%7b%20key3%20%7b%20data%20value3%20%7d%20key4%20%7b%20data%20value4%20%7d%20%7d |jq .I used notepad++ to decode and encode the URL and it looks like:
curl -ku "admin:admin" -X PATCH -H 'Content-type: application/json' -d '{ "name":"test_domain" }' https://localhost/mgmt/tm/ltm/data-group/internal/test_domain?options=records add { key3 { data value3 } key4 { data value4 } } |jq .Hope it works.
Hi Brad_Baker
You can additionally edit the content for a specific register
curl -ku "admin:Logi-123" -X PATCH -H 'Content-type: application/json' -d '{ "name":"test_domain" }' https://localhost/mgmt/tm/ltm/data-group/internal/test_domain?options=records%20modify%20%7B%20key3%20%7B%20data%20ke3new%20%7D%20%7D|jq .as tmos you can use all of this option to interact with the data-groups
add delete modify none replace-all-withto avoid mistakes use a tool to encode and decode the URL.
- Brad_Baker
Perfect - that's exactly what I was trying to do! I was just using add instead of modify - it seems that was the issue.
Thank you so much figuring that out! 🙂 Now I don't have to worry about accidentally messing up any other datagroup records during an update.
