Forum Discussion
saidshow
Cirrus
CirrusAPI Calls to F5 limited to 1024 KB download
Hi,
I am interacting with the F5 API in order to download ASM policies for the purpose of automating the backups. The process works fine however policies larger than 1024 KB are cut off at this size of 1024 KB. Initially I suspected that there was a default limit on the curl request however I have not been able to find information on how to increase this with the curl request.
Is this a limitation on the F5 API or the Curl request? wget is not an option as this is not natively supported on the F5 virtual appliance. My script lives on the appliance, downloads the relevant policies and then pushes them to a SMB share. The only issue is that the ASM policies that are larger than 1024 KB are being cut off at 1024 KB.
The API calls are as per the documentation here:
http://cdn.f5.com/websites/devcentral.f5.com/downloads/icontrol-rest-api-user-guide-13-0-0.pdf
specifically:
GET https://x.x.x.x/mgmt/tm/asm/policies
POST https://x.x.x.x/mgmt/tm/asm/tasks/export-policy
GET https://x.x.x.x/mgmt/tm/asm/file-transfer/downloads/$asmPolicy
Excluding the processing in my script the API calls I make are shown below: I expect the issue resides in the download api call. Is there a switch I can add to increase this limit?
curl -ku 'username:password' -X GET https://x.x.x.x/mgmt/tm/asm/policies | jq '.items[] | "pol_name:" + .name + ";api_id:" + .id' >> $wdir/asmDetails.txt
curl -ku 'username:password' -X POST https://x.x.x.x/mgmt/tm/asm/tasks/export-policy -H 'Content-Type: application/json' -d '{"filename":"'$asmPolicy'","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/'$asmIDs'"}}'
curl -ku 'username:password' -X GET https://x.x.x.x/mgmt/tm/asm/file-transfer/downloads/$asmPolicy > $wdir/asmBackup/"$folderName"/$number-$asmPolicy-$hostname-"$dateStamp".xml
Thanks
I found my files disappeared (including bogus test files I created) from the /var/ts/var/rest directory too. I haven't been able to find the process responsible for deletion .
saidshowI am trying to post my solution but get an error from the devcentral portal that reads "Maximum number of topic assignments have been exceeded"
- saidshow
- my working solution with creds removed is shown below. You were CRUCIAL to getting this to work. Thank you so very much!!!
Unfortunately I get an error when trying to post this as code... hopefully users can zoom in on the image below.
I have an iCall period handler kicking off an iCall script that executes this bash file. Note, this does push the exported asm policies in xml format to an external SMB share and forces a cleanup.
In the event that the unix box has rm pointing to rm -i, I specify /bin/rm to prevent prompting for the clean up.
I will leave this run in TEST every 15 mins for an hour, when I am comfortable all is well, I will have it run every 28 days at 2am starting on Sunday. This is controlled with the iCall period handler.
JGCumulonimbus
The underlying purpose of the question has been served by a work-around, and thanks for the sharing of the script.
However, the subject issue of curl failing to download beyond the first 1024KB remains.
In my tests, specifying the "Range:" header in the client request was also not successful; the server always returned the first 1024KB no matter what byte range was requested.
[Edit]
Satoshi Toyosawa has just clarified (in his linked post above) that "Content-Length" should be used rather than "Range:", as this is how F5 implemented it.
