Forum Discussion

saidshow_251381's avatar
saidshow_251381
Icon for Cirrostratus rankCirrostratus
Dec 06, 2016

iControlREST and Curl to save and download ASM policies

Hi,

I want to be able to save/export asm policies on the F5 and then download. I want to do this using iControlREST and curl.

I am able to save UCS files with the post shown below:

curl -v -sk -u admin:admin https://myF5/mgmt/tm/sys/ucs -H 'Content-Type: application/json' -X POST -d '{"command":"save","name":"blah.ucs"}' | jq

 

However if I try to do something similar for asm I get errors. Below is what I was trying with asm.

curl -v -sk -u admin:admin https://myF5/mgmt/tm/asm/policies/fn9GoMrandomGvoN2dD  -H 'Content-Type: application/json' -X POST -d '{"command":"save","name":"as_test.xml"}' | jq

 

The error I get is: { "code": 400, "message": "Could not parse/validate the Policy 'Security Policy /Common/as_test'. Unknown field 'command'", "originalRequestBody": "{\"command\":\"save\",\"name\":\"as_test.xml\"", "referer": "x.x.x.x", "restOperationId": 59083, "kind": ":resterrorresponse" }

Thank you

ASM Advanced WAF
curl
download
export
iControl
iControlREST
security
upload
  • Hannes_Rapp's avatar
    Hannes_Rapp
    Icon for Nimbostratus rankNimbostratus
    Dec 06, 2016

     

    1: Extract policy names and corresponding API IDs
  hannes@ubuntuBash$ curl -ku 'admin:admin' -X GET https://172.16.99.91/mgmt/tm/asm/policies | jq '.items[] | "pol_name:" + .name + ";api_id:" + .id'
  "pol_name:sp_fake.google.com;api_id:00TT7Lv-Yba2xcEZkjkc3A"
  "pol_name:sp_fake.github.com;api_id:cswJ2oCNNqbfdWMVWdzoTw"

2: Export policies natively on the appliance
  hannes@ubuntuBash$ curl -ku 'admin:admin' -X POST https://172.16.99.91/mgmt/tm/asm/tasks/export-policy -H 'Content-Type: application/json' -d '{"filename":"sp_fake.google.com.xml","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/00TT7Lv-Yba2xcEZkjkc3A"}}'
  hannes@ubuntuBash$ curl -ku 'admin:admin' -X POST https://172.16.99.91/mgmt/tm/asm/tasks/export-policy -H 'Content-Type: application/json' -d '{"filename":"sp_fake.github.com.xml","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/cswJ2oCNNqbfdWMVWdzoTw"}}'

3: Exported policies can be found on BigIP at '/var/ts/var/rest'
  [hannes@bip-01:Active:In-Sync] config  ls /var/ts/var/rest
  admin~sp_fake.github.com.xml  admin~sp_fake.google.com.xml

4: Download the exports: Issue a cURL to that policy, save output as a local file in your external file system.
  hannes@ubuntuBash$ curl -ku 'admin:admin' -X GET https://172.16.99.91/mgmt/tm/asm/file-transfer/downloads/sp_fake.google.com.xml >> sp_fake.google.com.xml
  hannes@ubuntuBash$ curl -ku 'admin:admin' -X GET https://172.16.99.91/mgmt/tm/asm/file-transfer/downloads/sp_fake.github.com.xml >> sp_fake.github.com.xml
  hannes@ubuntuBash$ ls
  sp_fake.google.com.xml sp_fake.github.com.xml

     

    Voila! Note that this was done on 12.1.1. I think all of the above will work on 11.6.x too. On 11.5.x the 4th step will fail (not implemented error).

     

    • saidshow_251381's avatar
      saidshow_251381
      Icon for Cirrostratus rankCirrostratus
      Mar 06, 2017

      Hi Hannes,

      Great advice again. I have taken what you provided and scripted it up. Just replace username and password and the IPs and off it goes. This script will create a folder with the current AEST date and time and place all the xml backups in that folder.

      Since I create 3 files in the script, I also remove them at the end. If I don't remove them they will grow every time the script is executed.

      I hope this helps someone else half as much as you helped me.

       

      !/bin/bash

curl -ku 'username:password' -X GET https://x.x.x.x/mgmt/tm/asm/policies | jq '.items[] | "pol_name:" + .name + ";api_id:" + .id' >> asmDetailsLAB.txt
cat asmDetailsLAB.txt |grep pol_name |cut -d":" -f2 |cut -d";" -f1 >> asmPoliciesLAB.txt
cat asmDetailsLAB.txt |grep pol_name |cut -d":" -f3 |cut -d'"' -f1 >> asmIDsLAB.txt


folderName="$(zdump AEST)"
mkdir -p asmLabBackup
mkdir "asmLabBackup/""$folderName"

paste -d'\n' asmPoliciesLAB.txt asmIDsLAB.txt | while read asmPolicy && read asmIDs;do
echo $asmPolicy $asmIDs
curl -ku 'username:password' -X POST https://x.x.x.x/mgmt/tm/asm/tasks/export-policy -H 'Content-Type: application/json' -d '{"filename":"'$asmPolicy'","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/'$asmIDs'"}}'
curl -ku 'username:password' -X GET https://x.x.x.x/mgmt/tm/asm/file-transfer/downloads/$asmPolicy >> asmLabBackup/"$folderName"/$asmPolicy.xml
done

rm asmDetailsLAB.txt
rm asmPoliciesLAB.txt
rm asmIDsLAB.txt

       

      • WildWeasel's avatar
        WildWeasel
        Icon for Cirrus rankCirrus
        May 12, 2023

        Thank You for sharing this..  It was a big help and got me halfway there.

        Anyway you have a similar script to now IMPORT all those .xml files?

    • saidshow_251381's avatar
      saidshow_251381
      Icon for Cirrostratus rankCirrostratus
      Dec 06, 2016

      Hannes Thank you! This is far more detailed than I expected. Thank you for your efforts. I have already upgraded to Ver12.1.1 so this will just perfect. I'll give this a try as soon as time allows. This really is appreciated. Well Done.

       

  • Wojciech_Wypior's avatar
    Wojciech_Wypior
    Icon for Nimbostratus rankNimbostratus
    Mar 30, 2017

    Hello Saidshow,

    For any export/import on ASM, you need to use its /tasks endpoint, and file transfer endpoint afterwards.

    In your case it is:

     

    /mgmt/tm/asm/tasks/export-policy
/mgmt/tm/asm/file-transfer/downloads

     

    NOTE: This is not possible in 11.5.4 as file_transfer endpoint for ASM was added in 11.6.x. You can attempt to use another endpoint for file transfer (see: this article )

    If you are familiar with python, using this library would be your best choice:

     

    https://github.com/F5Networks/f5-common-python

     

    Please have a look at the functional and unit tests as they should give you idea how to use it.

    If you really want to use CURL, then I guess something like this should work:

    1. Export to a file

     

    curl -v -sk -u admin:admin https://myF5/mgmt/tm/asm/tasks/export-policy -H 'Content-Type: application/json' -X POST -d '{"filename":"as_test.xml","policyReference":"{"link": "https://localhost/mgmt/tm/asm/policies/yutqTnGiAizDLpdd_mkRvg"}"}'

     

    • Policy link above would have to correspond to selfLink property of your intended exported policy.

    2. Download the file

     

    curl -v -sk -u admin:admin https://myF5/mgmt/tm/asm/file-transfer/downloads/as_test.xml -H 'Content-Type': 'application/octet-stream' -X GET

     

    I hope this helps