F5 Per-App AS3 Part 2 How to see if there are manual changes!
Code version: The code was tested on 17.1.5.3 AS3: 3.55 For more about AS3 and per-app AS3 see my previous code share Part 1 article: https://community.f5.com/kb/codeshare/f5-per-app-as3-part-1-how-share-tenant-specific-object/345072 First we will send Per-App AS3 declaration as shown below. { "id": "per-app-declarationn", "schemaVersion": "3.55.0", "controls": { "class": "Controls", "logLevel": "debug", "trace": true, "traceResponse": true }, "A2": { "class": "Application", "service": { "class": "Service_HTTP", "virtualAddresses": [ "10.0.3.10" ], "pool": "web2_pool" }, "web2_pool": { "class": "Pool", "monitors": [ "http" ], "members": [{ "servicePort": 80, "serverAddresses": [ "192.7.21.10", "192.7.21.11" ] }] } } } Then we will change for example virtual server ip from 10.0.3.10 to 10.0.3.11 and we will send the same declaration but with "dryRun" set to true as this will cause AS3 to validate the config but not to execute it and with trace and traceResponse we will get the difference 😎 { "id": "per-app-declarationn", "schemaVersion": "3.55.0", "controls": { "class": "Controls", "logLevel": "debug", "trace": true, "dryRun": true, "traceResponse": true }, "A2": { "class": "Application", "service": { "class": "Service_HTTP", "virtualAddresses": [ "10.0.3.10" ], "pool": "web2_pool" }, "web2_pool": { "class": "Pool", "monitors": [ "http" ], "members": [{ "servicePort": 80, "serverAddresses": [ "192.7.21.10", "192.7.21.11" ] }] } } } Now we see that the IP has been changed from 10.0.3.10 to 10.0.3.11 and here we go now we have the difference ! This can be added in CI/CD to always first do "dry-run" using the original declaration to see if there are changes before executing the new AS3 declaration that could be for example changing the IP address to 10.0.3.12 but using the official way. Look at the Json reply "diff" section that is seen thanks to trace and traceResponse options and an automation can just check this section and stop the new deployment if the manual changes need to be reviewed first. For AS3 basic declaration not Per-App actually the "dry-run" is a different. F5 likes changing the naming like Local Traffic policies to Endpoint Policies or naming of TLS profiles between GU/TMSH and AS3 😅 { "$schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/refs/heads/main/schema/3.55.0/as3-schema.json", "class": "AS3", "action": "dry-run", "logLevel": "debug", "trace": true, "traceResponse": true, "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.55.0", "id": "BIG-IP-Example-Tenant", "Example": { "class": "Tenant", "Shared": { "class": "Application", "template": "shared", "Example_Response": { "remark": "Used for F5 response", "class": "iRule", "iRule": { "base64": "d2hlbiBIVFRQX1JFUVVFU1Qgew0KICAgSFRUUDo6cmVzcG9uZCAyMDAgY29udGVudCB7DQogICAgICA8aHRtbD4NCiAgICAgICAgIDxoZWFkPg0KICAgICAgICAgICAgPHRpdGxlPkFwb2xvZ3kgUGFnZTwvdGl0bGU+DQogICAgICAgICA8L2hlYWQ+DQogICAgICAgICA8Ym9keT4NCiAgICAgICAgICAgIFdlIGFyZSBzb3JyeSwgYnV0IHRoZSBzaXRlIHlvdSBhcmUgbG9va2luZyBmb3IgaXMgdGVtcG9yYXJpbHkgb3V0IG9mIHNlcnZpY2U8YnI+DQogICAgICAgICAgICBJZiB5b3UgZmVlbCB5b3UgaGF2ZSByZWFjaGVkIHRoaXMgcGFnZSBpbiBlcnJvciwgcGxlYXNlIHRyeSBhZ2Fpbi4NCiAgICAgICAgIDwvYm9keT4NCiAgICAgIDwvaHRtbD4NCiAgIH0NCn0=" } } } } } } This will not show if someone has manually added a vlan for example as only changes on the apps that were deployed with AS3 will be seen. For those you will get error like the one below when you try to delete the partition. "" 0107082a:3: All objects must be removed from a partition "" https://my.f5.com/manage/s/article/K02718312 https://my.f5.com/manage/s/article/K000138638 Github link: https://github.com/Nikoolayy1/AS3-Per-App-Manual-Changes/tree/mainF5 Per-App AS3 Part 1 How Share Tenant specific object
Code version: The code was tested on 17.1.5.3 AS3: 3.55 For testing F5 AS3 you can use combination of Postman as Visual Studio has option to post AS3 declarations but not Per-App specific as that will be needed for executing my code. Still I recommend using Visual Studio for general AS3 or Fast Templates and you can use the reference links for how to use Visual Studio F5 extensions and maybe F5 will update the extension in the future to work for Per-App AS3 with options for adding tenant and as variables 🤔 For FAST nice example is at https://github.com/Nikoolayy1/fast-template-examples The same way the /Shared objects can be configured under /Common can be done under a specific tenant and this is really useful if Per-App AS3 is enabled as each AS3 app will use separate declaration and all of them can use a customer iRule defined under /<Tenant>/Shared. References for AS3 Declarative API: https://www.youtube.com/watch?v=rrjrcnwAe1g&t=433s AS3 Foundations: Beyond Imperatives - What the Heck is AS3? | DevCentral https://www.youtube.com/watch?v=OHSBpOtQg_0&list=PLrn3yqXftAPUPdSP_kbCX6BDicVprsnfI Per-App AS3 is enabled by default for the AS3 3.50.x and above but if needed enable it. POST Endpoint: /mgmt/shared/appsvcs/settings { "perAppDeploymentAllowed": true } POST Endpoint for Creating the Tenant and the Shared irule that is base64 endpoint: /mgmt/shared/appsvcs/declare { "schemaVersion": "3.45.0", "Shared": { "class": "Application", "template": "shared", "test": { "class": "iRule", "remark": "yes", "iRule": { "base64": "d2hlbiBIVFRQX1JFUVVFU1Qgew0KICAgSFRUUDo6cmVzcG9uZCAyMDAgY29udGVudCB7DQogICAgICA8aHRtbD4NCiAgICAgICAgIDxoZWFkPg0KICAgICAgICAgICAgPHRpdGxlPkFwb2xvZ3kgUGFnZTwvdGl0bGU+DQogICAgICAgICA8L2hlYWQ+DQogICAgICAgICA8Ym9keT4NCiAgICAgICAgICAgIFdlIGFyZSBzb3JyeSwgYnV0IHRoZSBzaXRlIHlvdSBhcmUgbG9va2luZyBmb3IgaXMgdGVtcG9yYXJpbHkgb3V0IG9mIHNlcnZpY2U8YnI+DQogICAgICAgICAgICBJZiB5b3UgZmVlbCB5b3UgaGF2ZSByZWFjaGVkIHRoaXMgcGFnZSBpbiBlcnJvciwgcGxlYXNlIHRyeSBhZ2Fpbi4NCiAgICAgICAgIDwvYm9keT4NCiAgICAgIDwvaHRtbD4NCiAgIH0NCn0=" } } } } POST Endpoint for sending Per-App AP that references the Tenant shared iRule I: /mgmt/shared/appsvcs/declare/Example/applications { "schemaVersion": "3.45.0", "A1": { "class": "Application", "service": { "class": "Service_HTTP", "virtualAddresses": [ "10.0.1.10" ], "pool": "web_pool", "iRules": [ { "use": "/Test/Shared/test" }, "insert_xff" ] }, "web_pool": { "class": "Pool", "monitors": [ "http" ], "members": [ { "servicePort": 80, "serverAddresses": [ "192.0.1.10", "192.0.1.12" ] } ] }, "insert_xff": { "class": "iRule", "iRule": "when HTTP_REQUEST { HTTP::header insert X-Forwarded-For [IP::client_addr] }" } } } Github link: Editing AS3-Per-App-Shared-Tenant-Objects/README.md at main · Nikoolayy1/AS3-Per-App-Shared-Tenant-Objects Note! To see the Tenant config use GET Endpoint /mgmt/shared/appsvcs/declare/Example or for the app /mgmt/shared/appsvcs/declare/Example/A1 For only the Tenant shared config use GET /mgmt/shared/appsvcs/declare/Example/applications/Shared By placing the pools in the /Shared under the Tenant all Per-AS3 declarations for Tenant can use the pool like for the iRule example I showed and also the pools can share the node IP addresses like the "shareNodes" option but only for the tenant. https://my.f5.com/manage/s/article/K88250015
Terraform AS3 code for GTM Only.
Hello All, I am really really suffering here :( Have been looking for GTM ONLY code in AS3 form, need a simple code hardcoded values will also work. I have seen documentation and couldn't see exact use case. We are doing POC for where VMs are direct;y added to GTM and NO LTM component are there. I can't post my LTM + GTM code as its in office. Would really appreciate any help and guidance here. Any simple code work snippet using only AS3 please.Less than 60 seconds lab setup
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’. Today I'll share with you my less than 60 seconds lab setup which I use for testing basic stuff. It's an AS3 declaration that will setup two virtuals, the first virtual that accepts any http traffic on port 80 and forwards it to a second virtual that will respond 200 OK to any HTTP request. The lab can easily be extended to add a https virtual. Purpose of this setup I use this configuration for many scenarios. With this setup I can test different profiles, TLS configurations (requires small adjustments), AWAF rules and iRules attached to the first virtual server without the requirement to setup any backend application. Deploying this AS3 declaration takes less than 20 seconds and I have a basic lab environment ready. Prerequisites In order to use this config, you must have AS3 installed on your BIG-IP. If you have not worked with AS3 yet and you are new to automation, I recommend you to start with Visual Studio Code and install The F5 Extension. From The F5 Extension you can connect your BIG-IP and install the AS3 extension and deploy the declaration. Furthermore: if you have not with AS3 yet - you're damn late to the party! My AS3 declaration The full declaration is available on GitHub, let's just look at the iRules. The iRules are the important part of this lab config. Don't get confused that you won't see the iRule code in the AS3 declaration. It's there, but it's base64 encoded. Forwarding iRule The iRule attached to the first virtual just forwards to the second virtual. Don't get confused by the path /simple_testing/responder_service/. AS3 works with Partitions, so called tenants. Therefore I must reference the second virtual with the name of its partition and application. when HTTP_REQUEST { virtual /simple_testing/responder_service/service_http_200 } HTTP Responder iRule The second iRule is attached to the second virtual server. It will just return a HTML page that says 200 OK to any request. when HTTP_REQUEST { HTTP::respond 200 content { <html> <head> <title>BIG-IP</title> </head> <body> 200 OK </body> </html> } } Deployment As said above, for starting with this you don't need anything but a BIG-IP and Visual Studio Code. After installing the F5 Extension you can connect (using the + symbol) to your BIG-IP from VS Code. After connecting you can install the AS3 extension on your BIG-IP. And then you are ready to deploy the AS3 declaration linked above. The deployment will take less than 60 seconds. Once the deployment is done, you will have a Partition called on your BIG-IP. There you will find the two virtual servers. The website is nothing special... What's next? In the next couple of days, I will share with you a simple website I made with the help of ChatGPT. It can run on any webserver, NGINX, Apache, IIS... The website has 4 flavors (red, blue, green and yellow) and I use it for testing LTM use-cases like persistence, priority groups, http profiles, SNAT, etc. This will be my less than 600 seconds lab.Less than 600 seconds lab
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’. In my previous post I shared with you, how you can deploy a lab environment in less than 60 seconds with AS3. This time let's take a look at another lab, that you can set up in less than 10 minutes. Purpose of this lab This lab requires a web server. And some minimal knowledge of Linux (Debian) and git. In my example, I use NGINX. The web application consists of four pages in four colours (red, blue, yellow and green) that are designed to demonstrate the load balancing functionality of the F5 Local Traffic Manager (LTM). You can use the app to familiarise yourself with load balancing functionalities such as: different load balancing methods and priority groups different types of persistence caching HTTP, SSL and other profiles SNAT The web application has a couple of nice features real-time server information display like Server hostname Request timestamp (ISO 8601 format) Request URI Source IP address X-Forwarded-For (XFF) header User-Agent informatio modern, responsive UI picture gallery Prerequisites First you need to set up and configure the web server. Add multiple IPs to the web server (Debian 11+). Edit /etc/network/interfaces: sudo nano /etc/network/interfaces Add the following: allow-hotplug eth0 iface eth0 inet static address 192.168.1.10/24 gateway 192.168.1.1 auto eth0:1 allow-hotplug eth0:1 iface eth0:1 inet static address 192.168.1.11/24 auto eth0:2 allow-hotplug eth0:2 iface eth0:2 inet static address 192.168.1.12/24 auto eth0:3 allow-hotplug eth0:3 iface eth0:3 inet static address 192.168.1.13/24 Restart networking: sudo systemctl restart networking Note: Replace eth0 with your actual interface name. Generate SSL Certificate Create a self-signed SSL certificate with RSA 2048-bit key (no password): openssl req -x509 -nodes -days 365 -newkey rsa:2048 \ -keyout nginx-selfsigned.key -out nginx-selfsigned.crt \ -subj "/C=US/ST=State/L=City/O=Organization/CN=example.com" Installing the web application Example for NGINX 1. Clone the repository git clone https://github.com/webserverdude/ltm-demo-html.git cd webpages 2. Deploy to your web server sudo cp -r * /var/www/ltm-demo-html 3. Configure your web server see below NGINX Configuration The configuration includes HTTP as well as HTTPS listeners. Add this configuration to your NGINX server block: server { listen 192.168.1.10:8000 default_server; root /var/www/ltm-demo-html; index index_red.html; server_name _; add_header X-Backend-Server 1; add_header Set-Cookie "X-Backend-Server=1; Max-Age=10"; location / { try_files $uri $uri/ =404; } # Enable the substitution filter sub_filter_once off; # Allow multiple substitutions # Replace template variables with actual NGINX variables sub_filter '{{server_name}}' '$hostname'; sub_filter '{{time_iso8601}}' '$time_iso8601'; sub_filter '{{request_uri}}' '$request_uri'; sub_filter '{{remote_addr}}' '$remote_addr'; sub_filter '{{http_x_forwarded_for}}' '$http_x_forwarded_for'; sub_filter '{{http_user_agent}}' '$http_user_agent'; } server { listen 10.0.2.71:443 ssl default_server; ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt; ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key; # SSL configuration ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384; ssl_prefer_server_ciphers off; ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; root /var/www/ltm-demo-html; index index_red.html; server_name _; add_header X-Backend-Server 1; add_header Set-Cookie "X-Backend-Server=$request_id; Max-Age=10; Secure; SameSite=Strict"; location / { try_files $uri $uri/ =404; } # Enable the substitution filter sub_filter_once off; # Allow multiple substitutions # Replace template variables with actual NGINX variables sub_filter '{{server_name}}' '$hostname'; sub_filter '{{time_iso8601}}' '$time_iso8601'; sub_filter '{{request_uri}}' '$request_uri'; sub_filter '{{remote_addr}}' '$remote_addr'; sub_filter '{{http_x_forwarded_for}}' '$http_x_forwarded_for'; sub_filter '{{http_user_agent}}' '$http_user_agent'; } Note: This is just a snippet for one HTTP and one HTTPS virtual. The full config for all four pages is available at my Git repository in the nginx_config folder. Once this is done, check the web pages from your browser. Make sure they work as expected. Configure your BIG-IP After the web server is running and serving all 4 pages with HTTP and HTTPS, you can configure your BIG-IP. My AS3 declaration includes an HTTP and an HTTPS virtual server, two pools and some http and persistence profiles. Here is a snippet: { "$schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/main/schema/latest/as3-schema.json", "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.0.0", "LTM_Demo": { "class": "Tenant", "LTM_Demo": { "class": "Application", "vs_http": { "class": "Service_HTTP", "virtualAddresses": [ "192.168.3.80" ], "persistenceMethods": [], "profileHTTP": { "use": "pr_http_xff" }, "pool": "pl_ltm-demo_http", "snat": { "use": "pl_SNAT_addresses" } }, ... The complete AS3 configuration can be found in my Git repository. The repository also contains an additional AS3 declaration with further configuration options. Note: You should not deploy the second declaration with the optional configurations; instead, merge the snippets you want to use into ltm_demo.json. Deployment The deployment of the AS3 declaration works similar like I described in my previous post. What's next? You can try differnet load balancing algorithms, persistence methods, caching, SSL configurations. Once you set up the web app and the LTM config, play around - the sky is the limit. Have fun!AS3 Foundations: Top 10 Features to Know in the VSCode F5 Extension
I joined in on the fun at several stops on the F5 Academy BIG-IP Next roadshow tour this fall, and in talking to customers, everyone is at various stages of their F5 automation journey, and some aren't automating at all yet. I'm kicking off a six-part series to set some foundational understanding of what AS3 is, how it works, what tools you might use to interact with it, and we'll finish it off with some best practices. If you want to learn or if you want to share your own experience with the class, join me over the next three weeks to finish 2023 in style! Episode Two Why VSCode for AS3? Top Features to Know for AS3 DevelopmentAS3 Foundations: Migrating and Deploying Applications in VSCode
I joined in on the fun at several stops on the F5 Academy BIG-IP Next roadshow tour this fall, and in talking to customers, everyone is at various stages of their F5 automation journey, and some aren't automating at all yet. I'm kicking off a six-part series to set some foundational understanding of what AS3 is, how it works, what tools you might use to interact with it, and we'll finish it off with some best practices. If you want to learn or if you want to share your own experience with the class, join me over the next three weeks to finish 2023 in style! Episode Three Installing VSCode Install the F5 extension for VSCode Install the ACC extension for VSCode Installing AS3 Adding and connecting to a BIG-IP Use config explorer to explore the existing configuration Select a few apps and review the diagnostics Use ACC to convert a few different types of applications Deploy these appsAS3 Foundations: Creating New Apps and Using Shared Objects
I joined in on the fun at several stops on the F5 Academy BIG-IP Next roadshow tour this fall, and in talking to customers, everyone is at various stages of their F5 automation journey, and some aren't automating at all yet. I'm kicking off a six-part series to set some foundational understanding of what AS3 is, how it works, what tools you might use to interact with it, and we'll finish it off with some best practices. If you want to learn or if you want to share your own experience with the class, join me over the next three weeks to finish 2023 in style! Episode Four Demonstrate snippet templates and grab examples from CloudDocs Demonstrate iRule/Cert strings to base 64 encoding Demonstrate cert management options Demonstrate use cases for shared objects, and make clear any gotchas with such approachesAS3 Foundations: Best Practices
I joined in on the fun at several stops on the F5 Academy BIG-IP Next roadshow tour this fall, and in talking to customers, everyone is at various stages of their F5 automation journey, and some aren't automating at all yet. I'm kicking off a six-part series to set some foundational understanding of what AS3 is, how it works, what tools you might use to interact with it, and we'll finish it off with some best practices. If you want to learn or if you want to share your own experience with the class, join me over the next three weeks to finish 2023 in style! Episode Five Tips for System Tweaking Tips for Structuring Declarations FAST - Well, hello there!Advanced WAF v16.0 - Declarative API
Since v15.1 (in draft), F5® BIG-IP® Advanced WAF™ can import Declarative WAF policy in JSON format. The F5® BIG-IP® Advanced Web Application Firewall (Advanced WAF) security policies can be deployed using the declarative JSON format, facilitating easy integration into a CI/CD pipeline. The declarative policies are extracted from a source control system, for example Git, and imported into the BIG-IP. Using the provided declarative policy templates, you can modify the necessary parameters, save the JSON file, and import the updated security policy into your BIG-IP devices. The declarative policy copies the content of the template and adds the adjustments and modifications on to it. The templates therefore allow you to concentrate only on the specific settings that need to be adapted for the specific application that the policy protects. This Declarative WAF JSON policy is similar to NGINX App Protect policy. You can find more information on the Declarative Policy here : NAP : https://docs.nginx.com/nginx-app-protect/policy/ Adv. WAF : https://techdocs.f5.com/en-us/bigip-15-1-0/big-ip-declarative-security-policy.html Audience This guide is written for IT professionals who need to automate their WAF policy and are familiar with Advanced WAF configuration. These IT professionals can fill a variety of roles: SecOps deploying and maintaining WAF policy in Advanced WAF DevOps deploying applications in modern environment and willing to integrate Advanced WAF in their CI/CD pipeline F5 partners who sell technology or create implementation documentation This article covers how to PUSH/PULL a declarative WAF policy in Advanced WAF: With Postman With AS3 Table of contents Upload Policy in BIG-IP Check the import Apply the policy OpenAPI Spec File import AS3 declaration CI/CD integration Find the Policy-ID Update an existing policy Video demonstration First of all, you need a JSON WAF policy, as below : { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false } } } 1. Upload Policy in BIG-IP There are 2 options to upload a JSON file into the BIG-IP: 1.1 Either you PUSH the file into the BIG-IP and you IMPORT IT OR 1.2 the BIG-IP PULL the file from a repository (and the IMPORT is included) <- BEST option 1.1 PUSH JSON file into the BIG-IP The call is below. As you can notice, it requires a 'Content-Range' header. And the value is 0-(filesize-1)/filesize. In the example below, the file size is 662 bytes. This is not easy to integrate in a CICD pipeline, so we created the PULL method instead of the PUSH (in v16.0) curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/file-transfer/uploads/policy-api.json' \ --header 'Content-Range: 0-661/662' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --header 'Content-Type: application/json' \ --data-binary '@/C:/Users/user/Desktop/policy-api.json' At this stage, the policy is still a file ​​​​​​​in the BIG-IP file system. We need to import it into Adv. WAF. To do so, the next call is required. This call import the file "policy-api.json" uploaded previously. An CREATE the policy /Common/policy-api-arcadia curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/javascript' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "filename":"policy-api.json", "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 1.2 PULL JSON file from a repository Here, the JSON file is hosted somewhere (in Gitlab or Github ...). And the BIG-IP will pull it. The call is below. As you can notice, the call refers to the remote repo and the body is a JSON payload. Just change the link value with your JSON policy URL. With one call, the policy is PULLED and IMPORTED. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }' A second version of this call exists, and refer to the fullPath of the policy. This will allow you to update the policy, from a second version of the JSON file, easily. One call for the creation and the update. As you can notice below, we add the "policy":"fullPath" directive. The value of the "fullPath" is the partition and the name of the policy set in the JSON policy file. This method is VERY USEFUL for CI/CD integrations. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 2. Check the IMPORT Check if the IMPORT worked. To do so, run the next call. curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ You should see a 200 OK, with the content below (truncated in this example). Please notice the "status":"COMPLETED". { "kind": "tm:asm:tasks:import-policy:import-policy-taskcollectionstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy?ver=16.0.0", "totalItems": 11, "items": [ { "isBase64": false, "executionStartTime": "2020-07-21T15:50:22Z", "status": "COMPLETED", "lastUpdateMicros": 1.595346627e+15, "getPolicyAttributesOnly": false, ... ​ From now, your policy is imported and created in the BIG-IP. You can assign it to a VS as usual (Imperative Call or AS3 Call). But in the next session, I will show you how to create a Service with AS3 including the WAF policy. 3. APPLY the policy As you may know, a WAF policy needs to be applied after each change. This is the call. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/apply-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{"policy":{"fullPath":"/Common/policy-api-arcadia"}}' 4. OpenAPI spec file IMPORT As you know, Adv. WAF supports OpenAPI spec (2.0 and 3.0). Now, with the declarative WAF, we can import the OAS file as well. The BEST solution, is to PULL the OAS file from a repo. And in most of the customer' projects, it will be the case. In the example below, the OAS file is hosted in SwaggerHub (Github for Swagger files). But the file could reside in a private Gitlab repo for instance. The URL of the project is : https://app.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 The URL of the OAS file is : https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 This swagger file (OpenAPI 3.0 Spec file) includes all the application URL and parameters. What's more, it includes the documentation (for NGINX APIm Dev Portal). Now, it is pretty easy to create a WAF JSON Policy with API Security template, referring to the OAS file. Below, you can notice the new section "open-api-files" with the link reference to SwaggerHub. And the new template POLICY_TEMPLATE_API_SECURITY. Now, when I upload / import and apply the policy, Adv. WAF will download the OAS file from SwaggerHub and create the policy based on API_Security template. { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3" } ] } } 5. AS3 declaration Now, it is time to learn how we can do all of these steps in one call with AS3 (3.18 minimum). The documentation is here : https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/declarations/application-security.html?highlight=waf_policy#virtual-service-referencing-an-external-security-policy With this AS3 declaration, we: Import the WAF policy from a external repo Import the Swagger file (if the WAF policy refers to an OAS file) from an external repo Create the service { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.2.0", "id": "Prod_API_AS3", "API-Prod": { "class": "Tenant", "defaultRouteDomain": 0, "API": { "class": "Application", "template": "generic", "VS_API": { "class": "Service_HTTPS", "remark": "Accepts HTTPS/TLS connections on port 443", "virtualAddresses": ["10.1.10.27"], "redirect80": false, "pool": "pool_NGINX_API_AS3", "policyWAF": { "use": "Arcadia_WAF_API_policy" }, "securityLogProfiles": [{ "bigip": "/Common/Log all requests" }], "profileTCP": { "egress": "wan", "ingress": { "use": "TCP_Profile" } }, "profileHTTP": { "use": "custom_http_profile" }, "serverTLS": { "bigip": "/Common/arcadia_client_ssl" } }, "Arcadia_WAF_API_policy": { "class": "WAF_Policy", "url": "http://10.1.20.4/root/as3-waf-api/-/raw/master/policy-api.json", "ignoreChanges": true }, "pool_NGINX_API_AS3": { "class": "Pool", "monitors": ["http"], "members": [{ "servicePort": 8080, "serverAddresses": ["10.1.20.9"] }] }, "custom_http_profile": { "class": "HTTP_Profile", "xForwardedFor": true }, "TCP_Profile": { "class": "TCP_Profile", "idleTimeout": 60 } } } } } 6. CI/CID integration As you can notice, it is very easy to create a service with a WAF policy pulled from an external repo. So, it is easy to integrate these calls (or the AS3 call) into a CI/CD pipeline. Below, an Ansible playbook example. This playbook run the AS3 call above. That's it :) --- ​ - hosts: bigip connection: local gather_facts: false vars: my_admin: "admin" my_password: "admin" bigip: "10.1.1.12" ​ tasks: - name: Deploy AS3 WebApp uri: url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic YWRtaW46YWRtaW4=" body: "{{ lookup('file','as3.json') }}" body_format: json validate_certs: no status_code: 200 7. FIND the Policy-ID When the policy is created, a Policy-ID is assigned. By default, this ID doesn't appear anywhere. Neither in the GUI, nor in the response after the creation. You have to calculate it or ask for it. This ID is required for several actions in a CI/CD pipeline. 7.1 Calculate the Policy-ID We created this python script to calculate the Policy-ID. It is an hash from the Policy name (including the partition). For the previous created policy named "/Common/policy-api-arcadia", the policy ID is "Ar5wrwmFRroUYsMA6DuxlQ" Paste this python code in a new waf-policy-id.py file, and run the command python waf-policy-id.py "/Common/policy-api-arcadia" Outcome will be The Policy-ID for /Common/policy-api-arcadia is: Ar5wrwmFRroUYsMA6DuxlQ #!/usr/bin/python ​ from hashlib import md5 import base64 import sys pname = sys.argv[1] print 'The Policy-ID for', sys.argv[1], 'is:', base64.b64encode(md5(pname.encode()).digest()).replace("=", "") 7.2 Retrieve the Policy-ID and fullPath with a REST API call Make this call below, and you will see in the response, all the policy creations. Find yours and collect the PolicyReference directive. The Policy-ID is in the link value "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0" You can see as well, at the end of the definition, the "fileReference" referring to the JSON file pulled by the BIG-IP. And please notice the "fullPath", required if you want to update your policy curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Range: 0-601/601' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ { "isBase64": false, "executionStartTime": "2020-07-22T11:23:42Z", "status": "COMPLETED", "lastUpdateMicros": 1.595417027e+15, "getPolicyAttributesOnly": false, "kind": "tm:asm:tasks:import-policy:import-policy-taskstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy/B45J0ySjSJ9y9fsPZ2JNvA?ver=16.0.0", "filename": "", "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "endTime": "2020-07-22T11:23:47Z", "startTime": "2020-07-22T11:23:42Z", "id": "B45J0ySjSJ9y9fsPZ2JNvA", "retainInheritanceSettings": false, "result": { "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "message": "The operation was completed successfully. The security policy name is '/Common/policy-api-arcadia'. " }, "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }, 8 UPDATE an existing policy It is pretty easy to update the WAF policy from a new JSON file version. To do so, collect from the previous call 7.2 Retrieve the Policy-ID and fullPath with a REST API call the "Policy" and "fullPath" directive. This is the path of the Policy in the BIG-IP. Then run the call below, same as 1.2 PULL JSON file from a repository, ​​​​​​​but add the Policy and fullPath directives Don't forget to APPLY this new version of the policy 3. APPLY the policy curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' TIP : this call, above, can be used in place of the FIRST call when we created the policy "1.2 PULL JSON file from a repository". But be careful, the fullPath is the name set in the JSON policy file. The 2 values need to match: "name": "policy-api-arcadia" in the JSON Policy file pulled by the BIG-IP "policy":"fullPath" in the POST call 9 Video demonstration In order to help you to understand how it looks with the BIG-IP, I created this video covering 4 topics explained in this article : The JSON WAF policy Pull the policy from a remote repository Update the WAF policy with a new version of the declarative JSON file Deploy a full service with AS3 and Declarative WAF policy At the end of this video, you will be able to adapt the REST Declarative API calls to your infrastructure, in order to deploy protected services with your CI/CD pipelines. Direct link to the video on DevCentral YouTube channel : https://youtu.be/EDvVwlwEFRw
