iWorkflow 201 (episode #03) - Calling REST from scripting languages (Javascript and Python)

NOTE: there’s a video at the end if you don’t want to read this!

Thus far, we’ve had a lot of focus on POSTMAN collections. For those who haven’t been following the series, POSTMAN is a great REST client that we’ve been using to demonstrate the F5 iWorkflow REST API. Take a look at first two iWorkflow 201 articles to understand more of that:

Now you’ve had some time to familiarize yourself with REST, its time to start communicating with iWorkflow via some popular scripting languages. If you are new to REST, and even newer to scripting, then here’s a trick that will save you some headaches. Yes, I’m talking about POSTMAN again… So, in POSTMAN, once you’ve worked out the REST transaction you want to perform, you can click the “Generate Code” button near the top right of the screen. Here’s an example using the Auth Token transaction:

Step 1 - With the desired transaction open, click “Generate Code”.

Step 2 - In the window that appears, select the desired language from which you require to execute the REST transaction.

Step 3 - Copy that data and start scripting! Or hand it to someone in your organization who’s looking to self-serve their L4 - L7 service templates via REST.

NOTE: I do not own shares in POSTMAN. Its just really cool, and free.

Side note - Need to brush up on scripting?

Whether you’ve scripted before and just want a refresh, or you’re starting as a beginner and are eager to dive right in, there’s great, free resources available to you. I’ve heard good things about the on-line courses by Codecademy.com, where there are free course on Javascript and Python. Once you've completed those, take a look at the Codecademy.com REST course where you’ll learn to communicate with REST API's using real API services from Youtube, NPR and more:

Ok, lets get scripting!

Scripting Languages

While I’ve put Javascript first, and Python second, this is not an F5 prioritization – but, yes, I prefer Javascript :) For both examples we will a) request a token, b) modify the Auth Token timeout, and c) list the Tenant’s L4 - L7 Service Templates. I’ve chosen these three transactions as they provide examples of POST, PATCH, and GET.

As covered in previous articles, iWorkflow employs a Provider/Tenant model. The Provider (administrator role) configures the iWorkflow platform. The Tenant deploys services using the iWorkflow platform. All of the examples in this article will be Tenant-based functions.

Javascript

Either you were a Javascript pro at the start of the article, or you’ve just returned from a few days at Codecademy.com. Whichever of the two, we’ll now start with an Auth Token request to the iWorkflow REST interface. In this exercise I’ll be using Node v4.4.0 running on my Macbook. Node is a very popular Javascript runtime.

NOTE: in my lab I am using the default, self-signed SSL certificates that are generated during the iWorkflow install and setup. By default (and rightly so, for security reasons) Node will barf at the self-signed SSL Cert. To circumvent this security behavior I have added the following line to the top of my Javascript file: process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

Part 1 - Request an Auth Token

Below is the full script, generated by POSTMAN with the addition of the SSL cert-check bypass, that I saved as “Javascript-Request_Auth_Token.js”:

$ cat ./Javascript-Request_Auth_Token.js

process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

var http = require("https");

var options = {

"method": "POST",

"hostname": "10.128.1.130",

"port": null,

"path": "/mgmt/shared/authn/login",

"headers": {

"content-type": "application/json",

"cache-control": "no-cache"

}

};

var req = http.request(options, function (res) {

var chunks = [];

res.on("data", function (chunk) {

chunks.push(chunk);

});

res.on("end", function () {

var body = Buffer.concat(chunks);

console.log(body.toString());

});

req.write(JSON.stringify({ username: 'User1',

password: 'admin',

loginProvidername: 'tmos' }));

req.end();

Most of the contents of “var options {}” will look familiar to you already. We also have the JSON payload that will be sent as part of the POST transaction (NOTE: username, password, and loginProvidername). All of the above were used in iWorkflow 201 episodes #01 and #02.

To execute the transaction, type:

node ./Javascript-Request_Auth_Token.js

Here are the results from this transaction:

$ node ./Javascript-Request_Auth_Token.js

{"username":"User1","loginReference":{"link":"https://localhost/mgmt/cm/system/authn/providers/local/login"},"loginProviderName":"local","token":{"token":"3YRVEXFR7UXM2ZMI3MEF5URVJV","name":"3YRVEXFR7UXM2ZMI3MEF5URVJV","userName":"User1","authProviderName":"local","user":{"link":"https://localhost/mgmt/shared/authz/users/User1"},"groupReferences":[{"link":"https://localhost/mgmt/shared/authn/providers/local/groups/05a36de3-4c85-4599-bfed-bfa45649df85"}],"timeout":1200,"startTime":"2016-08-10T17:38:40.556-0700","address":"10.128.1.1","partition":"[All]","generation":1,"lastUpdateMicros":1470875920556194,"expirationMicros":1470877120556000,"kind":"shared:authz:tokens:authtokenitemstate","selfLink":"https://localhost/mgmt/shared/authz/tokens/3YRVEXFR7UXM2ZMI3MEF5URVJV"},"generation":0,"lastUpdateMicros":0}

You’ll note its not presented as pretty as POSTMAN, but it is the same data!

Next action, modify the Auth Token resource.

Part 2 - Modify the Auth Token resource (timeout)

Next, we are going to change the timeout value of the Auth Token resource. While we do not require an extended timeout, it is an example of PATCH(ing) a resource. Here is the code to modify the Auth Token (but something is missing from it):

$ cat Javascript-Modify_Auth_Token_Timeout.js

process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

var http = require("https");

var options = {

"method": "PATCH",

"hostname": "10.128.1.130",

"port": null,

"path": "/mgmt/shared/authz/tokens/[Auth Token]",

"headers": {

"x-f5-auth-token": "[Auth Token]",

"content-type": "application/json",

"cache-control": "no-cache"

}

};

var req = http.request(options, function (res) {

var chunks = [];

res.on("data", function (chunk) {

chunks.push(chunk);

});

res.on("end", function () {

var body = Buffer.concat(chunks);

console.log(body.toString());

});

req.write(JSON.stringify({ timeout: '36000' }));

req.end();

For this to work we need to enter the Auth Token resource that we are modifying. You will need to replace [Auth Token] on lines 7 and 9 with the value of the Auth Token returned from our first transaction in "Part 1 - Request an Auth Token”. My Auth Token from the original transaction is “3YRVEXFR7UXM2ZMI3MEF5URVJV”. So the script will now look like the following (note lines 7 and 9):

$ cat Javascript-Modify_Auth_Token_Timeout.js

process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

var http = require("https");

var options = {

"method": "PATCH",

"hostname": "10.128.1.130",

"port": null,

"path": "/mgmt/shared/authz/tokens/3YRVEXFR7UXM2ZMI3MEF5URVJV",

"headers": {

"x-f5-auth-token": "3YRVEXFR7UXM2ZMI3MEF5URVJV",

"content-type": "application/json",

"cache-control": "no-cache"

}

};

var req = http.request(options, function (res) {

var chunks = [];

res.on("data", function (chunk) {

chunks.push(chunk);

});

res.on("end", function () {

var body = Buffer.concat(chunks);

console.log(body.toString());

});

req.write(JSON.stringify({ timeout: '36000' }));

req.end();

Ok, now we are ready to execute. As per the second to last line in this script, the value of Timeout will be increased from the default of 1200 seconds to the new value of 36000 seconds. The result:

$ node Javascript-Modify_Auth_Token_Timeout.js

{"token":"2NTTOAJBHEUWMFTLZ3KQY5R6SP","name":"3YRVEXFR7UXM2ZMI3MEF5URVJV","userName":"User1","authProviderName":"local","user":{"link":"https://localhost/mgmt/shared/authz/users/User1"},"groupReferences":[{"link":"https://localhost/mgmt/shared/authn/providers/local/groups/05a36de3-4c85-4599-bfed-bfa45649df85"}],"timeout":36000,"startTime":"2016-08-11T08:29:32.307-0700","address":"10.128.1.1","partition":"[All]","generation":3,"lastUpdateMicros":1470929566962077,"expirationMicros":1470965372307000,"kind":"shared:authz:tokens:authtokenitemstate","selfLink":"https://localhost/mgmt/shared/authz/tokens/3YRVEXFR7UXM2ZMI3MEF5URVJV"}

In the response we now see "timeout":36000

So far we’ve performed the two more complicated transactions, a POST and a PATCH. Now we’ll end with a GET.

Part 3 - List the Tenant’s L4 - L7 Service Templates

This final transaction, lets use our Auth Token to obtain a list of the L4 - L7 Service Templates available to User1. As this is a GET transaction there is no JSON payload being sent (no req.write at the end of the script). Don’t forget to edit the script and replace [Auth Token] with the Auth Token you generate earlier for the “x-f5-auth-token” header on line 9. Failing to do this will result in a "401 - Unauthorized" response.

$ cat Javascript-List_L4-L7_Service_Templates.js

process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

var http = require("https");

var options = {

"method": "GET",

"hostname": "10.128.1.130",

"port": null,

"path": "/mgmt/cm/cloud/tenant/templates/iapp/",

"headers": {

"x-f5-auth-token": “[Auth Token]",

"cache-control": "no-cache"

}

};

var req = http.request(options, function (res) {

var chunks = [];

res.on("data", function (chunk) {

chunks.push(chunk);

});

res.on("end", function () {

var body = Buffer.concat(chunks);

console.log(body.toString());

});

req.end();

And here’s the result. I have 1 L4 - L7 Service Template named “f5.http_ServiceTypeA":

$ node Javascript-List_L4-L7_Service_Templates.js

{"items":[{"name":"f5.http_ServiceTypeA","sections":[{"description":"Virtual Server and Pools","displayName":"pool"}],"vars":[{"name":"pool__addr","isRequired":true,"defaultValue":"","providerType":"NODE","serverTier":"Servers","description":"What IP address do you want to use for the virtual server?","displayName":"addr","section":"pool","validator":"IpAddress"}],"tables":[{"name":"pool__hosts","isRequired":false,"description":"What FQDNs will clients use to access the servers?","displayName":"hosts","section":"pool","columns":[{"name":"name","isRequired":true,"defaultValue":"","description":"Host","validator":"FQDN"}]},{"name":"pool__members","serverTier":"Servers","isRequired":false,"description":"Which web servers should be included in this pool?","displayName":"members","section":"pool","columns":[{"name":"addr","isRequired":false,"defaultValue":"","providerType":"NODE","description":"Node/IP address","validator":"IpAddress"},{"name":"port","isRequired":true,"defaultValue":"8080","providerType":"PORT","description":"Port","validator":"PortNumber"}]}],"properties":[{"id":"cloudConnectorReference","isRequired":true,"value":"https://localhost/mgmt/cm/cloud/tenants/myTenant1/connectors/bea388b8-46f8-4363-9f89-d8920ea8931f"}],"generation":2,"lastUpdateMicros":1468991604443417,"kind":"cm:cloud:tenant:templates:iapp:tenantiapptemplateworkerstate","selfLink":"https://localhost/mgmt/cm/cloud/tenant/templates/iapp/f5.http_ServiceTypeA"}],"generation":0,"lastUpdateMicros":0}

Pro Tips

Tip 1 - Passing options via the command line

Programmers are allergic to inefficiencies. Some saying it causes them physical pain! Having to edit your scripts all the time to update IP Addresses, Auth Tokens, or other data, can get a little tedious. So, here is how we pass that information to the script from the command line and then place that information in the right parts of the script using variables. I'll use the “Javascript - Modify Auth Token Timeout.js” script as an example.

At the beginning of the script we’re going to add the following four lines (between process.env… and var http…):

process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

var myArgs = process.argv.slice(2);

if (myArgs.length < 3) {

console.log("Usage: Host Token Timeout")

};

var http = require("https");

Without going into laborious detail, this creates an array which stores the arguments you’ve provided. The if condition checks that you have provided three arguments. If there are less than three arguments it prints a statement saying that you must provide the Host, Auth Token, and Timeout for it to work.

Then we modify the following four lines in the script to use the arguments that we passed in:

"hostname": myArgs[0],

"path": "/mgmt/shared/authz/tokens/" +myArgs[1],

"x-f5-auth-token": myArgs[1],

req.write(JSON.stringify({ timeout: myArgs[2] }));

The number in square braces marks the position in the array (list). Yes, computers start from zero! So, now the whole script looks like this:

$ cat ./Javascript-Modify_Auth_Token_Timeout.js

#!/usr/bin/env node

process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

var myArgs = process.argv.slice(2);

if (myArgs.length < 3) {

console.log("Usage: Host Token Timeout")

};

var http = require("https");

var options = {

"method": "PATCH",

"hostname": myArgs[0],

"port": null,

"path": "/mgmt/shared/authz/tokens/" +myArgs[1],

"headers": {

"x-f5-auth-token": myArgs[1],

"content-type": "application/json",

"cache-control": "no-cache"

}

};

var req = http.request(options, function (res) {

var chunks = [];

res.on("data", function (chunk) {

chunks.push(chunk);

});

res.on("end", function () {

var body = Buffer.concat(chunks);

console.log(body.toString());

});

req.write(JSON.stringify({ timeout: myArgs[2] }));

req.end();

And is executed like this:

$ node ./Javascript-Modify_Auth_Token_Timeout.js 10.128.1.130 3YRVEXFR7UXM2ZMI3MEF5URVJV 36000

Tip 2 - Make your script executable

More on the theme of efficiency, you can eliminate the requirement to type ”node” before the script. This requires two things. 1) Tell the command shell what runtime to execute against, and 2) change the attribute of the file to executable.

Step 1: add this as the first line in the script:

#!/usr/bin/env node

Step 2: Change the attribute of the file so that it is executable (on OS X 10.11.6 - same for *nix)

$ chmod u+x Javascript-Modify_Auth_Token_Timeout.js

Now you can execute the script like this (without the preceding ’node’):

$ ./Javascript-Modify_Auth_Token_Timeout.js 10.128.1.130 H43I46IBZKEIDQIKZRREN3VITS 36000

Javascript Summary

We’ve review the three most common types of transaction you will perform against the iWorkflow REST API: GET, POST, and PATCH.

Versions of each of the scripts above, with support for command line arguments, have been posted to Github and can be found here:

Get Auth Token: https://github.com/npearce/F5_iWorkflow_REST_API_Commands/blob/master/Javascript-Request_Auth_Token.js

Modify Auth Token Timeout: https://github.com/npearce/F5_iWorkflow_REST_API_Commands/blob/master/Javascript-Modify_Auth_Token_Timeout.js

List L4 - L7 Service Templates: https://github.com/npearce/F5_iWorkflow_REST_API_Commands/blob/master/Javascript-List_L4-L7_Service_Templates.js

Python

Ok, without the Javascript examples out of the way, lets do the same in Python.

NOTE: in my lab I am using the default, self-signed SSL certificates that are generated during the iWorkflow install and setup. By default (and rightly so, for security reasons) Python will barf at the self-signed SSL Cert. To circumvent this security behavior I have added the following line to the top of my python scripts: verify=False to the request (see below in Part1)

Part 1 - Request an Auth Token

Below is the full script, generated by POSTMAN with the addition of the SSL cert-check bypass, that I saved as “Javascript - Request Auth Token.py”:

$ cat Python-Request_Auth_Token.py

import requests

url = "https://10.128.1.130/mgmt/shared/authn/login"

payload = "{\n \"username\": \"User1\",\n \"password\": \"admin\",\n \"loginProvidername\": \"tmos\"\n}"

headers = {

'authorization': "Basic YWRtaW46YWRtaW4=",

'content-type': "application/json",

'cache-control': "no-cache"

}

response = requests.request("POST", url, data=payload, headers=headers, verify=False)

print(response.text)

NOTE: You’ll get a warning on the response about the ‘insecure mode’ due to the ‘verify=False’ option but it will work all the same. As an alternative, you could avoid using self-signed SSL certs.

This is what the response will look like:

$ python Python-Request_Auth_Token.py

/Library/Python/2.7/site-packages/requests-2.7.0-py2.7.egg/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html

InsecureRequestWarning)

{"username":"User1","loginReference":{"link":"https://localhost/mgmt/cm/system/authn/providers/local/login"},"loginProviderName":"local","token":{"token":"JX74CUNSBUHJX7JKGM7RYB64PV","name":"JX74CUNSBUHJX7JKGM7RYB64PV","userName":"User1","authProviderName":"local","user":{"link":"https://localhost/mgmt/shared/authz/users/User1"},"groupReferences":[{"link":"https://localhost/mgmt/shared/authn/providers/local/groups/05a36de3-4c85-4599-bfed-bfa45649df85"}],"timeout":1200,"startTime":"2016-08-11T17:08:12.662-0700","address":"10.128.1.1","partition":"[All]","generation":1,"lastUpdateMicros":1470960492662099,"expirationMicros":1470961692662000,"kind":"shared:authz:tokens:authtokenitemstate","selfLink":"https://localhost/mgmt/shared/authz/tokens/JX74CUNSBUHJX7JKGM7RYB64PV"},"generation":0,"lastUpdateMicros":0}

Ignoring the InsecureRequestWarning, you'll notice some familiar JSON. Specifically, the Auth Token: 5MN24CKDPCR7POARJ3FWTIALF5

Next action, modify the Auth Token resource.

Part 2 - Modify the Auth Token resource (timeout)

Next, we are going to change the 'timeout' value of the Auth Token resource. While we do not require an extended timeout, it is an example of PATCH(ing) a resource. Here is the code to modify the Auth Token (but something is missing from it):

$ cat Python-Modify_Auth_Token_Timeout.py

import requests

url = "https://10.128.1.130/mgmt/shared/authz/tokens/[Auth Token]"

payload = "{\n \"timeout\":\"36000\"\n}"

headers = {

'x-f5-auth-token': "[Auth Token]",

'content-type': "application/json",

'cache-control': "no-cache"

}

response = requests.request("PATCH", url, data=payload, headers=headers, verify=False)

print(response.text)

Replace [Auth Token] on lines 3 and 7 with the Auth Token we received in the previous transaction. In this case: 5MN24CKDPCR7POARJ3FWTIALF5

Execute with $ python Python-Modify_Auth_Token_Timeout.py

You’ll now see the Timeout value has been increased to 36000 seconds.

$ python Python-Modify_Auth_Token_Timeout.py

InsecureRequestWarning)

{"token":"5MN24CKDPCR7POARJ3FWTIALF5","name":"5MN24CKDPCR7POARJ3FWTIALF5","userName":"User1","authProviderName":"local","user":{"link":"https://localhost/mgmt/shared/authz/users/User1"},"groupReferences":[{"link":"https://localhost/mgmt/shared/authn/providers/local/groups/05a36de3-4c85-4599-bfed-bfa45649df85"}],"timeout":36000,"startTime":"2016-08-11T17:25:11.383-0700","address":"10.128.1.1","partition":"[All]","generation":2,"lastUpdateMicros":1470961824663992,"expirationMicros":1470997511383000,"kind":"shared:authz:tokens:authtokenitemstate","selfLink":"https://localhost/mgmt/shared/authz/tokens/5MN24CKDPCR7POARJ3FWTIALF5"}

Part 3 - List the Tenant’s L4 - L7 Service Templates

In this final transaction of the exercise lets use our Auth Token to obtain a list of the L4 - L7 Service Templates available to User1. As this is a GET transaction there is no JSON payload being sent. Just a GET with the x-f5-auth-token header. Don’t forget to edit the script and replace [Auth Token] with the Auth Token you generated earlier for the “x-f5-auth-token” header on line 9. Failing to do this will result in a "401 - Unauthorized” response.

This is the script:

$ cat Python-List_L4-L7_Service_Templates.py

import requests

url = "https://10.128.1.130/mgmt/cm/cloud/tenant/templates/iapp/"

headers = {

'x-f5-auth-token': "5MN24CKDPCR7POARJ3FWTIALF5",

'cache-control': "no-cache"

}

response = requests.request("GET", url, headers=headers, verify=False)

print(response.text)

This is the response:

$ python Python-List_L4-L7_Service_Templates.py

InsecureRequestWarning)

Again, we see a single L4 - L7 Service Template named f5.http_ServiceTypeA.

Pro Tips

Tip 1 - Passing options via the command line

As stated in the Javascript section, programmers are allergic to inefficiencies. Some saying it causes them physical pain! Having to edit your scripts all the time to update IP Addresses, Auth Tokens, or other data, can get a little tedious. So, here is how we pass that information to the script from the command line and then place that information in the right parts of the script using variables. I'll use the “Python - Modify Auth Token Timeout.py” script as an example.

Changes made to the script:

Add: import sys

To check if all the options are provided (Host, Auth Token, and Timeout) Add:

if len(sys.argv) < 4:

print "Usage: Host Token Timeout"

Substitute the Host Address with " + sys.argv[1] +", the Auth Token with " + sys.argv[2], and the Timeout with " + sys.argv[3] + “. Example below:

$ cat Python-Modify_Auth_Token_Timeout.py

import sys

import requests

if len(sys.argv) < 4:

print "Usage: Host Token Timeout"

sys.exit()

url = "https://" + sys.argv[1] + "/mgmt/shared/authz/tokens/" + sys.argv[2]

payload = "{\n \"timeout\":\"" + sys.argv[3] + "\"\n}"

headers = {

'x-f5-auth-token': sys.argv[2],

'content-type': "application/json",

'cache-control': "no-cache"

}

response = requests.request("PATCH", url, data=payload, headers=headers, verify=False)

print(response.text)

Now you can execute the following:

$ python Python-Modify_Auth_Token_Timeout.py 10.128.1.130 5MN24CKDPCR7POARJ3FWTIALF5 34000

This allows you to change the Host, Token and Timeout as required and can be used for many other operations beyond Auth Token Timeout.

Tip 2 - Make your script executable

More on the theme of efficiency, you can eliminate the requirement to type ‘python' before the script. This requires two things. 1) Tell the command shell what runtime to execute against, and 2) change the attribute of the file to executable.

Step 1: add this as the first line in the script:

#!/usr/bin/env python

Step 2: Change the attribute of the file so that it is executable (on OS X 10.11.6 - same for *nix)

$ chmod u+x Python-Modify_Auth_Token_Timeout.py

Now you can execute the script like this (without the preceding ’node’):

$ ./Python-Modify_Auth_Token_Timeout.py 10.128.1.130 5MN24CKDPCR7POARJ3FWTIALF5 36000

Neat, right?!

Python Summary

We’ve reviewed the three most common types of transaction you will perform against the iWorkflow REST API: GET, POST, and PATCH.

Versions of each of the scripts above, with support for command line arguments, have been posted to Github and can be found here:

Get Auth Token: https://github.com/npearce/F5_iWorkflow_REST_API_Commands/blob/master/Python-Request_Auth_Token.py

Modify Auth Token Timeout: https://github.com/npearce/F5_iWorkflow_REST_API_Commands/blob/master/Python-Modify_Auth_Token_Timeout.py

List L4 - L7 Service Templates: https://github.com/npearce/F5_iWorkflow_REST_API_Commands/blob/master/Python-List_L4-L7_Service_Templates.py

Conclusion

In this article we’ve extended upon what we learned using POSTMAN collections in iWorkflow 201 episodes #01 and #02, and are now performing some basic actions from the popular scripting languages, Javascript and Python. Hopefully this has explained how you can start automating basic tasks, which will free you up to do more! These introduction steps are the first phase of automation that can then be leveraged with 3rd party management and orchestration systems.

Here’s a video of the exercises above: