iControl REST + jq Cookbook - Part 3: Advanced Topics

In Part 3, some useful filters from actual customer cases are presented: i.e., replication of an existing object and data-group modification. The article also shows easy ways to summarize the iControl REST log files. In the last section, a method to access the Github API to retrieve information about F5 Automation Toolchain.

Replicate an object with slight modifications

Sometimes you want to replicate a configuration object with a slight changes to a few properties. jq can do that for you. Just call a GET request against the source object, pipe the response to a jq filter to modify the values of properties, and POST the result back to the BIG-IP.

In the example that follows, I will create a virtual server object from the existing virtual "vs" with the name, destination and pool properties changed. Let's check the existing properties first.

curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \
  jq -r '.name, .destination, .pool'
vs
/Common/172.16.10.50:80
/Common/Pool-CentOS80

Use the = assignment operator to modify the name property value, say to "vs8080". jq outputs the entire JSON text with only that property value changed.

curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \
  jq '.name="vs8080"'
{
  "kind": "tm:ltm:virtual:virtualstate",
  "name": "vs8080",                                            # name changed.
  "fullPath": "vs",
  ...

By repeatedly piping the result to the next assignment, you can modify other properties too: i.e.,

curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \
  jq '.name="vs8080" | .destination="/Common/172.16.10.50:8080" | .pool="/Common/Pool-CentO8080"'
{
  "kind": "tm:ltm:virtual:virtualstate",
  "name": "vs8080",                                            # name changed
  ...
  "destination": "/Common/172.16.10.50:8080",                  # destination changed
  ...
  "pool": "/Common/Pool-CentO8080",                            # pool changed
  ...

Redirect the output to a file (e.g., virtual8080.json). Post it to the BIG-IP to create the new virtual server object.

curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual \
  -H "Content-type: application/json" -X POST \
  -d@virtual8080.json

The virtual8080.json file contains some references to the original configurations: For example, the selfLink property is still pointing to the virtual vs, and the poolReferences to the Pool-CentOS80. No worry. iControl REST will relink the paths to the new objects. Optionally, you can selectively remove the properties using the del function. Specify the property names you would like to get rid of in the argument. Use the comma (,) for multiple properties.

curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \
  jq '.name="vs8080" | .destination="/Common/172.16.10.50:8080" | .pool="/Common/Pool-CentO8080" |
    del(.selfLink, .poolReference, .policiesReference, .profilesReference)'

Note that you do not add \ to the end of second line. This part is not a shell.

Modify internal data-group records

One of the frequently asked questions is an iControl REST method to partially add/modify/delete data-group records. Unfortunately, there is none. You need to obtain the current record set, modify it locally, then submit a PATCH request. jq can make this process easier.

You can get the records property value from the following jq filter. As you can see, the call returns an array.

curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/TestDg | \
  jq '.records'
[
  {
    "name": "name",
    "data": "Cha Siu Bao"
  },
  {
    "name": "type",
    "data": "large"
  }
]

You can add a record object (consisting of the name and data properties) to the array using the + operator. Note that the right-hand operand must also be an array.

curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg | \
  jq '.records + [{"name": "price", "data": 7}]'
[
  {
    "name": "name",
    "data": "Cha Siu Bao"
  },
  {
    "name": "type",
    "data": "large"
  },
  {
    "name": "price",
    "data": 7
  }
]

The patch data you want to send is an object that consists of the records property with the name and value: i.e., "records":[ {...}, {...}, {...}]. You can create the object by specifying the property name, colon (:) and the value and enclosing them with the {}.

curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/TestDg | \
  jq '{"records": (.records + [{"name": "price", "data": 7}])}'
{
  "records": [
    {
      "name": "name",
      "data": "Cha Siu Bao"
    },
    {
      "name": "type",
      "data": "large"
    },
    {
      "name": "price",
      "data": 7
    }
  ]
}

Note that the parentheses around the value are required to keep the value in one blob. Now, pipe the output to a file (here tmpData) and PATCH it to the endpoint.

curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg \
  -X PATCH -H "Content-type: application/json" -d@tmpData | jq .
{
  "kind": "tm:ltm:data-group:internal:internalstate",
  "name": "TestDg",
  "fullPath": "TestDg",
  "generation": 77,
  "selfLink": "https://localhost/mgmt/tm/ltm/data-group/internal/TestDg?ver=15.1.2",
  "type": "string",
  "records": [
    {
      "name": "name",
      "data": "Cha Siu Bao"
    },
    {
      "name": "price",
      "data": "7"
    },
    {
      "name": "type",
      "data": "large"
    }
  ]
}

Use the select function to remove a record by name. The function selectively yields the objects if the statement is the argument returns true. To exclude an object, use !=. In the following example, the record with the name "price" is removed.

curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg | \
  jq '{"records": [(.records[] | select(.name != "price"))]}'
{
  "records": [
    {
      "name": "name",
      "data": "Cha Siu Bao"
    },
    {
      "name": "type",
      "data": "large"
    }
  ]
}

Now, pipe the output to a file and PATCH it.

curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg \
  -X PATCH -H "Content-type: application/json" \
  -d@tmpData | jq .
{
  "kind": "tm:ltm:data-group:internal:internalstate",
  "name": "TestDg",
  "fullPath": "TestDg",
  "generation": 79,
  "selfLink": "https://localhost/mgmt/tm/ltm/data-group/internal/TestDg?ver=15.1.2",
  "type": "string",
  "records": [
    {
      "name": "name",
      "data": "Cha Siu Bao"
    },
    {
      "name": "type",
      "data": "large"
    }
  ]
}

Similarly, you can replace a record using the if-then-else-end syntax. The if statement accepts standard conditional operators such as ==. Note that jq's if-then-else-end always requires the else block. You need to explicitly put . in the else block to output the existing value. Also, it requires end at the end of statement. The following example modifies the object's data value to "medium" if the name property value is "type":

$ curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg | \
  jq '{"records": [(.records[] | if .name == "type" then .data = "medium" else . end)]}'
{
  "records": [
    {
      "name": "name",
      "data": "Cha Siu Bao"
    },
    {
      "name": "type",
      "data": "medium"
    }
  ]
}

Process restjavad-autit.log

/var/log/restjavad-audit.*.log (where * is a number) keeps records of REST calls as shown in the example below (manually edited for readability):

[I][784][12 May 2020 05:17:34 UTC][ForwarderPassThroughWorker]
{"user":"local/admin","method":"GET","uri":"http://localhost:8100/mgmt/tm/sys/available",
"status":400,"from":"com.f5.rest.common.RestAvailabilityEntry"}

Successful GETs (200) are not recorded by default. To include these calls, change the value of theaudit.logging.FileHandler.level property in /etc/restjavad.log.conf to FINE.

The log messages are handy for gathering the iControl REST statistics: How often do the calls fail? Which endpoint is used most extensively? Such intelligence is helpful when you need to investigate performance issues or intermittent failure.

First, use grep to extract only the audit records as the file may contain other messages (such as boot markers). Then, apply cut to trim the first timestamp part. Now you have a sequence of JSON texts.

fgrep '"user":' restjavad-audit.0.log | cut -d ' ' -f6-
{"user":"local/admin","method":"GET","uri":"http://localhost:8100/mgmt/tm/sys/tmm-stat","status":400,"from":"192.168.184.10"}
{"user":"local/admin","method":"GET","uri":"http://localhost:8100/mgmt/tm/sys/disk/stat","status":400,"from":"192.168.184.10"}
{"user":"local/admin","method":"POST","uri":"http://localhost:8100/mgmt/tm/util/bash","status":200,"from":"192.168.184.10"}
...

With the help of sort and uniq, you can generate a statistical report on the methods and status:

fgrep '"user":' restjavad-audit.0.log | cut -d ' ' -f6- | \
 jq -r '[.method, (.status|tostring)] | join(" ")' | \
 sort | uniq -c
      3 DELETE 200
   2160 GET 400
     20 GET 401
     14 GET 404
      1 GET 500
      1 GET 503
     21 PATCH 200
     12 PATCH 400
      5 PATCH 403
    147 POST 200
      2 POST 202
     46 POST 400
     14 POST 401
      4 POST 403
      4 POST 404
      2 PUT 202
      7 PUT 400

The first part of the filter extracts the method and status properties. The status value is a number, hence it must be converted to string using the tostring function (needs the parentheses to group the two operations). Note that the join function only accepts an array with string elements.

Similarly, you can obtain an endpoint-based summary. In the following example, long URLs are summarized into a component level (e.g., endpoints with actual virtual server object names are reduced to just /mgmt/tm/ltm/virutal) by picking only the first four path components (mgmt, tm, ltm and virtual).

fgrep '"user":' restjavad-audit.0.log | cut -d ' ' -f6- | \
  jq -r '.uri | split("/")[3:7] | join("/")' | \
  sort | uniq -c
      1 mgmt/cm/autodeploy/software-image-downloads
     64 mgmt/shared/authn/login
      2 mgmt/shared/authz/roles
     14 mgmt/shared/authz/tokens
      9 mgmt/shared/authz/users
...

The first part of the filter extracts the .uri property. The values are paths in string. The split function in the next stage splits the string by / character and the following slice ([3:7]) extracts only the first four path elements. Why starting from 3? Because the URIs start from https://localhost:8010/... and the first meaningful path element (mgmt) starts from the 3rd (counting from 0).

Process restjavad-api-usage.log

Another source of iControl REST statistics is /var/log/restjavad-api-usage.log. The file keeps the access counts per endpoint and per method in JSON format since the initial deployment of the system.

The restjavad-api-usage feature was introduced in BIG-IP v13.1.

The property apiUsageStartTimeMicrosUtc tells you when it started to tally the accesses. The value is a Unix epoch in microseconds.

cat restjavad-api-usage.json | \
  jq -r '.apiUsageStartTimeMicrosUtc | ./(1000*1000) | todate'
2019-08-25T22:05:46Z

The usage property keeps the access counts in an array. Each array element is an object that contains the usage per endpoint. Access counts are categorized by the access sources:

• external: Accesses from outside (via management interface).
• internal: Accesses from inside the box (via localhost)
• messages: iControl REST internal message passing.

The above access source properties keep the counts for each access method in an array: i.e., GET, POST, PUT, DELETE, PATCH and OPTIONS. Unlike the audit log, it counts the accesses irrespective of success or failure.

The example below shows the 1000th element, which, in this case, contains the access counts for the /mgmt/cm endpoint.

cat restjavad-api-usage.json | jq -r '.usage[1000]'            # 1000th element
{
  "path": "/mgmt/cm",                                          # The endpoint
  "external": [                                                # Access from outside
    1,                                                         # GET
    0,                                                         # POST
    0,                                                         # PUT
    0,                                                         # DELETE
    0,                                                         # PATCH
    0                                                          # OPTIONS
  ],
  "internal": [                                                # Access from inside (localhost)
    0,
    0,
    0,
    0,
    0,
    0
  ],
  "message": [                                                 # Interthread communications
    0,
    0,
    0,
    0,
    0,
    0
  ]
}

The example below extracts the number of GET requests against /mgmt/cm from external clients:

cat restjavad-api-usage.json | jq -r '.usage[1000].external[0]'
1

jq can do more for you. The example below extracts the sum of the external access counts (for all the methods) for the endpoints that start with /mgmt/tm/sys.

cat restjavad-api-usage.json | \
  jq -r '.usage[] | select(.path | startswith("/mgmt/tm/sys")) | 
    [.path, (.external | add | tostring)] | join("\t")'
/mgmt/tm/sys    4
/mgmt/tm/sys/clock      1
/mgmt/tm/sys/cpu        1
/mgmt/tm/sys/crypto     1
/mgmt/tm/sys/crypto/cert-order-manager  2
/mgmt/tm/sys/crypto/csr 1
/mgmt/tm/sys/crypto/key 6
/mgmt/tm/sys/db 2930
/mgmt/tm/sys/disk       5
/mgmt/tm/sys/disk/application-volume    1
/mgmt/tm/sys/disk/directory     9
/mgmt/tm/sys/file/external-monitor      7
/mgmt/tm/sys/global-settings    35
/mgmt/tm/sys/ha-group   2
/mgmt/tm/sys/hardware   4
/mgmt/tm/sys/host-info  1
/mgmt/tm/sys/provision  5
/mgmt/tm/sys/ready      16
/mgmt/tm/sys/software   4
/mgmt/tm/sys/software/image     1
/mgmt/tm/sys/tmm-info   1
/mgmt/tm/sys/ucs        4
/mgmt/tm/sys/version    94

Let's explore this filter. The select function extracts the usage object with the endpoints that start with /mgmt/tm/sys. .path | startswith() in the select argument feeds the .path value to the startswith function, which returns true when the input starts with the string in the function argument. The .external part of (.external | add | tostring) filter extracts the array with 6 numbers. The add function adds them together, then the last tostring does the number to string conversion (because join requires string elements).

Get a list of the latest Automation Toolchain

F5 Automation Toolchain is a collection of remote deployment services (See F5 Automation Toolchain). The software, which you need to install on BIG-IP to utilize the service(s), is updated with enhancements and bug fixes fairly often, hence you may want to check Github from time to time to stay up to date (The Github pages are AS3, DO and TS respectively).

You can automate the checking by accessing the following URI.

• AS3 (Application Services 3): https://api.github.com/repos/F5Networks/f5-appsvcs-extension/releases/latest
• DO (Declarative Onboarding): https://api.github.com/repos/F5Networks/f5-declarative-onboarding/releases/latest
• TS (Telemetry Streaming): https://api.github.com/repos/F5Networks/f5-telemetry-streaming/releases/latest

A GET request to the URL returns a JSON formatted text that contains information about the latest software. Here is an example from AS3.

curl -sk https://api.github.com/repos/F5Networks/f5-appsvcs-extension/releases/latest | \
  jq -r '.'
{
  ...
  "node_id": "MDc6UmVsZWFzZTM0MjA3MjM4",
  "tag_name": "v3.24.0",
  "target_commitish": "master",
  "name": "v3.24.0",
  "draft": false,
  "prerelease": false,
  "created_at": "2020-11-20T02:03:49Z",
  "published_at": "2020-11-20T02:12:39Z",
  ...
}

The top-level properties show the generic information about the software package. The version number is shown in the name property and the date of publication is found in the published_at property. Here's a script to grab these information for all the services:

for service in f5-appsvcs-extension f5-declarative-onboarding f5-telemetry-streaming; do
  echo -n "$service ..."
  curl -sk https://api.github.com/repos/F5Networks/$service/releases/latest | \
    jq -r '[.name, .published_at] | join(" ")'
done
f5-appsvcs-extension ...v3.24.0 2020-11-20T02:12:39Z
f5-declarative-onboarding ...v1.17.0 2020-11-20T01:45:51Z
f5-telemetry-streaming ...v1.16.0 2020-11-20T01:22:22Z

Conclusion

iControl REST was introduced in BIG-IP version 11.4 and it is now a must-know feature for automated configurations and maintenance. It represents the information in JSON, hence jq is a natural choice for casual data processing. If you did not find the method you need in this series, please take a look at the jq manual. You should be able to find the functions you are after. Having said so, jq is not as flexible as a general-purpose programming language. If you have complex requirements, you should consider using iControl REST SDKs such as F5 Python SDK.