iControl REST Cookbook - Virtual Server (ltm virtual)
This cookbook lists selected ready-to-use iControl REST curl commands for virtual-server related resources. Each recipe consists of the curl command, it's tmsh equivalent, and sample output. In this cookbook, the following curl options are used. Option Meaning ______________________________________________________________________________________ -s Suppress progress meter. Handy when you want to pipe the output. ______________________________________________________________________________________ -k Allows "insecure" SSL connections. ______________________________________________________________________________________ -u Specify user ID and password. For the start, you should use the "admin" account that you normally use to access the Configuration Utility. When you specify the password at the same time, concatenate with ":". e.g., admin:admin. ______________________________________________________________________________________ -X <method> Specify the HTTP method. When omitted, the default is GET. In the REST framework, POST means create (tmsh create), PATCH means overwriting the existing resource with the data sent (tmsh modify), and PATCH is for merging (ditto). ______________________________________________________________________________________ -H <Header> Specify the request header. When you send (POST, PATCH, PUT) data, you need to tell the server that the data is in JSON format. i.e., -H "Content-Type: application/json. ______________________________________________________________________________________ -d 'data' The JSON data to send. Note that you need to quote the entire json blob, and each "name":"value" pairs must be quoted. When you have nested quotes, make sure you escape (\) them. Get information of the virtual <vs> tmsh list ltm <vs> curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vs> Sample Output { kind: 'tm:ltm:virtual:virtualstate', name: 'vs', fullPath: 'vs', generation: 1109, selfLink: 'https://localhost/mgmt/tm/ltm/virtual/vs?ver=12.1.0', addressStatus: 'yes', autoLasthop: 'default', cmpEnabled: 'yes', connectionLimit: 0, description: 'TestData', destination: '/Common/192.168.184.226:80', enabled: true, gtmScore: 0, ipProtocol: 'tcp', mask: '255.255.255.255', mirror: 'disabled', mobileAppTunnel: 'disabled', nat64: 'disabled', pool: '/Common/vs-pool', poolReference: { link: 'https://localhost/mgmt/tm/ltm/pool/~Common~vs-pool?ver=12.1.0' }, rateLimit: 'disabled', rateLimitDstMask: 0, rateLimitMode: 'object', rateLimitSrcMask: 0, serviceDownImmediateAction: 'none', source: '0.0.0.0/0', sourceAddressTranslation: { type: 'automap' }, sourcePort: 'preserve', synCookieStatus: 'not-activated', translateAddress: 'enabled', translatePort: 'enabled', vlansDisabled: true, vsIndex: 4, rules: [ '/Common/irule' ], rulesReference: [ { link: 'https://localhost/mgmt/tm/ltm/rule/~Common~iRuleTest?ver=12.1.0' } ], policiesReference: { link: 'https://localhost/mgmt/tm/ltm/virtual/~Common~vs/policies?ver=12.1.0', isSubcollection: true }, profilesReference: { link: 'https://localhost/mgmt/tm/ltm/virtual/~Common~vs/profiles?ver=12.1.0', isSubcollection: true } } Get only specfic field of the virtual <vs> The naming convension for the parameters is slightly different from the ones on tmsh, so look for the familiar names in the GET response above. The example below queris the Default Pool (pool). tmsh list ltm <vs> pool curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vs>?options=pool Sample Output { kind: 'tm:ltm:virtual:virtualstate', name: 'vs', fullPath: 'vs', generation: 1, selfLink: 'https://localhost/mgmt/tm/ltm/virtual/vs?options=pool&ver=12.1.1', pool: '/Common/vs-pool', poolReference: { link: 'https://localhost/mgmt/tm/ltm/pool/~Common~vs-pool?ver=12.1.1' } } Get all the information of the virtual <vs> Unlike the tmsh equivalent, iControl REST GET does not return the configuration information of the attached policies and profiles. To see them, use expandSubcollections tmsh list ltm <vs> curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vs>?expandSubcollections=true Sample Output { "addressStatus": "yes", "autoLasthop": "default", "cmpEnabled": "yes", "connectionLimit": 0, "destination": "/Common/192.168.184.240:80", "enabled": true, "fullPath": "vs", "generation": 291, "gtmScore": 0, "ipProtocol": "tcp", "kind": "tm:ltm:virtual:virtualstate", "mask": "255.255.255.255", "mirror": "disabled", "mobileAppTunnel": "disabled", "name": "vs", "nat64": "disabled", "policiesReference": { "isSubcollection": true, "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~vs/policies?ver=13.1.0" }, "pool": "/Common/CentOS-all80", "poolReference": { "link": "https://localhost/mgmt/tm/ltm/pool/~Common~CentOS-all80?ver=13.1.0" }, "profilesReference": { "isSubcollection": true, "items": [ { "context": "all", "fullPath": "/Common/http", "generation": 291, "kind": "tm:ltm:virtual:profiles:profilesstate", "name": "http", "nameReference": { "link": "https://localhost/mgmt/tm/ltm/profile/http/~Common~http?ver=13.1.0" }, "partition": "Common", "selfLink": "https://localhost/mgmt/tm/ltm/virtual/~Common~vs/profiles/~Common~http?ver=13.1.0" }, { "context": "all", "fullPath": "/Common/tcp", "generation": 287, "kind": "tm:ltm:virtual:profiles:profilesstate", "name": "tcp", "nameReference": { "link": "https://localhost/mgmt/tm/ltm/profile/tcp/~Common~tcp?ver=13.1.0" }, "partition": "Common", "selfLink": "https://localhost/mgmt/tm/ltm/virtual/~Common~vs/profiles/~Common~tcp?ver=13.1.0" } ], "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~vs/profiles?ver=13.1.0" }, "rateLimit": "disabled", "rateLimitDstMask": 0, "rateLimitMode": "object", "rateLimitSrcMask": 0, "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vs?expandSubcollections=true&ver=13.1.0", "serviceDownImmediateAction": "none", "source": "0.0.0.0/0", "sourceAddressTranslation": { "type": "automap" }, "sourcePort": "preserve", "synCookieStatus": "not-activated", "translateAddress": "enabled", "translatePort": "enabled", "vlansDisabled": true, "vsIndex": 2 } Get stats of the virtual <vs> tmsh show ltm <vs> curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vs>/stats Sample Output { kind: 'tm:ltm:virtual:virtualstats', generation: 1109, selfLink: 'https://localhost/mgmt/tm/ltm/virtual/vs/stats?ver=12.1.0', entries: { 'https://localhost/mgmt/tm/ltm/virtual/vs/~Common~vs/stats': { nestedStats: { kind: 'tm:ltm:virtual:virtualstats', selfLink: 'https://localhost/mgmt/tm/ltm/virtual/vs/~Common~vs/stats?ver=12.1.0', entries: { 'clientside.bitsIn': { value: 12880 }, 'clientside.bitsOut': { value: 34592 }, 'clientside.curConns': { value: 0 }, 'clientside.evictedConns': { value: 0 }, 'clientside.maxConns': { value: 2 }, 'clientside.pktsIn': { value: 26 }, 'clientside.pktsOut': { value: 26 }, 'clientside.slowKilled': { value: 0 }, 'clientside.totConns': { value: 6 }, cmpEnableMode: { description: 'all-cpus' }, cmpEnabled: { description: 'enabled' }, csMaxConnDur: { value: 37 }, csMeanConnDur: { value: 29 }, csMinConnDur: { value: 17 }, destination: { description: '192.168.184.226:80' }, 'ephemeral.bitsIn': { value: 0 }, 'ephemeral.bitsOut': { value: 0 }, 'ephemeral.curConns': { value: 0 }, 'ephemeral.evictedConns': { value: 0 }, 'ephemeral.maxConns': { value: 0 }, 'ephemeral.pktsIn': { value: 0 }, 'ephemeral.pktsOut': { value: 0 }, 'ephemeral.slowKilled': { value: 0 }, 'ephemeral.totConns': { value: 0 }, fiveMinAvgUsageRatio: { value: 0 }, fiveSecAvgUsageRatio: { value: 0 }, tmName: { description: '/Common/vs' }, oneMinAvgUsageRatio: { value: 0 }, 'status.availabilityState': { description: 'available' }, 'status.enabledState': { description: 'enabled' }, 'status.statusReason': { description: 'The virtual server is available' }, syncookieStatus: { description: 'not-activated' }, 'syncookie.accepts': { value: 0 }, 'syncookie.hwAccepts': { value: 0 }, 'syncookie.hwSyncookies': { value: 0 }, 'syncookie.hwsyncookieInstance': { value: 0 }, 'syncookie.rejects': { value: 0 }, 'syncookie.swsyncookieInstance': { value: 0 }, 'syncookie.syncacheCurr': { value: 0 }, 'syncookie.syncacheOver': { value: 0 }, 'syncookie.syncookies': { value: 0 }, totRequests: { value: 4 } } } } } } Change one of the configuration options of the virtual <vs> The command below changes the Description field of the virtual ("description" in tmsh and iControl REST). tmsh modify ltm virtual <vs> description "Hello World!" curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vs> \ -X PATCH -H "Content-Type: application/json" \ -d '{"description": "Hello World!"}' Sample Output { kind: 'tm:ltm:virtual:virtualstate', name: 'vs', ... description: 'Hello World!', <==== Changed. ... } Disable the virtual <vs> The command syntax is same as above: To change the state of a virtual from "enabled" to "disabled", send "disabled":true. For enabling the virtual, use "enabled":true. Note that the Boolean type true/false does not require quotations. tmsh modify ltm virtual <vs> disabled curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vs> \ -X PATCH -H "Content-Type: application/json" \ -d '{"disabled": true}' \ Sample Output { kind: 'tm:ltm:virtual:virtualstate', name: 'vs', fullPath: 'vs', ... disabled: true, <== Changed ... } Add another iRule to <vs> When the virtual has iRules already attached, you need to send the existing ones too along with the additional one. For example, to add /Common/testRule1 to the virtual with /Common/testRule1, specify both in an array (square brackets). Note that the /Common/testRule2 iRule object should be already created. tmsh modify ltm virtual <vs> rules {testRule1 testRule2} curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vs> \ -X PATCH -H "Content-Type: application/json" \ -d '{"rules": ["/Common/testRule1", "/Common/testRule2"] }' Sample Output { kind: 'tm:ltm:virtual:virtualstate', name: 'vs', fullPath: 'vs', ... rules: [ '/Common/test1', '/Common/test2' ], <== Changed rulesReference: [ { link: 'https://localhost/mgmt/tm/ltm/rule/~Common~test1?ver=12.1.1' }, { link: 'https://localhost/mgmt/tm/ltm/rule/~Common~test2?ver=12.1.1' } ], ... } Create a new virtual <vs> You can create a skeleton virtual by specifying only Destination Address and Mask. The remaining parameters such as profiles are set to default. You can later modify the parameters by PATCH-ing. tmsh create ltm virtual <vs> destination <ip:port> mask <ip> curl -sku admin:admin -X POST -H "Content-Type: application/json" \ -d '{"name": "vs", "destination":"192.168.184.230:80", "mask":"255.255.255.255"}' \ https://<host>/mgmt/tm/ltm/virtual Sample Output { kind: 'tm:ltm:virtual:virtualstate', name: 'vs', partition: 'Common', fullPath: '/Common/vs', ... destination: '/Common/192.168.184.230:80', <== Created ... mask: '255.255.255.255', <== Created ... } Create a new virtual <vs> with a lot of parameters You can specify all the essential parameters upon creation. This example creates a new virtual with pool, default persistence profile, profiles, iRule, and source address translation. The call fails if any of the parameters conflicts. For example, you cannot specify "Cookie Persistence" without specifying appropriate profiles. If you do not specify any profile, it falls back to the default fastL4 , which is not compatible with Cookie Persistence. tmsh create ltm virtual <vs> destination <ip:port> mask <ip> pool <pool> persist replace-all-with { cookie } profiles add { tcp http clientssl } rules { <rule> } source-address-translation { type automap } curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual -H "Content-Type: application/json" -X POST -d '{"name": "vs", \ "destination": "10.10.10.10:10", \ "mask": "255.255.255.255", \ "pool": "CentOS-all80", \ "persist": [ {"name": "cookie"} ], \ "profilesReference": {"items": [ {"context": "all", "name": "http"}, {"context": "all", "name": "tcp"}, {"context": "clientside", "name": "clientssl"}] }, \ "rules": [ "ShowVersion" ], \ "sourceAddressTranslation": {"type": "automap"} }' Sample Output { "addressStatus": "yes", "autoLasthop": "default", "cmpEnabled": "yes", "connectionLimit": 0, "destination": "/Common/10.10.10.10:10", "enabled": true, "fullPath": "/Common/test", "generation": 592, "gtmScore": 0, "ipProtocol": "tcp", "kind": "tm:ltm:virtual:virtualstate", "mask": "255.255.255.255", "mirror": "disabled", "mobileAppTunnel": "disabled", "name": "vs", "nat64": "disabled", "partition": "Common", "persist": [ { "name": "cookie", "nameReference": { "link": "https://localhost/mgmt/tm/ltm/persistence/cookie/~Common~cookie?ver=13.1.0" }, "partition": "Common", "tmDefault": "yes" } ], "policiesReference": { "isSubcollection": true, "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~test/policies?ver=13.1.0" }, "pool": "/Common/CentOS-all80", "poolReference": { "link": "https://localhost/mgmt/tm/ltm/pool/~Common~CentOS-all80?ver=13.1.0" }, "profilesReference": { "isSubcollection": true, "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~test/profiles?ver=13.1.0" }, "rateLimit": "disabled", "rateLimitDstMask": 0, "rateLimitMode": "object", "rateLimitSrcMask": 0, "rules": [ "/Common/ShowVersion" ], "rulesReference": [ { "link": "https://localhost/mgmt/tm/ltm/rule/~Common~ShowVersion?ver=13.1.0" } ], "selfLink": "https://localhost/mgmt/tm/ltm/virtual/~Common~test?ver=13.1.0", "serviceDownImmediateAction": "none", "source": "0.0.0.0/0", "sourceAddressTranslation": { "type": "automap" }, "sourcePort": "preserve", "synCookieStatus": "not-activated", "translateAddress": "enabled", "translatePort": "enabled", "vlansDisabled": true, "vsIndex": 52 } Delete a virtual <vs> tmsh delete ltm virtual <vs> curl -sku admin:admin https://192.168.226.55/mgmt/tm/ltm/virtual/<vs> -X DELETE Sample Output No output (just 200 OK and no response body) References curl.1 the man page curl Releases and Downloads ... including the port for Windows Jason Rahm's "Demystifying iControl REST" series(DevCentral) -- This is Part I of 7 at the time of this article. iControl REST API reference (DevCentral) iControl® REST API User Guide (DevCentral) -- Link is for 12.1. Search for the older versions.
F5 Automated Backups - The Right Way
Hi all, Often I've been scouring the devcentral fora and codeshares to find that one piece of handywork that will drastically simplify my automated backup needs on F5 devices. Based on the works of Jason Rahm in his post "Third Time's the Charm: BIG-IP Backups Simplified with iCall" on the 26th of June 2013, I went ahead and created my own iApp that pretty much provides the answers for all my backup-needs. Here's a feature list of this iApp: It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (using smbclient, with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl [clock] command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute I wasn't able to upload the iApp template to this article, so I threw it on pastebin: http://pastebin.com/YbDj3eMN Enjoy! Thomas SchockaertWorking with MasterKeys
Author : Arnaud Fauvel (Obiane – Orange Group – France) Introduction : As explained in “SOL9420: Installing a UCS file containing an encrypted passphrase”: Passphrases used for configuration items, such as monitors, profiles, and Secure Sockets Layer (SSL) keys, are stored in the configuration file in encrypted format. The BIG-IP system uses a hardware-key encrypted master key to encrypt and decrypt passphrases contained in the configuration file. These hardware-key encrypted passwords can be identified with a prefix of $M$. Prior to BIG-IP 11.5.0, only the passphrases used for SSL private keys are stored in encrypted format. In BIG-IP 11.5.0 and later, passphrases used for other configuration objects, such as monitors and profiles, are also stored in encrypted format. To complete the description, the master key unit is: - Different on each standalone device but shared within a cluster. - Different on each vCMP guest and is dissociated from vCMP host. How to modify MasterKey As explained in the SOL it’s possible to modify the master key of the device with the following command: f5mku -r There are two bad behaviors of this command: - If there are already configuration items with encrypted parameter, the bigip is unable to load the configuration. We have to remove SSL key passphrase encryption as explained in the SOL14302: Replacing a VIPRION chassis that has one or more blades installed. - On a vCMP Host or Guest after executing the command the device become unstable. F5 support provides me the following commands explained in the following “SOL13508: ConfigSync operations fail to complete and generate a validation message”: modify /sys crypto master-key prompt-for-password This command is magic: - A new masterkey is defined based on a provided password - Saving the configuration automatically re-encrypts any encrypted-SSL-key passphrases, using the new master key, prior to saving them in the configuration file. - It works on BIGIP or vCMP guest. Considering the masterkey of the vCMP Host it’s not so simple. The precedent command can be used but all vCMP Guest will be unable to retrieve their master key: notice mcpd[6230]: 01071029:5: Cannot open unit key store notice mcpd[6230]: 01070406:5: Removed publication with publisher id ha_table_publish warning mcpd[6230]: 012a0004:4: halStorageRead: unable to read storage on this platform The masterkey of the vCMP host seems to be used for a unit key store shared with all vCMP Guest. You will find bellow a scheme which tries to represent the master key architecture: How to restore archive configuration without removing SSL key passphrase encryption The “SOL9420: Installing a UCS file containing an encrypted passphrase” is not really satisfactory because as explained before the f5mku -r commands doesn’t work with vCMP guest. But by using the magic commands it’s works very well J. 1. After installing a BIGIP or vCMP Guest, log in on to de device and force the master key with a password by typing the following command: # tmsh # modify /sys crypto master-key prompt-for-password enter password: password again: # save /sys config Saving running configuration... /config/bigip.conf /config/bigip_base.conf /config/bigip_user.conf 2. Save regulary the configuration (using iApp or remote expect script): save /sys ucs passphrase 3. Log in to the RMA BIG-IP system command line. 4. Install the master key with the password you enter in step 1 to the RMA BIG-IP system using the following command syntax: # tmsh # modify /sys crypto master-key prompt-for-password enter password: password again: # save /sys config Saving running configuration... /config/bigip.conf /config/bigip_base.conf /config/bigip_user.conf 5.Restore the UCS file to the RMA BIG-IP system using the following command syntax: tmsh load sys ucs .ucs no-licenseiControl REST Cookbook - Virtual Server Profile (LTM Virtual Profiles)
This cookbook lists selected ready-to-use iControl REST curl commands for virtual serverprofilerelated resources (the tmsh command xxx ltm virtual <vs> profiles not xxx ltm profile ). Each recipe consists of the curl command, it's tmsh equivallent, and sample output (default hidden: toggle "Expand to see sample output" (may not work on some browsers)). Please refer toiControl REST Cookbook - Virtual Server (ltm virtual) for the list of curl options. Get the profiles of the virtual <vss> list ltm virtual <vss> profiles curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vss>/profiles Expand to see sample output (may not work on some browsers) { "items": [ { "context": "all", "fullPath": "/Common/http", "generation": 405, "kind": "tm:ltm:virtual:profiles:profilesstate", "name": "http", "partition": "Common", "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vss/profiles/~Common~http?ver=13.1.0" }, { "context": "all", "fullPath": "/Common/tcp", "generation": 405, "kind": "tm:ltm:virtual:profiles:profilesstate", "name": "tcp", "partition": "Common", "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vss/profiles/~Common~tcp?ver=13.1.0" } ], "kind": "tm:ltm:virtual:profiles:profilescollectionstate", "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vss/profiles?ver=13.1.0" } Show only the specific profile <profile> of the virtual <vss> list ltm virtual <vss> profiles { <profile> } curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vss>/profiles/<profile> Expand to see sample output (may not work on some browsers) { "context": "all", "fullPath": "http", "generation": 454, "kind": "tm:ltm:virtual:profiles:profilesstate", "name": "http", "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vss/profiles/http?ver=13.1.0" } Add theprofile <profile> to the virtual <vss> modify ltm virtual <vss> profiles add { <profile> } curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vss>/profiles \ -X POST -H "Content-Type: application/json" \ -d '{"name":"<profile>"}' Expand to see sample output (may not work on some browsers) { "context": "clientside", "fullPath": "clientssl", "generation": 409, "kind": "tm:ltm:virtual:profiles:profilesstate", "name": "clientssl", "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vss/profiles/clientssl?ver=13.1.0" } You will get an error if the specified profile is not compatible with the existing ones. For example, you cannot add clientssl to a virtual with fastL4 (if you have created a virtual without explicitly specifying the profiles, it defaults to fastL4). Expand to see sample output (may not work on some browsers) { "code":400, "message":"01070734:3: Configuration error: Found disallowed profile on /Common/vss: Not Any Of (FastL4 Profile, FastHTTP Profile)", "errorStack":[],"apiError":3 } Replace all the existing profiles of the virtual <vss> Unlike the above, the endpoint is not .../<vss>/profiles but .../<vss>. The method is PATCH because you are overwriting the existing components of the virtual. The data is nested deeply: Each profile is represented as an object {key:value, ...}; the profiles are stored in a list []; and the profiles list is the value for the key 'items' in the 'profilesReference'. modify ltm virtual <vss> profiles replace-all-with { clientssl http } curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vss> \ -X PATCH -H "Content-Type: application/json" \ -d '{"profilesReference":{"items":[{"name":"clientssl"}, {"name":"http"}]}}' Expand to see sample output (may not work on some browsers) { "addressStatus": "yes", "autoLasthop": "default", "cmpEnabled": "yes", "connectionLimit": 0, "destination": "/Common/192.168.184.242:80", "enabled": true, "fullPath": "vss", "generation": 422, "gtmScore": 0, "ipProtocol": "tcp", "kind": "tm:ltm:virtual:virtualstate", "mask": "255.255.255.255", "mirror": "disabled", "mobileAppTunnel": "disabled", "name": "vss", "nat64": "disabled", "policiesReference": { "isSubcollection": true, "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~vss/policies?ver=13.1.0" }, "pool": "/Common/CentOS-all80", "poolReference": { "link": "https://localhost/mgmt/tm/ltm/pool/~Common~CentOS-all80?ver=13.1.0" }, "profilesReference": { "isSubcollection": true, "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~vss/profiles?ver=13.1.0" }, "rateLimit": "disabled", "rateLimitDstMask": 0, "rateLimitMode": "object", "rateLimitSrcMask": 0, "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vss?ver=13.1.0", "serviceDownImmediateAction": "none", "source": "0.0.0.0/0", "sourceAddressTranslation": { "type": "automap" }, "sourcePort": "preserve", "synCookieStatus": "not-activated", "translateAddress": "enabled", "translatePort": "enabled", "vlansDisabled": true, "vsIndex": 44 } You can add more fields to the request. e.g., context and partition. {"name":"clientssl", "context":"clientside", "partition":"Common"} Changing the protocol and profile of the virtual <vss> To change either protocol or profile, you need to change both in one shot because some profiles are not compatible with some protocols ( ip-protocol in tmsh; e.g., Any, TCP or UDP). modify ltm virtual <vss> ip-protocol udp profiles replace-all-with { fastL4 } curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vss> \ -X PATCH -H "Content-Type: application/json" \ -d '{"ipProtocol":"udp", "profilesReference":{"items":[{"name":"fastL4"}]}}' Expand to see sample output (may not work on some browsers) { "addressStatus": "yes", "autoLasthop": "default", "cmpEnabled": "yes", "connectionLimit": 0, "destination": "/Common/192.168.184.242:80", "enabled": true, "fullPath": "vss", "generation": 448, "gtmScore": 0, "ipProtocol": "udp", "kind": "tm:ltm:virtual:virtualstate", "mask": "255.255.255.255", "mirror": "disabled", "mobileAppTunnel": "disabled", "name": "vss", "nat64": "disabled", "policiesReference": { "isSubcollection": true, "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~vss/policies?ver=13.1.0" }, "pool": "/Common/CentOS-all80", "poolReference": { "link": "https://localhost/mgmt/tm/ltm/pool/~Common~CentOS-all80?ver=13.1.0" }, "profilesReference": { "isSubcollection": true, "link": "https://localhost/mgmt/tm/ltm/virtual/~Common~vss/profiles?ver=13.1.0" }, "rateLimit": "disabled", "rateLimitDstMask": 0, "rateLimitMode": "object", "rateLimitSrcMask": 0, "selfLink": "https://localhost/mgmt/tm/ltm/virtual/vss?ver=13.1.0", "serviceDownImmediateAction": "none", "source": "0.0.0.0/0", "sourceAddressTranslation": { "type": "automap" }, "sourcePort": "preserve", "synCookieStatus": "not-activated", "translateAddress": "enabled", "translatePort": "enabled", "vlansDisabled": true, "vsIndex": 44 } Deleting the profile <profile> from the virtual <vss> modify ltm virtual <vss> delete { <profile> } curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vss>/profiles/<profile> -X DELETE Expand to see sample output (may not work on some browsers) No output Remove all the profiles from the virtual <vss> There is no 'all' keyword in iControl REST. Use an empty object to indicate 'no data'. In this case, the profiles are represented as a list, so use the empty list []. modify ltm virtual <vss> delete { all } curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/<vss> \ -X PATCH -H "Content-Type: application/json" \ -d '{"profiles":[]}}' Note that the call will leave one profile because a virtual should have at least one profile.BIG-IP Interface Stats in Real Time with a TMSH Script
For the savants among us, calculating bits in and bits out over a delta from two snapshots of the interface counters is a walk in the park. For the rest of us, it's nice to have a tool to look at the current traffic load on an interface while working in the command line interface. This article will walk you through creating a TMSH script to do just that. Source Data You can get at interface data via snmp and icontrol, but is also available with the tmsh show net interface command. --------------------------------------------------------- Net::Interface Name Status Bits Bits Pkts Pkts Drops Errs Media In Out In Out --------------------------------------------------------- 1.1 up 59.4T 5.0T 6.2G 2.4G 0 0 none Yep, that's data. But when you get to terabits, the dial doesn't move quite so quickly, so taking a diff every few seconds won't amount to much. Specifying the raw option on the show net interface command helps out in that regard. (raw) ----------------------------------------------------------------------------------------- Net::Interface Name Status Bits Bits Pkts Pkts Drops Errs Media In Out In Out ----------------------------------------------------------------------------------------- 1.1 up 59485486972968 5080727699544 6291600606 2488751052 0 0 none That's better, but a little more challenging to parse than adding the field-fmt option, which puts it in a nice key value pair list. The bits-in and bits-out counters are the focus of this script. net interface 1.1 { counters.bits-in 59486479580896 counters.bits-out 5080875828888 counters.drops-all 0 counters.errors-all 0 counters.pkts-in 6291722759 counters.pkts-out 2488812198 media-active none name 1.1 status up } Now that we have key value pairs, and already separated by whitespace, this is a simple extraction once we split the entire string by newline. % split $x "\n" net\ interface\ 1.1\ \{ \ { counters.bits-in 59500356294368} \ { counters.bits-out 5082163022832} \ { counters.drops-all 0} \ { counters.errors-all 0} \ { counters.pkts-in 6293231170} \ { counters.pkts-out 2489470246} \ { media-active none} \ { name 1.1} \ { status up} \} \ {} % lindex [split $x "\n"] 1 counters.bits-in 59500356294368 % lindex [split $x "\n"] 2 counters.bits-out 5082163022832 % lindex [lindex [split $x "\n"] 1] 1 59500356294368 % lindex [lindex [split $x "\n"] 2] 1 5082163022832 Now that the data is extracted in proper form, we can move on to the script! Goals & Workflow The goals for this script are simple: take the values from counters.bits-in and counters.bits-out from a specified interface and display them at a specified refresh interval. We'll get from goals to a script by first working through some workflow: The Script Since we need to get data from the user (interface and interval specifications), let's start with the standard input. We'll use the getFeedback proc below. proc getFeedback { question } { puts -nonewline $question flush stdout return [gets stdin] } This proc pulls is then used in the initial script setup as shown next. tmsh::clear_screen if { $tmsh::argc == 1 } { set int [getFeedback "Please enter the interface number (ie, 1.1): "] } else { set int [lindex $tmsh::argv 1] } set l1 [] set l2 [] set interval [getFeedback "Please enter refresh rate for the stats (in seconds): "] set delay [expr $interval * 1000] Here we see the screen has been cleared, and then if the only argument in the script initialization is the script itself, then we ask for the interface name. Otherwise, we take the second argument value and set it as the interface name. Then, we initialize the l1 and l2 variables as lists. Finally, we ask for the desired refresh interval and set that delay for the after command use as it's argument is in milliseconds, not seconds. Next, we need to go ahead and take the data and dump it into the l1 variable we initialized: lappend l1 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 1] 1] lappend l1 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 2] 1] It looks a little scary, but this is an exact copy of the structure shown above in the Tcl shell except that we're using the TMSH command output instead of the static "x" variable we used to get the syntax necessary to extract the data. This results in l1 having a list with the bits-in and bits-out values in indexes 0 and 1 respectively. Now, the loop that allows this script to display the bit rate real time. while { true } { after $delay lappend l2 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 1] 1] lappend l2 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 2] 1] tmsh::clear_screen set statsIn [expr ([lindex $l2 0] - [lindex $l1 0]) / $interval] set statsOut [expr ([lindex $l2 1] - [lindex $l1 1]) / $interval] puts "Interface\t\tInbound (bps)\t\tOutbound (bps)" puts "$int\t\t\t$statsIn\t\t\t$statsOut" set l1 $l2 unset l2 } This loop will continue until you break it with a ctrl-c. We start the loop condition with our specified delay, then do with the l2 variable what we did with the l1 variable: take a snapshot of the bits-in and bits-out of the interface. After again clearing the screen, now we take the delta of the new snapshot and the old snapshot, and divide by the interval to get the bits transferred in and out on that interface, per second. Next, we display that to the screen with the puts command. Finally, in order to maintain the latest snapshot for the next interval, we set the l2 data to the l1 variable and unset the l2 variable. And that's it. Not that complicated, right? Going Forward This is a very simple throwaway script that needs a lot of work to have "arrived." Error checking, extensibility, etc, are missing, and are all left to the reader to develop for those purposes. This met a very specific troubleshooting need in my environment, and I would be remiss if I didn't share. I'd love to see someone take on error checking, or maybe displaying the bitrates for all interfaces if none is specified, or going a step further, summarizing all interfaces per vlan and showing vlan bitrates. Any takers? The script in its entirety is here in the TMSH codeshare.
iControl REST 101: Getting Started
So you’ve heard of this iControl REST thing, have you? You’re excited at the idea of a simple, light weight API that allows you programmatic access via tried and true tmsh commands with which you may already be familiar? Whether it was in the earlier iControl REST 101 article or elsewhere, your interest is piqued and you’re now looking to start digging in. Excellent, let’s get started. Links of Interest First you’ll want to save a couple of very handy links. These are sure to aid you in all of your future iControl REST endeavors, so take note: iControl REST On Cloud Docs– DevCentral is the place for all things programmatic and beyond the box when it comes to F5 technology. iControl REST is no different in this regard. The documentation, articles and examples on iControl REST available on DevCentral are already handy and will continue to grow as the community brings the power of this fully functional API to bear. No Death Stars were harmed in the making of that last sentence. Whether you’re looking for intro level docs or more advanced examples, this is a good place to start, so keep it close at hand. User’s Guide – If you’re looking into iControl REST programming and haven’t seen the user’s guide yet, you’re doing yourself a disservice. Chalk full of helpful examples, explanations and code snippets, the user’s guide will be your faithful companion on your iControl REST journey. Put together by the PD team responsible for bringing you iControl REST, you’ll find huge amounts of useful information here. Now you’ve got the documentation at the ready and you’re eager to get started. Excellent! The first step is to take a look at what an iControl REST call actually looks like. You can write all the code in the world in whatever language you’re most comfortable with, but you’ll need to know what calls to send and what to expect as a result before it amounts to much. Fortunately getting started with iControl REST is immensely simple. Because this is a REST based API there’s very little overhead or setup needed. All you really need to be able to do is send GET and POST requests to a web service. This can be done either directly from the command line with cURL, via a browser plug-in, a script, or anything else that allows you to build a custom body and send it via HTTP. Examples For these examples we’ll be using cURL, so let’s take a look at what that looks like. The first thing you’ll want to do, most likely, is get a listing of some different commands you can run. Of course, as I’ve said many times now iControl REST follows the tmsh structure, so if you’re familiar with that it will be nothing new. The rest of the setup, however, may be interesting. First we’ll use cURL to send a basic command to the BIG-IP asking it for a list of management resources. For this we use the –s (silent) and –k (insecure) cURL flags to strip out all status/usage info and allow for insecure SSL negotiation. We’ll also use –u to supply a username:password combo. This will get us into the test device and allow us to get an actual response. The constructed command looks like this (Keep in mind the output will vary per device based on version, configuration, etc.): curl –sk –u admin:admin https://dev.rest.box/mgmt/tm/ That isn’t hugely useful to a human, however, since it’s not overly readable. Given the formatting, however, it is pretty simple to break at each comma and insert a newline. We can do that easily from the command line to make this much more human readable. Here’s the appropriate sed string appended to the original command from above. Note that this is a bit different given that I’m running on a Mac, so I tweaked the regex replace a bit to suit my needs, though it should function fine on either Linux or OSX. curl -sk -u admin:admin https://dev.rest.box/mgmt/tm/ | sed 's/,/\'$'\n/g' What you’re seeing here is a list of available resources that you can query via iControl REST. Each of these is a part of the underlying tmsh system. This list goes on and on, the above is just a truncated snippet. If you’re looking for specific commands, check out the tmsh documentation and have at it. From this point it’s trivial to start sending commands that might get you useful information back from your device. Whatever you can think of you can send, for the most part. How about something handy like listing all virtual servers on your device and their associated info? Be warned that the below command will likely get you a huge amount of output on a device with a decent config. To list all virtuals you would do something like: curl -sk -u admin:admin https://dev.rest.box/mgmt/tm/cloud/ltm/virtual-servers | sed 's/,/\'$'\n/g' Again, this goes on and on, the above is just a snippet. Seriously, Note the familiar JSON formatting here. You have an item, identified by the address, and a list of name value pairs of info related to that item, including sub lists where necessary. This should look familiar to anyone that’s dealt in this kind of structure before. It is this simple, easy to parse formatting that is part of what makes iControl REST as widely applicable and easy to pick up or integrate as it is. There you are, a couple easy commands and you’re already connecting to your BIG-IP via iControl REST, grabbing data, and ready to start making changes and writing code. When I said iControl REST was light weight and simple, I wasn’t exaggerating. In the next installment of this series we’ll continue with the cURL usage but begin to make modifications to objects on the device.
Migrating to iSeries From Older Platforms: It's About Time
Migrating to iSeries From Older Platforms: It’s About Time Imagine this situation: It’s a great day because the new F5 gear has come in and you can take advantage of the years of software and hardware advancements that have been included in the latest iSeries models. Decommissioning old gear is a great way to save space, power, and administrative costs. Applications can run faster, smoother, more reliably. The future is bright with the flexibility of TurboFlex for use case-oriented FPGA acceleration at line rate. There’s a great opportunity to clean up old configurations, getting rid of redundant virtual servers and profiles which were necessary on older gear due to performance limitations. Overall, change can be a really good time to update troubleshooting and shake down the whole incident resolution process. There’s still a little bit of unease that comes with change. Moving applications to the new gear means migration. Migration can mean a large change window and the potential for regressions and having to back out the change. Memories of migrations past (and the accompanying all-nighters) cause shudders when scheduling a maintenance window. Copy and paste errors, incomplete certificate bundles, missing files, and incompatible interface numbering are all hallmarks of a long outage during a platform migration. What if there was a way to reduce the work and increase the accuracy of a migration? How much would you pay? What about removing any doubts that migration could be done within a maintenance window? What about now? Ladies and Gentlemen, allow me to introduce the platform-migrate flag, taking the load ucs command into new territory. Best thing about it, if you’ve got TMOS 12.1.1 or later, it’sbuilt right in! Amazing! (Not as amazing as winning the lottery, but go with it for the moment…) What is ‘platform-migrate’? What does it do? Will it make a difference? The full command we’re discussing is ‘ load ucs platform-migrate ’. Here’s a quick overview of how easy it is to use the command once you’ve uploaded the source User Configuration Set (UCS) to the new platform. This screenshot is from a real-time video capture: The configuration import took about 75 seconds (might have been faster without my slow typing). The UCS used was rather simple (only a few nodes, virtual servers, profiles), so it becomes a Your Mileage May Vary (YMMV) situation with larger configurations. However, the process is easily documented so administrators of all levels can easily follow the process and ensure it completes properly for each migration. This screenshot shows the migration of a configuration from a BIG-IP 8800 (D88) to a vCMP guest (Z101), demonstrating a leap-frog of two full generations of BIG-IP hardware in one quick keystroke. The actual platform migration takes a very short time, but the upfront preparation, as with any activity related to upgrading or migrating, is the key to making this work smoothly. There are several prerequisites to accomplish, but these are not any different than setting up a new device or vCMP guest For those familiar with some of the other flags for the load ucs command, platform-migrate is a combination of the no-license and no-platform-check along with the capability to ignore the L2 configuration stanzas which may not import properly when moving from one platform to another. A not-so-secret secret: Some of the interface numbers are different between platforms, meaning the L2 configuration, at best, needs to be renumbered, though most commonly recreated. Here’s a basic list of the things it does and does not do: Included in platform-migrate Not included in platform-migrate Functionality in ‘no-license’ and ‘no-platform-check’ (see: AskF5 K14096 and K16795) Migration of unencrypted UCS files from 10.2.4+ and encrypted UCS files with user-set passphrases from 11.0+ to 12.1.1talk-26316 Ignorance of the L2 portion of the configuration, meaning differences in interface numbering don’t block configurations from loading Automatic import and necessary conversion of configuration items such as: Virtual Server information Node information Profile information Any db variables that have been manually set Certificate bundles Monitors/EAV/Alerting configuration (alertd) Remote Authentication Credentials (LDAP, etc.) Migration of basic L2 configuration, as listed at K8240512, such as: VLANs (both host and guest) Management and Self-IP addresses Config sync trust group information Trunk group configuration Migration of module configurations other than those for LTM and GTM Migration of 10.2.4 configurations which are encrypted (because they are encrypted based on the appliance’s master key, not a user-specified passphrase)talk-26321 Reestablishing Device Trust relationships vCMP Host configuration (there’s a different method for that, contained in K13132, though it restores everything, including guests which may or may not be wanted) For platform-migrateto work well, there are a few prerequisites that must be accomplished on the destination system: 1) Prepare destination platform (provision vCMP if migrating to a vCMP guest) 2) Configure (and publish, for vCMP) VLANs 3 Ensure that Self-IP addresses and VLANs have the same name on the source and destination platforms If using vCMP, create guest and assign VLANs to guest 4) Copy UCS from source BIG-IP to destination platform 5)Run migration command on new platform 6) Make source BIG-IP stop serving traffic 7) Test virtuals and other config on destination platform If you are migrating from other platforms, such as from an older appliance to a iSeries appliance or from an older appliance to a BIG-IP Virtual Edition (VE), the initial configuration is similar to the migration to a vCMP guest, listed above. You need to set up Self-IP addresses, a Management address, and have VLANs that are similarly named for the migration to work properly. The platform-migrate option is best used with configurations that are relatively straightforward but would be time-consuming to replicate by hand. This can be a time saver with configurations that have hundreds of nodes, monitors, profiles, and VIPs. Sure, you could recreate them to some degree by using a script such as “create ltm pool members” on DevCentral, but that doesn’t really do much for specific config items for each node. Summary: platform-migrate saves time, period. A Word On Encrypted UCS Files… Since UCS files contain quite a bit of important information about a BIG-IP, such as user credentials, SSL Certificates, etc., it’s important to store them in as secure a manner as possible. UCS files can be encrypted with a passphrase which allows them to be stored on an accessible server for quick restorations or reversions to prior configurations. Because they contain all the files needed to recreate a BIG-IP, they’re very convenient. Just one small item that must be considered: UCS files from before 11.0 cannot be moved from one machine to another if they contain SSL certificates generated on a BIG-IP. Makes platform migration a bit more difficult without a portable archive. Why? Between 10.x and 11.0, a change was made as to the source of the encryption passphrase used to create SSL certificates. Before 11.0, the passphrase was set by the machine itself, using the machine-generated master key of the device. This solution was convenient in that you didn’t need to write down or remember the passphrase to any SSL certificates from those devices, since the machine itself was the passphrase, so to speak. Unfortunately, with that convenience came an inability to use the UCS on any other device. After 11.0, you could specify your own passphrase, thereby removing some of the portability issues that came with encrypting UCS files. Where does that leave the platform-migrate process? If you plan to migrate a device from 10.2.4 to 12.1.1+, you must create an unencrypted UCS archive to do so. This action is well-documented in the overview of UCS files available on AskF5.com. It also involves having the source platform up and running to do so. If you are working with 11.0+, you can specify a user-defined passphrase when you export and import the UCS as documented in the overview of the platform-migrate capability. Are There Other Options To Migrate From One Platform To Another? There most certainly are other options for migration. BIG-IP’s reputation as a “Swiss Army Knife” extends into the possible methods of moving from one platform to another. Similar to a Swiss Army Knife, each one of these methods is great for solving a specific problem or concern but may not cover off every instance or use case that may occur during a migration. Here are some of the other methods possible: Using a Single Configuration File (SCF) with Tarball of Additional Files:K13408 The SCF was originally developed as an easy way to present an overview of a configuration in one place, enabling easier auditing and change tracking using tools likediff and the like. It is entirely ASCII text and includes unencrypted hash blocks for device certificates as well as sections which have encrypted tokens, such as the password hashes for the different users on the device. As such, it’s not ideal for insecure storage, such as on unauthenticated file shares or FTP, but is great to use for simple search-and-replace operations to move configurations or configuration stanzas. With some scripting, SCF templates could be used to create new configurations almost dynamically. Pros Cons Can be used to migrate from one platform to another; has nothing that links it to a specific device aside from hostnames Configuration is laid out in a plain-text file which can be printed for offline review Templates can be created by copying and pasting or merging stanzas into new configurations via scripting or other command-line text manipulation Platform-to-Platform migration requires hand editing of the SCF and other files in the accompanying archive which opens the process to errors Does not handle differences in L2 configuration, such as a different management IP address but instead will try to overwrite the running config instead, possibly causing IP address collisions if on the same VLAN as the source device Device trust for sync groups will have to be rebuilt as with other methods Doesn’t include SSL keys, license information, external monitors Any device trust keys are included in the SCF in an unencrypted manner Cannot be secured with a passphrase for encrypted storage As is mentioned in the platform-migrate overview document, SCF files may be useful for simple configurations or where additional modules beyond LTM and DNS(ne é GTM) need to be migrated to the new platform. Loading a UCS From the Source Platform Directly on the New Platform:K13132 As mentioned above, the UCS was originally meant to be a great way to archive configurations and ensure that everything needed to restore a device is present in one place. Loading a UCS by itself onto a BIG-IP device without the platform-migrate flagdoes have the issue of interface numbering incompatibility when moving from one platform to another. Pros Cons Ensures all important files are collected in a single archive so they are not missing at startup after being loaded File list can be modified to add customized files into the UCS archive that are not part of the standard manifest Can move configuration from all modules including the supporting data stores and policies Encrypted UCS files either need to have a user-defined passphrase or no passphrase for successful migration (changes between 10.x and 11.0, as noted above) Requires hand editing of configuration files Using the “no-license” flag (formerly known as the RMA option) requires console access because the appliance or guest must be restored to factory defaults before the new UCS can be applied (K16795) Using Config Sync with an Existing Device to Copy Configuration Items There is a feature within the device service clustering and Device Group capabilities known as ConfigSync. When you’ve created a cluster of BIG-IPs, VEs, or vCMP guests to share the workload of an application, this is a great way to create high-availability across platforms of different speeds and capabilities. In the background, configuration changes to the clustered applications are replicated automatically, ensuring that all devices supporting that application are in lock-step and there won’t be any discrepancy between them as far as performance and capabilities are concerned. Synchronizing configuration objects you say? Wouldn’t that be a great way to replicate a configuration from one platform to another? Configurations are copied in the background and ensured that they are kept up to date from trusted sources– that sounds almost ideal! Well, no. It really isn’t ideal. ConfigSync has its limitations, and for good reason. If allof the items in a UCS or SCF were synchronized to another device, the Management IP and its related settings, the hostname, and other information that is best used when its unique, would be overwritten, causing address collisions and general mayhem. Device trust between the hosts needs to be established for synchronization to work. This may be a manageable situation if you’re setting up 2 discrete devices with different hostnames, etc. where there could be a period where both platforms could run simultaneously without issue. If you’re decommissioning an old device and want to keep the hostname and IP addresses on a new device, you’l have a more difficult time, since the trust certificates are X.509 certificates with the hostname in them. Aside from that mess, the list of things that, by design, don’t synchronize between devices is essentially the list of items you’d have to recreate on a new vCMP guest or other platform regardless of the method you used to migrate. Add on top of that the remaining need to import SSL certificates and other config files by hand which would be in the UCS or SCF and ConfigSync isn’t really the answer. I’m not going to have aPro and Con section here, because it’s not the right tool for platform migration. I did want to mention it in the interest of completeness and of avoiding a large mistake which could eat up a maintenance window quite quickly. There’s a link in the Resources section which has all the items that don’t get synchronized, if you’re interested. Manually Entering Configuration Pros Cons It’s a great use of a weekend to bond with your co-workers, especially one that’s sunny and warm The satisfaction of knowing you and your team typed in each and every node of the several thousand nodes and profiles that make up each of your data centers… You find yourself not wanting it to end! Seriously? You wouldn’t be looking at any automation or other ways to make things faster if this was they way you wanted things to run. I’m not judging you, but someone else may be… The Best Way To Migrate Configurations From Platform To Platform Is…? The one that ticks as many boxes as it can. There is no single right way to do it, similar to programming in Perl (cue the Programming Language Flame Wars, episode 27 ). The most important part of any migration is preparation. Make some dry runs if possible and time the dry runs of the migrations for more accurate change window estimation. Regardless of how configuration and consolidation happens, the fact is that the new iSeries provides many advancements that make getting off of older devices is more appealing than ever. That, and they’re fast. Really, really fast. Resources Replacing or migrating a BIG-IP or BIG-IQ hosted on a CSP (discusses considerations for moving a VE to a physical device including licensing) Overview of vCMP configuration considerations Device Trust Certificate overview Overview of Single Configuration Files Overview of User Configuration Sets Creating LTM Pool Members in tmsh Overview of platform-migrateiControl REST 101: What is iControl REST?
tl;dr - iControlREST provides a representation state transfer interface to BIG-IP management plane functions. iControl Background iControl is F5’s API that allows for programmatic access to pretty much every system that F5 has put out for quite some time now. This API is based on SOAP/XML. It is robust, granular, and allows a huge range of functionality. Whether you’re looking to tweak a feature buried under 4 sub menus or spin up 1,000 new pool members, iControl is up to the task. Anything you can do via the command line or GUI, you can accomplish via iControl. What’s more, you can do it programmatically so you don’t have to enter in every single command in the chain, or have an admin up at 3am for the change control window just to bleed the servers off a pool and toggle it, etc. This has proved extremely powerful and useful over the years, with many F5 users leveraging iControl to great success.There has been, however, a burning need to continue to build on this tried and true technology. Namely by adding a different, lighter weight means of accessing the BIG-IP remotely. Enter iControl REST. It'ssimpler, leaner, and easier to pick up with minimal additional programming or API specific knowledge. It’s built on, as the name suggests, REST (Representational State Transfer), which is a simple, stateless architecture delivered over HTTP(S). It is a simpler approach to passing data to and from an endpoint than the well-known SOAP architecture. Where SOAP is near bullet-proof due to its robust definitions and structure, REST is nimble and fleet of foot. Making use of JSON (JavaScript Object Notation), which is rapidly becoming the most commonly used format for transferring data pairs, allows iCR to require very little structure at all. Combine this with the fact that iControl REST is effectively an extension of tmsh, and you have an extremely low overhead solution, both in terms of operational expense, as well as time to learn. More Methods to learn? If you have any exposure to the SOAP based iControl API you’re no doubt aware of how extensive the list of available methods, or commands, is. There are thousands of possible commands, each unique to iControl SOAP. This is amazingly powerful, in that it is highly granular and allows you to perform the specific action in the precise manner that you need. That’s powerful, and there is definitely a place for that. For many tasks, however, such as something simple like creating a virtual server, it isn’t really necessary. Rather than introduce an entirely new set of methods to learn, iCR acts as effectively an interpreter layer that lives on top of the already powerful tmsh infrastructure on all F5 devices. This means that anything you can do via tmsh, you can do via iCR. And what’s more, you do it the same way. No more learning how to add a virtual via the command line and re-learning how to do it in your code. Once you’ve learned it in tmsh, you’ve learned it in iControl REST. So…it’s just remote tmsh? Well, sort of. It’s not quite as simple as that, but nearly. You still have to connect to the system, properly format your desired arguments into JSON so that the server side can understand what you’re instructing it to do, and send those commands. Other than that, though…yes, it really does act a heck of a lot like remote tmsh. This is a good thing, trust me. Not only does this make for excellent command and code re-use, which we love, but it also means a couple other important things. First, it means command and feature parity. The fact that iControl REST rides on top of tmsh means that any command, for any module that ever gets added to tmsh is immediately available to iControl REST. By collapsing the development needs from tmsh plus iControl to only tmsh, we’re not just reducing the workload on our internal devs, which means more goodness for you. We’re also eliminating any delay or feature gaps between command line and API, and that seems like a very good thing indeed. Second, and just as importantly, there is less to learn. If you already know how to manipulate your device via the command line, it’s going to be conceptually very simple to step into doing so remotely via iCR. For instance, in SOAP iControl, if you want to add a virtual server, it might look something like: 1: #!/bin/env python 2: 3: import sys 4: import pycontrol.pycontrol as pc 5: import time 6: 7: if pc.__version__ == '2.0': 8: pass 9: else: 10: print "Requires pycontrol version 2.x!" 11: sys.exit() 12: 13: if len(sys.argv) < 4: 14: print "Usage %s ip_address username password" % sys.argv[0] 15: sys.exit() 16: 17: a = sys.argv[1:] 18: 19: b = pc.BIGIP( 20: hostname = a[0], 21: username = a[1], 22: password = a[2], 23: fromurl = True, 24: wsdls = ['LocalLB.VirtualServer']) 25: 26: 27: # Setup a shortcut 28: v = b.LocalLB.VirtualServer 29: 30: # create() takes four params: 31: # definitions, a Common.VirtualServerSequence, 32: # wildmasks, a Common.IPAddressSequence, 33: # resources, a LocalLB.VirtualServer.VirtualServerResourceSequence, 34: # profiles, a LocalLB.VirtualServer.VirtualServerProfileSequenceSequence 35: 36: name = 'PC2' + str(int(time.time())) # the name of our vs.ww 37: 38: # Setup types. 39: vs_def = v.typefactory.create('Common.VirtualServerDefinition') 40: 41: vs_def.name = name 42: vs_def.address = '10.100.100.92' 43: vs_def.port = 8888 44: 45: proto = v.typefactory.create('Common.ProtocolType') 46: vs_def.protocol = proto.PROTOCOL_TCP 47: 48: vs_def_seq = v.typefactory.create('Common.VirtualServerSequence') 49: vs_def_seq.item = [vs_def] 50: 51: … 52: 53: # Resource has a 'type' attribute, which must point to a 'VirtualServerType' 54: # object, so let's create that. 55: vs_type = v.typefactory.create('LocalLB.VirtualServer.VirtualServerType') 56: resource = v.typefactory.create('LocalLB.VirtualServer.VirtualServerResource') 57: 58: resource.type = vs_type.RESOURCE_TYPE_POOL 59: resource.default_pool_name = 'dummyServer' # Change to whatever pool you want 60: 61: # A resource sequence we can add the resource to. 62: resource_seq = v.typefactory.create( 63: 'LocalLB.VirtualServer.VirtualServerResourceSequence' 64: ) 65: resource_seq.item = [resource] 66: 67: 68: context = v.typefactory.create('LocalLB.ProfileContextType') 69: prof = v.typefactory.create('LocalLB.VirtualServer.VirtualServerProfile') 70: prof.profile_context = context.PROFILE_CONTEXT_TYPE_ALL 71: prof.profile_name = 'tcp' 72: 73: prof_http = v.typefactory.create('LocalLB.VirtualServer.VirtualServerProfile') 74: prof_http.profile_name = 'http' 75: 76: prof_oneconn = v.typefactory.create('LocalLB.VirtualServer.VirtualServerProfile') 77: prof_oneconn.profile_name = 'oneconnect' 78: 79: # We need to create a 'profile sequence' to add this profile to... 80: prof_seq = v.typefactory.create( 81: 'LocalLB.VirtualServer.VirtualServerProfileSequence' 82: ) 83: 84: prof_seq.item = [prof, prof_http, prof_oneconn] 85: try: 86: v.create( 87: definitions = vs_def_seq, 88: wildmasks=['255.255.255.255'], 89: resources=resource_seq, 90: profiles=[prof_seq] 91: ) 92: except Exception, e: 93: print "Error creating virtual server %s" % name 94: print e Not horrible, but not exactly light on its feet, either. Shall we see what doing effectively the same thing in iControl REST looks like? 1: # create virtual 2: create_http_virtual(bigip, VS_NAME, VS_ADDRESS, VS_PORT, POOL_NAME) 3: print "created virtual server \"%s\" with destination %s:%s..." % (VS_NAME, VS_ADDRESS, VS_PORT) 4: 5: 6: def create_http_virtual(bigip, name, address, port, pool): 7: payload = {} 8: 9: # define test virtual 10: payload['kind'] = 'tm:ltm:virtual:virtualstate' 11: payload['name'] = name 12: payload['description'] = 'A Python REST client test virtual server' 13: payload['destination'] = '%s:%s' % (address, port) 14: payload['mask'] = '255.255.255.255' 15: payload['ipProtocol'] = 'tcp' 16: payload['sourceAddressTranslation'] = { 'type' : 'automap' } 17: payload['profiles'] = [ 18: { 'kind' : 'ltm:virtual:profile', 'name' : 'http' }, 19: { 'kind' : 'ltm:virtual:profile', 'name' : 'tcp' } 20: ] 21: payload['pool'] = pool 22: 23: bigip.post('%s/ltm/virtual' % BIGIP_URL_BASE, data=json.dumps(payload)) I don’t know about you, but that looks decidedly shorter to me. And not just shorter, but more legible. If you’ve used tmsh before you can almost certainly understand what’s going on here, and that’s the beauty of it. Where are all the commands? Remember how this runs off tmsh? That means there isn’t a huge laundry list of extra commands. There is not, and will not be a command by command iterative list of documentation on DevCentral that steps through every possible iControl REST command. That’s because you’re pretty much running tmsh commands, packaged in a JSON format and sent over HTTP. Because of this, if you’re looking for specific commands to execute in your scripts I recommend checking out the tmsh section on DevCentral. If what you’re looking for is how to format and send commands, that will be covered in the next couple installments of this series, so stay tuned! So who is this for? The entire idea is to make scripting more accessible to everyone. Whether you’re a seasoned programmer that’s been hankering for a more modern approach to API access, or you have extensive BIG-IP knowledge and have been hoping to dabble a bit in scripting, iCR is definitely for you. For the programming types: Let’s face it, REST is in. The trend for most APIs is to head towards REST as opposed to other available architectures. So much so that if you aren’t running on REST, your API probably isn’t going to get much traction amongst the coding types of today. It’s easier to use, faster, and is generally what is expected when approaching a device to automate or manage via code. For the networking types: iCR presents way less of a programming hurdle than the SOAP based architecture. There is hugely less setup to begin actually executing commands, and the commands you’re using are mostly the commands you already know, if you’re familiar with BIG-IP. You can learn the formatting requirements, how to set up and tear down connections in your code, and you’re off. It really is a short jump from command line guru to iControl REST writing wizard, and that’s very much by design. What now? Gocheck it out! Go to DevCentral, check out the wiki, the examples, and start poking around. Read through the user’s guide, and if you’ve got them, start asking questions. The community will no doubt be eager to help wherever it can. If your needs go beyond that, or if you’re experiencing difficulties, there is always product support to save the day. Also, make sure to keep a look out for more iControl REST information coming your way, and check back next week for the next installment, iControl REST 101: Getting Started – which will walk you through just what it will take to make the leap to iCR coding yourself, if you haven’t already gotten there. Again, if you’re looking for specific commands to use within your iControl REST scripts, check out the tmsh section on DevCentral.Configuring Decision Logging for the F5 BIG-IP Global Traffic Manager
I was working on a GTM solution and with my limited lab I wanted to make sure that the decisions that F5 BIG-IP Global Traffic Manager made at the wideIP and pool level were as evident in the logs as they were consistent in my test results. It turns out there are some fancy little checkboxes in the wideIP configuration that you can check to enable such logs. You might notice, however, that upon enabling these checkboxes the logs are nowhere to be found. This is because there are other necessary steps. You need to configure a few objects to get those logs flowing. Log Publisher The first object is the log publisher. For as much detail as flows in the decision logging, I’d highly recommend using an HSL profile to log to a remote server, but for the purposes of testing I used the local syslog. This can also be done with tmsh. sys log-config publisher gtm_decision_logging { destinations { local-syslog { } } } DNS Logging Profile Next, create a DNS logging profile, make sure to select the Log Publisher you created in the previous step. For testing purpose I enabled the log responses and query ID as well, but those are disabled by default. This also can be created in tmsh. ltm profile dns-logging gtm_decision_logging { enable-response-logging yes include-query-id yes log-publisher gtm_decision_logging } Custom DNS Profile Now create a custom DNS profile. The only custom properties necessary are at the bottom of the profile where you enable logging and select the logging profile. This can also be configured in tmsh. ltm profile dns gtm_decision_logging { app-service none defaults-from dns enable-logging yes log-profile gtm_decision_logging } Apply the DNS Profile Now that all the objects are created, you can reference the DNS profile in the listener. in tmsh, you can modify the listener by adding the profile or if one already exists, replacing it. modify gtm listener gtmlistener profiles replace-all-with { udp_gtm_dns gtm_decision_logging } Log Details Once you have all the objects configured and the DNS profile referenced in your listener, the logging should be hitting /var/log/ltm now. For this first query, the emea pool is selected, but there is no probe data for my primary load balancing method, and the none alternate method skips to the fallback, which uses the configured fallback IP to respond to the client. 2015-06-03 08:54:21 ltm1.dc.test qid 11139 from 192.168.102.1#64536: view none: query: my.example.com IN A + (192.168.102.5%0) 2015-06-03 08:54:21 ltm1.dc.test qid 11139 from 192.168.102.1#64536 [my.example.com A] [round robin selected pool (emea)] [pool member check succeeded (vip3:192.168.103.12) - pool member state is available (green)] [QoS skipped pool member (vip3:192.168.103.12) - path has unmeasured RTT] [pool member check succeeded (vip4:192.168.103.13) - pool member state is available (green)] [QoS skipped pool member (vip4:192.168.103.13) - path has unmeasured RTT] [failed to select pool member by preferred load balancing method] [Using none load balancing method] [failed to select pool member by alternate load balancing method] [selected configured fallback IP] 2015-06-03 08:54:21 ltm1.dc.test qid 11139 to 192.168.102.1#64536: [NOERROR qr,aa,rd] response: my.example.com. 30 IN A 192.168.103.99; In this second request, the emea pool is again selected, but now there is probe data, so the pool member is selected as appropriate. 2015-06-03 08:55:43 ltm1.dc.test qid 6201 from 192.168.102.1#61503: view none: query: my.example.com IN A + (192.168.102.5%0) 2015-06-03 08:55:43 ltm1.dc.test qid 6201 from 192.168.102.1#61503 [my.example.com A] [round robin selected pool (emea)] [pool member check succeeded (vip3:192.168.103.12) - pool member state is available (green)] [QoS selected pool member (vip3:192.168.103.12) - QoS score (2082756232) is higher] [pool member check succeeded (vip4:192.168.103.13) - pool member state is available (green)] [QoS skipped pool member (vip4:192.168.103.13) from two pool members with equal scores] [QoS selected pool member (vip3:192.168.103.12)] 2015-06-03 08:55:43 ltm1.dc.test qid 6201 to 192.168.102.1#61503: [NOERROR qr,aa,rd] response: my.example.com. 30 IN A 192.168.103.12; In this final request, the americas pool is selected, but there is no valid topology score for the pool members, so query is refused. 2015-06-03 08:55:53 ltm1.dc.test qid 23580 from 192.168.102.1#59437: view none: query: my.example.com IN A + (192.168.102.5%0) 2015-06-03 08:55:53 ltm1.dc.test qid 23580 from 192.168.102.1#59437 [my.example.com A] [round robin selected pool (americas)] [pool member check succeeded (vip1:192.168.103.10) - pool member state is available (green)] [QoS selected pool member (vip1:192.168.103.10) - QoS score (0) is higher] [pool member check succeeded (vip2:192.168.103.11) - pool member state is available (green)] [QoS skipped pool member (vip2:192.168.103.11) from two pool members with equal scores] [QoS selected pool member (vip1:192.168.103.10)] [topology load balancing method failed to select pool member (vip1:192.168.103.10) - topology score is 0] [failed to select pool member by preferred load balancing method] [selected configured option Return To DNS] 2015-06-03 08:55:53 ltm1.dc.test qid 23580 to 192.168.102.1#59437: [REFUSED qr,rd] response: empty Yeah, yeah, skip all that and give me the good stuff If you want to test it quickly, you can save the config below to a file (/var/tmp/gtmlogging.txt in this example) and then merge it in. Finally, modify the wideIP and listener and you’re good to go! ### ### configuration: /var/tmp/gtmlogging.txt ### sys log-config publisher gtm_decision_logging { destinations { local-syslog { } } } ltm profile dns-logging gtm_decision_logging { enable-response-logging yes include-query-id yes log-publisher gtm_decision_logging } ltm profile dns gtm_decision_logging { app-service none defaults-from dns enable-logging yes log-profile gtm_decision_logging } ### ### Merge Command ### tmsh load sys config merge file /var/tmp/gtmlogging.txt ### ### Modify wideIP and Listener ### tmsh modify gtm wideip my.example.com load-balancing-decision-log-verbosity { pool-member-selection pool-member-traversal pool-selection pool-traversal } tmsh modify gtm listener gtmlistener profiles replace-all-with { udp_gtm_dns gtm_decision_logging } tmsh save sys configiCall Triggers - Invalidating Cache from iRules
iCall is BIG-IP's all new (as of BIG-IP version 11.4) event-based automation system for the control plane. Previously, I wrote up the iCall system overview, as well as an article on the use of a periodic handler for automating backups. This article will feature the use of the triggered iCall handler to allow a user to submit a http request to invalidate the cache served up for an application managed by the Application Acceleration Manager. Starting at the End Before we get to the solution, I'd like to address the use case for invalidating cache. In many cases, the team responsible for an application's health is not the network services team which is the typical point of access to the BIG-IP. For large organizations with process overhead in generating tickets, invalidating cache can take time. A lot of time. So the request has come in quite frequently..."How can I invalidate cache remotely?" Or even more often, "Can I invalidate cache from an iRule?" Others have approached this via script, and it has been absolutely possible previously with iRules, albeit through very ugly and very-not-recommended ways. In the end, you just need to issue one TMSH command to invalidate the cache for a particular application: tmsh::modify wam application content-expiration-time now So how do we get signal from iRules to instruct BIG-IP to run a TMSH command? This is where iCall trigger handlers come in. Before we hope back to the beginning and discuss the iRule, the process looks like this: Back to the Beginning The iStats interface was introduced in BIG-IP version 11 as a way to make data accessible to both the control and data planes. I'll use this to pass the data to the control plane. In this case, the only data I need to pass is to set a key. To set an iStats key, you need to specify : Class Object Measure type (counter, gauge, or string) Measure name I'm not measuring anything, so I'll use a string starting with "WA policy string" and followed by the name of the policy. You can be explicit or allow the users to pass it in a query parameter as I'm doing in this iRule below: when HTTP_REQUEST { if { [HTTP::path] eq "/invalidate" } { set wa_policy [URI::query [HTTP::uri] policy] if { $wa_policy ne "" } { ISTATS::set "WA policy string $wa_policy" 1 HTTP::respond 200 content "App $wa_policy cache invalidated." } else { HTTP::respond 200 content "Please specify a policy /invalidate?policy=policy_name" } } } Setting the key this way will allow you to create as many triggers as you have policies. I'll leave it as an exercise for the reader to make that step more dynamic. Setting the Trigger With iStats-based triggers, you need linkage to bind the iStats key to an event-name, wacache in my case. You can also set thresholds and durations, but again since I am not measuring anything, that isn't necessary. sys icall istats-trigger wacache_trigger_istats { event-name wacache istats-key "WA policy string wa_policy_name" } Creating the Script The script is very simple. Clear the cache with the TMSH command, then remove the iStats key. sys icall script wacache_script { app-service none definition { tmsh::modify wam application dc.wa_hero content-expiration-time now exec istats remove "WA policy string wa_policy_name" } description none events none } Creating the Handler The handler is the glue that binds the event I created in the iStats trigger. When the handler sees an event named wacache, it'll execute the wacache_script iCall script. sys icall handler triggered wacache_trigger_handler { script wacache_script subscriptions { messages { event-name wacache } } } Notes on Testing Add this command to your arsenal - tmsh generate sys icall event <event-name> context none</event-name> where event-name in my case is wacache. This allows you to troubleshoot the handler and script without worrying about the trigger. And this one - tmsh modify sys db log.evrouted.level value Debug. Just note that the default is Notice when you're all done troubleshooting.
