Enumerate All Modules

Problem this snippet solves:

The iControl REST Users Guide documents a way to discover all components by using the curl command to manually scroll through all the different modules. I wanted something a bit more automated, so I came up with this short program to enumerate through all the possible modules. The program loads in all the top-level API modules into a stack, then proceeds to query the iControl REST API daemon (tmapid) for the contents of those containers. As the sub-modules are discovered, they are pushed down on to the stack to be queried as well. Since the REST API makes use of the same command processor as TMSH, you will notice that the item REST paths just about match up with TMSH. If you know the TMSH path, you can just about guess what the REST URI should look like.

Code :

#!/usr/bin/ruby
#---------------------------------------------------------------------------- 
#  iControl_REST_Enum.rb
#---------------------------------------------------------------------------- 
#
#
#  Before you can run this program, you need to enable the tmapid daemon:
#tmsh modify /sys service tmapid enable
# tmsh start /sys service tmapid
#
#  You can query the iControl REST API from the command line using curl:
#
# $ curl -s -k -u admin:admin https://10.147.29.117/mgmt/tm/sys/application | python -mjson.tool
# {
#     "apiPartition": "/Common/",
#     "currentItemCount": 4,
#     "items": [
#         {
#             "apiName": "apl-script",
#             "kind": "tm:sys:application:apl-script:apl-scriptstate",
#             "selfLink": "https://localhost/mgmt/tm/sys/application/apl-script"
#         },
#         {
#             "apiName": "custom-stat",
#             "kind": "tm:sys:application:custom-stat:custom-statstate",
#             "selfLink": "https://localhost/mgmt/tm/sys/application/custom-stat"
#         },
#         {
#             "apiName": "service",
#             "kind": "tm:sys:application:service:servicestate",
#             "selfLink": "https://localhost/mgmt/tm/sys/application/service"
#         },
#         {
#             "apiName": "template",
#             "kind": "tm:sys:application:template:templatestate",
#             "selfLink": "https://localhost/mgmt/tm/sys/application/template"
#         }
#     ],
#     "kind": "tm:sys:application:applicationstate",
#     "nextLink": null,
#     "pageIndex": 1,
#     "previousLink": null,
#     "selfLink": "https://localhost/mgmt/tm/sys/application",
#     "startIndex": 1,
#     "totalItems": 4,
#     "totalPages": 1
# }
#
#  The python json.tool on the end of the command line just formats the JSON into something
#  more readable. 
#---------------------------------------------------------------------------- 
# Software is distributed on an "AS IS" basis,
# WITHOUT WARRANTY OF ANY KIND, either express or implied. See
# the License for the specific language governing rights and limitations
# under the License.
#
# The Initial Developer of the Original Code is F5 Networks,
# Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 2013 F5 Networks,
# Inc. All Rights Reserved.
#
# Author: John D. Allen, Principal Solutions Engineer
# Email: john.allen@f5.com
#---------------------------------------------------------------------------- 
#
# Version:
#1.0INITIAL Version
#1.1Corrected to work with new icrd daemon JSON format.
#
#----------------------------------------------------------------------------

require 'rubygems'
require 'net/http'
require 'net/https'
require 'uri'
require 'json'

REST_host = "//10.147.29.121"    ##  Your BIGIP v11.4 host IP or Name here.  The '//' at the start need to be there.

def RESTcall(x)                  ##  Yes there are other ways to do this call, but I already had the tried and true code in hand.
   uri = URI.parse(x)
   http = Net::HTTP.new(uri.host, uri.port)
   req = Net::HTTP::Get.new(uri.request_uri)
   req.basic_auth "admin", "admin"                     ## You will need to change this if you have a different password.
   http.use_ssl = true
   http.verify_mode = OpenSSL::SSL::VERIFY_NONE        ## if your like me, you don't have a valid cert on your BIGIP
   return http.request(req)
end

##-----  Load the query stack -----##
stack = []
stack.push "https:#{REST_host}/mgmt/tm/sys"
stack.push "https:#{REST_host}/mgmt/tm/net"
stack.push "https:#{REST_host}/mgmt/tm/ltm"
stack.push "https:#{REST_host}/mgmt/tm/cm"
stack.push "https:#{REST_host}/mgmt/tm/cli"
stack.push "https:#{REST_host}/mgmt/tm/auth"
stack.push "https:#{REST_host}/mgmt/tm/pem"
stack.push "https:#{REST_host}/mgmt/tm/security"
stack.push "https:#{REST_host}/mgmt/tm/afm"
stack.push "https:#{REST_host}/mgmt/tm/analytics"       
stack.push "https:#{REST_host}/mgmt/tm/apm"
stack.push "https:#{REST_host}/mgmt/tm/gtm"
stack.push "https:#{REST_host}/mgmt/tm/wam"
##  If you don't have a module installed, you will get a '500' error, but the program will continue.

resp = ""

##
## Check our host to see if it is ready
##   For some reason the initial REST API calls usually timeout while the tmapid daemon works on the 
##   first API packet. Once going, it seems to be fine. This call just checks to see if it is 
##   responding back with a correct '200' status code.  If not, tmapid responds back with a '000' 
##   status code. We just loop until we get a 200, or die trying.
##
i = 0
loop do
   i += 1
   resp = RESTcall("https:#{REST_host}/mgmt/tm/ltm/available")
   break if resp.code == "200" or i > 5
   puts "--> Waking up tmapid daemon"
end
if i > 5
   puts "--->Unable to wake up tmapid daemon on BIGIP host."
   exit
end

##
## Main Loop
##
##  This program uses a simple FILO stack to enumerate through all the items exposed from the iControl REST API.
##  Some containers have large numbers of items, so if there are more than 50, they are not pushed onto the stack.
##  There are a few containers that seem to reference themselves, and thus were causing loops in the program. So
##  if the selfLink field of the item matches the selfLink field of the container, the item is ignored.  Some of
##  the containers seem to have items defined that cause errors when queried for. Not sure if that is by design,
##  or if it is a bug in the v11.4 Beta code.
##
loop do
   uri = stack.pop
   uu = URI.parse(uri)
   puts uu.path
   uri.gsub!(/\/\/localhost/, REST_host)    ## BIGIP REST interface returns 'localhost' in its URI's for some strange reason.
   i = 0
   loop do
      i += 1
      resp = RESTcall(uri)
      break if resp.code == "200" or i > 3
   end
   if i > 3
      case resp.code               ## This is the HTTP response code from the REST API call.
         when "400"
            puts "--->[400] Item(s) not currently available?"
         when "401"
         puts "--->[401] Not Authorized to Access."
         when "403"
         puts "--->[403] Verboten"
         when "404"
         puts "--->[404] Item(s) Not Found."   
         when "500"
            puts "--->[500] Not Definded to API server?"
      end
   else
      api = JSON.parse(resp.body)
      if api.has_key?('items')                     ## if container, not end parameter
         if api['totalItems'].to_i < 50            ## some containers have hundreds!!
            api['items'].each do |r|
               if r.has_key?('selfLink')
                   stack.push r['selfLink'] if api['selfLink'] != r['selfLink'].chop   ## some items reference themselves?!?!
                else
                   stack.push r['reference']['link']  ## v11.4 Build 145 has a different format from Beta I.
                end
            end
         else
            puts "---> Too many Items to store (#{api['totalItems']})"
         end
      end
   end
   break if stack.empty?
end

exit