Forum Discussion

ives_39316's avatar
ives_39316
Icon for Nimbostratus rankNimbostratus
Jan 18, 2013

python and iControl to verify load balancer configurations

 

Title...

 

Using python and iControl API to automatically verify F5 LTM LB VIP configurations....

 

tags: python iControl pyControl suds verify verifying validating configuration settings F5 LTM load balancer VIP automation provisioning

 

 

Problem...

 

If you're in a large company, you might have a shared services model for IT services like networking. You work with several centralized IT teams to implement firewall ACLs, Load Balancer VIPs, storage, etc.. Unless you have a solid provisioning process, and request forms that translate into implementation scripts, then you've likely run into issues resulting from human error. This is unavoidable if new requests are created manually through the UI. Wrong protocols, mismatching ports, incorrect pool members, wrong vlan / subnets, incorrect IPs, or incorrect partitions. The list goes on.

 

Provisioning errors introduce days to weeks of delays for new requests. They have to go through verification, troubleshooting, additional rounds of approval, scheduling for change control and QA, particularly for compliance environments. This only gets worse if there are many individuals or time zones between the person making the request and the person implementing the change. If you're migrating datacenters, and have multiple environments, this will likely leave your project managers very unhappy.

 

An automated provisioning process would certainly help, but can be expensive to build and implement. A short-term fix is to provide network engineers (those implementing the request) with a means of automatically verifying the request was implemented correctly. This will significantly reduce the turnaround time for new requests, saving time for both you and your networking team. This will also allow junior staff or interns to verify the work and free up senior staff.

 

 

Solution...

 

What's needed is a script or application that can load and verify a dataset containing new or existing F5 LTM load balancer request definitions. This script will read the dataset, store these in structured data types, and use the iControl API to verify the requests have been implemented properly. The output of the scripts will alert on misconfigured load balancer settings. This can be run by anyone on your ops or networking team, and can even be set up in Jenkins for single-click testing / reporting / alerting / history. Similarly this can be implemented as JUnit / TestNG, or puppet / chef / etc. Your network engineer or intern ops engineer can now verify implementations at the click of a button, immediately after the network engineer completes their work.

 

NOTE: Once this is in place, you will find that you're already half-way there to automatic provisioning, as the same definition files can be used to generate the implementation scripts.

 

 

To do this you'll need the following:

 

  • pyControl + suds (or java + xml libs if you go that route)
  • iControl SDK
  • a read-only user that is available to networking and operations staff

Here's a list of what our scripts do:

 

  1. read vip names to check from a file (including the LTMs they reside on)
  2. connect to each of LTMs referenced in the load balancer requests
  3. load the LTM's virtual server and pool member configuration via iControl
  4. for each virtual server and pool, verify LTM configuration matches the request and expected values
  5. output related information and alert or warn on any mismatches or missing VIPs

Here's a list of the types of things we check for:

 

... General ...

 

  • check version of iControl modules on the LTM, alert if mismatch (script might need to be updated)

... Virtual Server ...

 

  • exists on the LTM we expect it to
  • was created in the correct partition for the BU/FG/application
  • name matches the expected pattern (), where is typically fqdn or vip
  • ip address of LB VIP matches
  • virtual server port matches virtual server name (thisvip_80 is configured for port 80, thisvip_443 is 443, etc.)
  • resource pool name matches virtual server (pool name matches virtual server name, including ports)
  • state is enabled
  • correct protocol
  • http/https profile
  • connection limits
  • source NAT enabled / disabled
  • SSL offloading enabled / disabled
  • vlan matches (if used)
  • all other virtual server settings

... Pools and Pool Members ...

 

  • display pool member IPs and ports in output
  • pool member ports match virtual server / vip (80 to 80, 443 to 443)
  • pool member IPs match expected for LB VIP (from input)
  • pool options match default profile (from input)
  • health check matches virtual server type (TCP only)
  • timeouts are correct
  • minimum number of members matches
  • minimum member action matches
  • slow ramp time matches
  • members are reporting as available

... Other ...

 

  • perform forward and reverse DNS checks on virtual server VIP
  • keep script execution history / output for reporting purposes

 

 

My first draft of such a script in python served us well, if a bit rudimentary. It can run from Jenkins and a network engineer can click on the job to verify all VIPs in our environment (including the new ones they just added). If they see they made a mistake, they can just fix it and rerun the script (rinse and repeat until no more errors). If I schedule that Jenkins job to run daily, I get an alert if any of the LB VIPs or pool members are misconfigred. The same can be done for firewall requests.

 

Next step is to add LB VIP profiles and refactor the code so it can be shared.

 

 

 

  • I highly recommend moving forward with bigsuds instead instead of pyControl. Not only is it easier to work with native python objects, but it doesn't appear as there will be future support for pyControl:

     

    https://devcentral.f5.com/articles/getting-started-with-bigsuds-ndasha-new-python-library-for-icontrol

     

    "Community member L4L7, the author of the pycontrol v2 library, is no longer with F5 and just doesn’t have the cycles to maintain the library going forward. Bigsuds author Garron Moore, however, works in house and will fix bugs and enhance as time allows. Note that all iControl libraries are considered experimental and are not officially supported by F5 Networks."

     

    • retr0h_120800's avatar
      retr0h_120800
      Icon for Nimbostratus rankNimbostratus
      I'll worry about that when the time comes. I personally like pycontrol better ATM.
  • I highly recommend moving forward with bigsuds instead instead of pyControl. Not only is it easier to work with native python objects, but it doesn't appear as there will be future support for pyControl: https://devcentral.f5.com/s/articles/getting-started-with-bigsudsndasha-new-python-library-for-icontrol "Community member L4L7, the author of the pycontrol v2 library, is no longer with F5 and just doesn‚Äôt have the cycles to maintain the library going forward. Bigsuds author Garron Moore, however, works in house and will fix bugs and enhance as time allows. Note that all iControl libraries are considered experimental and are not officially supported by F5 Networks."
  • Just curious but is there a reason to use pycontrol over bigsuds? Bigsuds seems like a really simple interface to iControl.

     

    • retr0h_120800's avatar
      retr0h_120800
      Icon for Nimbostratus rankNimbostratus
      I like how pycontrol easially maps to the documented iControl API spec. As I learn more about iControl, I may find it useful to use something like Bigsuds or roll my own, but for now I don't need any extra sugar.
  • OK, Retr0h... I have the code installed and when I launch the load_balancer.py file, I get what seems to be a prevailing stack trace, around setup.py:

     

    "Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository."

     

    stemming from line 118 of load_balancer.py. Googling this error, I get all kinds of patch discussions, but no details on how to fix this issue. Any ideas?

     

    • Gregg_Lowrimore's avatar
      Gregg_Lowrimore
      Icon for Nimbostratus rankNimbostratus
      glowrimore@hostname:~/Downloads/bigpyp-master$ pip install -r requirements.txt Requirement already satisfied (use --upgrade to upgrade): d2to1>=0.2.10,<0.3 in /usr/local/lib/python2.7/dist-packages/d2to1-0.2.10-py2.7.egg (from -r requirements.txt (line 1)) Requirement already satisfied (use --upgrade to upgrade): pbr>=0.5,<0.6 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 2)) Requirement already satisfied (use --upgrade to upgrade): pycontrol in /usr/local/lib/python2.7/dist-packages/pycontrol-2.0.1a-py2.7.egg (from -r requirements.txt (line 3)) Requirement already satisfied (use --upgrade to upgrade): termcolor in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 4)) Requirement already satisfied (use --upgrade to upgrade): docopt in /usr/local/lib/python2.7/dist-packages/docopt-0.6.1-py2.7.egg (from -r requirements.txt (line 5)) Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/dist-packages/setuptools-1.0-py2.7.egg (from d2to1>=0.2.10,<0.3->-r requirements.txt (line 1)) Requirement already satisfied (use --upgrade to upgrade): pip>=1.0 in /usr/lib/python2.7/dist-packages (from pbr>=0.5,<0.6->-r requirements.txt (line 2)) Requirement already satisfied (use --upgrade to upgrade): suds>=0.3.9 in /usr/local/lib/python2.7/dist-packages/suds-0.3.9-py2.7.egg (from pycontrol->-r requirements.txt (line 3)) Cleaning up... glowrimore@hostname:~/Downloads/bigpyp-master$ cd bigpyp/ glowrimore@hostname:~/Downloads/bigpyp-master/bigpyp$ python load_balancer.py Traceback (most recent call last): File "load_balancer.py", line 119, in version = 'Load Balancer {0}'.format(version.VersionInfo('bigpyp')) File "/usr/local/lib/python2.7/dist-packages/pbr/version.py", line 38, in __str__ return self.version_string() File "/usr/local/lib/python2.7/dist-packages/pbr/version.py", line 72, in version_string for part in self.release_string().split('.'): File "/usr/local/lib/python2.7/dist-packages/pbr/version.py", line 64, in release_string self.release = self._get_version_from_pkg_resources() File "/usr/local/lib/python2.7/dist-packages/pbr/version.py", line 57, in _get_version_from_pkg_resources return packaging.get_version(self.package) File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py", line 819, in get_version raise Exception("Versioning for this project requires either an sdist" Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Same resulting error...
    • retr0h_120800's avatar
      retr0h_120800
      Icon for Nimbostratus rankNimbostratus
      Hi Gregg - Did you install all the libraries in requirements.txt? Probably want to run: pip install -r requirements.txt Also, keep in mind this makes quite a bit of my own assumptions. Was intended as examples for others, since I had to dig a lot of this up in the API specs.
  • I wrote some code not too long ago to build out our high-level bigip configurations. It is an idempotent approach to configuring an F5. There are some things I haven't figured out how to configure via iControl (e.g. set UTC system time). Also, the cert handling is a bit too opinionated. However, this should contain enough information to get you on your way.

     

    It is powered by a yaml file to handle the Virtual Server and pool members configuration.

     

    https://github.com/retr0h/bigpyp