Forum Discussion
NimbostratusRegarding optimization of iControl REST- PYTHON code
This code calls the iControl REST API provided by BIG-IP LTM. Trying to get the list of the pools, it's current status and the pool members associated with the pool.
My code works like this, based on three calls:
Get the pool names (/mgmt/tm/ltm/pool) Based on the pool names (path), get its status (mgmt/tm/ltm/pool/~pool~name/stats) Based on the pool names (path), get its pool members (/mgmt/tm/ltm/pool/~pool~names/members) If I do only the first call, I get the output within milliseconds. After adding the second call, it becomes worse. With a third call, it takes good amount of time (more than 6 minutes) to get all the output for about 400 pools.
How can I optimize this code? Basically, I am trying to draw a Network Map. Any help is appreciated.
import requests
from requests.auth import HTTPBasicAuth
BASE_URL = "https://bigip/mgmt/tm"
username = "admin"
password = "admin"
def makeRequest(username, password, url):
response_data = requests.get(url, auth = HTTPBasicAuth(username, password), verify = False)
return response_data.json()
pool_data = makeRequest(username, password, BASE_URL + "/ltm/pool")
for pools in pool_data['items']:
print pools['name']
tildPath = pools['fullPath'].replace('/','~')
GET the Pool stats
pool_stats = makeRequest(username, password, BASE_URL + "/ltm/pool/" + tildPath + "/stats")
print pool_stats['entries']['status.availabilityState']['description']
GET the Pool Members
pool_members = makeRequest(username, password, BASE_URL + "/ltm/pool/" + tildPath + "/members")
for members in pool_members['items']:
print members['name'] +" " + members['address'] + " " + members['state']6 Replies
Hannes_Rapp_162Nacreous
Your problem is serial-execution, or in other words - lack of parallelism.
If you have Python2.5 or older, use threading.Thread: https://docs.python.org/2/library/threading.htmlthreading.Thread
If you have Python2.6 or newer, use multiprocessing https://docs.python.org/2/library/multiprocessing.html- Threading will enable you to run multiple IO-wait streams but you're still stuck to a single-core processing on the client-side. For this task, you are limited on IO-wait, and not on client-side processing power, so it should still help you a lot.
- Multiprocessing offers the same as Threading, but also allows you to take advantage of more than one CPU core on the client-side.
Beware that running too many parallel API calls can easily utilize 100% of your F5 appliance CPU. (Test in QA first, and look at performance graphs to find the sweet spot)
DevBabuCirrus
Thanks Hannes I will try that one too.
Hannes_RappYour problem is serial-execution, or in other words - lack of parallelism.
If you have Python2.5 or older, use threading.Thread: https://docs.python.org/2/library/threading.htmlthreading.Thread
If you have Python2.6 or newer, use multiprocessing https://docs.python.org/2/library/multiprocessing.html- Threading will enable you to run multiple IO-wait streams but you're still stuck to a single-core processing on the client-side. For this task, you are limited on IO-wait, and not on client-side processing power, so it should still help you a lot.
- Multiprocessing offers the same as Threading, but also allows you to take advantage of more than one CPU core on the client-side.
Beware that running too many parallel API calls can easily utilize 100% of your F5 appliance CPU. (Test in QA first, and look at performance graphs to find the sweet spot)
- DevBabuThanks Hannes I will try that one too.
laraMultiprocessing Vs Threading
The threading module uses threads, the multiprocessing module uses processes. The difference is that threads run in the same memory space, while processes have separate memory. This makes it a bit harder to share objects between processes with multiprocessing. Since threads use the same memory, precautions have to be taken or two threads will write to the same memory at the same time. This is what the global interpreter lock is for. Spawning processes is a bit slower than spawning threads. Once they are running, there is not much difference.
You can get the pool member references for each pool by adding the 'expandSubcollections=true' option to the first call (getting the pools). Then, you can skip the third one (getting the pool member references). e.g.,
GET /mgmt/tm/ltm/pool?\expandSubcollections=true
