Testing for iControl auth errors

Hi folks,

one of my clients noticed API auth errors after upgrading the systems to TMOS v15.1.5.1.
When trying to use a newly created auth token to access LTM objects via API the systems returns a 401.
After retrying with the same token the access is successful most times.
As a workaround a latency was applied in his management framework.
The issue can be observed in all possible combinations under TMOS v151.1.5.1:
- using admin or specific user
- using local and remote authentication/authorization
- using mgmt interface and inband management
Even you might run the script on the F5 itself, I would recommend running it not on the device under test.
The script can be configured to use specific accounts, out of band or inband management and for a number of test cycles.
A latency (delay) can be configured to determine the safe amount of time between token generation and token usage.
In each test cycle a new token is created and the token is deleted afterwards as the number of active tokens is limited.
The script stops after a failed second attempt with the same token or after reaching the configured number of test cycles.

# python script: pyapitest.07.py
# version: 0.7 (2022-06-22)
# author: Stephan Manthey
# purpose:
#   retrieve auth token
#   list example pool configruation with token based auth via inband management IP (self IP)
#   use configurable delay between token generation and token usage
# module requests required (installed via Python PIP):
#   su -c 'yum install python-pip'
#   su -c 'sudo pip2 install requests'
#   su -c 'sudo pip3 install requests'
# or:
#   su -c 'yum install python-requests'
#   su -c 'yum install python3-requests'
# usage:
#   python pyapitest.07.py
# notes:
#   tested with python 2.7 and python 3.6

import sys
import time
import json
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

# specify username
username = '<username>'
# specify passphrase
password = '<passphrase>'
# specify BIG-IP self IP address (inband management IP) or
# specify BIG-IP mgmt IP address (out of band management IP)
bigipdev = '<management-ip>'
# using a sample pool provided automatically by the API
# (do not change)
poolname = 'example'
# specify latency in milliseconds
# (delay between auth token generation and delay for retry)
sleep_ms = 2
# specify number of loops
loop_count = 2000

authpath = 'https://{}/mgmt/shared/authn/login'.format(bigipdev)
conthead = {'Content-Type': 'application/json'}
authdata = {'username': username, 'password': password}

session = requests.Session()

for loop in range(1,loop_count + 1):
  authtime = time.time()
  tokenrequest = session.post(url=authpath,data=json.dumps(authdata),headers=conthead,verify=False)
  # print('getting auth token: {:f}'.format(time.time() - authtime))
  if tokenrequest.status_code == 200:
    tokendata = tokenrequest.json()
    xauthhead = {'X-F5-Auth-Token': tokendata['token']['token'], 'Content-Type': 'application/json'}
    querypath = 'https://{}/mgmt/tm/ltm/pool/{}'.format(bigipdev,poolname)
    # print('sleeping: {} millisecond(s)'.format(sleep_ms))
    time.sleep(sleep_ms / 1000.0)
    poolcheck = session.get(url=querypath,headers=xauthhead,verify=False)
    # print('1st response: {:f}'.format(time.time() - authtime))
    if poolcheck.status_code == 200:
      pooldata = poolcheck.json()
      # print(pooldata)
    elif poolcheck.status_code == 401:
      print('pool list 1st auth error ({}), sleeping {} ms, retrying loop {}:'.format(poolcheck.status_code,sleep_ms,loop))
      time.sleep(sleep_ms / 1000.0)
      poolcheck = session.get(url=querypath,headers=xauthhead,verify=False)
      if poolcheck.status_code != 200:
        print('pool list 2nd auth error ({}), stopping in loop {}:'.format(poolcheck.status_code,loop))
        break
      else:
        print('pool list 2nd attempt succeeded ({}) in loop {}, continuing'.format(poolcheck.status_code,loop))
    else:
      print('pool list error ({}), stopping in loop {}'.format(poolcheck.status_code,loop))
      break
    tokendelpath = 'https://{}/mgmt/shared/authz/tokens/{}'.format(bigipdev,tokendata['token']['token'])
    tokendelete = session.delete(url=tokendelpath,headers=xauthhead,verify=False)
    # print('deleting auth token: {:f}'.format(time.time() - authtime))
    if tokendelete.status_code != 200:
      print('token delete error ({}), stopping in loop {}'.format(tokendelete.status_code,loop))
      break
  else:
    print('getting token auth error ({}), stopping in loop {}'.format(tokenrequest.status_code,loop))
    break
    # print('successful iterations: {}'.format(loop))