cancel
Showing results for 
Search instead for 
Did you mean: 

Testing for iControl auth errors

Hi folks,

one of my clients noticed API auth errors after upgrading the systems to TMOS v15.1.5.1.
When trying to use a newly created auth token to access LTM objects via API the systems returns a 401.
After retrying with the same token the access is successful most times.
As a workaround a latency was applied in his management framework.
The issue can be observed in all possible combinations under TMOS v151.1.5.1:
- using admin or specific user
- using local and remote authentication/authorization
- using mgmt interface and inband management
Even you might run the script on the F5 itself, I would recommend running it not on the device under test.
The script can be configured to use specific accounts, out of band or inband management and for a number of test cycles.
A latency (delay) can be configured to determine the safe amount of time between token generation and token usage.
In each test cycle a new token is created and the token is deleted afterwards as the number of active tokens is limited.
The script stops after a failed second attempt with the same token or after reaching the configured number of test cycles.

 

 

# python script: pyapitest.07.py
# version: 0.7 (2022-06-22)
# author: Stephan Manthey
# purpose:
#   retrieve auth token
#   list example pool configruation with token based auth via inband management IP (self IP)
#   use configurable delay between token generation and token usage
# module requests required (installed via Python PIP):
#   su -c 'yum install python-pip'
#   su -c 'sudo pip2 install requests'
#   su -c 'sudo pip3 install requests'
# or:
#   su -c 'yum install python-requests'
#   su -c 'yum install python3-requests'
# usage:
#   python pyapitest.07.py
# notes:
#   tested with python 2.7 and python 3.6

import sys
import time
import json
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

# specify username
username = '<username>'
# specify passphrase
password = '<passphrase>'
# specify BIG-IP self IP address (inband management IP) or
# specify BIG-IP mgmt IP address (out of band management IP)
bigipdev = '<management-ip>'
# using a sample pool provided automatically by the API
# (do not change)
poolname = 'example'
# specify latency in milliseconds
# (delay between auth token generation and delay for retry)
sleep_ms = 2
# specify number of loops
loop_count = 2000

authpath = 'https://{}/mgmt/shared/authn/login'.format(bigipdev)
conthead = {'Content-Type': 'application/json'}
authdata = {'username': username, 'password': password}

session = requests.Session()

for loop in range(1,loop_count + 1):
  authtime = time.time()
  tokenrequest = session.post(url=authpath,data=json.dumps(authdata),headers=conthead,verify=False)
  # print('getting auth token: {:f}'.format(time.time() - authtime))
  if tokenrequest.status_code == 200:
    tokendata = tokenrequest.json()
    xauthhead = {'X-F5-Auth-Token': tokendata['token']['token'], 'Content-Type': 'application/json'}
    querypath = 'https://{}/mgmt/tm/ltm/pool/{}'.format(bigipdev,poolname)
    # print('sleeping: {} millisecond(s)'.format(sleep_ms))
    time.sleep(sleep_ms / 1000.0)
    poolcheck = session.get(url=querypath,headers=xauthhead,verify=False)
    # print('1st response: {:f}'.format(time.time() - authtime))
    if poolcheck.status_code == 200:
      pooldata = poolcheck.json()
      # print(pooldata)
    elif poolcheck.status_code == 401:
      print('pool list 1st auth error ({}), sleeping {} ms, retrying loop {}:'.format(poolcheck.status_code,sleep_ms,loop))
      time.sleep(sleep_ms / 1000.0)
      poolcheck = session.get(url=querypath,headers=xauthhead,verify=False)
      if poolcheck.status_code != 200:
        print('pool list 2nd auth error ({}), stopping in loop {}:'.format(poolcheck.status_code,loop))
        break
      else:
        print('pool list 2nd attempt succeeded ({}) in loop {}, continuing'.format(poolcheck.status_code,loop))
    else:
      print('pool list error ({}), stopping in loop {}'.format(poolcheck.status_code,loop))
      break
    tokendelpath = 'https://{}/mgmt/shared/authz/tokens/{}'.format(bigipdev,tokendata['token']['token'])
    tokendelete = session.delete(url=tokendelpath,headers=xauthhead,verify=False)
    # print('deleting auth token: {:f}'.format(time.time() - authtime))
    if tokendelete.status_code != 200:
      print('token delete error ({}), stopping in loop {}'.format(tokendelete.status_code,loop))
      break
  else:
    print('getting token auth error ({}), stopping in loop {}'.format(tokenrequest.status_code,loop))
    break
    # print('successful iterations: {}'.format(loop))

 

 

1 ACCEPTED SOLUTION

Hi, on behalf of my client I opened a service request with the F5 support.

The engineer was able to reproduce the issue based on the script provided.

The Bug ID 1108181: "iControl REST call with token fails with 401 Unauthorized" was confirmed and an Engineering Hotfix for TMOS v15.1.5.1 (Hotfix-BIGIP-15.1.5.1.0.230.14-ENG.iso) was provided. I can confirm, the hotfix solves the issue.

I can also confirm, that TMOS v16.1.3 is not affected.

In case you are facing sporadic issues with the F5 Ansible modules after upgrading to TMOS v15.1.5.1, it might be worth to try the script and to open a service request with F5 support.

Thanks again to @Leslie_Hubertus

View solution in original post

5 REPLIES 5

Leslie_Hubertus
Community Manager
Community Manager

Hey @StephanManthey - thanks for sharing this. I'm escalating it internally to make sure the right team sees this and can reply. 

Leslie_Hubertus
Community Manager
Community Manager

One of my colleagues in PS (who mainly works on Rakuten stuff) said, 

"If the problem is related to REST API then the workaround is to add more memory to restjavad process:

tmsh modify sys db provision.extramb value 2000
tmsh modify sys db provision.restjavad.extramb value 1984
tmsh modify sys db restjavad.useextramb value true

tmsh save sys config

bigstart restart restjavad

bigstart restart restnoded"

They think that “provision.restjavad.extramb” was added in 15.1.3.something.

Here's the K30042148: Restjavad memory provisioning threshold limit article.

Hi @Leslie_Hubertus

thanks for your reply. I have tried it, but it doesn´t work. 

The following command isn´t available on my device under test (TMOS v15.1.5.1): 

tmsh modify sys db provision.restjavad.extramb value 1984

Btw, I also tried to provision MGMT with more resources. No luck. 

Last week I opened a service request with F5 support. As soon as a solution is provided, I will post it here. 

Best regards, Stephan 😊

Well, drat. I was hoping. 

Vielen Dank in advance for coming back to share the solution when you get one, @StephanManthey

Hi, on behalf of my client I opened a service request with the F5 support.

The engineer was able to reproduce the issue based on the script provided.

The Bug ID 1108181: "iControl REST call with token fails with 401 Unauthorized" was confirmed and an Engineering Hotfix for TMOS v15.1.5.1 (Hotfix-BIGIP-15.1.5.1.0.230.14-ENG.iso) was provided. I can confirm, the hotfix solves the issue.

I can also confirm, that TMOS v16.1.3 is not affected.

In case you are facing sporadic issues with the F5 Ansible modules after upgrading to TMOS v15.1.5.1, it might be worth to try the script and to open a service request with F5 support.

Thanks again to @Leslie_Hubertus