HTTPD error logs filling up. Enterprise Manager to blame?
Since upgrading to 13.0 (currently HF3), my HTTPD log has been filling with errors regarding EM auth tokens. Dec 11 10:12:49 slot1/bigip1 err httpd[23832]: [error] [client 172.16.0.230] Got exception while handling EM auth token request: Invalid Token: Could not decrypt authentication token: Could not RSA decrypt authentication token., referer: http://localhost:8100/shared/device-availability Dec 11 10:13:14 slot1/bigip1 err httpd[24494]: [error] [client 172.16.0.230] Got exception while handling EM auth token request: Invalid Token: Could not decrypt authentication token: Could not RSA decrypt authentication token., referer: http://localhost:8100/shared/gossip This is puzzling because we don't have BIGIQ or Enterprise Manager configured. These vCMP guests are pre-production with APM/LTM modules provisioned. Troubleshooting steps so far have been: Reset device trust Delete and recreate device trust Restart the HTTPD daemon and clear the .gz files that accumulate. Is anyone else seeing this behavior in their version 13 BIG-IPs?
Creating a tmsh script with iControl REST and using it to restart HTTPD
Problem this snippet solves: TMSH has the ability to create tcl scripts that can be used to run multiple commands and transactions. It is rare that you will want to create on with TMSH but there are a few cases where this may be desirable. One of these is to restart HTTPD, which is difficult to do from iControl REST because the REST API is running over HTTPD and the restart will not be clean. See K13292945. This Python script creates a tmsh script, and then runs it to restart HTTPD. How to use this snippet: Syntax is <program_name.py> host user password Sample output: ./rest_script_example.py10.155.117.12 admin admin Before httpd (pid 3186) is running... After httpd (pid 3289) is running... Code : #!/usr/bin/python #m.lloyd@f5.com #Makes tmsh script to restart HTTP #Syntax: host username password import json #allow python 2 and python 3 by loading the correct libraries. try: from http.client import BadStatusLine from urllib.parse import urlparse, urlencode from urllib.request import urlopen, Request from urllib.error import HTTPError except ImportError: from httplib import BadStatusLine from urlparse import urlparse from urllib import urlencode from urllib2 import urlopen, Request, HTTPError import ssl import sys import time #Internal calls will not verify certs so disable cert verification. ssl._create_default_https_context = ssl._create_unverified_context #Create request for token based authentication. This is in Bigip 12 and later: url = 'https://'+sys.argv[1]+'/mgmt/shared/authn/login' values = {'username' : sys.argv[2], 'password' : sys.argv[3], 'loginProviderName' : 'tmos'} values = json.dumps(values).encode('utf-8') Request(url,data=values) req = Request(url,data=values) req.add_header('Content-Type' , 'application/json') #Request authentication token. response = urlopen(req) #auth=result will be a json data structure. auth_result = response.read() #print (auth_result) #Json.loads makes an internal python data structure that is easier to extract auth token from json. #Now construct icontrol rest query for device-groups info. auth=json.loads(auth_result) token=(auth['token']['token']) #print(token) #Get current PID of HTTPD url = 'https://'+sys.argv[1]+'/mgmt/tm/sys/service/httpd/stats' req = Request(url) req.add_header('X-F5-Auth-Token',auth['token']['token']) response = urlopen(req,data=None) json_response=(response.read()) python_response=json.loads(json_response) print("Before") print(python_response["apiRawValues"]["apiAnonymous"]) #look for script with name to make sure that the script does not already exist url = 'https://'+sys.argv[1]+'/mgmt/tm/cli/script/example.tcl' #urllib2 raises an exception with an HTTP 404 req = Request(url) req.add_header('X-F5-Auth-Token',auth['token']['token']) try: response = urlopen(req,data=None) except HTTPError as err: if err.code==404: #print (err.code) #print("\nCreate cli script\n") #request create here url = 'https://'+sys.argv[1]+'/mgmt/tm/cli/script' req = Request(url) req.add_header('X-F5-Auth-Token',auth['token']['token']) req.add_header('Content-Type' , 'application/json') values = {"name":"example.tcl", "apiAnonymous": "proc script::init {} {\n}\n\nproc script::run {} {\n tmsh::run util bash -c 'killall -9 httpd' \n tmsh::start sys service httpd\n} \n\nproc script::help {} {\n}\n\nproc script::tabc {} {\n}\n"} values = json.dumps(values) response = urlopen(req,data=values) response_py=(json.load(response)) #print(json.dumps(response_py,sort_keys=True,indent=4)) #Now run script url = 'https://'+sys.argv[1]+'/mgmt/tm/cli/script/example.tcl' req = Request(url) req.add_header('X-F5-Auth-Token',auth['token']['token']) req.add_header('Content-Type' , 'application/json') values = {"kind":"tm:cli:script:runstate","command":"run"} values = json.dumps(values).encode('utf-8') # Killing HTTPD will abort the connection so catch the exception. try: urlopen(req,data=values) except BadStatusLine: pass #Wait for httpd to restart so you can query. time.sleep(5) #Get current PID of HTTPD after restart url = 'https://'+sys.argv[1]+'/mgmt/tm/sys/service/httpd/stats' req = Request(url) req.add_header('X-F5-Auth-Token',auth['token']['token']) response = urlopen(req,data=None) json_response=(response.read()) python_response=json.loads(json_response) print("After") print(python_response["apiRawValues"]["apiAnonymous"]) Tested this on version: 13.0Bug (ID 775845) Workaround; REST API httpd restart
So this is less of a question, but a post to help my fellow BIG-IP LTM administrators, since the solution I came up with is quite the hack, but it works for me, so your mileage may vary, and of course -- test in non-production environments. So some background: I am a F5 administrator and a automation engineer. My main focus is automating much of my work as an administrator to take mundane and repetitive tasks out of my and my colleagues/organizations workflow. So, when it came time to renew the device certificates for my F5 VMs and hosts, combined with the most recently reduction in SSL certificate term length and guidance to renew certs often, I set forth to automate the entire stack of processes that are required to renew device certificates (create key/csr, submit csr to CA and obtain cert, upload cert to F5 and restart the httpd service to read in the new certificates). I was able to script everything using Python and REST API calls to the F5s and InCommon CA to get the certificates created and put on the F5s. The problem I ran into was the feature to restart the httpd service via a REST API call was broken (aka Bug ID 775845). I tried using the REST API call: /tm/sys/service -X POST -d '{"name":"httpd", "command":"restart"}' I also attempted to use the bash command call: /mgmt/tm/util/bash -X POST -d "{ "command": "run", "utilCmdArgs": "-c 'service httpd restart'" } NONE worked, as documented in the is KB article: https://support.f5.com/csp/article/K13292945 So I needed a workaround, and my solution incorporates a batch script that basically preemptively kills off httpd and then restarts it (as you see in the KB shows as a fix). First, you need the following bash script (which is actually incorporated into the script below so one can ensure that it always present on the F5 VM or host that needs to have the httpd daemon restarted). #/bin/bash # Pause, restart httpd # Greg Jewett, 2021-08-26, jewettg@austin.utexas.edu # # A known bug (Bug ID 775845) when using the REST API to restart the httpd service. # The pause is to allow the REST API call to complete, as script will be launched # in background, and should have successful exit code. This script provides an # immediate fix to bring environment back up, without manually restarting the # httpd daemon on each VM or host. service httpd status | logger -p local0.notice -t RST_HTTPD logger -p local0.notice -t RST_HTTPD Waiting 2 seconds... sleep 2s logger -p local0.notice -t RST_HTTPD Restarting httpd daemon thepids=`pgrep -d " " -f "/usr/sbin/httpd"` echo "httpd pids are: $thepids" for aPid in $thepids; do echo "Killing PID $aPid" kill -9 $aPid done service httpd start | logger -p local0.notice -t RST_HTTPD service httpd status | logger -p local0.notice -t RST_HTTPD logger -p local0.notice -t RST_HTTPD Done NOTE: I am having to attach the rest of my solution via comments, as the platform was allowing me to post a big chuck of text (>10k chars). See below.
How to allow access to configuration utility GUI only on particular VLAN instead of all VLANs
Hi, I would need to restrict the access to configuration utility GUI on only one VLAN, how is this done? I don't want to restrict the access based on the source IP, but just to have the port 443 listening only on the VLAN that is used for O&M purposes. The software version is 11.6.0. Thanks!
httpd settings keep re-enabling
HTTPD settings keep re-enabling. Was testing out settings for the management UI login. I end up having to go into tmsh or vi to un-do them. (Per the documentation.) (ssl-client-verify and ssl-ocsp-enable) I save a /sys config after I un-do the setting however every time I sync the Active and Standby the settings come back and re-enable. Are those settings stored in another file somewhere that the sync process reads?
