BIG-IQ : Error when adding device

[Zuke]

I'm standing up a new pair of APM/LTM guests on a BIG-IQ CM. When I add the guests, the standby APM guest fails to import.

From the BIG-IQ GUI, I get this message:

Trust establishment failed; reason: Failed to connect to 192.168.1.101 : Bad Request(400)

The restjavad logs show the attempts to connect and failures:

[WARNING][128][28 Apr 2021 19:08:32 UTC][8100/mgmt ForwarderPassThroughWorker] URI:http://localhost:8100/mgmt/shared/identified-devices/config/device-info, Referrer:http://localhost:8100/cm/global/utility/bigip-device-check, Method:GET, Exception:java.util.concurrent.TimeoutException: remoteSender:192.168.1.181, method:GET 
	at com.f5.rest.common.RestWorker.logAndFailExpiredOperation(RestWorker.java:3163)
	at com.f5.rest.common.RestWorker.checkForExpiredOperations(RestWorker.java:3152)
	at com.f5.rest.common.RestServer.checkAndExpirePendingWorkerOperations(RestServer.java:1263)
	at com.f5.rest.common.RestServer.access$200(RestServer.java:44)
	at com.f5.rest.common.RestServer$3.run(RestServer.java:1236)
	at com.f5.rest.common.ScheduleTaskManager$2$1.run(ScheduleTaskManager.java:123)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
	at java.lang.Thread.run(Thread.java:748)

[WARNING][129][28 Apr 2021 19:08:32 UTC][8100/shared/identified-devices/config/device-info DeviceInfoWorker] URI:http://localhost:8100/shared/identified-devices/config/device-info, Referrer:http://localhost:8100/cm/global/utility/bigip-device-check, Method:GET, Exception:java.util.concurrent.TimeoutException: remoteSender:192.168.1.181, method:GET 
	at com.f5.rest.common.RestWorker.logAndFailExpiredOperation(RestWorker.java:3163)
	at com.f5.rest.common.RestWorker.checkForExpiredOperations(RestWorker.java:3152)
	at com.f5.rest.common.RestServer.checkAndExpirePendingWorkerOperations(RestServer.java:1263)
	at com.f5.rest.common.RestServer.access$200(RestServer.java:44)
	at com.f5.rest.common.RestServer$3.run(RestServer.java:1236)
	at com.f5.rest.common.ScheduleTaskManager$2$1.run(ScheduleTaskManager.java:123)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
	at java.lang.Thread.run(Thread.java:748)

I've already tried restarting restjavad but no luck there. Any other ideas?

[AlexBCT]

Hi Zuke,

Can you post what versions of BigIQ and BigIP you're using? Also any other distinct characteristics for your deployment? And I assume from your post that the import (Device/LTM/APM) from the active member DID work?

Especially with APM, BigIQ is/was a bit picky and only certain versions of BigIQ support certain versions of BigIP APM. For example BigIQ v7.0 supports BigIP LTM v14.1, but only APM v14.0. See here for more details on the versioning: https://support.f5.com/csp/article/K34133507

[Zuke]

Hi Alex. The versions I'm on are 7.1 and 15.1.2. Yes, the peer device was added no problem, and actually this device had previously been added to this BIG-IQ CM. I decided to remove all devices and start fresh.

[Zuke · Accepted solution]

curl --verbose -sku "admin:<redacted>" https://bigip2/mgmt/shared/identified-devices/config/device-info  | jq -r
 
> GET /mgmt/shared/identified-devices/config/device-info HTTP/1.1
> Authorization: Basic YWRtaW46XmlhTCpSXlBCaC54ajgzOHVLLzhA
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 OpenSSL/1.0.1l zlib/1.2.3 libidn/1.18
> Host: <redacted>
> Accept: */*
> 
< HTTP/1.1 400 Bad Request
< Date: Wed, 05 May 2021 15:02:27 GMT
< Server: Jetty(9.2.22.v20170606)
< Set-Cookie: BIGIPAuthCookie=SN8F4Fr0VS8JJ4KZDAobOZ4BkXkfRKADH22x2Hpa; path=/; Secure; HttpOnly
< Set-Cookie: BIGIPAuthUsernameCookie=admin; path=/; Secure; HttpOnly
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=16070400; includeSubDomains
< Content-Type: application/json; charset=UTF-8
< Pragma: no-cache
< Cache-Control: no-store
< Cache-Control: no-cache
< Cache-Control: must-revalidate
< Expires: -1
< Content-Length: 0
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Content-Security-Policy: default-src 'self'  'unsafe-inline' 'unsafe-eval' data: blob:; img-src 'self' data:  http://127.4.1.1 http://127.4.2.1
< Connection: close
< 
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
} [data not shown]

The issue was related to RPM processes being stuck on the guest. I ran the above curl command (per F5 support) from both BIG-IQ and the guest itself and the results were the same. Running a filesystems check and rebooting the guest fixed the issue. Thanks Lucy.