Forum Discussion
John_Stewart_10
Nimbostratus
Dec 22, 2011BIG3D
We have an LTM running 10.2.3.112 code. We ran the big3d -v command and get the following: root@rndltmcore:Active] config big3d -vbig3d version
big3d Version 10.2.3.112.0 for linuxbig3d
version big3d Version 10.2.3.112.0
for linuxWhat is concerning is that SCOM MGMT Pack told us it upgraded the big3d, but the LTM does not reflect that.We also know that when we upgrade the big3d, it will interrupt the communication between GTM's and LTM's, which will affect load balancing for that period. It will also affect communication between Enterprise manager and GTM's and LTM's as they are being discovered.We also know that when we do a code upgrade with a new ISO, it will replace the big3d that was manually upgraded. This could potentially impair SCOM monitoring.Looking for feedback, or comments on this subject.
We attempted discover this LTM with the upgrade box unchecked and it failed. We ran the discovery again with the box checked and it succeeded . The log message indicated that the big3d was upgraded to 1.4.
However, when we run the big3d -v command again, we get the following:
[root@rndltmcore:Active] config big3d -v
8 Replies
- Richard__HarlanHistoric F5 Accountrunning big3d -v will return the version of the file in /usr/bin/big3d run the following command ps aux | grep -i big3d. This will show you the full path of the running version of big3d then run path/big3d -v this will show you the running version of big3d.
yes you are correct when big3d is updated it has to restart the process and GTM communication will be interrupted while it restarts. - John_Stewart_10
Nimbostratus
Thanks,
Got the the following this time:
[root@rndltmcore:Active] config ps aux | grep -i big3d
root 3374 0.0 0.0 1536 364 ? S Dec22 0:00 runsv big3d
root 3428 0.0 0.1 10596 5064 ? S Dec22 2:18 /shared/bin/big3d
root 8463 0.0 0.0 3036 780 pts/0 R+ 15:32 0:00 grep -i big3d
[root@rndltmcore:Active] config /shared/bin/big3d -v
/shared/bin/big3d version big3d Version [b}10.4.0.1.0 for linux
[root@rndltmcore:Active] config
Still need to understand ramifications of a code upgrade to the LTM - John_Stewart_10
Nimbostratus
OK, upgraded one of our LTMs that we discovered with SCOM mgmt pack to version 10.2.3. HF1. Now we get this!!!!!!!!!!!!!!
Jan 13 04:02:05 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:06 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:07 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:08 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:09 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:10 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:11 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:12 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:13 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:14 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:15 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:16 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:17 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:18 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:19 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:20 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:21 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:22 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:23 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:24 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:25 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:26 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:27 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:28 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:29 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:30 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:31 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:32 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:33 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:34 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:35 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:36 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:37 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:38 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:39 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:40 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:41 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:42 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:43 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:44 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:45 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:46 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:47 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:48 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:49 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:50 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:51 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:52 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:53 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:54 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:55 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:56 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:57 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:58 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:02:59 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:00 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:01 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:02 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:03 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:04 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:05 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:06 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:07 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:08 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:09 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:10 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:11 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:12 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:13 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:14 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:15 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:16 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:17 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:18 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:19 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:20 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:21 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:22 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:23 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:24 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:25 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:26 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:27 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:28 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:29 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:30 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:31 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:32 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:33 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:34 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:35 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:36 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:37 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:38 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:39 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:40 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:41 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:42 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:43 local/rndltmcore emerg logger: Re-starting big3d
Jan 13 04:03:44 local/rndltmcore emerg logger: Re-starting big3d - John_Stewart_10
Nimbostratus
Does SCOM MGMT path change teh path from where big3d runs? - Alex__Applebaum
Employee
Hi John,
Sorry, that sucks. The way it works is big3d startup scripts first look to see if there is a newer version in /shared/bin/. Newer versions are installed by EM/GTM/etc. If one doesn't exist, it will use the default version of the install in /usr/sbin/
Historically, it could be starting due to various issues like signatures/permissions, etc. You can use the solution below to debug and see if there's any reason logged.
sol10999: Error Message: Re-starting big3d
http://support.f5.com/kb/en-us/solutions/public/10000/900/sol10999.html?sr=18711325
Or you could open a case with support to help find out what the exact issue is or if this is a known issue with that particular combination of Mgmt Pack + BIGIP version. I know Dev has been working on this issue so opening a case on this and having it attached to the appropriate bug ID will help draw attention to the issue.
if really needed to stop big3d from restarting right now, you can just remove the "newer one" and restart big3d (like below) so it uses the default one.
[root@ltm1:Active] config /usr/sbin/big3d -v
/usr/sbin/big3d version big3d Version 10.2.3.112.0 for linux
[root@ltm1:Active] config
[root@ltm1:Active] config ls -lart /shared/bin/big3d
-rwxr-xr-x 1 root root 3869783 Dec 15 06:05 /shared/bin/big3d
[root@ltm1:Active] config /shared/bin/big3d -v
/shared/bin/big3d version big3d Version 10.4.0.12.0 for linux
[root@ltm1:Active] config
[root@ltm1:Active] config
[root@ltm1:Active] config ps -ef | egrep -i big3d | egrep -v grep
root 3736 3553 0 2011 ? S 0:00 runsv big3d
root 3790 3736 0 2011 ? S 3:17 /shared/bin/big3d
[root@ltm1:Active] config rm /shared/bin/big3d
rm: remove regular file `/shared/bin/big3d'? y
[root@ltm1:Active] config ps -ef | egrep -i big3d | egrep -v grep
root 3736 3553 0 2011 ? S 0:00 runsv big3d
root 3790 3736 0 2011 ? S 3:17 /shared/bin/big3d
[root@ltm1:Active] config bigstart restart big3d
[root@ltm1:Active] config ps -ef | egrep -i big3d | egrep -v grep
root 3736 3553 0 2011 ? S 0:00 runsv big3d
root 16996 3736 32 05:40 ? D 0:00 /usr/sbin/big3d
-Alex - John_Stewart_10
Nimbostratus
Alex, This started when I upgraded 10.2.3 LTM to HF1. Said LTM had 1.4 big3d that was intsalled my the SCOM Management pack. I replaced the 10.4 big3d and all is well,,,,,except SCOM :-) - Alex__Applebaum
Employee
Yeah, think the problem is SCOM's big3d is technically older than the 10.2.3 HF1 big3d even though it has a higher code (10.4.xxxx).
I know you will want your SCOM back up soon so the only thing I can recommend is sending a case to support.
TITLE: SCOM's big3d "Version [b}10.4.0.1.0" restarts on 10.2.3 HF1.
Referencing this thread.
They might be able to give you the newer big3d (ex. the big3d my EM 2.3 installed "10.4.0.12.0" and make everyone happy (SCOM won't complain there's an "older" code and will have the most recent big3d code which works with the latest 10.2.3 HF1). Or you might even be able to put a v11 big3d on. However, you would need to get Dev's recommendation on that one.
Like I said, last I heard, they were working aggressively on addressing this versioning issue so we won't have to worry about this anymore. - Alex__Applebaum
Employee
Hi John,
I installed the EM big3d (10.4.0.1.0) on 10.2.3 and upgraded to 10.2.3 HF1 and didn't see the restart issue.
[root@ltm2:Active] config b version | head
Kernel:
Linux 2.6.18-164.11.1.el5.1.0.f5app
Package:
BIG-IP Version 10.2.3 123.0
Hotfix HF1 Edition
Hot fixes:
ID372864
ID372590
ID372804
[root@ltm2:Active] config ps -ef | egrep 'bin/big3d' | grep -v grep
root 3821 3802 0 17:11 ? S 0:00 /shared/bin/big3d
[root@ltm2:Active] config /shared/bin/big3d -v
/shared/bin/big3d version big3d Version 10.4.0.1.0 for linux
What I think may have happened is that a 2nd big3d process was started at the same time. For example, I accidently ran it again like:
[root@ltm2:Active] config /shared/bin/big3d
Jan 16 17:39:31 local/ltm2 emerg logger: Re-starting big3d
Jan 16 17:39:32 local/ltm2 emerg logger: Re-starting big3d
Jan 16 17:39:33 local/ltm2 emerg logger: Re-starting big3d
Jan 16 17:39:34 local/ltm2 emerg logger: Re-starting big3d
Jan 16 17:39:35 local/ltm2 emerg logger: Re-starting big3d
Jan 16 17:39:36 local/ltm2 emerg logger: Re-starting big3d
[root@ltm2:Active] config Jan 16 17:39:37 local/ltm2 emerg logger: Re-starting big3d
I hit Ctl-C and things returned to normal (with only default daemon running).
There is a Bug ID to make it exit/log if another instance running which might help in the future.
Bug 344228 - If big3d detects another big3d already running, exits without log message
So short of it is, both big3ds should run just fine on 10.2.3 HF1. You should have been able to bigstart stop big3d & bigstart start big3d and you would have been ok. However in order for MP not to complain again, you need to restore your MP big3d with higher version . Going forward, you should be able to re-discover or even copy from another unit that MP big3d (10.4.0.1.0) back to /shared/bin/ and get your MP back up and running. The one with latest timestamp wins so if you see the 10.2.3 HF1 "10.2.3.112.0" big3d pop back up as the running process, just
touch /shared/bin/big3d
Otherwise, please open a case (while your logs haven't rotated) so they can investigate further. Dev was looking at this post so just post the case and I can forward it to them.
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects
