Forum Discussion
soymanue
Nimbostratus
Jul 19, 2012ConfigSync issue with LTM 11.2 HF1
Hello
I have upgraded two BIG-IP 3600 from version 11.1 to 11.2 HF1.
High Availability seems to be working, but ConfigSync fails.
Both devices show "Disconnected" state.
If you go to Device Management > Devices the other peer appears with its older information.
I've tried to force ConfigSync from console:
tmsh run cm config-sync to-group /Common/device-group-failover-67cbdf7d77ca
tmsh run cm config-sync to-group device-group-failover-67cbdf7d77ca
But nothing happens.
In /var/log/ltm file there are several messages like this:
Can't connect to CMI peer 192.168.96.251, port:6699, Transport endpoint is not connected
Telent to the other peer's 6699 port doesn't work:
telnet 192.168.96.251 6699
Trying 192.168.96.251...
telnet: connect to address 192.168.96.251: Connection refused
telnet: Unable to connect to remote host: Connection refused
Any idea?
31 Replies
- soymanue
Nimbostratus
I have more information:
tmsh run cm config-sync recover-sync
[root@dartagnan-cpdn:ModuleNotLicensed:Standby] ssl.crt tmsh run cm config-sync recover-sync
Backup Data...
Saving active configuration...
Clean system configuration files...
Create new bigip_base.conf...
Restore basic configuration...
Loading system configuration...
/defaults/app_template_base.conf
/defaults/config_base.conf
/config/low_profile_base.conf
/defaults/wam_base.conf
/usr/share/monitors/base_monitors.conf
/config/daemon.conf
/config/profile_base.conf
/defaults/fullarmor_gpo_base.conf
/defaults/classification_base.conf
Loading configuration...
/config/bigip_base.conf
0107134a:3: File object by name (dtdi.crt) is missing.
Unexpected Error: Loading configuration process failed.
Actually dtdi.crt is missing in /config/ssl/ssl.crt - nitass
Employee
it is a known issue which is fixed in next major version i.e. 11.3.
Bug 390715 - recover-sync has error; 'File object by name (dtdi.crt) is missing'
actually, i think file is not missing but configuration is i.e. cm cert and cm key in bigip_base.conf. you may check whether file is there by running "tmsh list cm cert dtdi.crt".
to fix, you have to add cm cert and cm key configuration to bigip_base.conf e.g. copy it from backup bigip_base.conf.
hope this helps. - soymanue
Nimbostratus
Thank you
The dtdi.crt issue has been fixed following your instructions.
But they're still disconnected.
And the same message is still there:
Jul 19 11:31:30 notice mcpd[4972]: 01071431:5: Attempting to connect to CMI peer 192.168.96.251 port 6699
Jul 19 11:31:30 err mcpd[4972]: 0107142f:3: Can't connect to CMI peer 192.168.96.251, port:6699, Transport endpoint is not connected - nitass
Employee
But they're still disconnected. have you rebooted/restarted? - soymanue
Nimbostratus
I have rebooted node 2 (secondary) and now de primary accepts connection to 6699 port. But the reloaded node doesn't answer yet. I can't find an explanation for that.
Where can I find information regarding the bug?
Thank you again. - Techgeeeg
Nimbostratus
Dear Manuel,
Have you checked the device certificates i hope they are not expired??????What was the situation of the certificate expiry when u upgraded the boxes???? if all was fine i advice you to open a case with F5 support and do share with us how it got resolved.
Regards, - Cory_50405
Noctilucent
We ran into this issue when upgrading our LTMs to 11.2.0 HF1. Check the big3d daemon version. /usr/sbin/big3d is where the running version should be (/usr/sbin/big3d -v). This should report 11.2.0.2446. If you use Enterprise Manager, then there will be another directory containing big3d (/shared/bin/big3d) that was pushed to the device by EM. Copy the version from /usr/sbin/big3d to /shared/bin/big3d and then restart big3d.
From CLI (bash), run watch_devicegroup_device to see which device thinks it has the latest version of the configuration. We had synchronization issues because both devices thought they did. Check solution 13590 to guide you through manually setting a device as the sync group leader and forcing a synchronization if needed. - Steven_Le_Roux_
Nimbostratus
This issue seems not resolved in 11.2.0 HF2.
On two devices with the same upgrade :
dev1 /usr/sbin/big3d -v
/usr/sbin/big3d version big3d Version 11.2.0.2446.0 for linux
dev1 /shared/bin/big3d -v
/shared/bin/big3d version big3d Version 11.2.0.2446.0 for linux
dev2 /usr/sbin/big3d -v
/usr/sbin/big3d version big3d Version 11.2.0.2446.0 for linux
dev2 /shared/bin/big3d -v
/shared/bin/big3d version big3d Version 11.1.0.2268.0 for linux
This is the weirdest part here... the upgrade does not comply identicaly on two devices...
I replaced the /shared/bin/big3d in version 11.1.0.2268 but without success.
Dev 1 and 2 still can't talk each other.
Sep 10 21:41:24 dev1 notice mcpd[6111]: 01071431:5: Attempting to connect to CMI peer 10.0.0.2 port 6699
Sep 10 21:41:24 dev1 err mcpd[6111]: 0107142f:3: Can't connect to CMI peer 10.0.0.2, port:6699, Transport endpoint is not connected
Sep 10 21:42:13 dev2 notice mcpd[5787]: 01071431:5: Attempting to connect to CMI peer 10.0.0.1 port 6699
Sep 10 21:42:13 dev2 err mcpd[5787]: 0107142f:3: Can't connect to CMI peer 10.0.0.1, port:6699, Transport endpoint is not connected
There is a TCP RST sent just after a SYN to mcpd :
dev2 > dev1
21:46:13.107789 IP 10.0.0.2.49994 > 10.0.0.1.6699: S 2928859848:2928859848(0) win 5840
21:46:13.107959 IP 10.0.0.1.6699 > 10.0.0.2.49994: R 0:0(0) ack 2928859849 win 0
dev1 > dev2
21:46:14.549942 IP 10.0.0.1.53332 > 10.0.0.2.6699: S 909210147:909210147(0) win 5840
21:46:14.549963 IP 10.0.0.2.6699 > 10.0.0.1.53332: R 0:0(0) ack 909210148 win 0
So even the device with good version of big3d don't accept connections.
I have both cm cert and cm key, with sha1 sum verified and matching the certs in the filestore.
This really seems a broken release here... any information for a fix or a date for a HF3 ? - Steven_Le_Roux_
Nimbostratus
Restarting big3d process is not enough. With a reboot of the two devices, I have a two devices "In Sync". - Alwin_Evans
Nimbostratus
solution 13887:Forcing a BIG-IP device group member to initiate a ConfigSync operation. did it for me
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects