Forum Discussion
sdavids5670_166
Nimbostratus
Aug 07, 2014Health check failing - tcp/54321 - is this more than a coincidence?
We are rolling out a new web application in a couple of weeks and the developers have noticed issues with the application that are very intermittent and difficult to quantify. Testers have reported ...
Jason_Adams_124
Dec 10, 2014Historic F5 Account
The scope of this bug has been increased to include the following:
11.5.0 Release Notes Known Issue
https://support.f5.com/kb/en-us/products/big-ip_ltm/releasenotes/product/relnote-ltm-11-5-0.htmlrn_ki
11.5.1 Release Notes Known Issue
11.6.0 Release Notes Known Issue
https://support.f5.com/kb/en-us/products/big-ip_ltm/releasenotes/product/relnote-ltm-11-6-0.htmlrn_ki
Known-Issue Solution Article:
SOL15907: The BIG-IP system may incorrectly send an ICMP destination unreachable message to a server responding to health monitor traffic on TCP source port 54321 - https://support.f5.com/kb/en-us/solutions/public/15000/900/sol15907.html
- JGDec 10, 2014
Cumulonimbus
SOL15907: "This issue occurs because the default iptables rules on the BIG-IP system contain a rule that is configured to protect a BIG-IP APM service listening on TCP port 54321, even though APM or SWG is not provisioned." Does this mean if the system has APM provisioned it is not affected? - Jason_Adams_124Dec 10, 2014Historic F5 AccountIt does not. If you are running APM, you are still affected by this bug. This means that BIG-IP software is affected, no matter what modules are provisioned.
- JGDec 10, 2014
Cumulonimbus
Thanks. But of the 3 commands provided in the sol, the 2nd one gives an error when run: "iptables: No chain/target/match by that name" - Jason_Adams_124Dec 11, 2014Historic F5 AccountYeah...that's actually expected. The 2nd command is supposed to delete the third rule from the list (if it exists hence the "-D" instead of the "-A" in the third command). If you ran those same three commands again, you would get the same error for the first command, then commands 2 and 3 would complete without any feedback. This will make sure that, if somebody runs those commands multiple times, they will always have the same end result.
- dfosborne2_2224Jan 26, 2015
Nimbostratus
Just so you know, the /config/startup hack is failing for me now on 11.6 HF3 which I upgraded to over this past weekend from 11.5.1. I originally had the issue with F5 back in August with 11.5 and after about a week of going back and forth with them, they provided that iptables fix. It started failing again after the upgrade to 11.6 this weekend even though I persisted the iptables directives in /config/startup. Tried running by /config/startup by hand and that still did not help. Eventually, I resorted to running each iptables. Reading the information here I noticed that SOL15907's ordering of the iptables rules are off as compared to mine below: My incorrectly entered order: /sbin/iptables -D INPUT -p tcp --dport 54321 -j REJECT --reject-with icmp-port-unreachable /sbin/iptables -A INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset /sbin/iptables -D INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset And sol15907's order: /sbin/iptables -D INPUT -p tcp --dport 54321 -j REJECT --reject-with icmp-port-unreachable /sbin/iptables -D INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset /sbin/iptables -A INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset Seems I may have put them in the wrong order. But the funny thing is that this didnt matter for 11.5.1 as the problem went away until the upgrade and even after just correcting and re-running, the failures still pop. I have a ticket in with F5 and will report back findings.. - Jason_Adams_124Jan 27, 2015Historic F5 AccountMy theory is that you incorrectly entered the rules into /config/startup on v11.5.1, but you did not reboot the system until you planned for the upgrade. The behavior of iptables did not change between versions. You just exposed the problem by restarting your system.
- dfosborne2_2224Jan 27, 2015
Nimbostratus
Hi Jason. Wish it was that simple. I rebooted both units many times in the interim between 11.5.1 and 11.6 upgrade. We were on 11.5 back in July and both units have been rebooted multiple times during that time. Are you running 11.6 yet? Have you not encountered this problem? I also rebooted both units with the correct startup file in place and the problem did not go away. - Jason_Adams_124Jan 27, 2015Historic F5 AccountHi dfosborne2. I have v11.5.1 and v11.6 in my lab. I just tested the behavior in the following order: On the command line: /sbin/iptables -D INPUT -p tcp --dport 54321 -j REJECT --reject-with icmp-port-unreachable /sbin/iptables -D INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset /sbin/iptables -A INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset - Verify the iptables rule is in place: iptables -nL|grep 54321 REJECT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:54321 flags:0x12/0x02 reject-with tcp-reset - Add the following lines to /config/startup (Incorrect Order): /sbin/iptables -D INPUT -p tcp --dport 54321 -j REJECT --reject-with icmp-port-unreachable /sbin/iptables -A INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset /sbin/iptables -D INPUT -p tcp -m tcp --dport 54321 --tcp-flags ACK,SYN SYN -j REJECT --reject-with tcp-reset - Reboot reboot - Check the status of my iptables rules: iptables -nL|grep 54321 [root@ltm9a:INOPERATIVE:] config - As expected, my iptables rule regarding port 54321 are all gone. -- -- -- -- So with that information, I would suspect that the iptables rules had been removed after the first reboot. -- -- -- To test if you are still experiencing the same failure, run a tcpdump to listen on port 54321: tcpdump -s0 -ni 0.0:nnn port 54321 -w /var/tmp/54321.tmm.$HOSTNAME.01.pcap -v And to listen between TMM and the Linux Host, use the :h flag: tcpdump -s0 -ni VLAN_NAME:nnnh port 54321 -w /var/tmp/54321.linux.$HOSTNAME.01.pcap -v
- dfosborne2_2224Jan 27, 2015
Nimbostratus
Hi again. So I ran the first tcpdump and saw a ton of traffic across several VLANs. Picked one of those VLANs and ran the second tcpdump and captured 6 packets between the VLAN 11 self IP floater and a pool member...Tried pasting the packets and its being id'd as spam.Sheesh.
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects