Forum Discussion

JN_AU's avatar
JN_AU
Icon for Altostratus rankAltostratus
Jun 16, 2023

Two virtual servers go down after an upgrade

Hi Everyone, I'm wondering if anyone has seen this behaviour before. After I perform an upgrade, two virtual servers out of about 10 go down. The applications using these VSs stop working, from the ...
application delivery
BIG-IP
upgrade
  • JN_AU's avatar
    JN_AU
    Oct 27, 2023

    Hi All,

    This looks to have possibly been this bug: https://my.f5.com/manage/s/article/K85805058

    The actual issue was on pool members behind the F5 - they used the F5s as a gateway to get to the internet using an IP forwarding VS. When the issue occurred these pool members were unable to get to the internet. It looks like the standard HTTPS health checks failed because the pool members were timing out trying to load internet content.

    After further examination of packet captures it was observed there was possibly async traffic (based on MACs observed).

    The fix was to create a new FastL4 profile and make sure 'loose init' and 'loose close' were enabled. This profile was then used on the ip forwarding VS, and it looks like this has solved the issue.

    14.1.5.6 was installed and is so far working fine.

JN_AU's avatar
JN_AU
Icon for Altostratus rankAltostratus

Hi All,

Thanks for the replies.

Paulius:

The health monitor and everything works fine when using the working versions - 14.1.4 or 15.1.3.1. It's only when a different version is made active that the issues start. Nothing else changes in the environment, the only difference is the F5 upgraded version (no routing/firewall changes).

I went and compared the versions - 14.1.4 to 14.1.5.4, and 15.1.3.1 to 15.1.4.1 (also tested not working).

Comparing 14.1.4 (working) to 14.1.5.4 (non-working), the only difference I could see in the config was under the virtual addresses for the VIPs, they had 'icmp-echo enabled' in 14.1.5.4 (non-working version).

Comparing 15.1.3.1 (working) to 15.1.4.1 (non-working) I couldn't see any differences in the config.

Ben_Novak:

I uploaded QKViews to iHealth but they didn't shed any light on the issue unfortunately. I've had a case open for quite some time with F5 for this issue. No errors come up in the logs when this issue happens that I've found so far.

Unfortunately every version I've tried in 14.x over 14.1.4 has not worked - same thing with anything over 15.1.3.1 in the 15.x branch. I also can't leave these F5s running 15.1.3.1 as they are 2000s, and only up to 15.1.2 is officially supported.

Mohamed_Ahmed_Kansoh:

Thanks for letting me know about the bug tracker. I did search this for the versions listed but unfortunately couldn't find anything related to these issues I'm facing.

Paulius's avatar
Paulius
Icon for MVP rankMVP
Jun 21, 2023

JN_AU If you are not able to share your configuration the only other thing that I can recommend is performing a code upgrade without copying the configuration over to the new installation and then configuring the F5 from scratch to see if it works after that. You should be able to load the configuration quickly by gathering all the configuration through the CLI and then loading it using the following command. Aside from this as a last resort, this might be something that you have to wait for F5 to respond on.

load sys config from-terminal merge