Forum Discussion

JN_AU's avatar
JN_AU
Icon for Altostratus rankAltostratus
Jun 16, 2023

Two virtual servers go down after an upgrade

Hi Everyone, I'm wondering if anyone has seen this behaviour before. After I perform an upgrade, two virtual servers out of about 10 go down. The applications using these VSs stop working, from the ...
application delivery
BIG-IP
upgrade
  • JN_AU's avatar
    JN_AU
    Oct 27, 2023

    Hi All,

    This looks to have possibly been this bug: https://my.f5.com/manage/s/article/K85805058

    The actual issue was on pool members behind the F5 - they used the F5s as a gateway to get to the internet using an IP forwarding VS. When the issue occurred these pool members were unable to get to the internet. It looks like the standard HTTPS health checks failed because the pool members were timing out trying to load internet content.

    After further examination of packet captures it was observed there was possibly async traffic (based on MACs observed).

    The fix was to create a new FastL4 profile and make sure 'loose init' and 'loose close' were enabled. This profile was then used on the ip forwarding VS, and it looks like this has solved the issue.

    14.1.5.6 was installed and is so far working fine.

Ben_Novak's avatar
Ben_Novak
Icon for Employee rankEmployee
Jun 16, 2023

In addition to Paulius, I suggest you start with uploading a qkview to iHealth, from there it might be able to identy any issues.

With code upgrades, the ciphers also get upgraded.  This can cause some server connectivity issues.

My first stop would be the ltm logs to see if there are any iRule errors, cipher errors, or health monitor timeouts.

I also suggest getting up to the latest 15.1.x train so you have all the latest hotfixes.

  • Mohamed_Ahmed_Kansoh's avatar
    Mohamed_Ahmed_Kansoh
    Icon for MVP rankMVP
    Jun 18, 2023

    Hi JN_AU , 
    in addition to Ben_Novak and Paulius , 
    use F5 Bug scrub portal to identify any Bugs before proceeding on your upgrade , maybe you hit in a bug. 
    This is the URI to use it : 
    https://my.f5.com/manage/s/bug-tracker

    Let me give you brief about Bug tracker , you will add your target virsion and filter on (blocking , Critical , High ) issues also filter on ( TMOS , LTM ) Bugs. 
    so Based on your design and scenario I think you may find some thing related to your issue.

    >> Note , Running Bug Scrub for a target version you aim to use it for upgrade is very crucial before Upgrade , indeed !