Forum Discussion

dhubick's avatar
dhubick
Icon for Altocumulus rankAltocumulus
May 04, 2022

High Server CPU after 16.2.1 Upgrade

 Our SysAdmin team has reported a sharp increase in CPU usage on the nodes in our ltm pools after upgrading to BIG-IP 16.2.1. from 14.2.1. The servers are maintained by a different team, but I ...
application delivery
JRahm's avatar
JRahm
Icon for Admin rankAdmin

Some good feedback here. I'd add:

  • First off, do you mean release 16.1.2? There isn't a 16.2.x.
  • Secondly, using default profiles in the config can create unexpected behaviors during major upgrades, as options within them can and do change over time. I'd review the tcp and server ssl profiles first, and perhaps your monitors as well to see if any of those are different after the upgrade. 
  • Lastly, if you haven't upgraded the standby box yet, take a packet capture on your upgraded active device, then fail over to the not yet upgraded standby device and take another packet capture so you can compare and analyze between the two environments.
dhubick's avatar
dhubick
Icon for Altocumulus rankAltocumulus
May 06, 2022

THE Jason Rahm? I'm honoured! I enjoy your Lightboard Lessons.

  • Yes, I do mean 16.1.2.1.  Although, I see 16.1.2.2 was released a couple weeks ago and I am preparing to upgrade to that.
  • We primarily use iApps from our BIG-IP 12 and 14 days. Most are using using custom TCP profiles derived from tcp-lan-optimized and they appear unchanged before/after the upgrade.
  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    May 06, 2022

    good deal. Keep us posted, would love to hear the resolution on this!

    And thank you, I appreciate that!

    • dhubick's avatar
      dhubick
      Icon for Altocumulus rankAltocumulus
      May 12, 2022

      We updated from 16.1.2.1 to 16.1.2.2 this week. Can you guess what day it was?

      So, the resolution is updating to 16.1.2.2, but I am still investigating to determine a root cause. I have not had a chance to compare before/after packet captures yet.

      One detail I left out of the initial post- these apache webservers are Identity proxies that sit in our DMZ and forward traffic between our external f5 hosts and our internal f5 hosts.

      I had assumed that the high CPU was caused by the external f5 hosts forwarding to the Identity proxies. But, perhaps it was between the Identity proxies and our internal f5 hosts.