Forum Discussion

kgaigl's avatar
kgaigl
Icon for Cirrocumulus rankCirrocumulus
Jun 20, 2023

high cpu usage independent from Traffic

Hello,

we've recognised since a few weeks every day for about 4 hours from 9 to 13 very high cpu-usage on Control-Plane and Analysis-Plane.

Overall concurrent Client-side connections between 1200 and 1800

That's also on the standby-Machine, so it's independent from Traffic (this F5 is for Traffic from Web and terminates ssl)

the hardware is i4800, but it's the same on our virtual Test-Machine

Version: 16.1.3.3, on Test: 16.1.3.4

Any hint, where to look for the cause?

Thank you

Karl

  • kgaigl's avatar
    kgaigl
    Icon for Cirrocumulus rankCirrocumulus

    Hi Mohamed,

    thank you, I got 30 Minutes ago the reply from F5 support with the same Info.

    The Background Task of Obfuscation startet every day at 9:38 am, now I resceduled it to 10 pm

    So it was apparently not realy a problem, but I was very concerned

    Thank you guys for your help

    Karl

    • Yes , 
      I expected that too from the graphs , it follows the same trend everyday , which me some tasks or processes runing in the background. 
      So it was the ASM Obfuscator 

      Great to find the needed clarifications. 🙂 

  • kgaigl's avatar
    kgaigl
    Icon for Cirrocumulus rankCirrocumulus

    from IHealth:

     

    strange thing: every day the same time period from 9 am to about 2  pm

  • kgaigl's avatar
    kgaigl
    Icon for Cirrocumulus rankCirrocumulus

     the process is always java

    I can see this with "top"

  • kgaigl's avatar
    kgaigl
    Icon for Cirrocumulus rankCirrocumulus

    for Info:

    disabled DOS-Profile, didn't help, now opened a support case.

    we will see...

    • kgaigl , 
      Could you please check the Data plane cores or go to >> Statictics ---> dashboard . 

      you can see the data plane and control clearly. 

      > also , did you issue Top command or not ? 
      we need to see if your bigip triggered idle enforcer mode or not , or check the process that consumes that CPU. 


    • Hello,

      I'm afraid so, if you have a valid support contract with F5 support, you can open a case and check whether they could provide you with a hotfix or if it is still being analyzed with the development team.

      Thanks,

  • kgaigl's avatar
    kgaigl
    Icon for Cirrocumulus rankCirrocumulus

    Thank you guys,

    I disabled AVR and uploaded a Snapshot to iheath.

    We'll try to find the Isssue with your help.

    Karl

  • In addition to what Mohamed_Ahmed_Kansoh  mentioned, if you're provisioning AVR module, it might cause high CPU utilization depending on how many virtual servers are using the analytics profile.

    Also, the odd and even cores can be extracted from the CPU graph on ihealth portal. you can generate a new QKVIEW file, import it to any of your F5 accounts and navigate to "graphs" section. then, select "Blade 0 CPU Usage By Core" and filter by the even cores as Kansoh mentioned.

    you can also try rebooting the device and check if the same CPU utilization occurring or not, because sometimes cleaning up files, and processes might hang out.

    check out this article:

    https://my.f5.com/manage/s/article/K48025015

    Thanks,

  • Hi kgaigl , 
    Follow this Article to troubleshoot on high CPU on your box : 

    https://community.f5.com/t5/technical-articles/troubleshooting-high-cpu-utilisation-on-big-ip-systems/ta-p/280806

    by the way, don't rely on Concurrent connections in your calculations , you have to monitor the Data plane itself on the ( Even-Cores ) and see it's percentage. 

    >> use TOP command script on Bigip bash prompit , and see which process consumes much of CPU or send a snap shot form Top Command if you wish.