Forum Discussion

yue_zheng1's avatar
Dec 13, 2024

syslog-ng anf errdefsd service err

When executing the errdefsd command on the 15.1.10.5 command line, mcpd resources are consumed by 95% and the web page cannot be opened. What is the problem? The ltm log indicates that errdefsd keeps restarting. Restarting syslog-ng and errdefsd services cannot solve this problem.

 

 

  • Hi yue_zheng1

    The reason could be...

    1. The errdefsd service is likely failing due to a corrupted or misconfigured log file, error definition file, or system logging configuration.
    2. The constant restarts of errdefsd could trigger cascading issues, causing the mcpd process to utilize excessive resources while trying to maintain stability.
    3. This could also lead to a feedback loop where syslog-ng and errdefsd interact incorrectly, generating an error storm.

    Likely solutions....

    • Clear Stale or Corrupted Logs:  - cd /var/log
      find . -type f -name '*.log' -size +500M -exec truncate -s 0 {} \; - 

    • Check the Error Definition Files, - Review and validate the errdefsd configuration and error definition files -

       

      - cd /var/log
      find . -type f -name '*.log' -size +500M -exec truncate -s 0 {} \; -  || Ensure the file is not corrupted. If needed, restore it from a backup.

    • Restart Services in Proper Order - Restart dependent services in this sequence - 
      • bigstart stop syslog-ng errdefsd
        bigstart start syslog-ng errdefsd
        bigstart restart mcpd

     

    • Verify System Logs - Use journalctl to check for specific errors causing errdefsd restarts - 
      • journalctl -xe | grep errdefsd

     

    If these do not work, reboot your device in maintenance window. 

    Please let me know if it works.

     

     

  • Hi yue_zheng1

    Check quickly if there are core files generated recently, follow this article : https://my.f5.com/manage/s/article/K10062

    if you found any of core files , you must open a case with F5 Support to allow them review the core dump file and let you know the RCA for this issue, no one other than F5 Support can help you in Core files analysis.



    By the way, I have reproduced this issue in my lab and as you said 
    errdfsd keeps restarting and mcpd takes most of CPU also I felt with some degradation overall. 

    It was solved by rebooting BIGIP, so I believe if you rebooted it this will free the CPU. 

    I don't see any other workarounds for this, but as I said you can reach out to F5 support team to know the RCA for this.

    • yue_zheng1's avatar
      yue_zheng1
      Icon for Cirrus rankCirrus

      hi :
      My device is out of maintenance time and can't be rebooted for a while, I killall syslog-ng and restarted syslog-ng to free up the CPU usage of the mcpd service. But I can't do anything about the constant rebooting of errdfsd, but I found that bigstart stop errdefsd stopping the errdefsd service stops the cyclic reboot problem.

       

       

      • Hi , 

        I believe that you need to prepare rebooting your BIGIP. 

        I think when you used bigstart stop errdefsd this stops the restarting but still mcpd takes the most of CPU.
        without errdefsd you have a problem in Logging.