Forum Discussion

yue_zheng1's avatar
yue_zheng1
Icon for Cirrus rankCirrus
Dec 13, 2024

syslog-ng anf errdefsd service err

When executing the errdefsd command on the 15.1.10.5 command line, mcpd resources are consumed by 95% and the web page cannot be opened. What is the problem? The ltm log indicates that errdefsd keeps...
application delivery
HarunTuna's avatar
HarunTuna
Icon for Cirrus rankCirrus
Dec 16, 2024

Hi yue_zheng1, 

The reason could be...

  1. The errdefsd service is likely failing due to a corrupted or misconfigured log file, error definition file, or system logging configuration.
  2. The constant restarts of errdefsd could trigger cascading issues, causing the mcpd process to utilize excessive resources while trying to maintain stability.
  3. This could also lead to a feedback loop where syslog-ng and errdefsd interact incorrectly, generating an error storm.

Likely solutions....

  • Clear Stale or Corrupted Logs:  - cd /var/log
    find . -type f -name '*.log' -size +500M -exec truncate -s 0 {} \; - 

  • Check the Error Definition Files, - Review and validate the errdefsd configuration and error definition files -

     

    - cd /var/log
    find . -type f -name '*.log' -size +500M -exec truncate -s 0 {} \; -  || Ensure the file is not corrupted. If needed, restore it from a backup.

  • Restart Services in Proper Order - Restart dependent services in this sequence - 
    • bigstart stop syslog-ng errdefsd
      bigstart start syslog-ng errdefsd
      bigstart restart mcpd

 

  • Verify System Logs - Use journalctl to check for specific errors causing errdefsd restarts - 
    • journalctl -xe | grep errdefsd

 

If these do not work, reboot your device in maintenance window. 

Please let me know if it works.

 

 

  • yue_zheng1's avatar
    yue_zheng1
    Icon for Cirrus rankCirrus
    Dec 18, 2024

    hi thank you so much for your reply!

    As you suggested
     1. find . -type f -name '*.log' -size +500M -exec truncate -s 0 {} \;

         no corrupted file


     2. bigstart stop syslog-ng errdefsd
         bigstart start syslog-ng errdefsd
         bigstart restart mcpd
    Executing the commands in this order will cause the device to go offline for a short time, ltm log will still keep outputting the errdefsd reboot log.

    3. journalctl -xe | grep errdefsd
    Executing the check command also keeps repeating the errdefsd reboot log.

    It seems that a reboot of the device is required to fully fix the problem. But I can't do that at the moment due to some limitations.

    so sad