Forum Discussion

EricTTEC's avatar
EricTTEC
Icon for Nimbostratus rankNimbostratus
Dec 10, 2019

Create/Repeat Alert When A Pool or Node is Down

I know that when there is a status change in a node or pool, an entry is made to the syslog. I have been asked if there is a way to repeat a message of a down condition periodically, say once every five minutes, while a node or pool is down. Preferable, I would be able to specify the nodes and or pools that this type of repeated alerting would happen on. Here is the scenario:

 

  • Prior to a maintenance window, error monitoring in 3rd party monitoring software is suppressed
  • Work is completed (patches for example) on certain servers
  • As part of the work, server goes offline
  • F5 knows this happens and creates alert, but since alerts being suppressed, it is not seen
  • Work completes, and monitoring is again enabled
  • F5 has already alerted, and does not repeat the alert, so it is missed in the monitoring software

 

Is there a setting, or possibly a way with iRules or something, that the down condition could be detected and reported on periodically?

 

Thanks!

 

Eric

 

application delivery
BIG-IP
  • jaikumar_f5's avatar
    jaikumar_f5
    Dec 11, 2019

    I think an external monitor would be the good solution for this. You have to write your script along with some logger, so that it constantly throws logs about the status. But it will flood your log file for x intervals.

     

    On good note, put the logging only for down state, for Up - do not put the logger.

  • jaikumar_f5's avatar
    jaikumar_f5
    Icon for MVP rankMVP
    Dec 11, 2019

    This is doable using an Icall script. You can throw logs to the ltm file, sleep for certain intervals and again throw logs to the same.

     

    But still, your requirement feels something like a process gap. Just because your monitoring software was paused for maintenance activity, you just shouldn't be throwing false alerts (because the pool member had come up after the activity).

     

    Let us know one thing, after the work is complete, does the pool member come up or is it still down.

    Also what kind of monitoring software are you using, Most of the monitoring platform follow MIB's and poll a certain set of intervals again and again. Even when you turn off & on later, it still makes the polling. It does not depend on the past logs of ltm.

     

    Maybe if you could explain more on the monitoring software, we can suggest accordingly. Still Icall is your best bet on this as far I know.

  • EricTTEC's avatar
    EricTTEC
    Icon for Nimbostratus rankNimbostratus
    Dec 11, 2019

    Thanks for the response!

     

    Unfortunately, it is a different branch of the organization that handles the monitoring and I do not know what the software is or how it works. The issue has been that after the patching, the servers in question have had issues coming back fully operational. The monitor I have on these is a SIP monitor. We are trying to see if there is a way for the F5 to periodically report the down condition. I agree that this should be handled elsewhere, not on the F5, but attempting to do my due diligence from an F5 perspective.

     

    Additionally, there is no VIP. I simply have the Nodes for the servers and then a Pool that they belong to. The sole purpose is for the F5 to monitor the servers and throw the syslog errors when they are not reachable.

     

    Thanks!

     

    Eric

    • jaikumar_f5's avatar
      jaikumar_f5
      Icon for MVP rankMVP
      Dec 11, 2019

      I think an external monitor would be the good solution for this. You have to write your script along with some logger, so that it constantly throws logs about the status. But it will flood your log file for x intervals.

       

      On good note, put the logging only for down state, for Up - do not put the logger.

  • EricTTEC's avatar
    EricTTEC
    Icon for Nimbostratus rankNimbostratus
    Dec 11, 2019

    Thanks again! I have not done this before, but I will look into this type of solution.

     

    Eric

  • 2_Jennifer's avatar
    2_Jennifer
    Icon for Nimbostratus rankNimbostratus
    Dec 12, 2019

    he issue has been that after the patching, the servers in question have had issues coming back fully operational. The monitor I have on these is a SIP monitor. We are trying to see if there is a way for the F5 to periodically report the down condition. telldunkin survey