Forum Discussion

Abed_AL-R's avatar
Abed_AL-R
Icon for Cirrostratus rankCirrostratus
Oct 04, 2020

email alert notification not working when member came up again

Hello guys

i've configured this:

https://support.f5.com/csp/article/K3667

https://support.f5.com/csp/article/K59616664

I receive alerts when node goes down, but not when going up again

What could be the problem?

Oct  3 23:02:57 my.website.com notice mcpd[8459]: 01070640:5: Node /Common/172.17.70.18 address 172.17.70.18 monitor status down. [ /Common/icmp: down ]  [ was up for 24hrs:2mins:5sec ]
Oct  3 23:02:57 my.website.com notice mcpd[8459]: 01070640:5: Node /Common/172.17.70.19 address 172.17.70.19 monitor status down. [ /Common/icmp: down ]  [ was up for 24hrs:2mins:6sec ]
Oct  3 23:02:57 my.website.com notice mcpd[8459]: 01071682:5: SNMP_TRAP: Virtual /Common/dev-myweb has become unavailable
Oct  3 23:02:59 my.website.com notice mcpd[8459]: 01070728:5: Node /Common/172.17.70.18 address 172.17.70.18 monitor status up. [ /Common/icmp: up ]  [ was down for 0hr:0min:2sec ]
Oct  3 23:03:02 my.website.com notice mcpd[8459]: 01070728:5: Node /Common/172.17.70.19 address 172.17.70.19 monitor status up. [ /Common/icmp: up ]  [ was down for 0hr:0min:5sec ]


  • Hello Abed.

    Could you share your 'user_alert.conf' file?

    Maybe you could try this instead:

    alert NODE_UP "(.*) monitor status up" {
       ...
    }
     
    alert NODE_DOWN "(.*) monitor status down" {
       ...
    }

    Regards,

    Dario.

    • Abed_AL-R's avatar
      Abed_AL-R
      Icon for Cirrostratus rankCirrostratus

      This is my config:

      alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS_UP "Pool member (.*?) monitor status up."{
          snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.11";
      email toaddress="my@email.com"
      fromaddress="f5"
      body="A pool member went up!"
      }
      alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS "Node (.*?) monitor status down."{
          snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.12";
      email toaddress="my@email.com"
      fromaddress="f5"
      body="Please notice the status of the node!"
      }
      alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS_UP "Node (.*?) monitor status UP."{
          snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.13";
      email toaddress="my@email.com"
      fromaddress="f5"
      body="Please notice the status of the node!"
      }
      alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS "Pool member (.*?) monitor status down."{
          snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.10";
      email toaddress="my@email.com"
      fromaddress="f5"
      body="A pool member went down!"
      }

      How should I adjust it?

      • Hello Abed.

        First of all, you are using mixing existing tags with your custom expressions. I don't recommend you to do that.

        Sometimes, existing tags have some little mistakes. As you can see below one tag expression has an space between the IP and the address and the other not.

        # cat /etc/alertd/bigip_mcpd_error_maps.h | grep MON_STATUS | grep Node
        0 LOG_NOTICE    01070640 BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS "Node %s address %s monitor status %s."
        0 LOG_NOTICE    01070728 BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS_UP "Node %saddress %s monitor status up."

        My recommendation is to use your own definition of tags in the user_alert.conf. This expressions should work in your case.

        alert NODE_UP "Node (.*) monitor status up" {
           ...
        }
         
        alert NODE_DOWN "Node (.*) monitor status down" {
           ...
        }
         
        alert POOL_UP "Pool (.*) monitor status up" {
           ...
        }
         
        alert POOL_DOWN "Pool (.*) monitor status down" {
           ...
        }

        BTW, you have to pay attention carefully to the expression to not miss some character. Take into account dots and remove this '?' in your (.*?) regex.

        ## POOLS
        notice mcpd[2964]: 01070638:5: Pool <Pool_name> member <ServerIP_port> monitor status down [ <MonitorA_name>: down, <MonitorB_name>: down ] [ was up for <#>hrs:<#>mins:<#>sec ]
        notice mcpd[2964]: 01070727:5: Pool <Pool_name> member <ServerIP_port> monitor status up. [ <MonitorA_name>: down, <MonitorB_name>: up ] [ was down for <#>hrs:<#>mins:<#>sec ]
         
        ## NODES
        notice mcpd[2964]: 01070640:5: Node <ServerIP> monitor status down.
        notice mcpd[2964]: 01070728:5: Node <ServerIP> monitor status up.

        REF - https://support.f5.com/csp/article/K12531

        Please, don't forget to mark this answer as the best to help me for this contribution.

        Regards,

        Dario.