cancel
Showing results for 
Search instead for 
Did you mean: 

email alert notification not working when member came up again

Abed_AL-R
Cirrostratus
Cirrostratus

Hello guys

 

i've configured this:

https://support.f5.com/csp/article/K3667

https://support.f5.com/csp/article/K59616664

 

I receive alerts when node goes down, but not when going up again

 

What could be the problem?

 

Oct  3 23:02:57 my.website.com notice mcpd[8459]: 01070640:5: Node /Common/172.17.70.18 address 172.17.70.18 monitor status down. [ /Common/icmp: down ]  [ was up for 24hrs:2mins:5sec ] Oct  3 23:02:57 my.website.com notice mcpd[8459]: 01070640:5: Node /Common/172.17.70.19 address 172.17.70.19 monitor status down. [ /Common/icmp: down ]  [ was up for 24hrs:2mins:6sec ] Oct  3 23:02:57 my.website.com notice mcpd[8459]: 01071682:5: SNMP_TRAP: Virtual /Common/dev-myweb has become unavailable Oct  3 23:02:59 my.website.com notice mcpd[8459]: 01070728:5: Node /Common/172.17.70.18 address 172.17.70.18 monitor status up. [ /Common/icmp: up ]  [ was down for 0hr:0min:2sec ] Oct  3 23:03:02 my.website.com notice mcpd[8459]: 01070728:5: Node /Common/172.17.70.19 address 172.17.70.19 monitor status up. [ /Common/icmp: up ]  [ was down for 0hr:0min:5sec ]
9 REPLIES 9

Hello Abed.

 

Could you share your 'user_alert.conf' file?

 

Maybe you could try this instead:

alert NODE_UP "(.*) monitor status up" { ... }   alert NODE_DOWN "(.*) monitor status down" { ... }

Regards,

Dario.

Regards,
Dario.

This is my config:

 

alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS_UP "Pool member (.*?) monitor status up."{ snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.11"; email toaddress="my@email.com" fromaddress="f5" body="A pool member went up!" } alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS "Node (.*?) monitor status down."{ snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.12"; email toaddress="my@email.com" fromaddress="f5" body="Please notice the status of the node!" } alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS_UP "Node (.*?) monitor status UP."{ snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.13"; email toaddress="my@email.com" fromaddress="f5" body="Please notice the status of the node!" } alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS "Pool member (.*?) monitor status down."{ snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.10"; email toaddress="my@email.com" fromaddress="f5" body="A pool member went down!" }

How should I adjust it?

 

Hello Abed.

 

First of all, you are using mixing existing tags with your custom expressions. I don't recommend you to do that.

 

Sometimes, existing tags have some little mistakes. As you can see below one tag expression has an space between the IP and the address and the other not.

# cat /etc/alertd/bigip_mcpd_error_maps.h | grep MON_STATUS | grep Node 0 LOG_NOTICE 01070640 BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS "Node %s address %s monitor status %s." 0 LOG_NOTICE 01070728 BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS_UP "Node %saddress %s monitor status up."

My recommendation is to use your own definition of tags in the user_alert.conf. This expressions should work in your case.

alert NODE_UP "Node (.*) monitor status up" { ... } alert NODE_DOWN "Node (.*) monitor status down" { ... }   alert POOL_UP "Pool (.*) monitor status up" { ... } alert POOL_DOWN "Pool (.*) monitor status down" { ... }

BTW, you have to pay attention carefully to the expression to not miss some character. Take into account dots and remove this '?' in your (.*?) regex.

## POOLS notice mcpd[2964]: 01070638:5: Pool <Pool_name> member <ServerIP_port> monitor status down [ <MonitorA_name>: down, <MonitorB_name>: down ] [ was up for <#>hrs:<#>mins:<#>sec ] notice mcpd[2964]: 01070727:5: Pool <Pool_name> member <ServerIP_port> monitor status up. [ <MonitorA_name>: down, <MonitorB_name>: up ] [ was down for <#>hrs:<#>mins:<#>sec ]   ## NODES notice mcpd[2964]: 01070640:5: Node <ServerIP> monitor status down. notice mcpd[2964]: 01070728:5: Node <ServerIP> monitor status up.

REF - https://support.f5.com/csp/article/K12531

 

Please, don't forget to mark this answer as the best to help me for this contribution.

 

Regards,

Dario.

Regards,
Dario.

Thank you.

I will update you if this works.

`Hi,

 

Sorry but still it is not working.

I have the same config in another machine and there its working.

The difference I noticed is that there I see "pool member" down/up, and here I see only "node" down although it is indeed a pool member and the monitor inherited from pool.

So two machines, same alert config, same ltm config, one working and one is not.

I'm not sure why

 

 

Hello Abed.

 

Have you restarted the alertd daemon?

tmsh restart /sys service alertd

 

Regards,

Dario.

Regards,
Dario.

Yes, of course.

If 'user_alert.conf' configuration is exactly the same in both devices, I recommend you to check some other basic stuff like DNS, SNMP or SMTP. I would try to test reachability of both devices and would check configuration as well.

 

A tcpdump fo checking if those packets are sending out of the box would also be a chance (for SMTP and SNMP Trap).

Regards,
Dario.

Hi Dario

So this has been resolved with F5 TAC

And this is the final conf:

alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS_UP { snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.11"; email toaddress="my@mail.com" fromaddress="root@f5mail.com" body="A pool member went up!" } alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS{ snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.12"; email toaddress="my@mail.com" fromaddress="root@f5mail.com" body="Please notice the status of the node!" } alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS_UP { snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.13"; email toaddress="my@mail.com" fromaddress="root@f5mail.com" body="Please notice the status of the node!" } alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS { snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.10"; email toaddress="my@mail.com" fromaddress="root@f5mail.com" body="A pool member went down!" }

Two things:

  • we changed the from address to a real address instead of just typing "root" or "f5"
  • we deleted all the (.*) or (.*?) that was going up there

 

Works like charm