Email notification of node/vs down

Thanks hoolio. I'll give it a shot.

Hi Aaron,

 

I tried this sample and noticed the following:

 

1) Only BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS is available from the /var/tmpfs/run/bigip_error_maps.dat file.

 

2) Only OID=".1.3.6.1.4.1.3375.2.4.0.10" never got activated (not .11)

 

3) I received multiple emails for the notification. Is this normal?

 

 

Thanks,

 

 

/Robert

Hello,

is there any difference between the two posted alerts,

both use the same OIDs but have different descriptions,

will it be ok to leave the first ones in alter.conf and just add the last ones to the user_alter.conf?

    
    /*    
     * from bigd (CR36393) -- changed from mcpd messages in v9.2.0 (CR46190) and hotfix v9.0.5    
     * For the following 4 alerts, messages need to be used in pattern match to    
     * distiguish different ipaddr, port, etc. So, if these messages are changed,    
     * the changes in alert system need to be done accordingly.    
     */    
    alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_DOWN "Pool member (.*?).*?) monitor status down." {    
            snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.10"    
    }    
    alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_UP "Pool member (.*?).*?) monitor status up." {    
            snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.11"    
    }    
        
    alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_DOWN "Node (.*?) monitor status down." {    
            snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.12"    
    }    
    alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_UP "Node (.*?) monitor status up." {    
            snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.13"    
    }    
        
    

   
   alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS {   
   snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.10";   
   email toaddress="support@example.com"   
   fromaddress="root@bigip.example.com"   
   body="Pool Member Status Down!"   
   }   
   alert BIGIP_MCPD_MCPDERR_POOL_MEMBER_MON_STATUS_UP {   
   snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.11";   
   email toaddress="support@example.com"   
   fromaddress="root@bigip.example.com"   
   body="Pool Member Monitor Status Up"   
   }   
      
   alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS {   
   snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.12";   
   email toaddress="support@example.com"   
   fromaddress="root@bigip.example.com"   
   body="Node Address Monitor Status Down!"   
   }   
   alert BIGIP_MCPD_MCPDERR_NODE_ADDRESS_MON_STATUS_UP {   
   snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.13";   
   email toaddress="support@example.com"   
   fromaddress="root@bigip.example.com"   
   body="Node Address Monitor Status Up"   
   }   
   

Is there any way to insert the poolname into the email body?

Hello,

 

I got this working, I really would like it,

 

but today I received 600 Emails from the BigIP where I added the email alerts.

 

This happend 2 times.

 

 

Is there a reason for this behaviour ?

 

 

greetings

 

 

Heiko

Hey guys. I have this setup working for all our health monitors, but i would like to change 2 things. 1) Add/change hostname (instead of ip). 2) (and this is the most important) I want to include which health monitor has marked the node down.

 

In our setup we have some webservers with multiple virtualhosts where the same node is in multiple pools with different health monitors.

ingard,

 

 

I don't think either request is possible. There isn't a lookup done in the alerting process to see what node name might be configured for a particular pool member's address. And I don't think the monitoring daemon, bigd, includes the monitor name that marked the pool member down in the alert, so alertd wouldn't be able to include it in the alert message. You could open a case with F5 Support to find out if there are workarounds though.

 

 

Aaron

I just did, its too bad really as it would be very usefull to know which monitor did the flapping that occured "last night"

You could enable logging on bigd, but I think you'd get too much logging for everyday use if you have a large number of pools. To enable logging, you can run 'b db bigd.debug enable'. The output is written to /var/log/bigdlog. To disable the logging, run 'b db bigd.debug.disable'.

 

 

If you get more info on this from support, can you reply back?

 

 

Thanks,

 

Aaron

Willdo, that debug output is really nice. I wish i knew about that option last time i was debugging a health monitor. All output from the health monitors goes there - really nice when u're trying to see whats going on.

After beeing pointed towards numerous links in the kb I think we finally got a breakthrough. Excerpt - mail from f5 support:

 

 

----------------------

 

 

Here is my test pool with 2 monitors applied to pool member 10.0.0.12:80:

 

 

pool mspiggy {

 

lb method member ratio

 

min active members 1

 

monitor all http

 

members

 

10.0.0.10:http

 

priority 2

 

10.0.0.11:http

 

priority 2

 

10.0.0.12:http

 

priority 2

 

monitor http and http_test

 

10.0.0.13:http

 

priority 2

 

10.0.0.14:http

 

}

 

 

 

Enable database key: bigd.lognodestatuschange

 

 

[root@pegleg:Active] ucs b db bigd.lognodestatuschange enable

 

[root@pegleg:Active] ucs b db bigd.lognodestatuschange

 

Bigd.LogNodeStatusChange = enable

 

 

 

[root@pegleg:Active] ucs tail -f /var/log/ltm

 

 

1:- if only apply 1 monitor http to pool member, 10.0.0.12:80

 

 

Oct 6 14:22:54 pegleg mcpd[1486]: 01070639:5: Pool member 10.0.0.12:80 session status enabled.

 

Oct 6 14:22:54 pegleg bigd[1599]: 01060001:5: Service detected UP for ::ffff:10.0.0.12:80 monitor http.

 

Oct 6 14:22:55 pegleg mcpd[1486]: 01070727:5: Pool member 10.0.0.12:80 monitor status up.

 

 

...

 

2:- If applied 2 monitors (see above pool mspiggy configurations), http_test monitor failed, you can see:

 

 

Oct 6 14:23:14 pegleg bigd[1599]: 01060001:5: Service detected UP for ::ffff:10.0.0.12:80 monitor http.

 

Oct 6 14:23:25 pegleg bigd[1599]: 01060001:5: Service detected DOWN for ::ffff:10.0.0.12:80 monitor http_test.

 

Oct 6 14:23:25 pegleg mcpd[1486]: 01070638:5: Pool member 10.0.0.12:80 monitor status down.

 

Oct 6 14:23:28 pegleg mcpd[1486]: 01070639:5: Pool member 10.0.0.12:80 session status enabled.

 

 

You can configure alertd to use which monitor strings written down in /var/log/ltm, and you can define to send snmp trap for the strings you defined to monitor, see SOL3727

 

 

-----------------------------

 

 

ingard