Forum Discussion
John_Harrison
Altostratus
AltostratusNode UP and Node Down messages do not get written
I have 2 x 3400 BigIP LTM's running 9.4.7 - I have combed the threads here for someone with a similar issue - I've seen how to fix smtp and email alerts but no one seems to have the same issue where the basic monitors simply do not write to the logs when a node becomes unavailable.
The system knows it's down and marks it unavailable in the pool but no entries appear in any of the local logs - I've grepped them all for Node and DOWN, etc.
I can write a custom log and use logger -p local0.error "message" to manually write to the log but I do not want to create a custom log for every pool and every situation when there are built in monitors. Specifically I would like the tcp monitor to work correctly.
I hope this is not a common problem that I've simply overlooked, wasting people's time - on the other hand it would be great if it was as simple as "b db bigd.lognodestatuschange enable" if it solved my issue (I've tried this btw!).
Let me know what other information you require and I'll post whatever is necessary.
Thanks in advance.
12 Replies
- nitass
Employee
what is your log.tmm.level?
this is mine.
[root@ve1023:Active] config b db log.tmm.level
Log.Tmm.Level = Notice 
John_HarrisonMine is the same as your.
[root@c9lb01:Active] config b db log.tmm.level
Log.Tmm.Level = Notice- nitassdid you customize syslog-ng?
- John_HarrisonNot on purpose but previous sysadmins may have. We did change the syslog-ng.conf file to get things sent to a remote syslog server. However this issue preexisted that change. This was added to the end of the file.
destination remote_server {
udp("10.x.x.x" port (514));
};
filter f_alllogs {
level (debug...emerg);
};
log {
source(local);
filter(f_alllogs);
destination(remote_server);
};
It's likely related that we were unable to get remote syslog working through the proper commands, we followed the directions in the threads for properly setting it up - all changes had no effect until we added the above and restarted syslog-ng
I appreciate your attention to this matter! Thanks. - nitass
what about log.mcpd.level?
[root@ve1023:Active] config b db Log.Mcpd.Level
Log.Mcpd.Level = notice
have you rebooted unit? did it help?
by the way, about remote syslog, please make sure you don't modify syslog-ng.conf directly. it will be gone after rebooting.
LTM 9.4.2+: Custom Syslog Configuration by Deb
https://devcentral.f5.com/s/articles/LTM-9-4-2-Custom-Syslog-Configuration
- John_HarrisonThat is also set to "notice"
We understand about the remote syslog, but since it was the only way to get it working we know we need to change it when we cycle.
It's not been rebooted, I suppose we should plan a reboot over the holidays but it's heavily used 24/7 - before it went to prod we tested this and say only a 7 second outage when the secondary took over but it's still nerve racking to do this at anytime!
Ideally we can figure out what's wrong without needing the reboot? :) - nitassIdeally we can figure out what's wrong without needing the reboot? :)i have no idea yet. anyway, have you ever opened a support case and provide them qkview? support engineer might be able to find something suspicious in qkview.
- John_HarrisonUnfortunately I do not have active support. The intention is to replace these with the F5 virtual offering sometime in the near future but for now we are just ensuring the F5 solution fits our customer requirements.
I'll let this sit until the new year then we'll try doing some of the same configurations on the backup system, if it works we'll plan a failover and reboot the primary.
I'll post those results.
I'm confident now this is not a simple configuration issue I've overlooked thanks to your inquiry's. So a reboot does seem to be the logical next step. - nitassUnfortunately I do not have active support.every unit serial number has one free support case even there is no active contract. anyway, it will be treated as standard support i.e monday to friday daytime.
- John_HarrisonI have not yet rebooted but I have found the issue with my syslog-ng config not being set through the b command, a previous sysadmin had renamed the syslog-ng.conf link in the etc/syslog-ng/ directory. Setting that back the way it's supposed to be has put me to the same functionality as before without the risk of a reboot reset.
Since the previous sysadmin was making changes I'm wondering if the alert.conf file is correct.
The end of mine shows:
alert BIGIP_LOG_EMERG "^[0-9]{8}:0: (.*)" {
snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.29"
}
alert BIGIP_LOG_ALERT "^[0-9]{8}:1: (.*)" {
snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.30"
}
alert BIGIP_LOG_CRIT "^[0-9]{8}:2: (.*)" {
snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.31"
}
/* we not alert those two until we make sure the log level for
each message is accurate.
alert BIGIP_LOG_ERR "^[0-9]{8}:3: (.*)" {
snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.32"
}
alert BIGIP_LOG_WARNING "^[0-9]{8}:4: (.*)" {
snmptrap OID=".1.3.6.1.4.1.3375.2.4.0.33"
}
How does this determine what traps are logged?
Could someone post a clean alert.conf for me to compare with?
Thanks in advance.
