Forum Discussion
RobM
Cirrus
Cirrus29 hour cycle in status monitors
I found a weird thing in the logs. Each of the various pools on the f5 has a status monitor, reporting whether it is up or not. Each of those pool monitors drops out for a short period every 29 hou...
RobMCirrus
CirrusSeems to be just http monitors, but that is most of what we have. Here's some (sanitised) example log lines:
[root@bigip_2:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80' ltm* | sort
Mar 11 15:29:32 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ]
Mar 14 01:29:45 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/14 01:29:45. ] [ was up for 58hrs:0min:13sec ]
Mar 14 01:29:47 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:2sec ]
Mar 15 06:29:50 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/15 06:29:50. ] [ was up for 29hrs:0min:3sec ]
Mar 15 06:30:00 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ]
Mar 16 11:30:00 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/16 11:30:00. ] [ was up for 29hrs:0min:0sec ]
Mar 16 11:30:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:17sec ]
Mar 17 16:30:12 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/17 16:30:12. ] [ was up for 28hrs:59mins:55sec ]
Mar 17 16:30:21 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:9sec ]
Mar 18 21:30:17 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/18 21:30:17. ] [ was up for 28hrs:59mins:56sec ]
Mar 18 21:30:22 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:5sec ]
Mar 20 02:30:28 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/20 02:30:28. ] [ was up for 29hrs:0min:6sec ]
Mar 20 02:30:32 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:4sec ]
Mar 21 07:30:39 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/21 07:30:39. ] [ was up for 29hrs:0min:7sec ]
[root@bigip_1:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80' ltm* | sort
Mar 21 07:30:37 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/21 07:30:37. ] [ was up for 29hrs:0min:1sec ]
Mar 21 07:30:47 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ]
[root@bigip_2:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-sdmp-apollo-pool' ltm* | sort
Mar 12 21:38:12 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/12 21:38:12. ] [ was up for 86hrs:58mins:54sec ]
Mar 12 21:39:22 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ]
Mar 12 22:33:07 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/12 22:33:07. ] [ was up for 57hrs:58mins:47sec ]
Mar 12 22:34:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ]
Mar 15 07:38:14 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/15 07:38:14. ] [ was up for 57hrs:58mins:52sec ]
Mar 15 07:39:24 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ]
Mar 15 08:33:04 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/15 08:33:04. ] [ was up for 57hrs:58mins:47sec ]
Mar 15 08:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:14sec ]
Mar 16 12:38:14 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/16 12:38:14. ] [ was up for 28hrs:58mins:50sec ]
Mar 16 12:39:27 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ]
Mar 16 13:33:05 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/16 13:33:05. ] [ was up for 28hrs:58mins:47sec ]
Mar 16 13:34:20 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:15sec ]
Mar 17 17:38:15 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/17 17:38:15. ] [ was up for 28hrs:58mins:48sec ]
Mar 17 17:39:28 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ]
Mar 17 18:33:06 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/17 18:33:06. ] [ was up for 28hrs:58mins:46sec ]
Mar 17 18:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ]
Mar 18 22:38:16 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/18 22:38:16. ] [ was up for 28hrs:58mins:48sec ]
Mar 18 22:39:27 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ]
Mar 18 23:33:06 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/18 23:33:06. ] [ was up for 28hrs:58mins:48sec ]
Mar 18 23:34:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ]
Mar 20 03:38:17 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/20 03:38:17. ] [ was up for 28hrs:58mins:50sec ]
Mar 20 03:39:29 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ]
Mar 20 04:33:07 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/20 04:33:07. ] [ was up for 28hrs:58mins:50sec ]
Mar 20 04:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ]
Mar 21 08:38:18 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 08:38:18. ] [ was up for 28hrs:58mins:49sec ]
Mar 21 08:39:31 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ]
Mar 21 09:33:08 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 09:33:08. ] [ was up for 28hrs:58mins:50sec ]
[root@bigip_1:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-sdmp-apollo-pool' ltm* | sort
Mar 21 08:38:18 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 08:38:18. ] [ was up for 28hrs:58mins:49sec ]
Mar 21 08:39:30 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ]
Mar 21 09:33:09 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 09:33:09. ] [ was up for 28hrs:58mins:51sec ]
Mar 21 09:34:18 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:9sec ]