Forum Discussion
RobM
Cirrus
Cirrus29 hour cycle in status monitors
I found a weird thing in the logs.
Each of the various pools on the f5 has a status monitor, reporting whether it is up or not.
Each of those pool monitors drops out for a short period every 29 hours, give-or-take a few seconds. (Mostly. Occasionally they’ll miss one, and drop out at the next multiple of 29 hours…) They stay down for a handful of seconds – up to a minute or so - then come back up for another 29 hours.
When the monitor sees a pool drop out, it sees it drop out from both f5s simultaneously (makes sense, but just checking…)
The different pools drop out at different times (so its presumably not the whole network going on the blink…)
Even though its less than a minute a day, I feel like our other status monitoring would be shouting at us if we were actually losing the service, so I wondered if we could be losing the monitor, not the pool. One of our sysadmins pointed out to me that 29 hours is the default Microsoft IIS recycling interval. Is the ststus monitor perhaps using Microsoft IIS somehow?
Reading up on this it does seem like if IIS is using the default 29 hour recycle interval this would be the expected behavior. A suggestion could be to look into scheduling the recycle to occur during a maintenance period, or tune the health check interval to check a little more slowly and allow for the app service to restart without failing the check.
Hi RobM ,
Can you share any details on what health monitors you are using? Does this apply to tcp/tcp-half-open monitors or others like http/https?
Thanks,
Josh
RobMSeems to be just http monitors, but that is most of what we have. Here's some (sanitised) example log lines:
[root@bigip_2:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80' ltm* | sort Mar 11 15:29:32 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ] Mar 14 01:29:45 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/14 01:29:45. ] [ was up for 58hrs:0min:13sec ] Mar 14 01:29:47 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:2sec ] Mar 15 06:29:50 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/15 06:29:50. ] [ was up for 29hrs:0min:3sec ] Mar 15 06:30:00 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ] Mar 16 11:30:00 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/16 11:30:00. ] [ was up for 29hrs:0min:0sec ] Mar 16 11:30:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:17sec ] Mar 17 16:30:12 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/17 16:30:12. ] [ was up for 28hrs:59mins:55sec ] Mar 17 16:30:21 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:9sec ] Mar 18 21:30:17 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/18 21:30:17. ] [ was up for 28hrs:59mins:56sec ] Mar 18 21:30:22 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:5sec ] Mar 20 02:30:28 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/20 02:30:28. ] [ was up for 29hrs:0min:6sec ] Mar 20 02:30:32 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:4sec ] Mar 21 07:30:39 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/21 07:30:39. ] [ was up for 29hrs:0min:7sec ] [root@bigip_1:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80' ltm* | sort Mar 21 07:30:37 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/21 07:30:37. ] [ was up for 29hrs:0min:1sec ] Mar 21 07:30:47 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ] [root@bigip_2:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-sdmp-apollo-pool' ltm* | sort Mar 12 21:38:12 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/12 21:38:12. ] [ was up for 86hrs:58mins:54sec ] Mar 12 21:39:22 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ] Mar 12 22:33:07 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/12 22:33:07. ] [ was up for 57hrs:58mins:47sec ] Mar 12 22:34:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ] Mar 15 07:38:14 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/15 07:38:14. ] [ was up for 57hrs:58mins:52sec ] Mar 15 07:39:24 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ] Mar 15 08:33:04 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/15 08:33:04. ] [ was up for 57hrs:58mins:47sec ] Mar 15 08:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:14sec ] Mar 16 12:38:14 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/16 12:38:14. ] [ was up for 28hrs:58mins:50sec ] Mar 16 12:39:27 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ] Mar 16 13:33:05 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/16 13:33:05. ] [ was up for 28hrs:58mins:47sec ] Mar 16 13:34:20 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:15sec ] Mar 17 17:38:15 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/17 17:38:15. ] [ was up for 28hrs:58mins:48sec ] Mar 17 17:39:28 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ] Mar 17 18:33:06 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/17 18:33:06. ] [ was up for 28hrs:58mins:46sec ] Mar 17 18:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ] Mar 18 22:38:16 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/18 22:38:16. ] [ was up for 28hrs:58mins:48sec ] Mar 18 22:39:27 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ] Mar 18 23:33:06 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/18 23:33:06. ] [ was up for 28hrs:58mins:48sec ] Mar 18 23:34:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ] Mar 20 03:38:17 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/20 03:38:17. ] [ was up for 28hrs:58mins:50sec ] Mar 20 03:39:29 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ] Mar 20 04:33:07 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/20 04:33:07. ] [ was up for 28hrs:58mins:50sec ] Mar 20 04:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ] Mar 21 08:38:18 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 08:38:18. ] [ was up for 28hrs:58mins:49sec ] Mar 21 08:39:31 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ] Mar 21 09:33:08 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 09:33:08. ] [ was up for 28hrs:58mins:50sec ] [root@bigip_1:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-sdmp-apollo-pool' ltm* | sort Mar 21 08:38:18 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 08:38:18. ] [ was up for 28hrs:58mins:49sec ] Mar 21 08:39:30 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ] Mar 21 09:33:09 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 09:33:09. ] [ was up for 28hrs:58mins:51sec ] Mar 21 09:34:18 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:9sec ]- RobM
Its http monitors that I'm seeing it with, but then thats most of what we have on the system. Here's some sample log data (redacted) so you can see what I'm talking about:
[root@bigip_2:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80' ltm* | sort Mar 11 15:29:32 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ] Mar 14 01:29:45 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/14 01:29:45. ] [ was up for 58hrs:0min:13sec ] Mar 14 01:29:47 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:2sec ] Mar 15 06:29:50 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/15 06:29:50. ] [ was up for 29hrs:0min:3sec ] Mar 15 06:30:00 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ] Mar 16 11:30:00 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/16 11:30:00. ] [ was up for 29hrs:0min:0sec ] Mar 16 11:30:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:17sec ] Mar 17 16:30:12 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/17 16:30:12. ] [ was up for 28hrs:59mins:55sec ] Mar 17 16:30:21 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:9sec ] Mar 18 21:30:17 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/18 21:30:17. ] [ was up for 28hrs:59mins:56sec ] Mar 18 21:30:22 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:5sec ] Mar 20 02:30:28 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/20 02:30:28. ] [ was up for 29hrs:0min:6sec ] Mar 20 02:30:32 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:4sec ] Mar 21 07:30:39 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/21 07:30:39. ] [ was up for 29hrs:0min:7sec ] [root@bigip_1:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80' ltm* | sort Mar 21 07:30:37 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status down. [ /NZP/nzp-aspxrptgen-http: down; last error: /NZP/nzp-aspxrptgen-http: Response Code: 200 (OK) @2022/03/21 07:30:37. ] [ was up for 29hrs:0min:1sec ] Mar 21 07:30:47 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-aspxrptgen-pool member /NZP/server_1:80 monitor status up. [ /NZP/nzp-aspxrptgen-http: up ] [ was down for 0hr:0min:10sec ] [root@bigip_2:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-sdmp-apollo-pool' ltm* | sort Mar 12 21:38:12 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/12 21:38:12. ] [ was up for 86hrs:58mins:54sec ] Mar 12 21:39:22 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ] Mar 12 22:33:07 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/12 22:33:07. ] [ was up for 57hrs:58mins:47sec ] Mar 12 22:34:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ] Mar 15 07:38:14 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/15 07:38:14. ] [ was up for 57hrs:58mins:52sec ] Mar 15 07:39:24 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:10sec ] Mar 15 08:33:04 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/15 08:33:04. ] [ was up for 57hrs:58mins:47sec ] Mar 15 08:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:14sec ] Mar 16 12:38:14 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/16 12:38:14. ] [ was up for 28hrs:58mins:50sec ] Mar 16 12:39:27 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ] Mar 16 13:33:05 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/16 13:33:05. ] [ was up for 28hrs:58mins:47sec ] Mar 16 13:34:20 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:15sec ] Mar 17 17:38:15 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/17 17:38:15. ] [ was up for 28hrs:58mins:48sec ] Mar 17 17:39:28 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ] Mar 17 18:33:06 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/17 18:33:06. ] [ was up for 28hrs:58mins:46sec ] Mar 17 18:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ] Mar 18 22:38:16 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/18 22:38:16. ] [ was up for 28hrs:58mins:48sec ] Mar 18 22:39:27 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ] Mar 18 23:33:06 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/18 23:33:06. ] [ was up for 28hrs:58mins:48sec ] Mar 18 23:34:17 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ] Mar 20 03:38:17 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/20 03:38:17. ] [ was up for 28hrs:58mins:50sec ] Mar 20 03:39:29 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ] Mar 20 04:33:07 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/20 04:33:07. ] [ was up for 28hrs:58mins:50sec ] Mar 20 04:34:18 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:11sec ] Mar 21 08:38:18 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 08:38:18. ] [ was up for 28hrs:58mins:49sec ] Mar 21 08:39:31 bigip_2 notice mcpd[7758]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:13sec ] Mar 21 09:33:08 bigip_2 notice mcpd[7758]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 09:33:08. ] [ was up for 28hrs:58mins:50sec ] [root@bigip_1:Active:Standalone] log # zgrep -hi ' Pool /NZP/nzp-sdmp-apollo-pool' ltm* | sort Mar 21 08:38:18 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 08:38:18. ] [ was up for 28hrs:58mins:49sec ] Mar 21 08:39:30 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_2:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:12sec ] Mar 21 09:33:09 bigip_1 notice mcpd[6631]: 01070638:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status down. [ /Common/http: down; last error: /Common/http: Unable to connect @2022/03/21 09:33:09. ] [ was up for 28hrs:58mins:51sec ] Mar 21 09:34:18 bigip_1 notice mcpd[6631]: 01070727:5: Pool /NZP/nzp-sdmp-apollo-pool member /NZP/server_3:80 monitor status up. [ /Common/http: up ] [ was down for 0hr:1min:9sec ]
