We have a GTM+LTM set up for our application which is running on 12 servers. This servers are separated in 4 LTMs with 3 servers each with monitoring set to a static page and "Action on service down" set to "None". We want to have a zero downtime deployment set up and currently we do it like this:
Mark half of the servers as down (results in 2 LTM having only down servers) but keep them running for as long as we can detect running requests
Deploy to this half and mark them as up
Mark the second half as down and deploy
Even after all of this some of our users are complaining about dropped requests when we deploy. From the logs https://krogerfeedback.nl https://talktosonic.onl https://talktowendys.vip https://whataburgersurvey.onl i see that the requests are being dropped immediately after we mark the servers as down even though they are still running and" Action on service down" is set to None.
So my question is might this be related with GTM marking the whole LTM pool as down and dropping all the running requests?
If the action on service down is set to none, then you should be able to bleed active and persistent connections, depending on what the pool members are set to. If GTM is using the bigip monitor for your LTMs, then new DNS lookups will lead to a different answer and thus a different VIP than the previous answer, but that should only impact new DNS lookups by your client's name servers.
If I were troubleshooting this actively, I'd take a packet capture on a client machine where the behavior is repeatable AND the BIG-IP where the servers are being shut down.
