Forum Discussion

pablo_juarez's avatar
pablo_juarez
Icon for Altocumulus rankAltocumulus
Sep 09, 2025
Solved

How to configure pool to go down if multiple members are down

Hello community,

I have a requirement related to pool health and its impact on BGP announcements. By default, a pool in BIG-IP is considered up as long as at least one member is still healthy. However, in my case, I need the pool to be marked down if a certain number of members are unhealthy.

For example:

Suppose I have a pool with 10 nodes.
I would like the pool to be considered down if 5 (or more) of those nodes are marked down.
The purpose is to ensure that when the pool is in this degraded state, the associated virtual server is also marked down, so that the VIP is no longer advertised via BGP.

In some specific cases, I have already applied monitors at the individual node level and configured the minimum number of monitors that must be available. While this works for isolated scenarios, I am looking for a more generic, scalable, and easy-to-maintain approach that could be applied across pools.

Has anyone implemented this type of behavior?

Is there a native configuration option in BIG-IP to achieve this?
Or would it require an external monitor script / custom solution?

Any guidance or best practices would be appreciated.

Thanks in advance!

  • Priority group activation: if at least one node is up, the pool will appear up, and therefore the virtual server will also show as green.
    iRule: this would work to reject traffic, but it’s not what I need.
    Individual health checks: this is how it’s currently working. It’s fine for a small pool, but I’m looking for something more scalable.

    Just to recap, the goal is to mark the virtual server as down so that this IP is not advertised via BGP. This way, traffic will be routed to another site where the VS has more nodes available. The issue was detected when a pool had fewer than 6 nodes available, since it was unable to handle the amount of DNS requests it was receiving.

    It seems I’ve managed to get it working with this solution, something like Injeyan_Kostas suggested:
    I created a new pool with an external monitor that checks the status of the original pool. On the VS, I applied this new pool and it appears to work correctly.

    It doesn’t seem like the most optimal solution, but at least if there are many nodes or they change regularly, not many modifications are required.

    Any other suggestions would be welcome.

     

    This is .sh file:

    #!/bin/bash
    #
    # External monitor genérico para marcar DOWN un pool si tiene menos de N miembros UP
    #

    # Variables de entorno pasadas desde el monitor
    #Ej.-
    #ltm monitor external monitor_externo_min_pool_member {
    #    defaults-from external
    #    description "Comprobacion de miembros arriba en un pool"
    #    interval 5
    #    run /Common/monitor_pool_min_members
    #    time-until-up 0
    #    timeout 16
    #    user-defined MIN_UP 3
    #    user-defined POOL Pool_A10_80
    #}


    POOL=$POOL
    MIN_UP=$MIN_UP

    # Obtenemos número de miembros activos del pool
    UP=$(tmsh show ltm pool $POOL | grep "Current Active Members" | awk '{print $5}')

    # Debug opcional en /var/log/ltm
    #logger -p local0.info "Monitor $POOL: $UP miembros UP, mínimo requerido $MIN_UP"

    if [ "$UP" -lt "$MIN_UP" ]; then
      # No mandamos nada a stdout -> el monitor marcará DOWN
      exit 1
    else
      # Mandamos algo (ej: "UP") -> el monitor marcará UP
      echo "UP"
      exit 0
    fi

     

    This is original pool:

    ltm pool Pool_A10_80 {
        description "Pool que son 4 VS en A10 2-pruebasPart con iRUle 200 OK"
        members {
            N1:http {
                address 10.10.100.1
                session monitor-enabled
                state up
            }
            N2:http {
                address 10.10.100.2
                session monitor-enabled
                state up
            }
            N3:http {
                address 10.10.100.3
                session monitor-enabled
                state up
            }
            N4:http {
                address 10.10.100.4
                session monitor-enabled
                state up
            }
        }
        monitor tcp
    }

     

    This is the new pool assigned to VS:

    ltm pool Pool_min_members {
        description "Pool de chequeo de numero de miembros disponibles"
        members {
            N1:http {
                address 10.10.100.1
                session monitor-enabled
                state down
            }
            N2:http {
                address 10.10.100.2
                session monitor-enabled
                state down
            }
            N3:http {
                address 10.10.100.3
                session monitor-enabled
                state down
            }
            N4:http {
                address 10.10.100.4
                session monitor-enabled
                state down
            }
        }
        monitor monitor_externo_min_pool_member
    }

     

     

10 Replies

  • Hello pablo_juarez​ 

    This is only achievable via external monitor
    you can use a script like this

    #!/bin/bash
    POOL="Your_Pool_name"
    THRESHOLD=5
    
    UP_COUNT=$(tmsh show ltm pool $POOL members | grep "Available Members" | awk '{print $4}')
    
    echo "[$(date)] Pool $POOL has $UP_COUNT members up" >> /var/log/pool_monitor.log
    
    if [ "$UP_COUNT" -lt "$THRESHOLD" ]; then
        echo "[$(date)] Pool DOWN: Only $UP_COUNT members up" >> /var/log/pool_monitor.log
        exit 1
    else
        echo "[$(date)] Pool UP: $UP_COUNT members up" >> /var/log/pool_monitor.log
        echo "Pool is healthy" 
        exit 0
    fi

     

  • I was working on this external monitor:

    #!/bin/bash
    #
    # External monitor genérico para marcar DOWN un pool si tiene menos de N miembros UP
    #

    # Variables de entorno pasadas desde el monitor
    POOL=${POOL}
    MIN_UP=${MIN_UP:-1}   # por defecto al menos 1

    # Obtenemos número de miembros activos del pool
    UP=$(tmsh show ltm pool $POOL | grep "Current Active Members" | awk '{print $5}')

    # Debug opcional en /var/log/ltm
    logger -p local0.info "Monitor $POOL: $UP miembros UP, mínimo requerido $MIN_UP"

    if [ "$UP" -lt "$MIN_UP" ]; then
      # No mandamos nada a stdout -> el monitor marcará DOWN
      exit 1
    else
      # Mandamos algo (ej: "UP") -> el monitor marcará UP
      echo "UP"
      exit 0
    fi

     

    Is it necessary to use at least 2 monitors?
    How can the monitor check itself?

    I would think that another monitor is needed to verify is members are up or down.

    • Injeyan_Kostas's avatar
      Injeyan_Kostas
      Icon for Nacreous rankNacreous

      Indeed this would end in an infinite loop because pool would never recover
      you should have one pool with specific monitor to mark each memeber up or down
      then create another pool with same members memebers and assign 2 monitors, one application specific and the external one

      note that in extenral monitor you should add the first pool and in VS use the second pool

  • Hi pablo_juarez​,

    you can also achieve this with pool member specific monitors. Here is an example with 4 pool members, if 2 or more members go down, the whole pool will go down.

    Monitors config, each monitor has a custom destination - its own IP and port.

    ltm monitor http mon_http_10_100_153_120 {
        adaptive disabled
        defaults-from http
        destination 10.100.153.120:http
        interval 5
        ip-dscp 0
        recv none
        recv-disable none
        send "GET /\r\n"
        time-until-up 0
        timeout 16
    }
    ltm monitor http mon_http_10_100_153_121 {
        adaptive disabled
        defaults-from http
        destination 10.100.153.121:http
        interval 5
        ip-dscp 0
        recv none
        recv-disable none
        send "GET /\r\n"
        time-until-up 0
        timeout 16
    }
    ltm monitor http mon_http_10_100_153_122 {
        adaptive disabled
        defaults-from http
        destination 10.100.153.122:http
        interval 5
        ip-dscp 0
        recv none
        recv-disable none
        send "GET /\r\n"
        time-until-up 0
        timeout 16
    }
    ltm monitor http mon_http_10_100_153_123 {
        adaptive disabled
        defaults-from http
        destination 10.100.153.123:http
        interval 5
        ip-dscp 0
        recv none
        recv-disable none
        send "GET /\r\n"
        time-until-up 0
        timeout 16
    }

     And here goes the pool config, each member has 4 monitors assigned. This way, if two members go down, all pool members go down.

    ltm pool pl_custom_monitors {
        members {
            10.100.153.120:http {
                address 10.100.153.120
                monitor min 3 of { mon_http_10_100_153_120 mon_http_10_100_153_121 mon_http_10_100_153_122 mon_http_10_100_153_123 }
                session monitor-enabled
                state up
            }
            10.100.153.121:http {
                address 10.100.153.121
                monitor min 3 of { mon_http_10_100_153_120 mon_http_10_100_153_121 mon_http_10_100_153_122 mon_http_10_100_153_123 }
                session monitor-enabled
                state up
            }
            10.100.153.122:http {
                address 10.100.153.122
                monitor min 3 of { mon_http_10_100_153_120 mon_http_10_100_153_121 mon_http_10_100_153_122 mon_http_10_100_153_123 }
                session monitor-enabled
                state up
            }
            10.100.153.123:http {
                address 10.100.153.123
                monitor min 3 of { mon_http_10_100_153_120 mon_http_10_100_153_121 mon_http_10_100_153_122 mon_http_10_100_153_123 }
                session monitor-enabled
                state up
            }
        }
    }

     

    Cheers

    Daniel

    • pablo_juarez's avatar
      pablo_juarez
      Icon for Altocumulus rankAltocumulus

      Hello Daniel,

      I have used this option in small pools, but I was thinking in something more scalable.

      Thanks anyway.

      • Daniel_Wolf's avatar
        Daniel_Wolf
        Icon for MVP rankMVP

        I have two more ideas, though not tested.
        1. Add all 10 members to a priority group, configure a minimum up requirement of "Less than..." 6 for this priority group. This should take down the whole priority group.
        2. Use an iRule similar to this one:

        when LB_SELECTED {
            set active_members [active_members [LB::server pool]]
            if { $active_members < 6 } {
                reject
            }
        }

         

  • I havent tested these 2 options that dont need irules, but they might work.

    Option1:

    Use pool member priority group with Activation less than X.

    Assign a high priority value, e.g. 99, to real pool members.
    Add dummy dead pool member and assign any lower prio value, e.g. 22

    Expectation: When less than X pool members up, vserver will switch to the dummy dead member
    but because it's dead then vserver status will be dead too.

    Also try without dummy dead.

     

    Option2 (it seems will work):

    Use pool's Availability Requirement = At least X,

    Create health monitor for each pool member
    and specify intended pool member ip addr and port in each of those health monitor's Alias address & Alias port.
    Then assign those health monitors into the pool.

     

     

  • Priority group activation: if at least one node is up, the pool will appear up, and therefore the virtual server will also show as green.
    iRule: this would work to reject traffic, but it’s not what I need.
    Individual health checks: this is how it’s currently working. It’s fine for a small pool, but I’m looking for something more scalable.

    Just to recap, the goal is to mark the virtual server as down so that this IP is not advertised via BGP. This way, traffic will be routed to another site where the VS has more nodes available. The issue was detected when a pool had fewer than 6 nodes available, since it was unable to handle the amount of DNS requests it was receiving.

    It seems I’ve managed to get it working with this solution, something like Injeyan_Kostas suggested:
    I created a new pool with an external monitor that checks the status of the original pool. On the VS, I applied this new pool and it appears to work correctly.

    It doesn’t seem like the most optimal solution, but at least if there are many nodes or they change regularly, not many modifications are required.

    Any other suggestions would be welcome.

     

    This is .sh file:

    #!/bin/bash
    #
    # External monitor genérico para marcar DOWN un pool si tiene menos de N miembros UP
    #

    # Variables de entorno pasadas desde el monitor
    #Ej.-
    #ltm monitor external monitor_externo_min_pool_member {
    #    defaults-from external
    #    description "Comprobacion de miembros arriba en un pool"
    #    interval 5
    #    run /Common/monitor_pool_min_members
    #    time-until-up 0
    #    timeout 16
    #    user-defined MIN_UP 3
    #    user-defined POOL Pool_A10_80
    #}


    POOL=$POOL
    MIN_UP=$MIN_UP

    # Obtenemos número de miembros activos del pool
    UP=$(tmsh show ltm pool $POOL | grep "Current Active Members" | awk '{print $5}')

    # Debug opcional en /var/log/ltm
    #logger -p local0.info "Monitor $POOL: $UP miembros UP, mínimo requerido $MIN_UP"

    if [ "$UP" -lt "$MIN_UP" ]; then
      # No mandamos nada a stdout -> el monitor marcará DOWN
      exit 1
    else
      # Mandamos algo (ej: "UP") -> el monitor marcará UP
      echo "UP"
      exit 0
    fi

     

    This is original pool:

    ltm pool Pool_A10_80 {
        description "Pool que son 4 VS en A10 2-pruebasPart con iRUle 200 OK"
        members {
            N1:http {
                address 10.10.100.1
                session monitor-enabled
                state up
            }
            N2:http {
                address 10.10.100.2
                session monitor-enabled
                state up
            }
            N3:http {
                address 10.10.100.3
                session monitor-enabled
                state up
            }
            N4:http {
                address 10.10.100.4
                session monitor-enabled
                state up
            }
        }
        monitor tcp
    }

     

    This is the new pool assigned to VS:

    ltm pool Pool_min_members {
        description "Pool de chequeo de numero de miembros disponibles"
        members {
            N1:http {
                address 10.10.100.1
                session monitor-enabled
                state down
            }
            N2:http {
                address 10.10.100.2
                session monitor-enabled
                state down
            }
            N3:http {
                address 10.10.100.3
                session monitor-enabled
                state down
            }
            N4:http {
                address 10.10.100.4
                session monitor-enabled
                state down
            }
        }
        monitor monitor_externo_min_pool_member
    }