Dynamic Intelligent Application Delivery in a Distributed Environment–Part 2

Designing for Fine Grained Control and Monitor in a Load Balanced Environment – Gathering Information

If you haven’t read Dynamic Intelligent Application Delivery Part 1, start there first, as this is not a standalone article and won’t make much sense without proper context. All caught up now? Splendid! We use separate apps and a number of protocols to gather information from both the load balancing devices (F5 BigIP) and pool members (SunONE Directory) to aid in the dispatching of clients to pools and pool members. By necessity the apps are specific to the devices and servers being monitored but could be readily adapted to gather information from other network devices (Such as firewalls) or servers (Such as Apache HTTP, Tomcat, or Weblogic).

ldapReporter

The ldapReporter perl application (Codeshare entry: ldapReporter) is a small server dedicated app that runs on the pool member^{^[1]} and reports pool member ‘load’ to the database. Pool member load (weight) is calculated as

weight=1000-$currentConnections - $backlog*10 - $waiters*10;

and clamped to the range 0 - 1000 (Where 1000 = no load and 0 = fully loaded)

where

These stats are all gathered from the slapd process using a search on the ‘cn=monitor’ base

The stats are presented to the database in the f5_poolmember table with the SQL

update f5_poolmember set dynamicratio=?,dnrsettime=? where ipaddr=? and portnum=?

where

dynamicratio=weight

dnrsettime=current epoch time

ipaddr and portnum are provided on the commandline to ldapReporter. Once provided, the app sleeps again for an interval defined on the command line (The default interval is 60 seconds).

^{^[1]} Although this is not a hard & fast rule. The ldapReporter app could run anywhere that has LDAP access to the poolmember being monitored as stats are gathered via LDAP over the network interface.

walkF5Stats

walkF5Stats is a medium sized perl application (Codeshare entry: walkF5Stats) that runs in a loop, and every 20 seconds (Configurable on the command line) polls a cluster of BigIP devices for statistics (Both system and those associated with the current configuration). Statistics are gathered via both iControl and SNMP.

The walkF5Stats application updates both the database and the RRD’s used to keep historical stats. The stats gathered include the areas of Pools, PoolMembers, Virtual Servers, Networks & Interfaces, CPU & Memory, Profiles (SSL), HA and configuration sync status.

Note: walkF5Stats is specific to F5 BigIP appliances

Pools - Gather pool availability, traffic and connection stats
PoolMembers - Gather poolmember availability, traffic and connection stats
Virtual Servers - Gather virtual server availability, traffic and connection stats
CPU & memory - Dependent on the BigIP version, gather memory and CPU counters for the host and individual CPU’s.
SSL Profiles - Gather statistics (counters) for SSL profiles for hardware of software offloading of SSL connections
Networks & Interfaces - Gather Byte counts and availability of networks and interfaces. Including VLAN’s and aggregated links
HA & Config Sync Status - Gather the HA status (Active or Standby) and configuration sync status (In sync, local or remote changed) for saving in the DB

Appendix – Database Tables

db_state_engine table

The db_state_engine table is the repository for the SE checks to be performed for the current running operational mode, and the actions involved based on the stateCheck and stateCheckTarget. The SQL access is by dblbsvcid (Service ID) and f5vsid (VS ID) sorted on priority for the current running operational mode (referenced from db_lbsvccluster.lbcurmodeind).

db_service_action table

db_service_action describes the actions to be taken in response to the flexible state engine checks, and also for operational mode changes (For both the fixed function and flexible state engine).

db_ll_action table

The db_ll_action table is used to describe multiple small actions that are part of a single larger service action. For example, when executing a CLASS_KEYVALUE action, the db_ll_action table describes the actual key/value pairs to be altered. Other service_actions also use the db_ll_action table to store parameters required for their specific actions (e.g. increment and decrement values, default pool names etc)