iRule Maintenance Windows
A fun, but not well known, feature on BIG-IP is the Statistics Profile. This tech tip is the second in a series on how the Statistics Profile and iRules, when working together, can save time, productivity, and your sleep!
The Scenario:
Imagine yourself, Mr./Mrs. IT Manager, working for a company that's business relies on updated content on it's back-end web servers. Your company has a web team that has decided that it's test cycle will always end at 3:00AM (your time) on Saturday morning. So, here you are, arriving at home after a night of wild karaoke and one too many drinks with cheap umbrellas and you get a call from the dev team that they need the site taken off line so they can push out the new content.
In the past, you gave the dev lead the keys to the castle (meaning the admin credentials for your BIG-IP) and he/she somehow thought "rm -rf /" was how you took a virtual server offline. After an emergency restore of the software you vowed to never give that department direct access to your servers - ever... So, now you are stuck with being on call whenever they deem updates are needed. If only, you could provide your dev team access to control taking their application offline while they are doing updates. Oh, and they should also be able to view the current status as well as being able to bring it back online.
Never fear, help is on the way! There are a couple of handy features within iRules that make this problem a thing of the past.
The Setup:
To get things going, you'll need a Virtual Server setup fronting your web application. Since your application is up and running, that step is already taken care of. Next, you'll need to create a Statistics Profile, assign it to your Virtual Server and write an iRule. So, first we'll create the statistics profile...
Create the Statistics Profile
- Login to the BIG-IP Administrative GUI.
- Select the Profiles option under Local Traffic and Virtual Servers
- Select the Statistics from the Other menu.
- Click the Create button
- Enter "maintenance_window" for the Profile name
- Add the following fields: day, start_time, end_time
- Click Update to create the profile
Create the iRule
Now that the virtual server is setup with the statistics profile, you'll need to create an iRule to to add the controlling functions as well as the maintenance window enforcement.
- Select Rules from the Local Traffic/Virtual Servers menu.
- Click the Create button.
- Enter "maintenance_window" for the iRule name and enter the iRule from below into the Definition text box.
- Click Finished to save the iRule
Apply the statistics profile to your virtual server
The statistics profile doesn't do much until you apply it to your virtual server properties. Here's how:
- Select Virtual Servers from the Local Traffic menu.
- Click on your Virtual Server to enter it's properties
- Make sure the Advanced Configuration option is selected, and scroll down to the Statistics Profile option
- Select the previously created profile maintenance_window
- Click the Update Button
- Select the Resources Menu
- Click the Manage Rules button
- Make sure the maintenance_window iRule is in the Enabled list box and click the Finished button.
The iRule
Now for the fun, here's the iRule that will give you peace, tranquility (and a good nights sleep). This iRule will look for commands in the URI to control the maintenance window settings (see the "/usage" section for details).
when HTTP_REQUEST {
set TITLE "iRule Maintenance Window Control"
set PROFILE_NAME "maintenance_window"
# Look for embedded commands to control the maintenance window.
switch -glob [string tolower [HTTP::uri]] {
"/enable*" {
# Extract the maintenance window day, start time, and end time from the URI.
# If not there, return an error message.
set params [split [HTTP::uri] "/"]
if { [llength $params] < 5 } {
HTTP::respond 200 content "
<head><center><title>$TITLE</title>
<body><h1>Usage: http://[HTTP::host]/enable/day_number/start_hour/end_hour</h1>
</body></html>"
} else {
# Update the maintenance window values in the statistics profile.
STATS::set $PROFILE_NAME "day" [lindex $params 2]
STATS::set $PROFILE_NAME "start_time" [lindex $params 3]
STATS::set $PROFILE_NAME "end_time" [lindex $params 4]
HTTP::respond 200 content "
<head><center><title>$TITLE</title>
<body><h1>Maintenance Window enabled</h1>
</body></html>"
}
}
"/disable" {
# By zero'ing out the values in the statistics profile, the maintenance window is not enforced.
STATS::set $PROFILE_NAME "day" 0
STATS::set $PROFILE_NAME "start_time" 0
STATS::set $PROFILE_NAME "end_time" 0
HTTP::respond 200 content "
<head><center><title>$TITLE</title>
<body><h1>Maintenance Window disabled</h1>
</body></html>"
}
"/check" {
# Return the current time and the current settings for the maintenance window
set day [STATS::get $PROFILE_NAME "day"]
set start_time [STATS::get $PROFILE_NAME "start_time"]
set end_time [STATS::get $PROFILE_NAME "end_time"]
set l [split [clock format [clock seconds] -format {%u %H %M}] " "]
set cur_day [lindex $l 0]
set cur_time [expr [expr [lindex $l 1]*100] + [lindex $l 2]]
HTTP::respond 200 content "
<head><center><title>$TITLE</title>
<body><h1>Current Date and time: $cur_day, $cur_time</h1>
<h1>Maintenance Window: $day, $start_time - $end_time</h1>
</body></html>"
}
"/usage" {
# Usage instructions for the commands
HTTP::respond 200 content "
<head><center><title>$TITLE</title>
<body><h1>Usage: http://[HTTP::host]/\[command\]</h1>
<table><tr><td><ul>
<li>/enable/day_of_week_number(1:mon-7:sun)/start_time(hhmm)/end_time(hhmm) - enable maintenance window.</li>
<li>/disable - disable maintenance window.</li>
<li>/check - check if in maintenance window.</li>
<li>/usage - display this usage message</li>
</ul></td></tr></table>
</body></html>"
}
default {
# no secret command entered, so check the maintenance window
# Get the values from the statistics profile.
set day [STATS::get $PROFILE_NAME "day"]
set start_time [STATS::get $PROFILE_NAME "start_time"]
set end_time [STATS::get $PROFILE_NAME "end_time"]
if { ($day ne 0) and ($start_time ne 0) and ($end_time ne 0) } {
# Use the TCL "clock" command to get the current time settings.
set l [split [clock format [clock seconds] -format {%u %H %M}] " "]
set cur_day [lindex $l 0]
# Math right here is a bit overkill. I've changed it to string concatenation.
#set cur_time [expr [expr [lindex $l 1]*100] + [lindex $l 2]]
set cur_time "[lindex $l 1][lindex $l 2]"
if { ($cur_day eq $day) &&
($cur_time >= $start_time) &&
($cur_time <= $end_time) } {
# The current time is in the maintenance window, so either issue a HTTP::redirect here to a
# prettied up page, or you can do as I've done here, and return a HTML response direction
# to the client.
HTTP::respond 200 content "
<head><center><title>$TITLE</title>
<body><h1>This site is down for maintenance, please come back at $end_time ...</h1>
</body></html>"
} else {
# Not in maintenance window, so allow connection to continue to application.
}
}
}
}
}
Conclusion:
So, now you should be all set. While your dev team now has control over their site's availability and the 3:00am calls are a thing of the past...
Published Apr 10, 2007
Version 1.0
25 Comments
Jatt_Dill_40963Nimbostratus
Is there any way we can disable this rule by default and enable when required? Looking for some line code in irule
DeVon_JarvisAltostratus
In my "modified" version, I had dropped the leading zeros to force numeric comparisons, thinking these would be faster. I have added back the leading zeros and you are right, string comparisons work just fine.
Here is my finished solution. I wanted only fixed outage windows, but wanted to allow more than one, so you will see the start and end times are in an array. I also optimozed the rule by only checking for the outage condition every n (currently 60) seconds. This made a huge difference on performance.
Let me know if you have any suggestions on how to improve it!
when RULE_INIT {
day 1=mon 7=sun, Times are HHMM with leading zeros!
These are list of day, start, end time for multiple outage windows
set ::day {7}
set ::start_time {0700}
set ::end_time {0900}
check_interval is in seconds
set ::check_interval 60
working variables
set ::check_time 0
set ::in_outage_window 0
}
when HTTP_REQUEST timing on {
Use the TCL "clock" command to get the current time settings.
set now [clock seconds]
if { $now >= $::check_time } {
set ::check_time [expr $now + $::check_interval]
set raw_time [clock format $now -format {%u %H %M}]
set raw_time [split [clock format [clock seconds] -format {%u %H %M}] " "]
set cur_day [lindex $raw_time 0]
set cur_time "[lindex $raw_time 1][lindex $raw_time 2]"
set temp_outage_window 0
foreach dayidx $::day startidx $::start_time endidx $::end_time {
if { ($cur_day eq $dayidx) &&
($cur_time >= $startidx) &&
($cur_time < $endidx) } {
set temp_outage_window 1
log local0. "In outage"
}
}
if {$temp_outage_window != $::in_outage_window } {
set ::in_outage_window $temp_outage_window
if {$temp_outage_window == 0} {
log local0. "End of outage"
} else {
log local0. "Start of outage"
}
}
}
if { $::in_outage_window} {
HTTP::respond 307 Location http://www.mysite.com/maintenance.html
return 0
}
}- How is it that string comparisons will lead to incorrect results. a string compare for 0800 will be less than 0900 and 1430 is greater then 0600. Can you provide an example of two times in HHMM format that will not work for a string comparison as opposed to a math comparison?
- jdspring_42045Would love to see the final product. Ours also does validation against the keepalives so that once keepalives < 1 it goes to maintenance too. The final version could be used to give user maintenance control and also prevent the "ugly" page cannot be found when all keepalives are down.
- DeVon_JarvisHere is what I found out. The octal concern is not an issue any more, but comparing anything to 0800 (leading zero) seems to do a string compare, which leads to incorrect results.
What I did was to change the clock format string to:
set t_array [split [clock format [clock seconds] -format {%u %k %M}] " "]
The %k gives you the hours in 24 hour clock without any leading zeros. Yay! Just what we wanted!
This way, numeric comparisons prevail and we get just what we wanted.
I also optimized my version to only do these comparisons once each minute, and store the result in a global variable. This saves lots of processing especially on a busy site! No need to check on every request, as the status of the outage window will not change anyway.
If you want my version of the code, let me know and I will submit it. - DeVon_JarvisSo, if I understand right, doing string manipulation I would get "0908" for the variable cur_time. Will the octal thing come up again when the comparison takes place? I thought I read that TCL tries to do a numerical comparison first. You may still need to remove the leading zeros. I'm sure I know much less about TCL than you do...
Let me know. - Good catch on the octal bug. I obviously didn't test this iRule in the morning B-).
I've got another solution that avoids regular expressions and string manipulation. You can throw in a little fun and games with the polymorphic nature of TCL to turn the values 08 into a number by placing a decimal point after the lindex command. Kinda funky but it works. The only downside is that the time of H=08, M=09 will be 809.0.
After looking back at this, it's probably overkill to do any math at all since all I was doing with the expr's was to shift the number over to form a HHMM format. Seems pretty easy to do with just plain old string concatenation. Ultimlately I think this is the easiest solution
set cur_time "[lindex $l 1][lindex $l 2]"
Now there is no math involved at all, just a string allocation.
Anyone got a better idea?
BTW, here's the solution with dots and decimal removal
set cur_time [format {%g} [expr [expr [lindex $l 1].*100] + [lindex $l 2].]]
-Joe
-Joe - DeVon_JarvisI tried to use part of this code to create an static outage window iRule this weekend. The problem is, there is a bug in this code.
When this line executes:
set cur_time [expr [expr [lindex $l 1]*100] + [lindex $l 2]]
It fails if the hour or minute is 08 or 09. The problem is, TCL assumes any string that starts with a '0' is octal, and 08 and 09 are invalid octal strings!
There are 2 options. Either use string manipulation or regexp to trim the zeros. I chose the string manipulation, thinking it would be faster. I didn't time it out though...
Here is the "fixed" code:
set cur_day [lindex $t_array 0]
set cur_hour [string trimleft [lindex $t_array 1] 0]
if { ![string length $cur_hour] } { set cur_hour 0 }
set cur_min [string trimleft [lindex $t_array 2] 0]
if { ![string length $cur_min] } { set cur_min 0 }
set cur_time [expr ( $cur_hour * 100 ) + $cur_min]
Hope this helps someone out there! - Is there a way to build some security into this?
-Yes, just check the IP::remote_addr against a list of IP address in a Data Group/Class. Use the Matchclass irule function.
Can this somehow be used to takedown a whole datacenter? That would be great!
-You could apply it to every virtual. You might be able to apply it to a GTM virtual server. It might need modifications though since a GTM virtual is not HTTP/HTML based.
Can I display the time format in HH:MM , rather than HHMM ( i.e. without : )
-Just use the regsub command to remove the ':'.
You have only one statistics profile. If you use this rule on multiple sites does that not cause problems (ie. one site zeros out statistics now the other site that is supposed to be "in maintenance" is no longer)
-Use multiple STATS profiles.
HTH. - jdspring_42045You have only one statistics profile. If you use this rule on multiple sites does that not cause problems (ie. one site zeros out statistics now the other site that is supposed to be "in maintenance" is no longer)
