HTTP prefetch insertion

Problem this snippet solves:

This iRule demonstrate how to use BIG-IP to insert prefetch hint/force to http response from server. "Prefetch hint" is the way that web server tell browser to continue downloading more objects after it finishes current page loading. Usually, once the page is loaded, user may spend sometimes reading page anyway. So during this time, browser can quietly prefetch some objects in the background. Then when user visit next page load time will be faster since some objects is already in browser cache.

While there are multiple prefetch hint/force technique such as

• inserting Link: header
• insert < link> tag
• insert < meta> tag
• insert javascript that prefetch object after the page load is done to invisible object (like iFrame or < DIV> tag)
• same as above without javascript

I picked the easiest one (:P) which is to insert "Link:" header to http response from server. I believe modern browser support. (I only tested with Firefox 3.5 though)

This iRule also keep tract of relationship while user navigates through website. For example, user may start with /, then visit /about which may include some images, javascript, etc. iRule will remember those images, javascript url and may insert hint to prefetch some of those urls next time that user visit /.

This is a prove-of-concept iRule so it may lack of many features such as

• filter prefetch url list by content-length (not to prefetch if it is too large)
• filter prefetch url list by content-type (i.e. prefetch only image)
• static prefetch list (dont keep track so we can save some cpu)
• CMP compatible (this irule use global variable for easy and fast development, we may use session command or other new upcoming feature instead)
• support more prefetch technique
• check useragent and decide which prefetch technique to be used (legacy browser may have limited support)
• etc

Any feedback is welcome.

Code :

when HTTP_REQUEST {
    if { [HTTP::header "X-Moz"] eq "prefetch" } {
        set xmoz_prefetch 1
        return
    } else {
        set xmoz_prefetch 0
    }
    set uri [HTTP::uri]
    if { [HTTP::cookie exists "prefetch_tracking"] } {
        set new_session 0
        set id [HTTP::cookie "prefetch_tracking"]
        if { [info exists ::previous_uri($id) ] } {
            log local0. "id=$id $::previous_uri($id)"
            set puri $::previous_uri($id)
            set name \{::next_uri_${puri}($uri)\}
            if { ! [info exists ::next_uri_${puri}($uri)] } {
                set ::next_uri_${puri}($uri) 1
            } else {
                incr ::next_uri_${puri}($uri)
            }
            log local0.alert "::next_uri_${puri}($uri)) = [subst \$$name]"
        }
    } else {
        set new_session 1
    }
}
when HTTP_RESPONSE {
    if { $xmoz_prefetch } { return }
    if { $new_session and [string tolower [HTTP::header "Content-Type"]] contains "text/html" } {
        # find unique tracking-id
        set id [b64encode [md5 [expr 65535*rand()]]]
        log local0. "id=$id"
        HTTP::cookie insert name "prefetch_tracking" value $id
    }
    if { [string tolower [HTTP::header "Content-Type"]] contains "text/html" } {
        log local0. [string tolower [HTTP::header "Content-Type"]]
        log local0. "set ::previous_uri($id) = $uri"
        if { ![info exists ::next_previous_uri($id) ] } {
            set ::next_previous_uri($id) $uri
        } else {
            set ::previous_uri($id) $::next_previous_uri($id)
            set ::next_previous_uri($id) $uri
        }
    }
    if { ! [info exists sizeof_uri($uri)] } {
        if { [HTTP::header exists "Content-Length"] } {
            set sizeof_uri($uri) [HTTP::header "Content-Length"]
        } else {
            set sizeof_uri($uri) 0
        }
    }
    if { [string tolower [HTTP::header "Content-Type"]] contains "text/html" } {
        set ll ""
        foreach u [array name ::next_uri_${uri}] {
            lappend ll [subst "$u \$\{::next_uri_${uri}($u)\}"]
        }
        set ll [lsort -index 1 -integer $ll]
        set max_prefetch 6
        for { set i 0 } { $i < $max_prefetch } {incr i } {
            set u [lindex [lindex $ll $i] 0]
            if { $u == "" } {
                break
            }
            # implement one of prefetch method here
            HTTP::header insert "Link" "<$u>; rel=prefetch"
            log local0. u=$u
        }
    }
}