Forum Discussion

Mike_Maher's avatar
Mike_Maher
Icon for Nimbostratus rankNimbostratus
Sep 05, 2017

Stateful applications and DC persistance

I have an application (stateful application) that gets sorted to 1 of 2 data centers by GTM at our perimeter using a region based topology. It is setup where half of the US goes to one DC and the other half of the US goes to the other DC. We have run into an issue with some users that their DNS uses anycast and in the middle of their session they get moved to the other DC. I have been asked to look into providing solution to maintain DC stickiness for this scenario. I know once I get the connection to LTM that I can do an internal cross DC connection based on something like a cookie. However my concern is that if this application move out to 3 or 4 DCs at some point these old infrastructure solutions to stickiness are not going to serve me well. I was wondering if anyone could share any of their solutions on how they manage stateful applications across multiple locations/DCs.

 

application delivery
BIG-IP DNS
LTM
  • RossVermette's avatar
    RossVermette
    Icon for Nimbostratus rankNimbostratus
    Sep 05, 2017

    I just went through something similar.

    A client comes into a VS (LTM/APM enabled) in one datacenter, and in “mid session”, the client does a dns lookup which resolves to some other DataCenter. (example, mobile devices, or certain carriers use anycast dns, or proxies the DNS traffic where GTM won’t be able to keep persistence, but because the dns query comes from a different “client local dns servers”, they can get sent to a different DC and GTM persistency will not work.) So we created new "fronting Vs" that fronted these existing LTM/APM vs that needed persistency. (The fronted vs was needed so it can read the cookie before APM gets processed, otherwise APM would consider it a new session)

    Our thinking for this iRule: (You set variable for "localDatacenter to the DCx that you designate") Logic: Look to see if you have a cookie and check its value for which Datacenter you are/were connected to. If there was no cookie, then set the cookie the localDatacenter.

    If you had a cookie and it's localdatacenter, you forward using the "virtual virtualServerName"(via lookup) if you're remote, you forward to a remote node via a lookup.

    iRule: Uses 2x DataGroups:

    1. Node-DG: (Used to MAP HostHeaderName to a NODE IP. (Note case sensitivity) Format:"hostheadername.fqdn.name DCx" := "IPAddress:port", ex: "test1.domain.name dc1" := "1.1.1.1:443", "test1.domain.name dc2" := "2.2.2.2:443",

    2. VS-DG: (Used to MAP HostHeaderName to the destination of virtual server you need to forward to) Format: "hostheadername.fqdn.name" := "VirtualServerName", ex: "test1.domain.name" := "TEST1-HTTPS_vs",

      when HTTP_REQUEST {
  set debug 0
  set LocalDataCenter XX
  set secret "SomePassPharse"
  set urihead [HTTP::host]
  set encrypted [HTTP::cookie value $urihead]
  if {$encrypted != ""} {
    set loc [AES::decrypt $secret [b64decode $encrypted]]
  } else {
    set loc ""
  }
  if {$loc == "$LocalDataCenter" || $loc == ""}  {
                set vs_value [class lookup [string tolower $urihead] VS-DG]
                if {$debug == 1}{
                    log local0.debug "Forwarding traffic to VS: $vs_value from [IP::client_addr]:[TCP::client_port]"
                }
                elseif { $vs_value == ""} {
                   if {$debug == 1} {
                        log local0.debug "NO match for vs_value lookup"
                    }
                    reject
                    return
                }
                virtual $vs_value
  } else {
                set node_value [class lookup [string tolower "$urihead $loc"] Node-DG]
                if {$debug == 1}{
                    log local0.debug "$loc cookie name $urihead dectected in $LocalDataCenter from client [IP::client_addr]"
                    log local0.debug "NODE value $node_value from [IP::client_addr]"
                }
                node $node_value
  }
  snatpool SNAT-Pool-NAMEXXX
}

when HTTP_RESPONSE {
  sets the location cookie to the local DC
  if { $loc == "" } {
    HTTP::cookie path $urihead /
    HTTP::cookie domain $urihead $urihead
    HTTP::cookie secure $urihead enable
    set encrypted [b64encode [AES::encrypt $secret $LocalDataCenter]]
    HTTP::cookie insert name $urihead value $encrypted
  }
}

    All our vs are https enabled and we use SNI, and instead of creating a new ssl-server profile for every vs we front, we have this second iRule which copies the SNI name from the client SSL to the Server side ssl.

    when CLIENTSSL_HANDSHAKE {
    if { [SSL::extensions exists -type 0] } then {
        set tls_sni_extension [SSL::extensions -type 0]
    } else {
        set tls_sni_extension ""
    }
}
when SERVERSSL_CLIENTHELLO_SEND {
    if { $tls_sni_extension ne "" } then {
        SSL::extensions insert $tls_sni_extension
    }
}

    Hope this helps.