Nimbostratus

Jun 11, 2015

Classes / Data Groups searching

This is on LTMs, v11.3...

Client URLs are of two possible forms: http://anyplace.com/streamX and http://anyplace.com/hls/streamX/playlist.m3u8

I've created a Data Group "streams" - the "String" values are all of the allowed streamX values; the "Value" values are the name of the pools that will handle that specific stream. E.g.:

    String      Value
    news1       pool1
    old-news    pool2
    story6b     pool1
    sport-nine  pool1
    sport9      pool2

...and so on (< 100 values)

So, from example 1 in the Data Group, we will see both of these URLs: http://anyplace.com/news1 and http://anyplace.com/hls/news1/playlist.m3u8.

Had been attempting to use findclass & 'contains' to generate an iRule that would let me parse the URL, and direct the client to the appropriate pool. Then I came across https://devcentral.f5.com/articles/v11-irules-data-group-updates and am now completely confused! Any pointers/help appreciated (there's very little in the way of real-world examples, for a mere mortal [such as myself], covering Classes/Data Groups in v11).

TIA, Jon

Rather than using scan, is it not less resource-hungry to do something like:

it seems you are right.

 scan

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    timing on
when HTTP_REQUEST {
  scan [HTTP::path] {/hls/%[^/]/playlist.m3u8} stream
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) show ltm rule qux raw

 (raw)
---------------------------------
Ltm::Rule Event: qux:HTTP_REQUEST
---------------------------------
Priority                    500
Executions
  Total                     500
  Failures                    0
  Aborts                      0
CPU Cycles on Executing
  Average                 31477
  Maximum                108708
  Minimum                 26020

 getfield

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    timing on
when HTTP_REQUEST {
  set test [getfield [HTTP::path] "/" 3]
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) show ltm rule qux raw

 (raw)
---------------------------------
Ltm::Rule Event: qux:HTTP_REQUEST
---------------------------------
Priority                    500
Executions
  Total                     500
  Failures                    0
  Aborts                      0
CPU Cycles on Executing
  Average                 26278
  Maximum                104980
  Minimum                 22000

nitass

Employee

Jun 11, 2015

is it something like this?

 configuration

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm virtual bar
ltm virtual bar {
    destination 200.200.200.10:80
    ip-protocol tcp
    mask 255.255.255.255
    pool foo
    profiles {
        http { }
        tcp { }
    }
    rules {
        qux
    }
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    vs-index 9
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    when HTTP_REQUEST {
  set path [HTTP::path]
  set stream ""

  if { $path starts_with "/hls/" and $path ends_with "/playlist.m3u8" } {
    if { [scan $path {/hls/%[^/]/playlist.m3u8} stream] != 1 } {
       use default pool
      return
    }
  } else {
    if { [scan $path {/%[^/]} stream] != 1 } {
       use default pool
      return
    }
  }

  if { [catch { pool [class match -value $stream equals stream_datagroup] }] } {
     use default pool
  }
}
when HTTP_RESPONSE {
  log local0. "path: $path stream: $stream pool: [LB::server pool]"
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm data-group internal stream_datagroup
ltm data-group internal stream_datagroup {
    records {
        news1 {
            data pool1
        }
    }
    type string
}

 /var/log/ltm

[root@ve11c:Active:In Sync] config  tail -f /var/log/ltm
Jun 11 21:31:21 ve11c info tmm1[10748]: Rule /Common/qux : path: / stream:  pool: /Common/foo
Jun 11 21:31:27 ve11c info tmm[10748]: Rule /Common/qux : path: /something stream: something pool: /Common/foo
Jun 11 21:31:33 ve11c info tmm1[10748]: Rule /Common/qux : path: /news1 stream: news1 pool: /Common/pool1
Jun 11 21:31:46 ve11c info tmm[10748]: Rule /Common/qux : path: /hls/something/playlist.m3u8 stream: something pool: /Common/foo
Jun 11 21:31:54 ve11c info tmm1[10748]: Rule /Common/qux : path: /hls/news1/playlist.m3u8 stream: news1 pool: /Common/pool1

jonhd_205628
Nimbostratus
Jun 11, 2015
Kinda does, nitass, thanks. How does that 'play' with a 2nd entry in the Data Group of (say) "old-news" "pool2" (as per my example Data Group)?... Cheers, Jon
nitass
Employee
Jun 11, 2015
multiple data group records should work too.

nitass_89166

Noctilucent

Jun 11, 2015

is it something like this?

 configuration

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm virtual bar
ltm virtual bar {
    destination 200.200.200.10:80
    ip-protocol tcp
    mask 255.255.255.255
    pool foo
    profiles {
        http { }
        tcp { }
    }
    rules {
        qux
    }
    source 0.0.0.0/0
    source-address-translation {
        type automap
    }
    vs-index 9
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    when HTTP_REQUEST {
  set path [HTTP::path]
  set stream ""

  if { $path starts_with "/hls/" and $path ends_with "/playlist.m3u8" } {
    if { [scan $path {/hls/%[^/]/playlist.m3u8} stream] != 1 } {
       use default pool
      return
    }
  } else {
    if { [scan $path {/%[^/]} stream] != 1 } {
       use default pool
      return
    }
  }

  if { [catch { pool [class match -value $stream equals stream_datagroup] }] } {
     use default pool
  }
}
when HTTP_RESPONSE {
  log local0. "path: $path stream: $stream pool: [LB::server pool]"
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm data-group internal stream_datagroup
ltm data-group internal stream_datagroup {
    records {
        news1 {
            data pool1
        }
    }
    type string
}

 /var/log/ltm

[root@ve11c:Active:In Sync] config  tail -f /var/log/ltm
Jun 11 21:31:21 ve11c info tmm1[10748]: Rule /Common/qux : path: / stream:  pool: /Common/foo
Jun 11 21:31:27 ve11c info tmm[10748]: Rule /Common/qux : path: /something stream: something pool: /Common/foo
Jun 11 21:31:33 ve11c info tmm1[10748]: Rule /Common/qux : path: /news1 stream: news1 pool: /Common/pool1
Jun 11 21:31:46 ve11c info tmm[10748]: Rule /Common/qux : path: /hls/something/playlist.m3u8 stream: something pool: /Common/foo
Jun 11 21:31:54 ve11c info tmm1[10748]: Rule /Common/qux : path: /hls/news1/playlist.m3u8 stream: news1 pool: /Common/pool1

jonhd_205628
Nimbostratus
Jun 11, 2015
Kinda does, nitass, thanks. How does that 'play' with a 2nd entry in the Data Group of (say) "old-news" "pool2" (as per my example Data Group)?... Cheers, Jon
nitass_89166
Noctilucent
Jun 11, 2015
multiple data group records should work too.

jonhd_205628
Nimbostratus
Jun 11, 2015
Ok, thanks. Rather than using scan, is it not less resource-hungry to do something like:

set requested_path [HTTP::path] . . if { $requested_path starts_with “/hls/” } then { set channel [getfield $requested_path “/” 3]

Just wondering.

Cheers, Jon

nitass

Employee

Jun 12, 2015

Rather than using scan, is it not less resource-hungry to do something like:

it seems you are right.

 scan

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    timing on
when HTTP_REQUEST {
  scan [HTTP::path] {/hls/%[^/]/playlist.m3u8} stream
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) show ltm rule qux raw

 (raw)
---------------------------------
Ltm::Rule Event: qux:HTTP_REQUEST
---------------------------------
Priority                    500
Executions
  Total                     500
  Failures                    0
  Aborts                      0
CPU Cycles on Executing
  Average                 31477
  Maximum                108708
  Minimum                 26020

 getfield

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    timing on
when HTTP_REQUEST {
  set test [getfield [HTTP::path] "/" 3]
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) show ltm rule qux raw

 (raw)
---------------------------------
Ltm::Rule Event: qux:HTTP_REQUEST
---------------------------------
Priority                    500
Executions
  Total                     500
  Failures                    0
  Aborts                      0
CPU Cycles on Executing
  Average                 26278
  Maximum                104980
  Minimum                 22000

jonhd_205628
Nimbostratus
Jun 12, 2015
Wow nitass - thanks for the comprehensive evidence. Your help is much appreciated. Jon

nitass_89166

Noctilucent

Jun 12, 2015

Rather than using scan, is it not less resource-hungry to do something like:

it seems you are right.

 scan

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    timing on
when HTTP_REQUEST {
  scan [HTTP::path] {/hls/%[^/]/playlist.m3u8} stream
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) show ltm rule qux raw

 (raw)
---------------------------------
Ltm::Rule Event: qux:HTTP_REQUEST
---------------------------------
Priority                    500
Executions
  Total                     500
  Failures                    0
  Aborts                      0
CPU Cycles on Executing
  Average                 31477
  Maximum                108708
  Minimum                 26020

 getfield

root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm rule qux
ltm rule qux {
    timing on
when HTTP_REQUEST {
  set test [getfield [HTTP::path] "/" 3]
}
}
root@(ve11c)(cfg-sync In Sync)(Active)(/Common)(tmos) show ltm rule qux raw

 (raw)
---------------------------------
Ltm::Rule Event: qux:HTTP_REQUEST
---------------------------------
Priority                    500
Executions
  Total                     500
  Failures                    0
  Aborts                      0
CPU Cycles on Executing
  Average                 26278
  Maximum                104980
  Minimum                 22000

jonhd_205628
Nimbostratus
Jun 12, 2015
Wow nitass - thanks for the comprehensive evidence. Your help is much appreciated. Jon

Forum Discussion

Classes / Data Groups searching

Recent Discussions

iRule for X509 Subject

AS3 Schema for per Application Deployment

How can I find the current connectivity sessions via SNMP?

Unable to create F5 in AWS via CFT - no internet access

BIGIP Next - SHARED-00001: Internal server error

Related Content

Certified Kubernetes Administrator - Study Group

External Data Group / Class Manipulation

How to search the latest security academic papers

Intermediate iRules: Data-Groups

Class match with starts_with - Data group matching order