Several months ago I wrote up the v10 formatting for internal and external datagroups: iRules Data Group Formatting Rules. In v11, however, there is a change to the format of the internal data group and the data group reference to external class files (the formatting in the external class file itself is unchanged). The formatting rules in v11 for data groups more closely resembles the tmsh commands necessary to build the class at the CLI (these command attributes are masked if you are using the GUI). I’ll follow the same format as the original write-up in showing the various data group types. The format is the same among internal data group types. If there is no value associated with the key, there is a curly bracket pair trailing the key on the same line. If there is an associated value with a key, the curly bracket opens the value, followed by a newline with the keyword data and the value, then another newline with the closing curly bracket. After the records are listed, the type is specified. For external data groups, the file name and the type are specified. If the filename is in /var/class, the path is omitted from the filename reference.
Address Data Groups
Internal Data Group
ltm data-group internal addr_testclass {
records {
192.168.1.1/32 { }
192.168.1.2/32 {
data "host 2"
}
192.168.2.0/24 { }
192.168.3.0/24 {
data "network 2"
}
}
type ip
}
External Data Group
ltm data-group external addr_testclass_ext { external-file-name addr_testclass.class type ip }
Integer Data Groups
Internal Data Group
ltm data-group internal int_testclass {
records {
1 {
data "test 1"
}
2 {
data "test 2"
}
}
type integer
}
External Data Group
ltm data-group external int_testclass_ext {
external-file-name int_testclass
type integer
}
String Data Groups
Internal Data Group
ltm data-group internal str_testclass {
records {
str1 {
data "value 1"
}
str2 {
data "value 2"
}
}
type string
}
External Data Group
ltm data-group external str_testclass_ext {
external-file-name str_testclass.class
type string
}
External Datagroup File Management
Beginning in v11, external datagroups are imported into a local filestore rather than simply existing someplace on the file system (/config/filestore). However, this filestore is not meant to edited manually. Please follow the steps below for creating or modifying external datagroups. Thanks to hoolio for the steps in this external datagroup section.
Create a New External Datagroup from the CLI Non-Interactively
1. Create or copy over to LTM a temporary file containing the external data group contents. If copying, make sure the line terminators are \n only not \r\n.
Whether imporing external datagroups in the GUI or from the CLI, the system does not accept files with \r\n line terminators, it only accepts \n. If you copy files over from windows, most likely you have the wrong terminator in your file format. To check, you can use the od command.
Datagroup Contents created in vi on LTM
[root@golgotha:Active] data_group_d # od -c /var/tmp/string_name_value_external_dg.txt
0000000 " n a m e 1 " : = " v a l u
0000020 e 1 " , \n " n a m e 2 " : =
0000040 " v a l u e 2 " , \n " n a m e 3
0000060 " : = " v a l u e 3 " , \n
0000077
Datagroup Contents created in Notepad on Windows
[root@golgotha:Active] data_group_d # od -c /var/tmp/notepad_dg.txt
0000000 " n a m e 1 " : = " v a l u
0000020 e 1 " , \r \n " n a m e 2 " : =
0000040 " v a l u e 2 " , \r \n " n a m
0000060 e 3 " : = " v a l u e 3 " ,
0000100
If your line terminators are incorrect, you can use the tr command to remove the \r's.
[root@golgotha:Active] tmp # cat /var/tmp/notepad_dg.txt | tr -d '\r' > /var/tmp/notepad_dg_update.txt
[root@golgotha:Active] tmp # od -c /var/tmp/notepad_dg_update.txt
0000000 " n a m e 1 " : = " v a l u
0000020 e 1 " , \n " n a m e 2 " : =
0000040 " v a l u e 2 " , \n " n a m e 3
0000060 " : = " v a l u e 3 " ,
0000076
In v11 to use an external file, you need to import it first. Here are the steps:
1. Create a text file containing the data group contents. Here is a sample string data group file:
"name1" := "value1",
"name2" := "value2",
"name3" := "value3",
2. If you're on Windows, use a utility like dos2unix.exe to convert the line endings from CRLF to LF (https://encrypted.google.com/search?q=dos2unix.exe). If you don't do this, you will get an error on import:
01070627:3: The string data group external file (/config/filestore/.stage_d/55_d/Common_d/data_group_d/:Common:name_value_external_dg_1) has an invalid format, line: 2.
3. Navigate to System | File Management | Data Group File List | Import
4. Specify a filename. Example:
File Name: C:\tmp\name_value_external_dg.dos.txt
Name: name_value_external_dg
File Contents: string
Key / Value Pair Seperator: =:
5. Navigate to Local Traffic | iRules | Data Group List | Create
I would be very interested in the sample code also. We had a working solution in version 10 that we could manage our data group files but it looks like in 11 this has changed and does not work anymore. If there is a code sample could you please post it here.
Syntax Error: "external-file-name" may not be specified in the context of the "modify" command. "external-file-name" may be specified using the following commands: create, list
[root@L22RRZPEP12:/S1-green-P:Active] tmp
Any ideas how I can modify the group without deleting and re-creating it?
In case you want to work with network address type external datagroups you may want to use the following structures:
Key definition options for host addresses (no quotes used in key definition):
host 10.21.21.21
network 10.22.22.22 prefixlen 32
network 10.23.23.23 mask 255.255.255.255
Key definition options for network addresses (no quotes used in key definition):
network 10.24.24.0 prefixlen 24
network 10.25.25.0 mask 255.255.255.0
To add a value to a key the following syntax applies (double quotes used for argument definition):
host 10.21.21.21 := "host 21",
network 10.22.22.22 prefixlen 32 := "network 22",
Further notes:
- trailing comma in last line is optional
- line feed between records is optional (all records can be put in a single line as the comma is used as record separator)
Thank you so much for the detailed explanation of the data group updates. I have a question. I am trying to save a data group file with some similar cookie elements. Using this data group i would like to reject the HTTP REQUEST. I mean if the HTTP::cookie has any one of the cookies in the data group i created. The HTTP request should be rejected. I will be adding new cookies to the data group on regular basis. So, Can you please giving me an idea of , how to approach or what might be the best way to put an iRule satisfying this situation.
Here is my iRule
when HTTP_REQUEST
{ if { class match -value [HTTP::cookie $aam_uuid] contains Cookie_Rejects } {reject}
}
Data group name is Cookie_Rejects and its internal with one string value.
Consider i updated the data group with one more cookie value today, if a http request come with the updated cookie value it must be rejected with out changing any iRule.
Thanks in advance
I've read through this thread, get to thinking I understand it but still no success. I've read older threads with script suggestions - I figured this would be a pretty simple process. I'm running 12.1.1.
I have an existing Address type data group, I need to add about 250 IP addresses to the group.
Can I add/import into my existing Address type data group?
Can I use the File Management function to do so?
Can you give an example of loading an external datagroup file non-interactively with the type "ip" please? All of the combinations I've tried thus far give me the error, "0107169a:3: The data group external file (/config/filestore/.stage_d/...) has an invalid format, line: 1.". Thank you.
Ok - turns out the format is similar to the V10 rules. The following worked - note that if you give the CIDR you have to use "network" and not "host" even if it's a 32-bit netmask.
