Forum Discussion
don_23889
Nimbostratus
Dec 28, 2009grep'ng for 'string' in bigip.conf
grep'ng for 'string' and returning entire block of object definitions from bigip.conf for each instance of 'string', starting with 'virtual' and ending with '}'
config less bigip.conf |grep -B5 -A5 string |more
This is cumbersome since you have to guess for -B(x) and -A(x)
So, as an example, I want to return all object definitions with iRule 'http-xff'. Basically, all lines between [virtual] and [}]
This example is syntactically incorrect, and only for illustrating what I am trying to achieve....ie...pseudo-code.
less bigip.conf |grep BEGIN[virtual] END[}] http-xff |more
virtual vip1.80 {
pool vip1.80
destination aaa.bbb.ccc.ddd:http
ip protocol tcp
rules SNAT2VIP
profiles
http-xff
tcp-lan-optimized
serverside
tcp-wan-optimized
clientside
persist source_addr_30min
}
virtual vip-2.80 {
pool vip-2.80
destination mmm.nnn.ooo.ppp:http
ip protocol tcp
rules SNAT2VIP
profiles
http-xff
tcp-lan-optimized
serverside
tcp-wan-optimized
clientside
persist source_addr_30min
}
What would really be sweet, would be to grep for a given 'string', and return entire block for 'any' object definition...where the preceding line and following line are are both blank. This would return all object definitions for a given string...virtual, pool, monitor...etc.
30 Replies
- Mark_Crosland_2Historic F5 AccountNote that the example demonstrates the data-driven nature of the tmsh scripting API. As new types of components are added to the system (e.g., virtual, pool, monitor, etc...), or, new properties/settings are added to existing components, the script will automatically search new elements. The script does not have to be updated, it just continues to work.
- hoolio
Cirrostratus
Just wanted to thank everyone for the examples for parsing a bigip.conf (for non-TMSH versions like 9.3.x). I used this post as a start for a script to reorder the bigip.conf objects alphabetically. It's a bit of a hack, but gets the job done.
The use case is:
Prepare bigip.conf files from several BIG-IP's in a customers set of environments. The configs are similar between environments, but not identical. It's a royal pain trying to diff two bigip.conf files from separate environments when all the iRules, VIPs, etc, are in different orders.
Note that I haven't tried loading the resulting bigip.conf--just comparing files so far.
So thanks guys,
Aaron!/usr/bin/perl v0.2 - 2010-01-27 This is a simple script to parse a bigip.conf and print out each instance of an object types in alphabetical order Ex: my_class, whose_class, your_class, my_rule_1, my_rule_2, your_rule_1, my_vip_1, my_vip_2, your_vip_1, your_vip_2, Order of the printing of the object types themselves is hardcoded for now using the @ObjectTypeNames array. This could be changed to print in the order the object types are found in the bigip.conf Object types not defined in $ObjectTypeRegex are considered "unknown" and are printed first. use strict; Need to declare the ObjectTypeName arrays first use warnings; use Switch; Get the filename from the command line arguments. Default to ./bigip.conf @ARGV = 'bigip.conf' unless @ARGV; my ($File) = @ARGV; Debug logging? (0=none, 1=debug on result printing, 2=debug on printing and file parsing) my $debug = 0; Initialise some variables my ($CurlyBracesCount, $Object) = (0, ""); my $ObjectDefLine = ""; Array of object type names. The names need to be in the order that object types are listed in the conf files to ensure the output order is valid. my @ObjectTypeNames = ('unknowns', 'routes', 'classes', 'monitors', 'auths', 'profiles', 'nodes', 'pools', 'snatpools', 'snats', 'rules', 'virtual_addresses', 'virtuals'); Regex which matches my $ObjectTypeRegex = qw/(^(?:auth|class|monitor|node|pool|profile|route|rule|snat|snatpool|virtual).*){$/; ToDo: Declare array names for each ObjectTypeName and enable 'use strict' Open file open (Fh, $File) || die "Cant open $File"; print "===========================\nStart...\n\n" if $debug > 1; Read in the file while ( ){ Append current line $Object .= $_; Count the number of open/close curly braces in the current line $CurlyBracesCount += (tr/\{//); $CurlyBracesCount -= (tr/\}//); If the curly brace count is 0, we've found a matching close brace if ($CurlyBracesCount == 0) { Look for an object type definition if ((split /\n/, $Object)[0] =~ /$ObjectTypeRegex/s) { print "Matched: $1\n" if $debug > 1; Save the match if there was one if ($1 ne "") { $ObjectDefLine = $1; $ObjectDefLine =~ s/\s+$//; print "Saving \$ObjectDefLine: $ObjectDefLine\n" if $debug > 1; } } elsif ($Object =~ /[a-z].*\{$/) { Found a line which starts with an alpha character and ends with an open curly brace print "\nUnknown object definition: $Object\n" if $debug > 1; $ObjectDefLine = "unknown"; } else { Found something else... print "\nDefault. Unknown object definition:\n$Object\n" if $debug > 1; $ObjectDefLine = "unknown"; } Save the current object to the corresponding object type array print "Evaluating object type definition: $ObjectDefLine\n" if $debug > 1; switch ((split / /, $ObjectDefLine)[0]) { case "auth" { Add the current class name and definition as a hash to an object type array print "Adding object name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@auths, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "class" { Add the current class name and definition as a hash to an object type array print "Adding class name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@classes, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "monitor" { print "Adding monitor: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@monitors, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "node" { print "Adding node: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@nodes, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "pool" { print "Adding pool: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@pools, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "profile" { print "Adding profile: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@profiles, {ObjectName => (split / /, $ObjectDefLine)[1]." ".(split / /, $ObjectDefLine)[2], ObjectDef => $Object} ); } case "route" { print "Adding route: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@routes, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "rule" { print "Adding rule: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@rules, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "snat" { print "Adding snat: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@snats, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "snatpool" { print "Adding snatpool: $ObjectDefLine, name: ".(split / /, $ObjectDefLine)[1]."\n" if $debug > 1; push(@snatpools, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "packet" { print "Adding packet filter: $ObjectDefLine\n, name: ".(split / /, $ObjectDefLine)[1]."" if $debug > 1; push(@packet_filters, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } case "virtual" { ToDo: could parse virtual servers and virtual addresses into separate arrays If so, check for second word of "address" to find virtual addresses if ((split / /, $ObjectDefLine)[1] eq "address") { print "Adding virtual address: $ObjectDefLine\n, name: ".(split / /, $ObjectDefLine)[2]."" if $debug > 1; push(@virtual_addresses, {ObjectName => (split / /, $ObjectDefLine)[2], ObjectDef => $Object} ); } else { print "Adding virtual server: $ObjectDefLine\n, name: ".(split / /, $ObjectDefLine)[1]."" if $debug > 1; push(@virtuals, {ObjectName => (split / /, $ObjectDefLine)[1], ObjectDef => $Object} ); } } else { print "No match: $ObjectDefLine\n\$Object: $Object" if $debug > 1; push(@unknowns, {ObjectName => "unknown", ObjectDef => $Object} ); } } $Object = ""; print "-----------------------------\n" if $debug > 1; } } close (Fh); Print the output print "===========================\nResults:\n" if $debug; Loop through the Object Types by name foreach $ObjectType (@ObjectTypeNames) { Sort the array by the ObjectType hash key name, ObjectName my @SortedObjectType = sort { $a->{'ObjectName'} cmp $b->{'ObjectName'} } @$ObjectType; print "===========================\n" if $debug; print "Object Type: $ObjectType\n\n" if $debug; Loop through the array of Object hashes foreach $ObjectHashRef (@SortedObjectType) { Loop through the Objects and print the definition foreach $Object ($ObjectHashRef) { print "---------------------------\n" if $debug; print "$Object->{'ObjectName'}:\n" if $debug; print $Object->{'ObjectDef'}; } } } - hoolio
Cirrostratus
The output is just each object instance ordered by name. The eventual goal is to use the same ordering of object types as they're listed in the original bigip.conf. For now it's just a way to reorder the bigip.conf to make diff'ing of two different bigip.conf's easier.
So if you had a few classes, pools and VIPs defined but listed in mixed alphabetical order:
whose_class {...}
your_class {...}
my_class {...}
your_rule_1 {...}
my_rule_2 {...}
my_rule_1 {...}
your_vip_2 {...}
my_vip_2 {...}
your_vip_1 {...}
my_vip_1 {...}
The script would write these out in alphabetical order with the definitions unchanged:
my_class {...}
whose_class {...}
your_class {...}
my_rule_1 {...}
my_rule_2 {...}
your_rule_1 {...}
my_vip_1 {...}
my_vip_2 {...}
your_vip_1 {...}
your_vip_2 {...}
At some point, I'd like to write out the object types in the same order as they were parsed from the bigip.conf. But for now, the order is hardcoded in the script. Once I fix that, I'll add it to the codeshare.
Aaron - smp_86112
Cirrostratus
Hey hoolio, did you ever post anything to codeshare on this topic? We are sorely in need of a parsing engine that can handle v10 config files. We wrote an engine ourselves that worked for v9 which severely broke when we moved to v10 yesterday. - hoolio
Cirrostratus
Hi SMP,
I haven't had a need to try it yet, but I would imagine that tmsh should provide more functionality for displaying the bigip.conf objects in a systematic way.
What specifically are you trying to update for v10 that you had for v9?
Aaron - smp_86112
Cirrostratus
The automation we developed works with a local copy of bigip.conf, so tmsh isn't an option.
The format of the config file changed (obviously) between 9 and 10. The v10 config file uses nested profile and iRule directives in the definition of the virtual server definition, versus what the v9 did which was to reference these items and define them elsewhere. We basically deconstruct a virtual, and use the data to populate an application a database. We had to write our own parser for v9, and we were hoping someone had already done this for v10. It doesn't look like that that is the case though... - hoolio
Cirrostratus
I haven't seen anything in the wild for parsing 10.x bigip.conf's.
If you could get live access to a BIG-IP running the config (or maybe a VE running 10.1.0?), I'd think iControl would be the best way to handle a requirement like this as you wouldn't be subject to configuration syntax changes. Else, you could take hwidjaja's method for parsing objects with nested braces like I did for Perl and use that to rework your 9.x parsing tool.
Aaron - JRahm
Admin
Just FYI, tmsh or icontrol will need to be part of your long range plans as bigpipe will not be around forever. - hoolio
Cirrostratus
. - hoolio
Cirrostratus
Posted By hwidjaja on 01/02/2010 09:35 AM It seems like this awk one-liner is better than my previous one:awk 'BEGIN {RS="\n}";FS=RS} /rules SNAT2VIP/ {print $1"\n}";} ' /config/bigip.confHey Humphrey,I ended up using this one liner to extract all of the monitors from the bigip.conf for a 9.x customer working on their upgrades. Just wanted to say thanks again for this gem.In case anyone is looking for an ugly script to update 9.x monitors to make them 10.x compatible, here you go:b save; awk 'BEGIN {RS="\n}";FS=RS} /monitor .*defaults from/ {print $1"\n}";}' /config/bigip.conf | perl -pe 's/(? /var/tmp/monitors.scf; b merge /var/tmp/monitors.scf; b saveIt basically replaces \n's that don't have a preceding \r with \r\n and appends up to two \r\n's to the end of all send strings if they're not there already. The input is the bigip.conf. The output is written to an SCF file which is then imported back into memory and saved to disk with a 'b save'.It's not thoroughly tested (yet), but should save a lot of time manually updating monitor send strings.Aaron
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects
