Forum Discussion

Darren_Walker_2's avatar
Oct 12, 2016

bulk edit data-group (bigip.conf )

I've seen some answers to this but I'm needing to make sure it's the right answer. We have a data-group key/value pair for redirects. We put in the source, destination, and other details, like start and end date so we don't have to edit the irule every time a new redirect is requested.

 

We are requested to do hundreds of redirects and want to avoid manually updating the data-group one at a time. I understand it is technically possible to edit the bigip.conf directly (soft reboot of the LTM, risk of syntax errors, spaces out of place, etc.). Regardless, we would like to know how to make edits and additions to a text file, then merge them into the section of the conf that has our data group.

 

I know how to find our data-group with this command: tmsh list ltm data-group internal name_of_redirect_data_group then I output that into a file >redirects (should this file rename extension-less?) So now I can edit that file with the additions and modifications.

 

This is where I'm confused. Some say I now do this: tmsh load /sys config file ~/redirects merge Another answer said: tmsh load /sys config merge file ~/redirects

 

But I want to use the verify command first to check syntax. And, I'm not sure the load /sys config is the right thing I want. How does it know where in the bigip.conf file to merge the changes to?

 

Why can't I edit the bigip.conf in vi, save changes in vi, save sys config, then load sys config? When I try that in my test environment the system reloads but my vi changes don't take effect. Thank you for helping me figure this out.

 

Also, we're on BIG-IP 12.1.1 Build 1.0.196 Hotfix HF1

 

  • There is no need to do this. There are two data-group sources: internal (meaning, part of the config) and external (meaning loaded from a separate file when the config is reloaded). They are equivalent in performance; they differ only in how they are loaded (well, and the differ in layout, as well). This article explains how to create and modify an external data-group:

     

  • Vernon_97235's avatar
    Vernon_97235
    Historic F5 Account

    There is no need to do this. There are two data-group sources: internal (meaning, part of the config) and external (meaning loaded from a separate file when the config is reloaded). They are equivalent in performance; they differ only in how they are loaded (well, and the differ in layout, as well). This article explains how to create and modify an external data-group:

     

    • Darren_Walker_2's avatar
      Darren_Walker_2
      Icon for Cirrus rankCirrus

      I am needing to edit the internal data-group that already exists. Either by editing the bigip.conf and saving changes or by editing a file to merge with the internal data-group. I don't know how to do either. Is that another option to accomplish what I'm asking that's safer? I saw you can verify the syntax before merging and I've been told editing the .conf is not preferred. I just want a way to mass edit lots of data instead of hand typing line by line everything from the GUI.

       

      So I'm looking for a tmsh command that merges an existing file into my existing internal data-group and loads the updated config or a list of commands I can use to edit the .conf and update in-line with my additions.

       

  • There is no need to do this. There are two data-group sources: internal (meaning, part of the config) and external (meaning loaded from a separate file when the config is reloaded). They are equivalent in performance; they differ only in how they are loaded (well, and the differ in layout, as well). This article explains how to create and modify an external data-group:

     

    • Darren_Walker_2's avatar
      Darren_Walker_2
      Icon for Cirrus rankCirrus

      I am needing to edit the internal data-group that already exists. Either by editing the bigip.conf and saving changes or by editing a file to merge with the internal data-group. I don't know how to do either. Is that another option to accomplish what I'm asking that's safer? I saw you can verify the syntax before merging and I've been told editing the .conf is not preferred. I just want a way to mass edit lots of data instead of hand typing line by line everything from the GUI.

       

      So I'm looking for a tmsh command that merges an existing file into my existing internal data-group and loads the updated config or a list of commands I can use to edit the .conf and update in-line with my additions.

       

  • "Why can't I edit the bigip.conf in vi, save changes in vi, save sys config, then load sys config? When I try that in my test environment the system reloads but my vi changes don't take effect. Thank you for helping me figure this out."

    the changes you made are to the config file not running memory, you would need to do a load sys config after the edits to load the changes into running config. You can do it this way but there is more risk. I would prefer merging changes..

    As long as your file contains the full context of the object you want to merge with the correct syntax as would be shown with the command below then the system will know where to merge it in, you should be ok to do a merge verify, then a merge (without verify), then a save sys config.

    (tmos) list ltm data-group xxxx

  • Why can you not move it to an external data group? They are functionally the same and the load is atomic.

    In any case, if you simply want to add and delete records, you can do it in a regular

    tmsh
    command, as in

    tmsh modify ltm data-group internal my-dg records \
       add { record4 { data def } record5 { data ghi } record6 { data mno } } \
       records delete { record1 record2 record3 }
    

    If you want to replace the entire data group all at once, you can use a

    load...merge
    :

    cat > /var/tmp/new-dg-def.txt << EOF
    ltm data-group internal my-dg {
        records {
            record1 {
                data def
            }
            record2 {
                data ghi
            }
            record3 {
                data there
            }
        }
        type string
    }
    EOF
    
    tmsh load sys config /var/tmp/new-dg-def.txt merge
    

    The last command is the important part. The rest is just to demonstrate what the file contents must be. It is !!! IMPERATIVE !!! that you include the

    merge
    keyword. If you forget, you will potentially blow away your entire config.

    This

    load
    will validate the syntax before attempting to add, so if there is an error in the source file, it won't replace the data-group.

    • Darren_Walker_2's avatar
      Darren_Walker_2
      Icon for Cirrus rankCirrus

      tmsh load sys config ~/uaredirects.txt verify Syntax Error: "/home/me/uaredirects" unknown property

       

      How do I get around the syntax error?

       

      Here's what my file looks like:

       

      ~ cat uaredirects.txt

       

      ltm data-group internal class_requestDirectorKV { records { _SourceURL { data "Type;Directive;Location;Implementation Date Ticket Number;Start Date;End Date" } ua.mywebsite.com/redirect2 { data "=;302; 123;2016-10-12;2016-12-12" } } type string }

       

      Thank you guys for the help!

       

  • Vernon_97235's avatar
    Vernon_97235
    Historic F5 Account

    Why can you not move it to an external data group? They are functionally the same and the load is atomic.

    In any case, if you simply want to add and delete records, you can do it in a regular

    tmsh
    command, as in

    tmsh modify ltm data-group internal my-dg records \
       add { record4 { data def } record5 { data ghi } record6 { data mno } } \
       records delete { record1 record2 record3 }
    

    If you want to replace the entire data group all at once, you can use a

    load...merge
    :

    cat > /var/tmp/new-dg-def.txt << EOF
    ltm data-group internal my-dg {
        records {
            record1 {
                data def
            }
            record2 {
                data ghi
            }
            record3 {
                data there
            }
        }
        type string
    }
    EOF
    
    tmsh load sys config /var/tmp/new-dg-def.txt merge
    

    The last command is the important part. The rest is just to demonstrate what the file contents must be. It is !!! IMPERATIVE !!! that you include the

    merge
    keyword. If you forget, you will potentially blow away your entire config.

    This

    load
    will validate the syntax before attempting to add, so if there is an error in the source file, it won't replace the data-group.

    • Darren_Walker_2's avatar
      Darren_Walker_2
      Icon for Cirrus rankCirrus

      tmsh load sys config ~/uaredirects.txt verify Syntax Error: "/home/me/uaredirects" unknown property

       

      How do I get around the syntax error?

       

      Here's what my file looks like:

       

      ~ cat uaredirects.txt

       

      ltm data-group internal class_requestDirectorKV { records { _SourceURL { data "Type;Directive;Location;Implementation Date Ticket Number;Start Date;End Date" } ua.mywebsite.com/redirect2 { data "=;302; 123;2016-10-12;2016-12-12" } } type string }

       

      Thank you guys for the help!

       

  • My apologies, it's actually:

    tmsh load sys config file /var/tmp/new-dg-def.txt merge
    

    Note that this will not work on 10.x.

    • Darren_Walker_2's avatar
      Darren_Walker_2
      Icon for Cirrus rankCirrus

      Thank you. That told me my directory wasn't white listed so I used the command to list those locations and put my redirects.txt file in /var/local/scf and now it verifies the file and I can merge it. I appreciate all your help!

       

  • Vernon_97235's avatar
    Vernon_97235
    Historic F5 Account

    My apologies, it's actually:

    tmsh load sys config file /var/tmp/new-dg-def.txt merge
    

    Note that this will not work on 10.x.

    • Darren_Walker_2's avatar
      Darren_Walker_2
      Icon for Cirrus rankCirrus

      Thank you. That told me my directory wasn't white listed so I used the command to list those locations and put my redirects.txt file in /var/local/scf and now it verifies the file and I can merge it. I appreciate all your help!