BIG-IP in a Vagrant Virtualbox Box

Tim_Rupp · ‎01-Apr-2016

I want to preface this and the following article with the remark that I am fully aware that this was posted on April Fools day. While others on the interwebs are surely having a great time being silly, this and the following post are totally legit.

So over the last year or so, I've had this burning desire to turn the VE version of the BIG-IP into a Vagrant box.

As I've become more aware of the inner workings of a BIG-IP, it seemed like a fun technical challenge that would produce something that I could use in my own day-to-day work.

Periodically I test integration-like things. Things that include a BIG-IP and....something else. I thought something like this would have found its way around to a variety of internal groups by the time I had shown up at F5 but, surprisingly, it hasn't.

So this post and another are going to dive into the details on how to make this work.

tl;dr. It works. It requires a VE OVA file and a hand or two to be used for such things as high-fiving, or (as Samuel Jackson so eloquently put it) holding onto your butt.

There are two parts to this operation.

In this article, we're going to address the building of the actual box. Along the way, I'll touch upon some observations on why you have to do things the way you have to. There are free resources that can make this process easier and I've included sources and artifacts to get you going.

In the next article we'll figure out how to bend vagrant to our will, to boot the BIG-IP and configure it. You'll see that this isn't as straightforward as it might at first seem.

Vagrant makes a number of assumptions about the box it is configuring that are just not valid in the context of a BIG-IP. I'll touch upon all of those and note where, frankly, I just kinda claimed victory even though I technically could have optimized it further. I'll leave it to the overachievers in the audience to address those optimizations.

Let's get started!

Tools

For this operation you're going to need the following

a VE image in the OVA format
a copy of packer from www.packer.io
a copy of vagrant from www.vagrantup.com

I initially tested this on a Mac, but then tried the procedure on an Ubuntu box with equal success. So I will leave it up to y'all to choose.

The Template

All of our work is going to involve us mucking with a packer template.

Some background on packer if you are not familiar with it. Packer is a tool from the same folks who brought you vagrant. It allows you to create VM and container images for a number of different platforms from a single JSON configuration file.

Packer templates include a builder and optionally a provisioner and post-processor.

{
  "builders": [
      {
          ...
      }
   ],
   "provisioners": [
      {
          ...
      }
   ],
   "post-processors": [
      {
          ...
      }
   ]
}

Builders are what allow packer to generate the images for a specific platform. There are builders for many different platforms.

Provisioners are run by packer to install software and configure the image prior to that machine being turned into a static image.

Post-processors take the output of a builder and turn it into another artifact. For instance, the "compress" post-processor zips the images.

The Builder

We're interested specifically in the Virtualbox builder. This builder accepts a type called "virtualbox-ovf" that takes an OVA as its input. Here's the builder that I used.

    "builders": [
    {
      "type": "virtualbox-ovf",
      "source_path": "BIGIP-11.6.0.0.0.401.LTM_1SLOT-ide.ova",
      "ssh_username": "root",
      "ssh_password": "default",
      "ssh_wait_timeout": "30000s",
      "headless": "false",
      "shutdown_command": "shutdown -h now",
      "import_flags": ["--eula", "accept"],
      "guest_additions_mode": "disable",
      "vm_name": "BIGIP-11.6.0.0.0.401.LTM_1SLOT-ide",
      "vboxmanage": [
        ["modifyvm","{{.Name}}","--memory","4096"],
        ["modifyvm","{{.Name}}","--cpus","2"],
        ["modifyvm","{{.Name}}","--nic1","NAT"]
      ]
    }
  ]

There are several things to note in this builder.

First, I have specified an

ssh_wait_timeout

. BIG-IP takes longer to boot than the standard Vagrant SSH wait timeout, so I override it here to account for that.

Second, note that I have made use of the vboxmanage dictionary for much more than a "normal" box would. These settings are to adjust for resource requirements that BIG-IP expects to be there by default. The amount of RAM specified there is what I typically allocate to BIG-IP to support the different combination of modules that I work with. The CPUs setting is there, again, to adjust for module needs.

For the third item in the list, the NAT item, it is needed for an interface to actually be allocated to BIG-IP to be used by vagrant for connecting to over SSH. When BIG-IP boots, the first interface (eth0) is always the mgmt address and is used by admins to connect to BIG-IP over SSH to manage it. If you do not do this, then Virtualbox will not give the BIG-IP a NIC to use for eth0. Interestingly, BIG-IP will still bring up the mgmt interface even without the adapter being there, (how?) but packer will generate a lot of SSH connection errors such as

TCP connection to SSH ip/port failed: dial tcp 127.0.0.1:3819: getsockopt: connection refused

Add this builder to your packer template and that will be sufficient for this section.

The Provisioner

We're going to specifically use this step to do some hacking on the BIG-IP image. To make it work with vagrant, we need to do some extra steps including

Install sudo
Create the "vagrant" user
Add the insecure vagrant key to that vagrant user's authorized_keys file

All of this is going to be done in our image before we make it a static image. This will prepare it to make it work automatically in vagrant when we send it through the post-processor in a moment.

Here's the provisioner block that I used for my box.

  "provisioners": [
  {
    "type": "shell",
    "inline": [
      "mount -o remount,rw /dev/mapper/vg--db--hda-set.1._usr /usr",
      "curl -o /tmp/sudo-1.8.16-1.el5.x86_64.rpm http://repo.centos.org/sudo-1.8.16-1.el5.x86_64.rpm",
      "rpm -Uvh /tmp/sudo-1.8.16-1.el5.x86_64.rpm",

      "tmsh create auth user vagrant password vagrant partition-access add { all-partitions { role admin } } shell bash",
      "echo 'vagrant ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/vagrant",
      "mkdir -p /home/vagrant/.ssh",
      "chmod 0700 /home/vagrant/.ssh",
      "curl --insecure -L -o /home/vagrant/.ssh/authorized_keys https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub",
      "chmod 0600 /home/vagrant/.ssh/authorized_keys",
      "chown -R vagrant /home/vagrant/.ssh",
      "tmsh save sys config"
    ]
  }
]

Let's go into the details.

Vagrant uses an unprivileged user account to provision a device when it boots. It makes privileged calls using sudo. The rub is that BIG-IP doesn't have sudo installed. So...we install it.

A little interrogation of the BIG-IP finds that it is loosely based on CentOS 5. So I went and found the sudo rpm from a CentOS 5 repository and magically it installed right away. Obviously you'll need to go to the CentOS repositories and find a similar rpm. I just used a generic placeholder in the config above.

Now, let me take a moment to point out that surely at this point we have gone far far beyond the point of no return in regards to Support on your image here. You were officially on your own when you pointed packer at our VE, but now you have surely crossed the line and will not get any help from support or anyone if you run into problems.

Vagrant requires that a user account, vagrant, exist on the remote box. Vagrant uses this account to log in over SSH and perform tasks on the box as part of the provisioning steps. BIG-IP doesn't come with this vagrant account, so we create it. We can't use the standard linux useradd tool to create it though, because mcpd controls the users on a BIG-IP system. So instead, we use the tmsh command to add the user. Be sure to have the users shell set to "bash".

Finally, after creating the vagrant user, we set up its SSH settings to accept the insecure public key provided by vagrant and then we save the whole configuration.

Now that we're done provisioning, let's move on to the post-processor.

The Post-processor

We are specifically interested in creating a Vagrant box, so we run the output through the vagrant post-processor to give us one.

  "post-processors": [
    {
      "type": "vagrant",
      "compression_level": 1,
      "output": "BIGIP-11.6.0.0.0.401.LTM_1SLOT-ide.box"
    }
  ],

When the above post-processor runs, it will drop a BIGIP-11.6.0.0.0.401.LTM_1SLOT-ide.box file in your current working directory.

That's it!

With that all assembled, you can run packer like so

PACKER_LOG=1 packer build template.json

And off it will go.

I've included the full template in the URL below for reference because JSON can be particular about formatting and commas (among other things).

Hope that points you in the right direction! Go read the second article I posted about using your new box in a Vagrantfile!

References

https://devcentral.f5.com/s/articles/packer-template-for-building-a-big-ip-box

Kevin_Davies_40 · ‎17-Apr-2016

Several kinds if awesome right here. Thank you for all the info.

Bart_Van_Bos · ‎19-Jan-2020

Is this method still working/supported on BIGIP-15.1.0-0.0.31.LTM_1SLOT?

After changing the password back to default (by doing an intermediate step to another password, as the first login requires a mandatory password change), packer seems to fail to connect with Big-IP over ssh. I can login with the credentials in the VirtualBox UI console though.

2020/01/20 05:24:53 packer-builder-virtualbox-ovf plugin: [DEBUG] reconnecting to TCP connection for SSH
2020/01/20 05:24:53 packer-builder-virtualbox-ovf plugin: [DEBUG] handshaking with SSH
2020/01/20 05:24:53 packer-builder-virtualbox-ovf plugin: [DEBUG] SSH handshake err: ssh: handshake failed: read tcp 127.0.0.1:59384->127.0.0.1:4169: read: connection reset by peer
2020/01/20 05:25:00 packer-builder-virtualbox-ovf plugin: [INFO] Attempting SSH connection to 127.0.0.1:4169...
2020/01/20 05:25:00 packer-builder-virtualbox-ovf plugin: [DEBUG] reconnecting to TCP connection for SSH
2020/01/20 05:25:00 packer-builder-virtualbox-ovf plugin: [DEBUG] handshaking with SSH

If I look into the Virtualbox UI, I see that mcpd and devmgmtd are in a crashloop.

logger[22111] Re-starting mcpd
logger[22818] Re-starting devmgmtd
logger[22332] Re-starting mcpd
...

The output of dmesg shows the tmm interface flapping.

IPv6: ADDRCONF(NETDEV_CHANGE): tmm: link becomes ready
IPv6: ADDRCONF(NETDEV_UP: tmm: link is not ready
traps: mcpd[17749] trap divide error ip:56c01093 sp:ffe15:0 in libmcpdcommon.so[5678f000+67c000]
IPv6: ADDRCONF(NETDEV_CHANGE): tmm: link becomes ready
IPv6: ADDRCONF(NETDEV_UP: tmm: link is not ready
traps: mcpd[18214] trap divide error ip:56c01093 sp:ffe15:0 in libmcpdcommon.so[5678f000+67c000]
...

DevCentral