Pushing Updates to BIG-IP w/ HashiCorp Consul Terraform Sync

HashiCorp Consul Terraform Sync (CTS) is a tool/daemon that allows you to push updates to your BIG-IP devices in near real-time (this is also referred to as Network Infrastructure Automation). This helps in scenarios where you want to preserve an existing set of network/security policies and deliver updates to application services faster.

Consul Terraform Sync

Consul is a service registry that keeps track of where a service is (10.1.20.10:80 and 10.1.20.11:80) and the health of the service (responding to HTTP requests). Terraform allows you to push updates to your infrastructure, but usually in a one and done fashion (fire and forget). NIA is a symbiotic relationship of Terraform and Consul. It allows you to track changes via Consul (new node added/removed from a service) and push the change to your infrastructure via Terraform.  

Putting CTS in Action

We can use CTS to help solve a common problem of how to enable a network/security team to allow an application team to dynamically update the pool members for their application. This will be accomplished by defining a virtual server on the BIG-IP and then enabling the application team to update the state of the pool members (but not allow them to modify the virtual server itself).  

Defining the Virtual Server

The first step is that we want to define what services we want. In this example we use a FAST template to generate an AS3 declaration that will generate a set of Event-Driven Service Discovery pools. The Event-Driven pools will be updated by NIA and we will apply an iControl REST RBAC policy to restrict updates.  

The FAST template takes the inputs of “tenant”, “virtual server IP”, and “services”.  

This generates a Virtual Server with 3 pools.

Event-Driven Service Discovery

Each of the pools is created using Event-Driven Service Discovery that creates a new API endpoint with a path of:

/mgmt/shared/service-discovery/task/ ~[tenant]~EventDrivenApps~[service]_pool/nodes

You can send a POST API call these to add/remove pool members (it handles creation/deletion of nodes). The format of the API call is an array of node objects:

[{“id”:”[identifier]”,”ip”:”[ip address]”,”port”:[port (optional)]}]

We can use iControl REST RBAC to limit access to a user to only allow updates via the Event-Driven API.

Creating a CTS Task

NIA can make use of existing Terraform providers including the F5 BIG-IP Provider. We create our own module that makes use of the Event-Driven API

...
resource "bigip_event_service_discovery" "pools" {
 for_each = local.service_ids
 taskid = "~EventDriven~EventDrivenApps~${each.key}_pool"
 dynamic "node" {
  for_each = local.groups[each.key]
  content {
   id = node.value.node
   ip = node.value.node_address
   port = node.value.port
  }
 }
}
...

Once NIA is run we can see it updating the BIG-IP

- Finding f5networks/bigip versions matching "~> 1.5.0"...
...
module.AS3.bigip_event_service_discovery.pools["app003"]: Creating...
…
module.AS3.bigip_event_service_discovery.pools["app002"]: Creation complete after 0s [id=~EventDriven~EventDrivenApps~app002_pool]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Scaling up the environment to go from 3 pool members to 10 you can see NIA pick-up the changes and apply them to the BIG-IP in near real-time.

module.AS3.bigip_event_service_discovery.pools["app001"]: Refreshing state... [id=~EventDriven~EventDrivenApps~app001_pool]
…
module.AS3.bigip_event_service_discovery.pools["app002"]: Modifying... [id=~EventDriven~EventDrivenApps~app002_pool]
…
module.AS3.bigip_event_service_discovery.pools["app002"]: Modifications complete after 0s [id=~EventDriven~EventDrivenApps~app002_pool]

Apply complete! Resources: 0 added, 3 changed, 0 destroyed.

NIA can be run interactively at the command-line, but you can also run it as a system service (i.e. under systemd).

Alternate Method

In the previous example you saw an example of using AS3 to define the Virtual Server resource. You can also opt to use Event-Driven API directly on an existing BIG-IP pool (just be warned that it will obliterate any existing pool members once you send an update via the Event-Driven nodes API). To create a new Event-Driven pool you would send a POST call with the following payload to /mgmt/shared/service-discovery/task

{
  "id": "test_pool",
  "schemaVersion": "1.0.0",
  "provider": "event",
  "resources": [
    {
      "type": "pool",
      "path": "/Common/test_pool",
      "options": {
        "servicePort": 8080
      }
    }
  ],
  "nodePrefix": "/Common/"
}

You would then be able to access it with the id of “test_pool”. To remove it from Event-Driven Service Discovery you would send a DELETE call to /mgmt/shared/service-discovery/task/test_pool

Separation of Concerns

In this example you saw how CTS could be used to separate network, security, and application tasks, but these could be easily combined using NIA just as easily. Consul Terraform Sync is now generally available, and I look forward to seeing how you can leverage it. For an example that is similar to this article you can take a look at the following GitHub repo that has an example of using NIA. You can also view another example on the Terraform registry as well.