Exploring F5OS automation features on VELOS

[J_McInnes]

Concepts

VELOS is F5's latest generation chassis platform, providing a high density, high performance hardware platform for organisations that need the ultimate performance for on-premise applications. With VELOS, a new hypervisor operating system called F5OS - based on Kubernetes (Openshift) - runs on the system controllers, and manages all manner of functions - from hardware functions such as power supplies, to software functions such as fine-grained resource allocation. F5OS has its own, new, REST API, which is quite different to the iControl API many might already be familiar with.

The resource allocation that is possible on F5OS/VELOS, is similar to vCMP virtualisation, but provides more control and expanded multi-tenant functionality. Some concepts that will make this article easier to understand:

• System Controller - the two dedicated hardware cards that run F5OS in a VELOS chassis, and supervise VELOS blades.
• System Controller UI (https,ssh) - this is the root level object for Chassis administrators of a VELOS system. The person(s) that own the actual hardware, and can fully administer the system would have this access. The main functions that would be performed at this level are configuring management settings, managing the system controller software and assigning resources to Chassis Partitions.
• Chassis Partition UI (https,ssh) - this is the root level object for Chassis Partition administrators on a VELOS system. The person(s) that have been assigned some amount of blade resources would have this access. The main functions that would be performed at this level are managing the allocation of resources within the Chassis Partition, and managing add/remove/boot of tenants.
• Tenant - the software/functions running on one or more traffic blades in a VELOS chassis. Today this is a classic TMOS v14 or v15 instance. In the near future this will be a containerised tmm pod/BIGIP Next instance.

The following diagram lays out the relationship of System Controller, Partition and Tenant quite nicely:

Please review the VELOS whitepaper here for a more thorough explanation of the concepts. In the following sections, we will explore the API calls required to create and boot up a tenant on a VELOS chassis.

Creating Chassis Partitions

The first step in the process is to make a series of API calls toward the System Controller that create and assign resources to a Chassis Partition.

Get an auth token

In all of the following requests, we will assume $name, $password & $ip are relevant for the System Controller. The API endpoint we are communicating with here is the System Controller.

curl -D - -sk -u "$name":"$password" -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: rctoken" -X HEAD https://$ip:8888/restconf/data/openconfig-system:system/aaa |egrep '^X-Auth-Token: '

The returned X-Auth-Token header will contain a token that can be used in subsequent requests instead of a user/pass.

Release slots from the default partition

By default, any blades in the chassis will be assigned to the default partition. We have to release them before we can use them with another partition.

curl -sk -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: $token" -X PATCH -d @release_default https://$ip:8888/restconf/data

The JSON sent will be similar to below:

{
    "f5-system-slot:slots": {
        "slot": [
            {
                "slot-num": 1,
                "enabled": true,
                "partition": "none"
            },
            {
                "slot-num": 2,
                "enabled": true,
                "partition": "none"
            },
            {
                "slot-num": 3,
                "enabled": true,
                "partition": "none"
            },
            {
                "slot-num": 4,
                "enabled": true,
                "partition": "none"
            }
        ]
    }
}

Create a new partitions

Next, we will create two new partitions; orange and purple.

curl -sk -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: $token" -X POST -d @partition https://$ip:8888/restconf/data/f5-system-partition:partitions

The JSON sent will be similar to below:

{
	"partition": [{
			"name": "orange",
			"config": {
				"enabled": true,
				"iso-version": "1.3.1-5968",
				"mgmt-ip": {
					"ipv4": {
						"address": "10.1.1.84",
						"prefix-length": 24,
						"gateway": "10.1.1.1"
					}
				}
			}
		},
		{
			"name": "purple",
			"config": {
				"enabled": true,
				"iso-version": "1.2.2-12471",
				"mgmt-ip": {
					"ipv4": {
						"address": "10.1.1.85",
						"prefix-length": 24,
						"gateway": "10.1.1.1"
					}
				}
			}
		}
	]
}

Assign blades to the new partition

In order for the new partitions to do anything, we have to assign them resources:

curl -sk -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: $token" -X PATCH -d @assign_slots https://$ip:8888/restconf/data

The JSON sent will be similar to below:

{
    "f5-system-slot:slots": {
        "slot": [
            {
                "slot-num": 1,
                "enabled": true,
                "partition": "orange"
            },
            {
                "slot-num": 2,
                "enabled": true,
                "partition": "purple"
            }
        ]
    }
}

Assigning resources to Chassis Partitions

Next, we will switch API endpoints to the Chassis Partition API and configure the network interfaces, VLANs and assign some resources to a Tenant.

Secure admin credentials

The first thing to do is change the admin credentials, as they will be created with defaults - which must be changed before we can do anything else.

curl -sk -u "$name":"$password" -H "Content-Type: application/yang-data+json" -X POST -d @pwd https://$ip:8888/restconf/operations/openconfig-system:system/aaa/authentication/users/user=admin/config/change-password

The JSON sent will be similar to below:

{
    "input": [
        {
            "old-password": "admin",
            "new-password": "Secure123!",
            "confirm-password": "Secure123!"
        }
    ]
}

Get an auth token

In all of the following requests, we will assume $name, $password & $ip are relevant for the Chassis Partition. The API endpoint we are communicating with here is the Chassis Partition. Otherwise, the API calls to get a token are the same as for the Chassis Partition.

Configure Partition VLANs

Definition of layer 1-2 network elements such as interfaces, LAGs and VLANs are performed at the Chassis Partition layer. First, we will define a few VLANs.

curl -sk -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: $o_token" -X PATCH -d @vlan https://$ip:8888/restconf/data/openconfig-vlan:vlans

The JSON sent will be similar to below:

{
    "openconfig-vlan:vlans": {
        "vlan": [
            {
                "vlan-id": "1",
                "config": {
                    "vlan-id": 1,
                    "name": "VLAN1"
                }
            },
            {
                "vlan-id": "110",
                "config": {
                    "vlan-id": 110,
                    "name": "VLAN110"
                }
            },
            {
                "vlan-id": "120",
                "config": {
                    "vlan-id": 120,
                    "name": "VLAN120"
                }
            }
        ]
    }
}

Configure Partition Network Interfaces

By default, the network interfaces in a VELOS blade are configured to be two logical 100G (or 40G - depending on tranceiver) interfaces. They can also be separated into 4* 25G (or 4* 10G). Note that reconfiguring them into seperate interfaces can take a few minutes, and you will need to poll the API to detect when they are available again to continue the configuration.

curl -sk -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: $o_token" -X PATCH -d @portgroup_orange https://$ip:8888/restconf/data/

The JSON sent will be similar to below:

{
	"f5-portgroup:portgroups": {
		"portgroup": [{
				"portgroup_name": "1/1",
				"config": {
					"name": "1/1",
					"mode": "MODE_4x25GB"
				}
			},
			{
				"portgroup_name": "1/2",
				"config": {
					"name": "1/2",
					"mode": "MODE_4x25GB"
				}
			}
		]
	}
}

Assign VLANs to the Network Interfaces

Now that the physical interfaces are ready, we can assign the appropriate VLANs to them.

curl -sk -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: $o_token" -X PATCH -d @interface_orange https://$ip:8888/restconf/data/

The JSON sent will be similar to below:

{
  "openconfig-interfaces:interfaces": {
    "interface": [{
      "name": "1/1.1",
      "openconfig-if-ethernet:ethernet": {
        "openconfig-vlan:switched-vlan": {
          "config": {
            "native-vlan": 1,
            "trunk-vlans": [110]
          }
        }
      }
    }]
  }
}

Upload BIGIP images to the Chassis Partitions

When a new Chassis Partition is created, it does not have any software images that you can use to provision a tenant. In order to actually provision and deploy a tenant, some TMOS software must be uploaded to the Chassis Partition. This can be achieved multiple ways (SFTP, HTTPS), and are documented in the API guide. As another, interesting way, images can be pushed onto the System Controller, via SSH, directly into the Partition container.

scp ~/BIGIP-14.1.4.6-0.0.8.T1-VELOS.qcow2.zip.bundle root@10.1.1.81:/var/F5/partition2/IMAGES

Note that there are several options for VELOS tenant images, and depending on which one you select, it can mean that the tenant has a small or larger filesystem size requirement (which has impact on what can be provisioned on the tenant). Read the docs here.

Configure Tenant Deployments

Once all the pieces of the puzzle are in place, we can actually define and start a tenant. Similar to a vCMP guest, we have a few different initial states; configured, provisioned (resources reserved) & deployed (started). 

curl -sk -H "Content-Type: application/yang-data+json" -H "X-Auth-Token: $o_token" -X POST -d @tenant_orange https://$ip:8888/restconf/data/f5-tenants:tenants

The JSON sent will be similar to below:

{
	"tenant": [{
			"name": "orange-bigip03-small",
			"config": {
				"image": "BIGIP-14.1.4.6-0.0.8.T1-VELOS.qcow2.zip.bundle",
				"nodes": [
					1
				],
				"mgmt-ip": "10.1.1.188",
				"gateway": "10.1.1.1",
				"prefix-length": 24,
				"vlans": [
					110
				],
				"vcpu-cores-per-node": 2,
				"memory": 7680,
				"cryptos": "enabled",
				"running-state": "deployed"
			}
		},
		{
			"name": "orange-bigip04-large",
			"config": {
				"image": "BIGIP-15.1.5.1-0.0.14.ALL-F5OS.qcow2.zip.bundle",
				"nodes": [
					1
				],
				"mgmt-ip": "10.1.1.189",
				"gateway": "10.1.1.1",
				"prefix-length": 24,
				"vlans": [
					110
				],
				"vcpu-cores-per-node": 8,
				"memory": 30720,
				"cryptos": "enabled",
				"running-state": "deployed"
			}
		}
	]
}

Exploring the System Controller via SSH

If you log in to the System Controller as root via SSH, you will be greeted with a bash prompt, as per what might be expected for any Linux system. It wont take you long to figure out that F5OS uses Openshift as the container orchestrator. Try some of these commands:

oc get pod --all-namespaces -o wide
oc -n partition-2 get pods
oc -n partition-3 get pods

Configuring tenants

The tenants that we have just provisioned will be booted up with very minimal configuration. Basically, they will have any VLANs assigned pre-populated. And, similar to a vCMP guest, they will inherit licensing from the Chassis Partition - so the licensing step is not required.

Otherwise, once you get to this point, all the automation steps required are the same as what would be required for any TMOS device. Things like Declarative Onboarding and AS3 can be used to apply a suitable configuration. If you are doing a migration exercise from an existing F5 device, you could use the new F5 Journeys tool to migrate the configuration from the old device to the new VELOS Tenant.

Future state

What is possible on F5OS and VELOS is just the start of the journey for both platforms. Today it is also possible to deploy F5 cNF on VELOS, and with the imminent release of BIGIP Next the dataplane Tenants will become pods running data plane elements only - no need for a combined control and data plane such as we have with classic TMOS. These BIGIP Next pods will be orchestrated by F5OS. Of course, it will be possible to have a mixture of classic TMOS and BIGIP Next tenants. 

More info

There is an F5OS API guide on Clouddocs - for both F5OS-C (for chassis systems) and F5OS-A (for appliance systems). Also, feel free to play witht the (very basic) shell scripts that I have been using to explore the F5OS API at my github.