Deploy Failover BIG-IP Cluster with New Stack in Google Cloud using v2 Templates

In this article, I plan to cover how-to deploy a pair of BIG-IP devices with failover and high availability features into a Google Cloud new network stack.

How-to v2 Failover Example: BIG-IP Cluster on Google Cloud with New Network Stack

The following section will be a walkthrough with steps to deploy two BIG-IP instances in GCP using the Failover example with a new network stack. When complete, the BIG-IP devices will form an Active/Standby cluster. You can use the same techniques in order to copy the templates, modify as needed, and deploy in your environment.

During a failover event on the BIG-IP devices, the new active device (was standby) will detect the problem via heartbeats. The new active BIG-IP device sends API calls to Google API to move objects like alias IP ranges, forwarding rule IP mappings (aka public IPs), and more. This API failover method is thanks to the magic in the F5 Cloud Failover Extension, part of the F5 Automation Toolchain.

Clone GitHub Repository

1. Navigate to GitHub repo for v2 BIG-IP GDM templates

2. Select the "Code" button and copy the HTTPS address

3. Open a terminal (ex. Visual Studio Code) and clone the repository

cd ~/Desktop git clone https://github.com/F5Networks/f5-google-gdm-templates-v2.git

Create Secret in GCP Secret Manager

One prerequisite for this deployment is to use a secret from GCP Secret Manager. Secret Manager stores API keys, passwords, certificates, and other sensitive data. The secret will be used for the BIG-IP administrator password during onboarding.

1. From your terminal, create a file called password.txt containing the BIG-IP password in one line.

2. Create the secret with gcloud using the password.txt file

Note: For demo purposes of "new stack", use the secret name mySecretId to avoid the need to customize the BIG-IP onboarding files.

gcloud secrets create mySecretId --data-file="password.txt"

# Sample Output Created version [1] of the secret [mySecretId]

Modify Parameters - New Stack

For this section, I will deploy two BIG-IP instances with a new network stack in GCP. The result will be the following:

4x New VPCs: mgmt, external, internal, app (and related components)
Application instance running docker, container=f5devcentral/f5-demo-app:latest
1x Storage bucket for BIG-IP cloud failover state
2x BIG-IP instances, version=16.1.2.2
4x CPU, 3-NICs
Using runtime-init-conf-3nic-payg-instance01-with-app.yaml and runtime-init-conf-3nic-payg-instance02-with-app.yaml from examples in F5 GitHub repo
BIG-IP cluster with HA, network, application, and WAF configurations

Here we go!

Note: For demo purposes of this article, we will use a tagged version of the repo to maintain article consistency with code.

1. From your terminal, set DEPLOYMENT_NAME, set CONFIG_FILE, change folders, and change branch to pinned tag

DEPLOYMENT_NAME=giroux-bigip-failover-new CONFIG_FILE=sample_failover.yaml cd f5-google-gdm-templates-v2/examples/failover git checkout tags/v2.4.0.0

2. Review the entire README for instructions and prerequisites

3. Edit the file sample_failover.yaml

Note: For demo purposes of "new stack", use the cfeTag value "bigip_high_availability_solution" to avoid the need to customize the BIG-IP onboarding files.

# sample_failover.yaml - BIG-IP Failover Cluster with new stack # Note: Commented and some Optional lines were removed from # the yaml below. This keeps the code block small # for easier illustration purposes. imports: - path: failover.py - path: ../modules/access/access.py - path: ../modules/application/application.py - path: ../modules/bastion/bastion.py - path: ../modules/bigip-standalone/bigip_standalone.py - path: ../modules/dag/dag.py - path: ../modules/network/network.py resources: - name: failover-py properties: bigIpImageName: f5-bigip-16-1-2-2-0-0-28-payg-best-plus-1gbps-220505081153 bigIpRuntimeInitConfig01: >- https://raw.githubusercontent.com/F5Networks/f5-google-gdm-templates-v2/v2.4.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-payg-instance01-with-app.yaml bigIpRuntimeInitConfig02: >- https://raw.githubusercontent.com/F5Networks/f5-google-gdm-templates-v2/v2.4.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-payg-instance02-with-app.yaml cfeTag: bigip_high_availability_solution cfeBucket: cfe-storage owner: giroux provisionPublicIp: true region: us-west1 restrictedSrcAddressApp: [0.0.0.0/0] restrictedSrcAddressMgmt: [0.0.0.0/0] uniqueString: giroux-ha zone: us-west1-a type: failover.py

Deploy the BIG-IP Failover Cluster - New Stack

1. From your terminal, launch the GDM template

gcloud deployment-manager deployments create ${DEPLOYMENT_NAME} --config ${CONFIG_FILE}

# Sample Output The fingerprint of the deployment is b'FauS5Cmgls9Tc39Q6Q8N6w==' Waiting for create [operation-1657906455110-5e3db6d53e48e-a588ab52-c520aa3a]...done. Create operation operation-1657906455110-5e3db6d53e48e-a588ab52-c520aa3a completed successfully. NAME TYPE STATE ERRORS INTENT giroux-ha-app-int-fw compute.v1.firewall COMPLETED [] giroux-ha-app-subnet compute.v1.subnetwork COMPLETED [] giroux-ha-app-vip-fw compute.v1.firewall COMPLETED [] giroux-ha-application-vm-01 compute.v1.instance COMPLETED [] giroux-ha-bigip-iam-bind gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding COMPLETED [] giroux-ha-bigip-sa iam.v1.serviceAccount COMPLETED [] giroux-ha-bigip-vm-01 compute.v1.instance COMPLETED [] giroux-ha-bigip-vm-01-ti compute.v1.targetInstance COMPLETED [] giroux-ha-bigip-vm-02 compute.v1.instance COMPLETED [] giroux-ha-bigip-vm-02-ti compute.v1.targetInstance COMPLETED [] giroux-ha-cfe-storage storage.v1.bucket COMPLETED [] giroux-ha-ext-network compute.v1.network COMPLETED [] giroux-ha-ext-subnet compute.v1.subnetwork COMPLETED [] giroux-ha-fr-01 compute.v1.forwardingRule COMPLETED [] giroux-ha-ha-fw compute.v1.firewall COMPLETED [] giroux-ha-http-hc compute.v1.healthCheck COMPLETED [] giroux-ha-https-hc compute.v1.healthCheck COMPLETED [] giroux-ha-int-network-02 compute.v1.network COMPLETED [] giroux-ha-int-subnet-02 compute.v1.subnetwork COMPLETED [] giroux-ha-mgmt-fw compute.v1.firewall COMPLETED [] giroux-ha-mgmt-network compute.v1.network COMPLETED [] giroux-ha-mgmt-subnet compute.v1.subnetwork COMPLETED [] giroux-ha-public-ip-01 compute.v1.address COMPLETED [] giroux-ha-tcp-hc compute.v1.healthCheck COMPLETED [] girouxhabigipaccessrole gcp-types/iam-v1:projects.roles COMPLETED []

Validating the Deployment

See the "Validating the Deployment" section or the "Further Exploring" section in the Failover README file for more validation commands.

1. Retrieve the parent template outputs

2. Copy the values for bigIpManagementPublicIp1, bigIpManagementPublicIp2, and vip1PublicIp

gcloud deployment-manager manifests describe --deployment=${DEPLOYMENT_NAME} --format="value(layout)" | yq .resources[0].outputs

# Sample Output ...snippet - finalValue: 35.199.156.64 name: bigIpManagementPublicIp1 - finalValue: 34.82.195.63 name: bigIpManagementPublicIp2 ...snippet... - finalValue: 35.233.137.160 name: vip1PublicIp

SSH to BIG-IP Cluster and Review Logs

1. Access the BIG-IP devices and enter bash mode (value from bigIpManagementPublicIp1 and bigIpManagementPublicIp2)

ssh admin@35.199.156.64 -i ~/.ssh/id_rsa admin@(giroux-ha-bigip1)(cfg-sync In Sync)(Active)(/Common)(tmos)# bash [admin@giroux-ha-bigip1:Active:In Sync] ~ #

2. Review BIG-IP Runtime Init onboard logs

[admin@giroux-ha-bigip1:Active:Standalone] ~ # cat /var/log/cloud/bigIpRuntimeInit.log # Sample Output ...snippet... 2022-08-11T19:04:42.573Z [3752]: info: Successfully validated declaration 2022-08-11T19:04:42.661Z [3752]: info: Resolving parameters 2022-08-11T19:04:45.259Z [3752]: info: Executing install operations. 2022-08-11T19:05:15.323Z [3752]: info: Executing service operations. 2022-08-11T19:05:15.369Z [3752]: info: Creating - do 1.31.0 .... 2022-08-11T19:08:38.510Z [3752]: info: Creating - cf 1.11.0 .... 2022-08-11T19:08:52.323Z [3752]: info: Creating - as3 3.38.0 .... 2022-08-11T19:09:25.127Z [3752]: info: Creating - do 1.31.0 .... 2022-08-11T19:10:07.330Z [3752]: info: Initializing telemetryClient 2022-08-11T19:10:08.193Z [3752]: info: Sending f5-teem report 2022-08-11T19:10:08.197Z [3752]: info: All operations finished successfully 2022-08-11T19:10:13.570Z [3752]: info: Successfully sent data to F5 Teem

Testing the Application and WAF

The following tests will work if you deploy the BIG-IP with Layer 4-7 settings. For my demo settings, the "new stack" creates an app server and adds a BIG-IP listener with WAF policy. You can still POST an AS3 declaration to the BIG-IP, or you can login manually and create a pool and listener. Check out the F5 AS3 Docs for more example declarations!

1. From your local machine, curl the demo application (value from vip1PublicIp)

curl http://35.233.137.160 -I

# Sample Output HTTP/1.1 200 OK ...snippet... Set-Cookie: BIGipServer~Tenant_1~Shared~Shared_Pool=xxxx; path=/; Httponly Set-Cookie: TS01aa0884=xxxx; Path=/

2. Perform a security violation by sending a disallowed 'method' of DELETE

curl http://35.233.137.160 -X DELETE

# Sample Output <html><head><title>Request Rejected</title></head><body>The requested URL was rejected. Please consult with your administrator.<br><br>Your support ID is: 9487250596978229314<br><br><a href='javascript:history.back();'>[Go Back]</a></body></html>

Testing Failover

Let's validate a few things and then force a failover.

1. From your terminal, run gcloud to list the forwarding rule and verify the target BIG-IP instance (1 or 2)

gcloud compute forwarding-rules list --filter="name=(giroux-ha-fr-01)"

# Sample Output NAME REGION IP_ADDRESS IP_PROTOCOL TARGET giroux-ha-fr-01 us-west1 35.233.137.160 TCP us-west1-a/targetInstances/giroux-ha-bigip-vm-01-ti

The output shows vm-01 which is BIG-IP instance 1. This matches the CLI on each BIG-IP in which instance 1 shows 'active' and instance 2 shows 'standby'.

# From instance 1 [admin@giroux-ha-bigip-vm-01:Active:In Sync] ~ # # From instance 2 [admin@giroux-ha-bigip-vm-02:Standby:In Sync] ~ #

Now, let's force a failover.

2. From the standby BIG-IP, start a 'tail -f' on the restnoded log file to watch live logs

[admin@giroux-ha-bigip-vm-02:Standby:In Sync] ~ # tail -f /var/log/restnoded/restnoded.log

3. From the active BIG-IP, force a failover

[admin@giroux-ha-bigip-vm-01:Active:In Sync] ~ # tmsh run sys failover standby [admin@giroux-ha-bigip-vm-01:Standby Sync]

4. Go back to the other BIG-IP instance...now 'active' and review the restnoded logs

# Sample Output ... Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Performing Failover - update Thu, 11 Aug 2022 19:46:19 GMT - fine: [f5-cloud-failover] Address discovery: .... ... ... Thu, 11 Aug 2022 19:46:19 GMT - finest: [f5-cloud-failover] Updating forwarding rule: giroux-ha-fr-01 to target: xxxx/giroux-ha-bigip-vm-02-ti Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Disassociate NICs successful. Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Associate NICs successful. Thu, 11 Aug 2022 19:46:20 GMT - finest: [f5-cloud-failover] updateFwdRule operation name: operation-xxxx Thu, 11 Aug 2022 19:46:45 GMT - info: [f5-cloud-failover] Updated forwarding rules successfully ... Thu, 11 Aug 2022 19:46:45 GMT - info: [f5-cloud-failover] Failover Complete Thu, 11 Aug 2022 19:46:48 GMT - finest: [f5-cloud-failover] Download stateFile: .... [admin@giroux-ha1122-bigip-vm-02:Active:In Sync] ~ #

The log entries validate the forwarding rule was updated to target instance 2.

5. From your terminal, run gcloud to list the forwarding rules again to validate a new target

gcloud compute forwarding-rules list --filter="name=(giroux-ha-fr-01)"

# Sample Output NAME REGION IP_ADDRESS IP_PROTOCOL TARGET giroux-ha-fr-01 us-west1 35.233.137.160 TCP us-west1-a/targetInstances/giroux-ha-bigip-vm-02-ti

Test traffic again (see earlier sections) and everything should still work like magic!

Delete and Clean Up

1. Delete the deployment

gcloud deployment-manager deployments delete ${DEPLOYMENT_NAME} -q

Summary

This article shows you how to use the BIG-IP Cloud Solution Templates to easily deploy services and BIG-IP in Google Cloud. I use sample configurations in this article to deploy two BIG-IP devices in a failover cluster with a new network stack and a demo application server. I then show you how to validate the deployment, review onboard logs, and test the application. This Failover example is an easy way to have your applications protected by BIG-IP running in HA on Google Cloud. Make sure to check out the other examples in the GitHub repository for scenarios like standalone or autoscale.