23-Aug-2022 05:00 - edited 23-Aug-2022 10:59
In this article, I plan to cover how-to deploy a pair of BIG-IP devices with failover and high availability features into a Google Cloud new network stack.
The following section will be a walkthrough with steps to deploy two BIG-IP instances in GCP using the Failover example with a new network stack. When complete, the BIG-IP devices will form an Active/Standby cluster. You can use the same techniques in order to copy the templates, modify as needed, and deploy in your environment.
During a failover event on the BIG-IP devices, the new active device (was standby) will detect the problem via heartbeats. The new active BIG-IP device sends API calls to Google API to move objects like alias IP ranges, forwarding rule IP mappings (aka public IPs), and more. This API failover method is thanks to the magic in the F5 Cloud Failover Extension, part of the F5 Automation Toolchain.
1. Navigate to GitHub repo for v2 BIG-IP GDM templates
2. Select the "Code" button and copy the HTTPS address
3. Open a terminal (ex. Visual Studio Code) and clone the repository
cd ~/Desktop
git clone https://github.com/F5Networks/f5-google-gdm-templates-v2.git
One prerequisite for this deployment is to use a secret from GCP Secret Manager. Secret Manager stores API keys, passwords, certificates, and other sensitive data. The secret will be used for the BIG-IP administrator password during onboarding.
1. From your terminal, create a file called password.txt containing the BIG-IP password in one line.
2. Create the secret with gcloud using the password.txt file
Note: For demo purposes of "new stack", use the secret name mySecretId to avoid the need to customize the BIG-IP onboarding files.
gcloud secrets create mySecretId --data-file="password.txt"
# Sample Output
Created version [1] of the secret [mySecretId]
For this section, I will deploy two BIG-IP instances with a new network stack in GCP. The result will be the following:
Here we go!
Note: For demo purposes of this article, we will use a tagged version of the repo to maintain article consistency with code.
1. From your terminal, set DEPLOYMENT_NAME, set CONFIG_FILE, change folders, and change branch to pinned tag
DEPLOYMENT_NAME=giroux-bigip-failover-new
CONFIG_FILE=sample_failover.yaml
cd f5-google-gdm-templates-v2/examples/failover
git checkout tags/v2.4.0.0
2. Review the entire README for instructions and prerequisites
3. Edit the file sample_failover.yaml
Note: For demo purposes of "new stack", use the cfeTag value "bigip_high_availability_solution" to avoid the need to customize the BIG-IP onboarding files.
# sample_failover.yaml - BIG-IP Failover Cluster with new stack
# Note: Commented and some Optional lines were removed from
# the yaml below. This keeps the code block small
# for easier illustration purposes.
imports:
- path: failover.py
- path: ../modules/access/access.py
- path: ../modules/application/application.py
- path: ../modules/bastion/bastion.py
- path: ../modules/bigip-standalone/bigip_standalone.py
- path: ../modules/dag/dag.py
- path: ../modules/network/network.py
resources:
- name: failover-py
properties:
bigIpImageName: f5-bigip-16-1-2-2-0-0-28-payg-best-plus-1gbps-220505081153
bigIpRuntimeInitConfig01: >-
https://raw.githubusercontent.com/F5Networks/f5-google-gdm-templates-v2/v2.4.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-payg-instance01-with-app.yaml
bigIpRuntimeInitConfig02: >-
https://raw.githubusercontent.com/F5Networks/f5-google-gdm-templates-v2/v2.4.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-payg-instance02-with-app.yaml
cfeTag: bigip_high_availability_solution
cfeBucket: cfe-storage
owner: giroux
provisionPublicIp: true
region: us-west1
restrictedSrcAddressApp: [0.0.0.0/0]
restrictedSrcAddressMgmt: [0.0.0.0/0]
uniqueString: giroux-ha
zone: us-west1-a
type: failover.py
1. From your terminal, launch the GDM template
gcloud deployment-manager deployments create ${DEPLOYMENT_NAME} --config ${CONFIG_FILE}
# Sample Output
The fingerprint of the deployment is b'FauS5Cmgls9Tc39Q6Q8N6w=='
Waiting for create [operation-1657906455110-5e3db6d53e48e-a588ab52-c520aa3a]...done.
Create operation operation-1657906455110-5e3db6d53e48e-a588ab52-c520aa3a completed successfully.
NAME TYPE STATE ERRORS INTENT
giroux-ha-app-int-fw compute.v1.firewall COMPLETED []
giroux-ha-app-subnet compute.v1.subnetwork COMPLETED []
giroux-ha-app-vip-fw compute.v1.firewall COMPLETED []
giroux-ha-application-vm-01 compute.v1.instance COMPLETED []
giroux-ha-bigip-iam-bind gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding COMPLETED []
giroux-ha-bigip-sa iam.v1.serviceAccount COMPLETED []
giroux-ha-bigip-vm-01 compute.v1.instance COMPLETED []
giroux-ha-bigip-vm-01-ti compute.v1.targetInstance COMPLETED []
giroux-ha-bigip-vm-02 compute.v1.instance COMPLETED []
giroux-ha-bigip-vm-02-ti compute.v1.targetInstance COMPLETED []
giroux-ha-cfe-storage storage.v1.bucket COMPLETED []
giroux-ha-ext-network compute.v1.network COMPLETED []
giroux-ha-ext-subnet compute.v1.subnetwork COMPLETED []
giroux-ha-fr-01 compute.v1.forwardingRule COMPLETED []
giroux-ha-ha-fw compute.v1.firewall COMPLETED []
giroux-ha-http-hc compute.v1.healthCheck COMPLETED []
giroux-ha-https-hc compute.v1.healthCheck COMPLETED []
giroux-ha-int-network-02 compute.v1.network COMPLETED []
giroux-ha-int-subnet-02 compute.v1.subnetwork COMPLETED []
giroux-ha-mgmt-fw compute.v1.firewall COMPLETED []
giroux-ha-mgmt-network compute.v1.network COMPLETED []
giroux-ha-mgmt-subnet compute.v1.subnetwork COMPLETED []
giroux-ha-public-ip-01 compute.v1.address COMPLETED []
giroux-ha-tcp-hc compute.v1.healthCheck COMPLETED []
girouxhabigipaccessrole gcp-types/iam-v1:projects.roles COMPLETED []
See the "Validating the Deployment" section or the "Further Exploring" section in the Failover README file for more validation commands.
1. Retrieve the parent template outputs
2. Copy the values for bigIpManagementPublicIp1, bigIpManagementPublicIp2, and vip1PublicIp
gcloud deployment-manager manifests describe --deployment=${DEPLOYMENT_NAME} --format="value(layout)" | yq .resources[0].outputs
# Sample Output
...snippet
- finalValue: 35.199.156.64
name: bigIpManagementPublicIp1
- finalValue: 34.82.195.63
name: bigIpManagementPublicIp2
...snippet...
- finalValue: 35.233.137.160
name: vip1PublicIp
1. Access the BIG-IP devices and enter bash mode (value from bigIpManagementPublicIp1 and bigIpManagementPublicIp2)
ssh admin@35.199.156.64 -i ~/.ssh/id_rsa
admin@(giroux-ha-bigip1)(cfg-sync In Sync)(Active)(/Common)(tmos)# bash
[admin@giroux-ha-bigip1:Active:In Sync] ~ #
2. Review BIG-IP Runtime Init onboard logs
[admin@giroux-ha-bigip1:Active:Standalone] ~ # cat /var/log/cloud/bigIpRuntimeInit.log
# Sample Output
...snippet...
2022-08-11T19:04:42.573Z [3752]: info: Successfully validated declaration
2022-08-11T19:04:42.661Z [3752]: info: Resolving parameters
2022-08-11T19:04:45.259Z [3752]: info: Executing install operations.
2022-08-11T19:05:15.323Z [3752]: info: Executing service operations.
2022-08-11T19:05:15.369Z [3752]: info: Creating - do 1.31.0 ....
2022-08-11T19:08:38.510Z [3752]: info: Creating - cf 1.11.0 ....
2022-08-11T19:08:52.323Z [3752]: info: Creating - as3 3.38.0 ....
2022-08-11T19:09:25.127Z [3752]: info: Creating - do 1.31.0 ....
2022-08-11T19:10:07.330Z [3752]: info: Initializing telemetryClient
2022-08-11T19:10:08.193Z [3752]: info: Sending f5-teem report
2022-08-11T19:10:08.197Z [3752]: info: All operations finished successfully
2022-08-11T19:10:13.570Z [3752]: info: Successfully sent data to F5 Teem
The following tests will work if you deploy the BIG-IP with Layer 4-7 settings. For my demo settings, the "new stack" creates an app server and adds a BIG-IP listener with WAF policy. You can still POST an AS3 declaration to the BIG-IP, or you can login manually and create a pool and listener. Check out the F5 AS3 Docs for more example declarations!
1. From your local machine, curl the demo application (value from vip1PublicIp)
curl http://35.233.137.160 -I
# Sample Output
HTTP/1.1 200 OK
...snippet...
Set-Cookie: BIGipServer~Tenant_1~Shared~Shared_Pool=xxxx; path=/; Httponly
Set-Cookie: TS01aa0884=xxxx; Path=/
2. Perform a security violation by sending a disallowed 'method' of DELETE
curl http://35.233.137.160 -X DELETE
# Sample Output
<html><head><title>Request Rejected</title></head><body>The requested URL was rejected. Please consult with your administrator.<br><br>Your support ID is: 9487250596978229314<br><br><a href='javascript:history.back();'>[Go Back]</a></body></html>
Let's validate a few things and then force a failover.
1. From your terminal, run gcloud to list the forwarding rule and verify the target BIG-IP instance (1 or 2)
gcloud compute forwarding-rules list --filter="name=(giroux-ha-fr-01)"
# Sample Output
NAME REGION IP_ADDRESS IP_PROTOCOL TARGET
giroux-ha-fr-01 us-west1 35.233.137.160 TCP us-west1-a/targetInstances/giroux-ha-bigip-vm-01-ti
The output shows vm-01 which is BIG-IP instance 1. This matches the CLI on each BIG-IP in which instance 1 shows 'active' and instance 2 shows 'standby'.
# From instance 1
[admin@giroux-ha-bigip-vm-01:Active:In Sync] ~ #
# From instance 2
[admin@giroux-ha-bigip-vm-02:Standby:In Sync] ~ #
Now, let's force a failover.
2. From the standby BIG-IP, start a 'tail -f' on the restnoded log file to watch live logs
[admin@giroux-ha-bigip-vm-02:Standby:In Sync] ~ # tail -f /var/log/restnoded/restnoded.log
3. From the active BIG-IP, force a failover
[admin@giroux-ha-bigip-vm-01:Active:In Sync] ~ # tmsh run sys failover standby
[admin@giroux-ha-bigip-vm-01:Standby Sync]
4. Go back to the other BIG-IP instance...now 'active' and review the restnoded logs
# Sample Output
...
Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Performing Failover - update
Thu, 11 Aug 2022 19:46:19 GMT - fine: [f5-cloud-failover] Address discovery: ....
...
...
Thu, 11 Aug 2022 19:46:19 GMT - finest: [f5-cloud-failover] Updating forwarding rule: giroux-ha-fr-01 to target: xxxx/giroux-ha-bigip-vm-02-ti
Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Disassociate NICs successful.
Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Associate NICs successful.
Thu, 11 Aug 2022 19:46:20 GMT - finest: [f5-cloud-failover] updateFwdRule operation name: operation-xxxx
Thu, 11 Aug 2022 19:46:45 GMT - info: [f5-cloud-failover] Updated forwarding rules successfully
...
Thu, 11 Aug 2022 19:46:45 GMT - info: [f5-cloud-failover] Failover Complete
Thu, 11 Aug 2022 19:46:48 GMT - finest: [f5-cloud-failover] Download stateFile: ....
[admin@giroux-ha1122-bigip-vm-02:Active:In Sync] ~ #
The log entries validate the forwarding rule was updated to target instance 2.
5. From your terminal, run gcloud to list the forwarding rules again to validate a new target
gcloud compute forwarding-rules list --filter="name=(giroux-ha-fr-01)"
# Sample Output
NAME REGION IP_ADDRESS IP_PROTOCOL TARGET
giroux-ha-fr-01 us-west1 35.233.137.160 TCP us-west1-a/targetInstances/giroux-ha-bigip-vm-02-ti
Test traffic again (see earlier sections) and everything should still work like magic!
1. Delete the deployment
gcloud deployment-manager deployments delete ${DEPLOYMENT_NAME} -q
This article shows you how to use the BIG-IP Cloud Solution Templates to easily deploy services and BIG-IP in Google Cloud. I use sample configurations in this article to deploy two BIG-IP devices in a failover cluster with a new network stack and a demo application server. I then show you how to validate the deployment, review onboard logs, and test the application. This Failover example is an easy way to have your applications protected by BIG-IP running in HA on Google Cloud. Make sure to check out the other examples in the GitHub repository for scenarios like standalone or autoscale.