Help
Technical Articles
F5 SMEs share good practice.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Technical Forum Quicklinks: No Replies | Active-Not Solved | Recent Solutions
Custom Alert Banner

F5 AppWorld 2024 session proposal deadline extended to Friday, December 8th!

Deploy Failover BIG-IP Cluster with New Stack in Google Cloud using v2 Templates

Jeff_Giroux_F5
Legacy Employee
Legacy Employee

‎23-Aug-2022 05:00 - edited ‎23-Aug-2022 10:59

In this article, I plan to cover how-to deploy a pair of BIG-IP devices with failover and high availability features into a Google Cloud new network stack.

How-to v2 Failover Example: BIG-IP Cluster on Google Cloud with New Network Stack

The following section will be a walkthrough with steps to deploy two BIG-IP instances in GCP using the Failover example with a new network stack. When complete, the BIG-IP devices will form an Active/Standby cluster. You can use the same techniques in order to copy the templates, modify as needed, and deploy in your environment.

During a failover event on the BIG-IP devices, the new active device (was standby) will detect the problem via heartbeats. The new active BIG-IP device sends API calls to Google API to move objects like alias IP ranges, forwarding rule IP mappings (aka public IPs), and more. This API failover method is thanks to the magic in the F5 Cloud Failover Extension, part of the F5 Automation Toolchain.

gcp-bigip-failover.gif

Clone GitHub Repository

1. Navigate to GitHub repo for v2 BIG-IP GDM templates

2. Select the "Code" button and copy the HTTPS address

Jeff_Giroux_0-1650947864747.png

3. Open a terminal (ex. Visual Studio Code) and clone the repository

cd ~/Desktop
git clone https://github.com/F5Networks/f5-google-gdm-templates-v2.git

Create Secret in GCP Secret Manager

One prerequisite for this deployment is to use a secret from GCP Secret Manager. Secret Manager stores API keys, passwords, certificates, and other sensitive data. The secret will be used for the BIG-IP administrator password during onboarding.

1. From your terminal, create a file called password.txt containing the BIG-IP password in one line.

2. Create the secret with gcloud using the password.txt file

Note: For demo purposes of "new stack", use the secret name mySecretId to avoid the need to customize the BIG-IP onboarding files.

gcloud secrets create mySecretId --data-file="password.txt"
# Sample Output
Created version [1] of the secret [mySecretId]

Modify Parameters - New Stack

For this section, I will deploy two BIG-IP instances with a new network stack in GCP. The result will be the following:

  • 4x New VPCs: mgmt, external, internal, app (and related components)
  • Application instance running docker, container=f5devcentral/f5-demo-app:latest
  • 1x Storage bucket for BIG-IP cloud failover state
  • 2x BIG-IP instances, version=16.1.2.2
  • 4x CPU, 3-NICs
  • Using runtime-init-conf-3nic-payg-instance01-with-app.yaml and runtime-init-conf-3nic-payg-instance02-with-app.yaml from examples in F5 GitHub repo
  • BIG-IP cluster with HA, network, application, and WAF configurations

Here we go!

Note: For demo purposes of this article, we will use a tagged version of the repo to maintain article consistency with code.

1. From your terminal, set DEPLOYMENT_NAME, set CONFIG_FILE, change folders, and change branch to pinned tag

DEPLOYMENT_NAME=giroux-bigip-failover-new
CONFIG_FILE=sample_failover.yaml

cd f5-google-gdm-templates-v2/examples/failover
git checkout tags/v2.4.0.0

2. Review the entire README for instructions and prerequisites

3. Edit the file sample_failover.yaml

Note: For demo purposes of "new stack", use the cfeTag value "bigip_high_availability_solution" to avoid the need to customize the BIG-IP onboarding files.

# sample_failover.yaml - BIG-IP Failover Cluster with new stack
# Note: Commented and some Optional lines were removed from
#       the yaml below. This keeps the code block small
#       for easier illustration purposes.

imports:
  - path: failover.py
  - path: ../modules/access/access.py
  - path: ../modules/application/application.py
  - path: ../modules/bastion/bastion.py
  - path: ../modules/bigip-standalone/bigip_standalone.py
  - path: ../modules/dag/dag.py
  - path: ../modules/network/network.py
resources:
  - name: failover-py
    properties:
      bigIpImageName: f5-bigip-16-1-2-2-0-0-28-payg-best-plus-1gbps-220505081153
      bigIpRuntimeInitConfig01: >-
        https://raw.githubusercontent.com/F5Networks/f5-google-gdm-templates-v2/v2.4.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-payg-instance01-with-app.yaml
      bigIpRuntimeInitConfig02: >-
        https://raw.githubusercontent.com/F5Networks/f5-google-gdm-templates-v2/v2.4.0.0/examples/failover/bigip-configurations/runtime-init-conf-3nic-payg-instance02-with-app.yaml
      cfeTag: bigip_high_availability_solution
      cfeBucket: cfe-storage
      owner: giroux
      provisionPublicIp: true
      region: us-west1
      restrictedSrcAddressApp: [0.0.0.0/0]
      restrictedSrcAddressMgmt: [0.0.0.0/0]
      uniqueString: giroux-ha
      zone: us-west1-a
    type: failover.py

Deploy the BIG-IP Failover Cluster - New Stack

1. From your terminal, launch the GDM template

gcloud deployment-manager deployments create ${DEPLOYMENT_NAME} --config ${CONFIG_FILE}
# Sample Output
The fingerprint of the deployment is b'FauS5Cmgls9Tc39Q6Q8N6w=='
Waiting for create [operation-1657906455110-5e3db6d53e48e-a588ab52-c520aa3a]...done.                            
Create operation operation-1657906455110-5e3db6d53e48e-a588ab52-c520aa3a completed successfully.
NAME                             TYPE                                                   STATE      ERRORS  INTENT
giroux-ha-app-int-fw         compute.v1.firewall  COMPLETED  []
giroux-ha-app-subnet         compute.v1.subnetwork  COMPLETED  []
giroux-ha-app-vip-fw         compute.v1.firewall  COMPLETED  []
giroux-ha-application-vm-01  compute.v1.instance  COMPLETED  []
giroux-ha-bigip-iam-bind     gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding COMPLETED  []
giroux-ha-bigip-sa           iam.v1.serviceAccount  COMPLETED  []
giroux-ha-bigip-vm-01        compute.v1.instance  COMPLETED  []
giroux-ha-bigip-vm-01-ti     compute.v1.targetInstance  COMPLETED  []
giroux-ha-bigip-vm-02        compute.v1.instance  COMPLETED  []
giroux-ha-bigip-vm-02-ti     compute.v1.targetInstance  COMPLETED  []
giroux-ha-cfe-storage        storage.v1.bucket  COMPLETED  []
giroux-ha-ext-network        compute.v1.network  COMPLETED  []
giroux-ha-ext-subnet         compute.v1.subnetwork  COMPLETED  []
giroux-ha-fr-01              compute.v1.forwardingRule  COMPLETED  []
giroux-ha-ha-fw              compute.v1.firewall  COMPLETED  []
giroux-ha-http-hc            compute.v1.healthCheck  COMPLETED  []
giroux-ha-https-hc           compute.v1.healthCheck  COMPLETED  []
giroux-ha-int-network-02     compute.v1.network  COMPLETED  []
giroux-ha-int-subnet-02      compute.v1.subnetwork  COMPLETED  []
giroux-ha-mgmt-fw            compute.v1.firewall  COMPLETED  []
giroux-ha-mgmt-network       compute.v1.network  COMPLETED  []
giroux-ha-mgmt-subnet        compute.v1.subnetwork  COMPLETED  []
giroux-ha-public-ip-01       compute.v1.address  COMPLETED  []
giroux-ha-tcp-hc             compute.v1.healthCheck  COMPLETED  []
girouxhabigipaccessrole      gcp-types/iam-v1:projects.roles  COMPLETED  []

Validating the Deployment

See the "Validating the Deployment" section or the "Further Exploring" section in the Failover README file for more validation commands.

1. Retrieve the parent template outputs

2. Copy the values for bigIpManagementPublicIp1, bigIpManagementPublicIp2, and vip1PublicIp

gcloud deployment-manager manifests describe --deployment=${DEPLOYMENT_NAME} --format="value(layout)" | yq .resources[0].outputs
# Sample Output
...snippet
- finalValue: 35.199.156.64
  name: bigIpManagementPublicIp1

- finalValue: 34.82.195.63
  name: bigIpManagementPublicIp2

...snippet...
- finalValue: 35.233.137.160
  name: vip1PublicIp

SSH to BIG-IP Cluster and Review Logs

1. Access the BIG-IP devices and enter bash mode (value from bigIpManagementPublicIp1 and bigIpManagementPublicIp2)

ssh admin@35.199.156.64 -i ~/.ssh/id_rsa

admin@(giroux-ha-bigip1)(cfg-sync In Sync)(Active)(/Common)(tmos)# bash
[admin@giroux-ha-bigip1:Active:In Sync] ~ #

2. Review BIG-IP Runtime Init onboard logs

[admin@giroux-ha-bigip1:Active:Standalone] ~ # cat /var/log/cloud/bigIpRuntimeInit.log

# Sample Output
...snippet...
2022-08-11T19:04:42.573Z [3752]: info: Successfully validated declaration
2022-08-11T19:04:42.661Z [3752]: info: Resolving parameters
2022-08-11T19:04:45.259Z [3752]: info: Executing install operations.
2022-08-11T19:05:15.323Z [3752]: info: Executing service operations.
2022-08-11T19:05:15.369Z [3752]: info: Creating - do 1.31.0 ....
2022-08-11T19:08:38.510Z [3752]: info: Creating - cf 1.11.0 ....
2022-08-11T19:08:52.323Z [3752]: info: Creating - as3 3.38.0 ....
2022-08-11T19:09:25.127Z [3752]: info: Creating - do 1.31.0 ....
2022-08-11T19:10:07.330Z [3752]: info: Initializing telemetryClient
2022-08-11T19:10:08.193Z [3752]: info: Sending f5-teem report
2022-08-11T19:10:08.197Z [3752]: info: All operations finished successfully
2022-08-11T19:10:13.570Z [3752]: info: Successfully sent data to F5 Teem

Testing the Application and WAF

The following tests will work if you deploy the BIG-IP with Layer 4-7 settings. For my demo settings, the "new stack" creates an app server and adds a BIG-IP listener with WAF policy. You can still POST an AS3 declaration to the BIG-IP, or you can login manually and create a pool and listener. Check out the F5 AS3 Docs for more example declarations!

1. From your local machine, curl the demo application (value from vip1PublicIp)

curl http://35.233.137.160 -I
# Sample Output
HTTP/1.1 200 OK
...snippet...
Set-Cookie: BIGipServer~Tenant_1~Shared~Shared_Pool=xxxx; path=/; Httponly
Set-Cookie: TS01aa0884=xxxx; Path=/

2. Perform a security violation by sending a disallowed 'method' of DELETE

curl http://35.233.137.160 -X DELETE
# Sample Output
<html><head><title>Request Rejected</title></head><body>The requested URL was rejected. Please consult with your administrator.<br><br>Your support ID is: 9487250596978229314<br><br><a href='javascript&colon;history.back();'>[Go Back]</a></body></html>

Testing Failover

Let's validate a few things and then force a failover.

1. From your terminal, run gcloud to list the forwarding rule and verify the target BIG-IP instance (1 or 2)

gcloud compute forwarding-rules list --filter="name=(giroux-ha-fr-01)"
# Sample Output
NAME                 REGION    IP_ADDRESS      IP_PROTOCOL  TARGET
giroux-ha-fr-01  us-west1  35.233.137.160  TCP          us-west1-a/targetInstances/giroux-ha-bigip-vm-01-ti

The output shows vm-01 which is BIG-IP instance 1. This matches the CLI on each BIG-IP in which instance 1 shows 'active' and instance 2 shows 'standby'.

# From instance 1
[admin@giroux-ha-bigip-vm-01:Active:In Sync] ~ # 

# From instance 2
[admin@giroux-ha-bigip-vm-02:Standby:In Sync] ~ #

Now, let's force a failover.

2. From the standby BIG-IP, start a 'tail -f' on the restnoded log file to watch live logs

[admin@giroux-ha-bigip-vm-02:Standby:In Sync] ~ # tail -f /var/log/restnoded/restnoded.log

3. From the active BIG-IP, force a failover

[admin@giroux-ha-bigip-vm-01:Active:In Sync] ~ # tmsh run sys failover standby
[admin@giroux-ha-bigip-vm-01:Standby Sync]

4. Go back to the other BIG-IP instance...now 'active' and review the restnoded logs

# Sample Output
...
Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Performing Failover - update
Thu, 11 Aug 2022 19:46:19 GMT - fine: [f5-cloud-failover] Address discovery: ....
...
...
Thu, 11 Aug 2022 19:46:19 GMT - finest: [f5-cloud-failover] Updating forwarding rule: giroux-ha-fr-01 to target: xxxx/giroux-ha-bigip-vm-02-ti
Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Disassociate NICs successful.
Thu, 11 Aug 2022 19:46:19 GMT - info: [f5-cloud-failover] Associate NICs successful.
Thu, 11 Aug 2022 19:46:20 GMT - finest: [f5-cloud-failover] updateFwdRule operation name: operation-xxxx
Thu, 11 Aug 2022 19:46:45 GMT - info: [f5-cloud-failover] Updated forwarding rules successfully
...
Thu, 11 Aug 2022 19:46:45 GMT - info: [f5-cloud-failover] Failover Complete
Thu, 11 Aug 2022 19:46:48 GMT - finest: [f5-cloud-failover] Download stateFile: ....

[admin@giroux-ha1122-bigip-vm-02:Active:In Sync] ~ #

The log entries validate the forwarding rule was updated to target instance 2.

5. From your terminal, run gcloud to list the forwarding rules again to validate a new target

gcloud compute forwarding-rules list --filter="name=(giroux-ha-fr-01)"
# Sample Output
NAME                 REGION    IP_ADDRESS      IP_PROTOCOL  TARGET
giroux-ha-fr-01  us-west1  35.233.137.160  TCP          us-west1-a/targetInstances/giroux-ha-bigip-vm-02-ti

Test traffic again (see earlier sections) and everything should still work like magic!

Delete and Clean Up

1. Delete the deployment

gcloud deployment-manager deployments delete ${DEPLOYMENT_NAME} -q

Summary

This article shows you how to use the BIG-IP Cloud Solution Templates to easily deploy services and BIG-IP in Google Cloud. I use sample configurations in this article to deploy two BIG-IP devices in a failover cluster with a new network stack and a demo application server. I then show you how to validate the deployment, review onboard logs, and test the application. This Failover example is an easy way to have your applications protected by BIG-IP running in HA on Google Cloud. Make sure to check out the other examples in the GitHub repository for scenarios like standalone or autoscale.

Resources

Article Series

Labels:
0 Kudos
Version history
Last update:
‎23-Aug-2022 10:59
Updated by:
Legacy Employee
Contributors
Copyright © 2023 Khoros, LLC