Automate Multicloud Networking w/ Terraform: routing and app connect on F5 Distributed Cloud

Introduction

Apps are comprised of many smaller services. Often, apps use services spread across different locations. Different regions might have companies specializing in specific services. An app developer might use a map service from a company in Europe, a payment processing service from the US, and cloud storage from Asia – all within a single app. Services in certain locations might be more cost-efficient. Developers can benefit from this by using them without sacrificing functionality. Regulations regarding data privacy can vary by location, and an app might use services that comply with specific regional regulations to ensure user data is protected. Services offered in multiple locations can provide better redundancy and scalability for the app.

The need for NetOps and DevOps teams to rapidly deploy infrastructure and deliver complex apps, including their dependent services from multiple locations while also keeping everything secure is paramount.

Here are some examples of apps with distributed services:

• Online Banking: Financial institutions leverage distributed systems for secure online banking services. This includes user authentication, transaction processing, and account management – all happening across geographically distributed servers.
• E-commerce Platforms: Amazon, eBay, and other online retailers use distributed services for various functions like product listings, shopping carts, and order processing. This enables them to handle high traffic volumes during peak shopping seasons.
• Payment Processing: Services like Stripe (US) and Adyen (Netherlands) handle secure online transactions for apps globally.
• Scientific Computing: Large-scale scientific research projects often utilize distributed computing for tasks like protein folding simulations or weather forecasting. These projects require immense processing power, which can be achieved by harnessing the combined power of multiple computers across different locations.

The next time you use an app, it may be the result of a complex collaboration between multiple service providers working together behind the scenes from various locations. Connecting many services belonging to a distributed app in a secure and automated way is no easy feat. F5 Distributed Cloud (XC) uniquely solves this challenge by making it ridiculously easy to connect any app, anywhere, and deliver it optimally to anyone regardless of region.

Solution Overview

This article shows how using automation with Distributed Cloud can deploy infrastructure, connect an app, secure it, and deliver it with services running in multiple public cloud environments. By orchestrating the integration of services in different cloud providers and by using Distributed Cloud Customer Edge (CE) Sites, Network Connect, App Connect, and F5’s Global Network, together with HTTP security and enhanced firewall rules, F5 XC streamlines the deployment and management in hybrid environments.

This is an automation-exclusive solution intended for NetOps and DevOps teams in large enterprise organizations.

This fully automated end-to-end solution workflow in GitHub uses F5 XC with Customer Edge (CE) sites deployed in AWS, Azure, and GCP to provide network-centric connectivity to the services in running on each site, and app-centric connectivity for traffic ingress.

To provide an example of the modern distributed app, Arcadia Finance, this workflow creates and/or manages resources in the following cloud providers:

• AWS
• Azure
• GCP

This workflow also has the following automation-dependent infrastructure requirements:

• Hashicorp Terraform Cloud
  • Account
• GitHub
  • Account
  • Fork/Clone f5-xc-terraform-examples repo
  • Add to your repo: secrets and variables, all cloud credentials, network settings, as defined in the workflow guide

This distributed app is comprised of four services, three of which run in different locations.

The main app and backend run in AWS; the money transfer app in Azure, and the refer a friend app in Google Cloud. Main calls out to both the backend and money transfer for data and formatting. The API in the money transfer app can be used independently, and when accessed directly, calls out to the remote backend for data.

Connectivity between the services is delivered two ways: 1) App-centric: users access the app with an App Connect HTTP LB that provides Layer7 connectivity to the main and money transfer apps, 2) Network-centric: the apps in different locations communicate internally and use a full-mesh Site Mesh Group. Routing between the sites is direct (CE to CE), however, if direct connectivity isn’t available (not depicted), traffic can go via the Global Network (CE to RE to CE).

Automation Workflow

Accounts and Settings

Distributed Cloud (XC)

A tenant in F5 XC is needed to orchestrate this workflow and deliver the application.  Under tenant administration, add new credentials with type API Certificate, and save both the p12 file and password to use later in GitHub.

Terraform Cloud

A HashiCorp account is required to maintain a state in Terraform Cloud. Using a HashiCorp account, go to app.terraform.io and create a new organization. Configure this organization with the setting: default execution mode “Local”. Next, create either an organization-based or a user-based API token. Both the token and name of the organization will be used by the automation workflow to create workspaces and store the infrastructure state.

GitHub

Using a personal or enterprise GitHub account, fork the main repo f5devcentral/f5-xc-terraform-examples. Next, configure all the variables to access your cloud accounts and deployment parameters, such as the region and which IP addresses to use, as well as the fully qualified domain-name (FQDN) to publicly access the app.

AWS

Both an access key id and its secret will be needed by GitHub and F5 XC to provision infrastructure and deploy a CE site. Create and save these to enter in GitHub as repository secrets.

Azure

Both GitHub and F5 XC use an app registration for Terraform to provision infrastructure and deploy a CE site. Create a new app registration and save the client id and secret to configure in GitHub. Additional permissions need to be added to the app registration and are listed in the GitHub workflow README.

GCP

A service account will be used by both F5 XC and GitHub. Create this and assign the needed IAM roles detailed in the workflow README.

 

The following tables are the complete list of the environment variables and secrets required to complete the workflow.

GitHub Action Secrets

 

GitHub Action Environment Variables

 

(Optional) Existing F5 XC Cloud Credentials (pre-configured for each cloud provider)

 

Deployment: Order of Operations

Using GitHub Actions, open the “Secure MCN Apply” action and run each of the following deployment modes consecutively:

1. aws-vpc-site
2. azure-vnet-site
3. gcp-vpc-site
4. deploy-resources

 

1. aws-vpc-site

 

• Run jobs
  • New/existing VPC
  • New/existing 3 IPv4 subnets
  • New F5 XC Cloud CE VM
  • Route table management

2. azure-vnet-site

 

• Run jobs
  • New/existing resource group
  • New/existing VNet
  • New/existing 3 IPv4 subnets
  • New F5 XC Cloud CE VM
  • User-defined route table management

3. gcp-vpc-site

 

• Run jobs
• New/existing VPC
• New/existing 4 IPv4 subnets
  • 3 subnets for Cloud CE
  • 1 subnet for K8s ingress proxy load balancer
• New/existing service account
• New F5 XC Cloud CE VM

4. deploy-resources

 

• Run jobs
  • AWS
    • New/existing EKS cluster
    • New NGINX+ k8s ingress controller
    • Remote services registration in EKS K8s coredns-custom configmap
    • Example distributed app workloads: Arcadia Finance (main and files)
  • Azure
    • New/existing AKS cluster
    • New ALB K8s ingress controller
    • Remote services registration in AKS K8s coredns-custom configmap
    • Example distributed app workload: Arcadia Finance (api)
  • GCP
    • New/existing GKE cluster
    • New GKE K8s Proxy Load Balancer
    • Example distributed app workload: Arcadia Finance (refer-a-friend)
  • Distributed Cloud (F5 XC)
    • Create/connect Global Network with all CE’s
    • Create/connect direct CE to CE network path with a full-mesh Site Mesh Group
    • Create/deploy Enhanced Firewall Policy to only allow specific app-related traffic between sites
    • Create app ingress HTTP LB on RE using local CE’s to proxy inbound connections to service origins in AWS and Azure

With all jobs completed successfully, we can now access the application at the FQDN configured for the workflow.

Bonus: When Distributed Cloud is also configured as a Primary DNS for the domain name, and HTTP LB managed records is enabled, the FQDN will automatically be registered in DNS and can be resolved publicly, enabling access the frontend app over the Internet with zero manual configuration.

 

Conclusion

This automation-based workflow is just one in a series of workflows available in our F5 XC Terraform Examples repo on GitHub. All the Terraform modules in the repo are open source and are designed to be reusable and can be customized and repurposed to work in even the most unique environments. More information is available in this technical article F5 Distributed Cloud, from manual configuration to Terraform automation, showing how to use and change the other workflows in this repo.

With the need to secure, connect, and deliver apps composed of services in multiple environments, an automation-based workflow saves time by routinely building and deploying infrastructure. Using Terraform with F5 XC to build hybrid cloud environments ensures accuracy by uniformly configuring the connectivity and security policies.

Additional Resources

GitHub

F5 XC Terraform Examples

F5 Hybrid Security Architectures

DevCentral Technical Articles (related)

Complete Multi-Cloud Networking walkthrough with F5 XC

Introducing Secure MCN features on F5 XC

Use F5 Distributed Cloud to control Primary and Secondary DNS

Product Documentation

F5 XC Terraform 

F5 XC App Connect

F5 XC Site Mesh Groups

F5 XC DNS Zone Management

Product Information

F5 XC Network Connect

F5 XC App Connect