Fortify HashiCorp Terraform Cloud automation for BIG-IP using Open Policy Agent - OPA

Introduction

The Open Policy Agent (OPA, pronounced “oh-pa”) is an open-source, general-purpose policy engine that unifies policy enforcement across the stack by providing a high-level declarative language that lets you specify policy as code and simple APIs to offload policy decision-making from your software. OPA decouples policy decision-making from policy enforcement and is becoming popular with Kubernetes ecosystems. When it comes to F5 infrastructures such as BIG-IP or NGINX, OPA can be used to provide guard rails around deployment automation and configuration files. HashiCorp Terraform is commonly used by many customers to deploy BIG-IP infrastructure and now Terraform Cloud allows its users to utilize OPA’s policy framework (as an alternative to HashiCorp's sentinel) to write and enforce policies for Terraform automation.

Here are some sample BIG-IP automation scenarios:

In the Terraform resource ‘security group’ if the user exposes the CIDR block as “0.0.0.0/0” open to the world for SSH traffic, before this configuration is applied, a rule in the OPA policy could flag this as an error in configuration, allowing the user to correct.
If the AS3 JSON has some sensitive information like “Private Key” which is getting exposed, OPA can flag those.

The screenshot below shows how OPA flagged a failure when the SSH security group for BIG-IP is accidentally left wide open. The OPA Policy will show the failure when the Terraform plan is run and will not allow the BIG-IP instance to be deployed in the cloud.

Sample OPA BIG-IP Policy

package terraform.policies.public_ingress

import input.plan as tfplan

deny[msg] {
  r := tfplan.resource_changes[_]
  r.type == "aws_security_group"
  r.change.after.ingress[_].cidr_blocks[_] == "0.0.0.0/0"
  msg := sprintf("%v has 0.0.0.0/0 as allowed ingress", [r.address])
}

How to use OPA with BIG-IP?

Here are the steps to follow to use an Open policy agent with Terraform Cloud and BIG-IP.

Fork the example Git repository https://github.com/scshitole/bigip-opa and Git clone to customize the terraform.tf file with your own Terraform Cloud organization name and desired workspace name
Login to Terraform Cloud https://app.terraform.io/app/organizations and click the new workspace button in the Workspaces list.

Select the CLI-driven workflow option. We are selecting the CLI-driven option so that we can issue Terraform CLI commands from the terminal that you are using on your laptop. The Terraform state information will be securely stored in Terraform Cloud.

Provide a workspace name matching what you have in the terraform.tf file as shown.

Staying on the workspace option select variables on the left as shown below, Add AWS Credentials to the workspace. Make sure the values are marked sensitive as shown.

Follow the instructions on the page displayed when the workspace is created. It instructs you to run Terraform CLI commands for this new workspace.
Next create an OPA policy set in Terraform Cloud, to create the OPA policy. Click on your organization name and then settings on the left as shown below:

Then click on Policy Sets just below the policies to create a new OPA Policy
Click on Connect a new policy set as shown below

Click on GitHub and connect your VCS repository which you did a git clone in Step 1

You can browse to locate the repository; in your case, it will be your forked repository

Then select the option Open Policy Agent and provide any name to the policy.

Click the option “Policies enforced on selected workspaces” and the new workspace you just created to the policy as shown below. Finally, click on connect policy set. This will apply the policy to the workspace.

Go to terminal CLI and issue command terraform init & terraform apply
The policy will be triggered, and you can see it on the terraform cloud
The OPA policy will fail and show the reason in the description

You will see the OPA policy in action and it will not allow the BIG-IP to be deployed in the cloud before we fix the issue, in this case, the security group needs to be something other than “0.0.0.0/0”

To fix this issue we must make changes to file modules/network/main.tf

data "aws_availability_zones" "available" {
  state = "available"

  filter {
    name   = "zone-type"
    values = ["availability-zone"]
  }
}

data "aws_ec2_instance_type" "bigip" {
  instance_type = var.bigip_instance_type
}


module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.14.0"

  cidr = var.vpc_cidr

  azs             = data.aws_availability_zones.available.names
  private_subnets = var.private_subnet_cidrs
  public_subnets  = var.public_subnet_cidrs

  enable_nat_gateway   = true
  enable_vpn_gateway   = false
  enable_dns_hostnames = true
}

resource "aws_security_group" "bigip" {
  name   = "bigip_ssh"
  vpc_id = module.vpc.vpc_id

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "bigip" {
  instance_type = var.bigip_instance_type
  ami = "ami-09ae9af26d2e96786"

  subnet_id              = module.vpc.public_subnets[0]
  vpc_security_group_ids = [aws_security_group.bigip.id]

  lifecycle {
    precondition {
      condition     = data.aws_ec2_instance_type.bigip.default_cores <= 4
      error_message = "Change the value of bigip_instance_type to a type that has 4 or fewer cores to avoid over provisioning."
    }
  }
}

Above replace "0.0.0.0/0" with "192.168.0.0/16" in the ingress cidr_blocks

Once those changes are made, we need to commit those changes into GitHub as the new changes will be picked by the OPA policy and Terraform Cloud.

Do git commit with those changes in main.tf file, are reflected on the GitHub repository.
Execute terraform apply and you can see that the OPA policy is passed and the deployment of BIG-IP moves forward.

OPA file structure on GitHub

You can see that all the policy files are under the OPA directory

The policies are defined in policies.hcl file, in the above example we saw the policy “public_ingress” was used

cat opa/policies.hcl

policy "public_ingress" {
  query = "data.terraform.policies.public_ingress.deny"
  enforcement_level = "mandatory"
}

To further see details about the public_ingress policy, we need to check the file under opa/policies/public_ingress.rego

package terraform.policies.public_ingress

import input.plan as tfplan

deny[msg] {
  r := tfplan.resource_changes[_]
  r.type == "aws_security_group"
  r.change.after.ingress[_].cidr_blocks[_] == "0.0.0.0/0"
  msg := sprintf("%v has 0.0.0.0/0 as allowed ingress", [r.address])
}

Here we are looking for “0.0.0.0/0” and denying the policy.

Conclusion

HashCorp’s sentinel is a great framework for setting automation guardrails. OPA provides a more flexible and portable way to enforce IT best practices while deploying application infrastructures such as BIG-IP in the public and private clouds.