ABOUT DEVCENTRAL

DevCentral News Technical Forum Technical Articles Technical CrowdSRC Community Guidelines DevCentral EULA Get a Developer Lab License Become a DevCentral MVP

RESOURCES

Product Documentation White Papers Glossary Customer Stories Webinars Free Online Courses F5 Certification LearnF5 Training

SUPPORT

Manage Subscriptions Professional Services Professional Services Create a Service Request Software Downloads Support Portal

PARTNERS

Find a Reseller Partner Technology Alliances Become an F5 Partner Login to Partner Central

---

©2024 F5, Inc. All rights reserved.

Background

I recently had a scenario that required me to do some testing and I thought it would be a good opportunity to share. A user told me that he wants to put BIG-IP in Azure, but he has a few requirements:

\n
1. He wants to use an Azure Load Balancer (ALB) to ensure HA for his BIG-IP pair. This makes failover times faster in Azure, compared to other options.
\n
2. He does not want to use an external Load Balancer. He has internet-facing firewalls that will proxy inbound traffic to BIG-IP, so there is no need to expose BIG-IP to internet. He needs internal BIG-IP's only to provide the app services he needs.
\n
3. He does not want his traffic SNAT'd. He wants app servers to see the true client IP.
\n
4. Ideally he does not want to automate an update of Azure routes at time of failover.
\n
5. He would like to run his BIG-IP pair Active/Active, but could also run Active/Standby.
\n

Quick side note: Why would we use ALB's when deploying BIG-IP? Isn't that like putting a load balancer in front of a load balancer? In this case we're using the ALB as a basic, Layer 3/4 traffic disaggregator to simply send traffic to multiple BIG-IP appliances and provide failover in the case of a VM-level failure. However we want our traffic to traverse BIG-IP's when we need advanced app services: TLS handling, authentication, advanced health monitoring, sophisticated load balancing methods, etc.

Let's analyze this!

Firstly, I put together a quick demo to easily show him how to deploy BIG-IP behind an ALB. My demo uses an external (internet-facing) ALB and an internal ALB. It is based on the official template provided by F5, but additionally deploys an app server and configures the BIG-IP's with a route and an AS3 declaration. If it wasn't for the internal-only part, we would have met his requirements with this set up:

Constraints

However, this user's case requires no internet-facing BIG-IP. And now we hit 2 problems:

\n
1. Azure will not allow 2x internal LB’s in the same Availability Set, or you’ll get a conflict with error:

   NetworkInterfacesInAvailabilitySetUseMultipleLoadBalancersOfSameType

   . So the diagram above cannot be re-created with 2x Internal Azure LB's.
\n
2. When using only 1x internal LB, you “

   should only have one inbound rule if that rule loadbalances across all ports and protocols.

   ” Since we want at least 1 rule for all ports (for outbound traffic from server), we cannot also have individual LB rules for our apps. Trying to do so will fail with the error message in quotes.
\n

Alternatives

This leaves us with a few options. We can meet most of his requirements but one that I have not been able to overcome is this: if your cluster of BIG-IP's is Active/Active, you will need to SourceNAT traffic to ensure the response traffic traverses the same BIG-IP. I'll discuss three options and how they meet the requirements.

\n
1. Use a single Azure internal LB. At BIG-IP, SNAT the traffic to the web server, and send XFF header in place of true client IP. Default route can be the Firewall, so server-initiated traffic like patch updates can still reach internet. Can be Active/Active or Active/Standby, but you must SNAT if you do not want to be updating a UDR at time of failover.
\n
2. Or, don’t SNAT traffic, and web server sees true source IP. You will need a UDR (User Defined Route) to point default route and any client subnets at the Active BIG-IP. You will need to automatically update this UDR at time of failover (automated via F5's Cloud Failover Extension or CFE). Can be Active/Standby only, as traffic will return following the default route.
\n
3. Use a single Azure internal LB, but with multiple LB rules. In our example we'll have 2x Front End IP's configured on the Azure LB, and these will be in different internal subnets. Then we'll have 2x back end pools that consist of the external SelfIP's for 1 rule, and internal SelfIP's for the other. Finally we'll have 2x LB rules. The rule for the \"internal\" side of the LB may listen on ALL ports (for outbound traffic) and the \"external\" side of the LB might listen on 80 and 443 only.
\n

Advanced use cases (not pictured)

\n
1. Single NIC. If you did not want to have a 3-NIC BIG-IP, it would be possible to achieve scenario C above with a single NIC or dual NIC VM: Use a 2-nic BIG-IP (1 nic for mgmt., 1 for dataplane). Put your F5 pair behind a single internal Azure LB with only 1 LB rule which has “HA” ports checked (all ports). We can then have the default route of the server subnet point to Azure LB, which will be served by a VIP 0.0.0.0/0 on F5. Because this only allows you 1 LB rule on the Azure LB, enable DSR on the Azure LB Rule. Designate an “alien subnet range” that doesn’t exist in VNET, but only on the BIG-IP. Create a route to this range, and point the next hop at the frontend IP on the only LB rule. Then have your FW send traffic to the actual VIP on F5 that's within the alien range (not the frontendIP), which will get forwarded to Azure LB, and to F5. I have tested this but see no real advantage and prefer multi-NIC VM's.
\n
2. Alien range. As mentioned above, an \"alien IP range\" - a subnet not within the VNET but configured only on the BIG-IP for VIPs - could exist. You can then create a UDR that points this \"alien range\" toward the FrontEnd IP on Azure LB. An alien range may help you overcome the limit of internal IP's on Azure NIC's, but with a limit of 256 private IP's per NIC, I have not seen a case requiring this. An alien range might also allow app teams to manage their own network without bothering network admins. I would not advise going \"around\" network teams however - cooperation is key. So I cannot find a great use for this in Azure, but I have written about how an alien range may help in AWS.
\n
3. DSR. Direct Server Return in Azure LB means that Azure LB will not perform Destination NAT on the traffic, and it will arrive at the backend pool member with the true destination IP address. This can be handy when you want to create 1 VIP per application in your BIG-IP cluster, and not worry about multiple VIP's per application, or VIP's with /30 masks, or VIP's that use Shared Address lists. However, given that we have multiple options to configure VIP's when Destination NAT is performed by Azure LB (as it is, by default), I generally don't recommend DSR on Azure LB unless it's truly desired.
\n

Personally, I'd recommend in this case to proceed with Option C below. I'd also point out:

\n
• I believe operating in the cloud requires automation, so you should not shy away from automated updates to UDR's when required. Given these can be configured by tools like F5's Cloud Failover Extension (CFE), they are a part of mature cloud operations.
\n
• I personally try to architect to allow for changes later. Making SNAT a requirement may be a limitation for app owners later on, so I try not to end up in a scenario where we enforce SNAT.
\n
• I personally like to see outbound traffic traverse BIG-IP, and not just because it allows apps to see true source IP. Outbound traffic can be analyzed, optimized, secured, etc - and a sophisticated device like BIG-IP is the place to do it.
\n
• Lastly, I recommend best practices we all should know:
\n
• Use template-based deployments for production so that you have Infrastructure as Code
\n
• Ideally keep your BIG-IP config off-box and deploy with AS3 templates
\n
• Get your app and dev teams configuring BIG-IP using declarative deployments to speed your deployment times
\n

Conclusion

There are multiple ways to deploy BIG-IP when your requirements dictate an architecture that is not deployed by default. By keeping in mind my priorities of application services, operational support and high availability, you can decide on the best architecture for a given scenario.

Thanks for reading, and let me know if you have any questions.

Background

I recently had a scenario that required me to do some testing and I thought it would be a good opportunity to share. A user told me that he wants to put BIG-IP in Azure, but he has a few requirements:

\n
1. He wants to use an Azure Load Balancer (ALB) to ensure HA for his BIG-IP pair. This makes failover times faster in Azure, compared to other options.
\n
2. He does not want to use an external Load Balancer. He has internet-facing firewalls that will proxy inbound traffic to BIG-IP, so there is no need to expose BIG-IP to internet. He needs internal BIG-IP's only to provide the app services he needs.
\n
3. He does not want his traffic SNAT'd. He wants app servers to see the true client IP.
\n
4. Ideally he does not want to automate an update of Azure routes at time of failover.
\n
5. He would like to run his BIG-IP pair Active/Active, but could also run Active/Standby.
\n

Quick side note: Why would we use ALB's when deploying BIG-IP? Isn't that like putting a load balancer in front of a load balancer? In this case we're using the ALB as a basic, Layer 3/4 traffic disaggregator to simply send traffic to multiple BIG-IP appliances and provide failover in the case of a VM-level failure. However we want our traffic to traverse BIG-IP's when we need advanced app services: TLS handling, authentication, advanced health monitoring, sophisticated load balancing methods, etc.

Let's analyze this!

Firstly, I put together a quick demo to easily show him how to deploy BIG-IP behind an ALB. My demo uses an external (internet-facing) ALB and an internal ALB. It is based on the official template provided by F5, but additionally deploys an app server and configures the BIG-IP's with a route and an AS3 declaration. If it wasn't for the internal-only part, we would have met his requirements with this set up:

Constraints

However, this user's case requires no internet-facing BIG-IP. And now we hit 2 problems:

\n
1. Azure will not allow 2x internal LB’s in the same Availability Set, or you’ll get a conflict with error:NetworkInterfacesInAvailabilitySetUseMultipleLoadBalancersOfSameType. So the diagram above cannot be re-created with 2x Internal Azure LB's.
\n
2. When using only 1x internal LB, you “should only have one inbound rule if that rule loadbalances across all ports and protocols.” Since we want at least 1 rule for all ports (for outbound traffic from server), we cannot also have individual LB rules for our apps. Trying to do so will fail with the error message in quotes.
\n

Alternatives

This leaves us with a few options. We can meet most of his requirements but one that I have not been able to overcome is this: if your cluster of BIG-IP's is Active/Active, you will need to SourceNAT traffic to ensure the response traffic traverses the same BIG-IP. I'll discuss three options and how they meet the requirements.

\n
1. Use a single Azure internal LB. At BIG-IP, SNAT the traffic to the web server, and send XFF header in place of true client IP. Default route can be the Firewall, so server-initiated traffic like patch updates can still reach internet. Can be Active/Active or Active/Standby, but you must SNAT if you do not want to be updating a UDR at time of failover.
\n
2. Or, don’t SNAT traffic, and web server sees true source IP. You will need a UDR (User Defined Route) to point default route and any client subnets at the Active BIG-IP. You will need to automatically update this UDR at time of failover (automated via F5's Cloud Failover Extension or CFE). Can be Active/Standby only, as traffic will return following the default route.
\n
3. Use a single Azure internal LB, but with multiple LB rules. In our example we'll have 2x Front End IP's configured on the Azure LB, and these will be in different internal subnets. Then we'll have 2x back end pools that consist of the external SelfIP's for 1 rule, and internal SelfIP's for the other. Finally we'll have 2x LB rules. The rule for the \"internal\" side of the LB may listen on ALL ports (for outbound traffic) and the \"external\" side of the LB might listen on 80 and 443 only.
\n

Advanced use cases (not pictured)

\n
1. Single NIC. If you did not want to have a 3-NIC BIG-IP, it would be possible to achieve scenario C above with a single NIC or dual NIC VM: Use a 2-nic BIG-IP (1 nic for mgmt., 1 for dataplane). Put your F5 pair behind a single internal Azure LB with only 1 LB rule which has “HA” ports checked (all ports). We can then have the default route of the server subnet point to Azure LB, which will be served by a VIP 0.0.0.0/0 on F5. Because this only allows you 1 LB rule on the Azure LB, enable DSR on the Azure LB Rule. Designate an “alien subnet range” that doesn’t exist in VNET, but only on the BIG-IP. Create a route to this range, and point the next hop at the frontend IP on the only LB rule. Then have your FW send traffic to the actual VIP on F5 that's within the alien range (not the frontendIP), which will get forwarded to Azure LB, and to F5. I have tested this but see no real advantage and prefer multi-NIC VM's.
\n
2. Alien range. As mentioned above, an \"alien IP range\" - a subnet not within the VNET but configured only on the BIG-IP for VIPs - could exist. You can then create a UDR that points this \"alien range\" toward the FrontEnd IP on Azure LB. An alien range may help you overcome the limit of internal IP's on Azure NIC's, but with a limit of 256 private IP's per NIC, I have not seen a case requiring this. An alien range might also allow app teams to manage their own network without bothering network admins. I would not advise going \"around\" network teams however - cooperation is key. So I cannot find a great use for this in Azure, but I have written about how an alien range may help in AWS.
\n
3. DSR. Direct Server Return in Azure LB means that Azure LB will not perform Destination NAT on the traffic, and it will arrive at the backend pool member with the true destination IP address. This can be handy when you want to create 1 VIP per application in your BIG-IP cluster, and not worry about multiple VIP's per application, or VIP's with /30 masks, or VIP's that use Shared Address lists. However, given that we have multiple options to configure VIP's when Destination NAT is performed by Azure LB (as it is, by default), I generally don't recommend DSR on Azure LB unless it's truly desired.
\n

Personally, I'd recommend in this case to proceed with Option C below. I'd also point out:

\n
• I believe operating in the cloud requires automation, so you should not shy away from automated updates to UDR's when required. Given these can be configured by tools like F5's Cloud Failover Extension (CFE), they are a part of mature cloud operations.
\n
• I personally try to architect to allow for changes later. Making SNAT a requirement may be a limitation for app owners later on, so I try not to end up in a scenario where we enforce SNAT.
\n
• I personally like to see outbound traffic traverse BIG-IP, and not just because it allows apps to see true source IP. Outbound traffic can be analyzed, optimized, secured, etc - and a sophisticated device like BIG-IP is the place to do it.
\n
• Lastly, I recommend best practices we all should know:
\n
• Use template-based deployments for production so that you have Infrastructure as Code
\n
• Ideally keep your BIG-IP config off-box and deploy with AS3 templates
\n
• Get your app and dev teams configuring BIG-IP using declarative deployments to speed your deployment times
\n

Conclusion

There are multiple ways to deploy BIG-IP when your requirements dictate an architecture that is not deployed by default. By keeping in mind my priorities of application services, operational support and high availability, you can decide on the best architecture for a given scenario.

Thanks for reading, and let me know if you have any questions.

Hoping this article and the diagrams can help out those looking to use Azure internal load balancer. Please leave a comment if you have any questions or feedback on this article.

Hi this article is useful for understanding. Can you please share the ARM template for Option B and Option A ? waiting for your kind response.

I have the same scenerio. I dont want to expose F5 WAF directly to the Internet. my WAF will be behind Perimeter firewall. Kindly advise

sajid.khan@qdsnet.com

I would like to share some points related to our scenario.

In Our case, we have an Internet-facing firewall that will proxy inbound traffic to BIGIP (in our case, there is no need to expose BIG-IP to the Internet).

We would like to run BIG IP pair Active/Active but could also run Active/Standby based on your suggestion

we do not want to automate the update of Azure routes at the time of failover

We want to use an Azure Load Balancer (ALB) to ensure HA for our BIG-IP pair. This makes failover times faster in Azure as compared to other options like API. (Please correct me if I am wrong)

Also, we would like to access both F5 pair separately using out of band management.

Based on our use case can you please share the custom ARM template that would help us to deploy BIG IP as per our scenario

Hi Sajid,

Here's a template that will deploy a perimeter firewall and F5 WAF behind it. There's a diagram in the ReadMe that shows the architecture of what will be deployed.

https://github.com/mikeoleary/azure-f5-palo

Here is a devcentral article that explains the demo too.

https://devcentral.f5.com/s/articles/Deploy-F5-and-PaloAlto-in-Azure-with-this-demo

Can you let me know if this helps you? -Mike.

Hi Micheal,

First of All I am really thankful to you for such a nice informative article and second thanks for the reply.

let me check this template and will update you asap.

Can you please share me that template which will deploy only F5WAF?

if I use this same template and after the deployment if i delete the Palo Alto VMs i think this will work for me as well please correct me if i am wrong

Also I have tested just now In this template how Can I assign Existing Vnets and subnets as per our environment ?

In My Case I have to deploy BYOL in Hub Vnet

Hi Sajid,

If you have an existing VNET, I recommend you start with a template from F5 Networks' official repo. This does deploy an External Azure Load Balancer, but you could edit the template to deploy an internal load balancer. Assuming you want an HA pair of 3-NIC devices licensed via BYOL into an existing VNET, this is the template I recommend starting with:

https://github.com/F5Networks/f5-azure-arm-templates/tree/master/supported/failover/same-net/via-lb/3nic/existing-stack/byol

You might deploy this template and then delete the external Load Balancer and deploy an internal Load Balancer instead, if you are uncomfortable with editing a template yourself. I am unable to customize a template specifically for your individual situation.

Does that make sense? You could also ask your F5 account about professional services if you need.