# BIG-IP VE in Red Hat OpenShift Virtualization **URL:** \ **Category:** F5 Technical Articles\ **Tags:** cloud, devops, big-ip, big-ip-ve, application-delivery\ **Created:** [September 4, 2024, 12:00pm UTC](https://community.f5.com/t/big-ip-ve-in-red-hat-openshift-virtualization/75039 "2024-09-04T12:00:00Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Ulises\_Alonso](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/ulises_alonso/32/21978_2.png) [@Ulises\_Alonso](https://community.f5.com/u/Ulises_Alonso)\ **Post date:** [September 4, 2024, 12:00pm UTC](https://community.f5.com/t/big-ip-ve-in-red-hat-openshift-virtualization/75039/1 "2024-09-04T12:00:00Z")
[Update: article updated for L2 UDN networks] [  **Update:** please check the article [VMware vSphere to OpenShift Virtualization Migration with F5 BIG-IP](https://community.f5.com/topic/334264) and the companion github repository [https://github.com/f5devcentral/f5-bd-openshift-virt-migration/tree/main](https://github.com/f5devcentral/f5-bd-openshift-virt-migration/tree/main) for more up to date info ] # Overview Running BIG-IP VE in OpenShift Virtualization allows for VM and modern apps/Kubernetes to convergence, simplifying management and operations. OpenShift Virtualization is Red Hat’s enterprise-ready KubeVirt offering which uses the well established QEMU+KVM Linux virtualization layers under the hood. OpenShift Virtualization/KubeVirt provides a Kubernetes declarative interface. Configuring & running BIG-IP VE in OpenShift Virtualization is very much the same as in any other hypervisor or cloud provider. The initial BIG-IP configuration is done likewise using [cloud-init](https://clouddocs.f5.com/cloud/public/v1/shared/cloudinit.html) and [Declarative Onboarding](https://clouddocs.f5.com/products/extensions/f5-declarative-onboarding/latest/). On the other hand, VM and platform configurations are specific of the environment. This article focuses in these latter. All the best practice recommendations in this article are aligned with the [OpenShift Virtualization Reference Implementation Guide](https://access.redhat.com/sites/default/files/attachments/openshift_virtualization_reference_implementation_guide.pdf) which is a recommended reading. Update: the manifests referenced in this article can be found in [f5-bd-openshift-virt-migration/bigip-vsphere-networking at main · f5devcentral/f5-bd-openshift-virt-migration · GitHub](https://github.com/f5devcentral/f5-bd-openshift-virt-migration/tree/main/bigip-vsphere-networking) These contain the Kubernetes manifests to setup a BIG-IP HA pair with the best practices. In the next sections it will be covered each of these highlighting the most important aspects of these as well as including relevant links for additional information. **Note** : In order to use BIG-IP VE in OpenShift Virtualization it is required to [use versions starting with 15.1.10, 16.1.5 and 17.1.1](https://cdn.f5.com/product/bugtracker/ID776117.html). # Platform configuration The base installation of OpenShift Virtualization is done following [these steps](https://docs.openshift.com/container-platform/4.16/virt/install/installing-virt.html). After this, it is best practice to additionally setup CPU Manager to guarantee resource allocation and also setup Topology Manager to set a single single-numa-node policy which guarantees all VM resources are in the same NUMA domain. This [configuration](https://docs.openshift.com/container-platform/4.16/scalability_and_performance/using-cpu-manager.html#setting_up_topology_manager_using-cpu-manager-and-topology_manager) is applied to machineConfigPool with the label custom-kubelet: cpumanager-enabled as it can be seen in the [official configuration example](https://docs.openshift.com/container-platform/4.16/scalability_and_performance/using-cpu-manager.html#setting_up_cpu_manager_using-cpu-manager-and-topology_manager). These configurations are done with a single KubeletConfig manifest which can be found in the attached bigip-platform-manifests.zip. OpenShift Virtualization/KubeVirt uses QEMU/KVM under the hood and typically the [QEMU Guest Agent](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/chap-qemu_guest_agent#sect-QEMU_Guest_Agent) is used to allow low level operations on the VMs. In the case of a F5 BIG-IP VE appliance these operations are not supported and therefore the agent is not needed. # VirtualMachine configuration The attached bigip-vm-manifests.zip contains recommended VirtualMachine configurations for a typical HA pair with network attachments with typical external, internal, ha and management interfaces. This zip file also contains the DataVolume manifests which specify where the qcow2 files of the VE will be residing. This sample manifest downloads the qcow2 file from a web server, [other methods are supported](https://docs.openshift.com/container-platform/4.16/virt/virtual_machines/creating_vms_custom/virt-creating-vms-from-custom-images-overview.html). Please accommodate these to your needs. Next it will be described what are the key sections: - **podAntiAffinity** rules to avoid two BIG-IPs of the same HA-pair to run in the same node - **Resource allocation** : CPU, memory and network queues - Annotations to hint the **CPU scheduler** to enable **low latency** to the BIG-IPs. See this [link for details](https://docs.openshift.com/container-platform/4.16/scalability_and_performance/low_latency_tuning/cnf-provisioning-low-latency-workloads.html). # Network configuration This guide follows the [OpenShift Virtualization Reference Implementation Guide](https://access.redhat.com/articles/7067871) recommendation of using NodeNetworkConfigurationPolicy with type OVS bridge and NetworkAttachmentDefinitions with type ovn-k8s-cni-overlay and topology localnet. Alternatively, It is also fully supported by BIG-IP VE to use any other VirtIO based networking. The ovn-k8s-cni-overlay+localnet networking provides access to the physical network and features micro-segmentation by means of using tags instead of IP addresses as well as better visibility of the traffic. This feature [needs to be enabled on the Operator](https://docs.openshift.com/container-platform/4.15/networking/multiple_networks/configuring-multi-network-policy.html#nw-multi-network-policy-enable_configuring-multi-network-policy) and configured using the [MultiNetworkPolicy](https://docs.openshift.com/container-platform/4.15/networking/multiple_networks/configuring-multi-network-policy.html#configuring-multi-network-policy_working-with-multi-network-policy) resource. It is important to remark that access to the physical network is provided by ovn-k8s-cni-overlay with access ports to the VMs, that is: VLAN trunking or tagging cannot be exposed to VMs. If VLAN trunking or tagging was used previously, then the interfaces and tagging configured in the VLANs will need to be modified. BIG-IP VE can have a total of 28 NICs[[1]](#community-333503-_ftn1). Considering one NIC for management, this means that BIG-IP VE can be connected to 27 VLANs with this network type. **Note: If using BIG-IP prior 17.5.1 it is required to apply a workaround for**  [ID 1492337](https://cdn.f5.com/product/bugtracker/ID1492337.html) . [[1]](#community-333503-_ftnref1) [Linux KVM: BIG-IP VE Setup](https://clouddocs.f5.com/cloud/public/v1/kvm/kvm_setup.html#virtual-machine-network-interfaces) # Considerations when using User-Defined-Networks (UDN) When using UDN with virtualisation, networking typically used is Cluster UDN (CUDN) of type L2 with localnet topology. Networks of this type allow to connect the VMs to physical networks directly but by default these don´t allow set custom IP addresses, or floating IPs in the interfaces of the VMs. In order to be able to use these, it is required to set the parameter ipam.mode to “Disabled” as shown in the next example: ```yaml apiVersion: k8s.ovn.org/v1 kind: ClusterUserDefinedNetwork metadata: name: data spec: namespaceSelector: matchExpressions: - key: kubernetes.io/metadata.name operator: In values: ["udn-test"] network: topology: Localnet localnet: role: Secondary ipam: mode: Disabled physicalNetworkName: cudn-interface vlan: mode: Access access: id: 3010 ``` At time of this writing (Jun26) this is not clearly documented in Red Hat and we have requested to document this publicly. Please use the following statement from Red Hat support in the interim: > Configuring a `ClusterUserDefinedNetwork` with `topology: Localnet` and `ipam.mode: Disabled` is the supported mechanism for relaxing IP anti-spoofing protections within a User Defined Network (UDN). > > With this configuration: > > - IP anti-spoofing protection is disabled for the Localnet UDN. > - Virtual machines are permitted to use specific IP addresses, including scenarios where an IP address may need to move or “float” between VM interfaces. > - This is the intended API-based approach for enabling floating IP use cases, such as network appliances that require ownership of the same IP address across different instances or interfaces. > - MAC spoofing protection remains enforced; setting `ipam.mode: Disabled` only affects IP anti-spoofing behavior. > > The primary consideration is that, when `ipam.mode: Disabled` is used, Multi-Network Policy functionality for that Localnet secondary UDN is reduced. Specifically, pod and namespace selectors are no longer available, and policy peers are limited to `ipBlock` definitions. Please note that because MAC spoofing protection remains enforced, MAC masquerading cannot be used. # Final remarks and next steps In future articles we will describe further how to expand further the possibilities of OpenShift Virtualization by covering the following topics: - Migrating BIG-IP VE from VMware to OpenShift Virtualization - Using BIG-IP VE and CIS connected to a POD network ---
**Author:** ![MichaelOLeary](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/michaeloleary/32/14931_2.png) [@MichaelOLeary](https://community.f5.com/u/MichaelOLeary)\ **Post date:** [September 18, 2024, 4:28pm UTC](https://community.f5.com/t/big-ip-ve-in-red-hat-openshift-virtualization/75039/2 "2024-09-18T16:28:06Z")
Thanks for the article, Ulises! ---
**Author:** ![Kay\_1](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/kay_1/32/18526_2.png) [@Kay\_1](https://community.f5.com/u/Kay_1)\ **Post date:** [November 7, 2024, 10:19am UTC](https://community.f5.com/t/big-ip-ve-in-red-hat-openshift-virtualization/75039/3 "2024-11-07T10:19:13Z")
We look forward to migrate BIG-IP VE from VMware to OpenShift Virtualization, but we have 40 VLANs on the current BIG-IP VE. As it’s stated max. 28 NICs are supported in BIG-IP VE, what other options do we have? Currently we are using VLAN trunking/tagging in VMware. ---
**Author:** ![Ulises\_Alonso](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/ulises_alonso/32/21978_2.png) [@Ulises\_Alonso](https://community.f5.com/u/Ulises_Alonso)\ **Post date:** [November 14, 2024, 11:25am UTC](https://community.f5.com/t/big-ip-ve-in-red-hat-openshift-virtualization/75039/4 "2024-11-14T11:25:55Z")
Hi Kay Please check the followup/more comprehensive PDF attached in [VMware to Red Hat OpenShift Virtualization Migration | DevCentral](https://community.f5.com/topic/334264) Trunking could be your option in OpenShift Virtualization as well: “If VLAN trunking is required towards the VMs, a L2 configuration consisting of a NodeNetworkConfigurationPolicy linux-bridge and a NetworkAttachmentDefinition with cnv-bridge can be used.” Regards Ulises ---
**Author:** ![Kay\_1](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/kay_1/32/18526_2.png) [@Kay\_1](https://community.f5.com/u/Kay_1)\ **Post date:** [August 20, 2025, 7:36am UTC](https://community.f5.com/t/big-ip-ve-in-red-hat-openshift-virtualization/75039/5 "2025-08-20T07:36:02Z")
We found a working NetworkAttachmentDefinition configuration for OpenShift without the need of additional linux-bridge(s). Here an example: > apiVersion: [k8s.cni.cncf.io/v1](http://k8s.cni.cncf.io/v1) > kind: NetworkAttachmentDefinition > metadata: >   annotations: >     description: example >     [k8s.v1.cni.cncf.io/resourceName:](http://k8s.v1.cni.cncf.io/resourceName:) [bridge.network.kubevirt.io/os-core](http://bridge.network.kubevirt.io/os-core) >   name: example >   namespace: example > spec: >   config: ‘{ >     “name”:“example”, >     “type”:“cnv-bridge”, >     “cniVersion”:“0.3.1”, >     “bridge”:“os-core”, >     “macspoofchk”:false, >     “ipam”:{}, >     “preserveDefaultVlan”:false, >     “vlanTrunk”: [{“id”: 120}, {“id”: 130}, {“id”: 140}] >   }’