Introduction
A GPU cloud sells GPU hours. What it can’t easily sell today is a network service in front of those GPUs that belongs to one tenant: a load balancer with that tenant’s own addresses, its own routing table, and its own health monitors. Ideally, it consumes no GPU-host compute resources and remains completely isolated from every other tenant.
NVIDIA’s Infrastructure Controller (NICo) already decides what a tenant’s network looks like. It owns tenancy, VPCs, IP address management, and the tenant overlay that runs on each server’s BlueField-3 DPU. F5 BIG-IP Next for Kubernetes (BNK) extends that architecture with tenant-scoped application delivery and load balancing: the F5 Traffic Management Microkernel (TMM) data plane runs inside the same BlueField-3 DPU, in the traffic path, delivered by the same framework that delivers the tenant network.
This article explains how the pieces fit together. It starts with the business view, then goes deep enough for the people who will have to run it. Two short videos (about five minutes each) show everything described here running live on BlueField-3 hardware:
Part 1 — The Platform
Part 2 — The Proof
Why the DPU is the right place for the load balancer
Every GPU server in an AI factory sits on two networks. The east-west rail fabric carries RoCE and NCCL collectives between GPUs. It is lossless, tightly engineered, and has no place for services. The north-south converged fabric carries everything that enters or leaves a tenant: inference requests, storage access, and provisioning. That traffic crosses the server’s BlueField-3 DPU, where NVIDIA DOCA HBN (Host-Based Networking) terminates the tenant’s EVPN/VXLAN overlay.
So the DPU already serves as the tenant’s front door. Putting the load balancer there has three consequences that matter to an operator:
- No new appliances, no new rack space. The service runs on Arm cores and ConnectX silicon the operator has already bought, inside servers the tenant is already paying for.
- The GPU host stays untouched. With NVIDIA DOCA Platform Framework (DPF) in zero-trust mode, the host sees a plain NIC. There is no agent, no privileged container, and no access to the DPU’s operating system.
- Isolation is enforced below the tenant’s workload. Every isolation mechanism described below runs on the DPU, outside the kernel the tenant controls.
How it fits into NICo
NICo stays in charge of what it already owns: the VPC, the tenant’s address space, and the overlay on the DPU. It does not reconfigure the physical leaf or spine switches. It only expects the fabric to speak BGP and EVPN. F5 adds a thin layer on top:
- A load-balancer service API next to NICo’s existing tenancy API. A tenant declares a load balancer against its VPC, and NICo allocates the virtual IP from that tenant’s own prefix.
- An operator that turns each declaration into standard Kubernetes Gateway API objects (Gateway, L4Route, and Pool) in a namespace dedicated to that tenant.
- The BNK control plane (f5ingress), which watches those objects and programs TMM over gRPC.
- TMM, the F5 data plane, running on every DPU and hardware-accelerated through DOCA.
From the tenant’s point of view this is one more NICo resource. From the operator’s point of view it is one more DPF service.
Deployed before the first tenant exists
The architecture’s main design choice is the order of operations: the load balancer is infrastructure, not a per-tenant install.
DPF provisions the DPUs without touching the host. It discovers them over Redfish, flashes the BlueField bundle, and joins the DPU Arm cores as Kubernetes nodes in a cluster managed from outside the server. TMM is then delivered as a DPUService, declared the same way as HBN and the rest of the DPU networking plumbing and reported ready the same way.
When TMM comes up on each DPU, it attaches to HBN through two trusted Scalable Functions (SFs). These are DPDK ports carved out of the ConnectX by DPF and wired into a DPF service chain:
- an external leg on which virtual-server traffic arrives from the tenant’s VRF, and
- an internal leg toward the tenant’s origin servers on their stretched L2 segment.
All of this exists before the first tenant is provisioned. The SFs, the service chain, and the TMM pod (data plane, routing stack, route-health injection, and telemetry sidecars) are part of the DPU’s baseline, like HBN itself.
Tenants don’t get new ports. They get 802.1Q VLANs multiplexed onto the same SF pair. Each tenant’s external and internal legs are a pair of tagged VLANs on TMM. HBN maps each tag into that tenant’s bridge domain and VRF. TMM pays a real cost per DPDK port, so the design multiplexes ports instead of multiplying them. As a result, the per-tenant footprint on the DPU is a few VLAN tags and some routing state, not new hardware resources.
Getting the VIP into the fabric
When a virtual server is programmed, TMM’s routing stack (ZebOS, fed by tmrouted route-health injection) advertises the VIP as a /32 to HBN over an eBGP session in the tenant’s VRF. HBN re-originates it as an EVPN type-5 route into the tenant’s L3VNI. Both DPUs do this, so the fabric sees the VIP as anycast with ECMP. If one DPU is lost, the tenant loses capacity, not availability.
One design rule we strongly recommend: run the BGP session over the same leg the traffic forwards on. If BGP runs on a separate control-plane link, the eBGP next-hop is that link’s address, and the fabric learns a VIP that leads nowhere. When BGP peers over the data-plane leg, the session doubles as a liveness signal: if the data plane dies, the session drops and the fabric withdraws the VIP within the hold time. HBN accepts the peer with a dynamic-neighbor listen-range, so TMM’s address on each leg comes from IPAM and never has to be hard-coded.
Multi-tenancy, layer by layer
Tenant isolation here does not rest on a single mechanism. It is a stack of independent ones, and none of them runs in the workload’s kernel:
- NICo VPC and IPAM: each tenant has its own VPC and its own VIP prefix.
- Kubernetes: each tenant has its own namespace and its own Gateway API objects.
- 802.1Q VLAN: a frame is classified into a tenant as soon as it leaves TMM’s port.
- Bridge domain / L2VNI: a tenant’s origin servers share one stretched L2 segment and can reach nothing else.
- VRF / L3VNI: each tenant has a separate routing table, end to end across the fabric. A tenant’s prefix does not exist outside its VRF.
To test this, we probed every tenant’s VIP from every tenant’s VRF. Only the diagonal answers: three tenants, nine probes, and zero cross-tenant reachability. This is not the result of a firewall rule that someone could edit. The routes simply aren’t present in the other tenants’ tables.
Adding and removing tenants is a non-event
Because the SFs, the service chain, and TMM itself exist before any tenant does, onboarding a tenant takes two API calls:
- Register the tenant in NICo. This creates its VPC and the VIP prefix its load balancers will draw from. The call is idempotent: run it again and you get the same identifiers back.
- Create the load balancer. NICo allocates a VIP from that prefix. The operator renders a Gateway, a route, and a pool into the tenant’s namespace. The BNK control plane programs TMM on both DPUs, and the VIP appears in the tenant’s VRF from both of them.
Here is what nobody had to touch: no new Scalable Function, no new service chain, no moved cable, no change on the GPU host, no hand-edited leaf configuration, and no TMM restart. The new tenant’s origin traffic rides a tagged sub-interface of a port another tenant was already using.
The same holds in reverse. Removing a tenant withdraws its VIP routes and its VLAN and VRF state. Nothing that other tenants depend on is restarted or re-wired, so adding or removing a tenant does not affect existing tenants. In the Part 2 video, a third tenant comes up while the other two keep serving, and the isolation matrix grows from 2×2 to 3×3 with a clean diagonal.
Where the bytes actually move
TMM makes the load-balancing decision, and the ConnectX moves the bytes. Once TMM has accepted a flow, DOCA offloads its forwarding into the BlueField’s embedded switch. The bulk of the traffic never enters TMM’s software path.
The Part 2 video shows this under live load. In that test setup, a single tenant pushes about 130 Gbit/s on the wire through the VIP. The DPU’s own eSwitch counters show that traffic passing through, while TMM’s byte counters, over exactly the same seconds, register about 100 kbit/s. TMM isn’t idle; those bytes simply never reached it. That is a gap of about six orders of magnitude, and it is the core argument for putting the service on the DPU.
In this test environment, we also observed:
- about 126–130 Gbit/s of bulk throughput per tenant, limited by the test clients rather than by the DPUs;
- about 256k–319k HTTP requests per second for small objects;
- a DPU load average of about 3–4 of 16 Arm cores for the whole stack under full load;
- zero failed requests out of 100,000, with a mean latency of 0.5 ms;
- a newly onboarded tenant reaching 129 Gbit/s on its first benchmark, with no tuning.
Operations: centralized visibility
None of this has to be watched from a terminal. Both DPUs and every DPF service placed on them can be monitored from a single console. Per-tenant gateways, traffic flow, and byte counters are read directly from the TMM data plane. For the operator, the DPUs are infrastructure to manage, not machines to log into.
Availability
F5 is developing this integration with NVIDIA NICo and DPF, and we’re targeting early access around NVIDIA GTC Berlin, October 21–22, 2026. If you’re building a GPU cloud on NICo and want tenant-scoped load balancing and ingress on BlueField-3, contact your F5 account team to discuss early-access participation.
Summary
- NICo owns the tenant’s network on the DPU. F5 BNK adds the tenant’s load balancer to that same DPU.
- DPF zero-trust delivers TMM as a DPUService. The GPU host is never touched.
- TMM attaches to HBN through two Scalable Functions before the first tenant exists. Tenants are VLANs and VRFs multiplexed onto that pair, not new hardware.
- VIPs are anycast EVPN type-5 routes from every DPU, advertised over the data-plane leg itself.
- Isolation is layered: VPC, namespace, VLAN, L2VNI, and VRF.
- Onboarding takes two API calls, and adding or removing a tenant doesn’t affect existing tenants.
- The ConnectX moves the bytes: in our test setup, about 130 Gbit/s per tenant, with TMM’s software path seeing about 100 kbit/s.
Related content
- Video — Per-Tenant LB on NVIDIA NICo via DPF Zero-Trust, Part 1: The Platform
- Video — Per-Tenant LB on NVIDIA NICo via DPF Zero-Trust, Part 2: The Proof
- BIG-IP Next for Kubernetes Nvidia DPU deployment walkthrough
- NVIDIA Infrastructure Controller (NICo)
- BIG-IP Next for Kubernetes on NVIDIA BlueField DPUs