Host-Based Networking on GPU Compute Nodes: Bringing Network Leaf Switch Functions to the DPUs with Aviz ONES

TL;DR
Host-Based Networking (HBN) brings VTEP termination, tenant VRF, and BGP routing closer to the server by offloading VTEPs to the NVIDIA BlueField-3 DPU. Aviz ONES helps orchestrate the entire operational lifecycle, starting with Day-0 DPU and switch configuration through per-tenant VF allocation, taking a sophisticated feature and automating it to repeatable operations.
Introduction
Networking architectures must scale with bigger GPU clusters and more tenants. Today's modern AI infrastructure demands this. When the BGP routing, tenant VRFs, and VTEP termination live in each leaf switch, all the networking state lives on the leaf.
Host-Based Networking (HBN) puts these functions onto a BlueField-3 DPU within each host. Network functions are distributed across your rack, with routing, VXLAN encapsulation, and tenant separation living closer to where your computation occurs.
Of course, moving networking function into each host brings its own operational challenges. How do you configure FRR, BGP EVPN, and VRF-to-VNI bindings? How do you automate VRF to VNI, VF setup, and binding for tens, hundreds, or thousands of hosts and DPUs without manual configuration?
That's where the Aviz ONES orchestration layer comes in. ONES automates the networking fabric's fabric setup and network function life cycle and offers a unified orchestration experience for switches, DPUs, tenants, and virtual functions.
What Is Host-Based Networking?
Host-Based Networking (HBN) is a DOCA service that is implemented as a container on the BlueField-3 DPU Arm cores, which effectively turn the DPU into a server-side BGP router. Instead of relying on hypervisor vSwitches to provide routing and VXLAN tunnel termination, HBN allows you to build an all L3 underlay fabric that doesn't add a burden on the host. At the heart of HBN is the Free Range Routing stack, FRR. FRR is provided as a container running in an isolated environment on the DPU, where BGP peering to the upstream leaf switches is established on the DPU uplink ports. All routing decisions, VXLAN encapsulation, and tenant VRF enforcement are performed on the DPU, before packets arrive at the host CPUs.
Routing and bridging tables in the Linux kernel networking stack on the DPU are translated from software to hardware tables on the DPU via the nl2docad driver. Traffic is forwarded in hardware, not software, on the BlueField-3.
This results in a server-side networking architecture, where leaf switches can act as a simple L3 transport layer, with all BGP, EVPN, VXLAN, and tenant isolation occurring on the NIC itself.
What Changes When HBN Is Enabled?
You have this traditional non-HBN setup where the leaf switch is in charge of all of it. Inbound VTEP termination, inbound tenant V RFs, the tenant SVI Vlan interfaces – all those tenant boundaries are on your leaf switch. When you start scaling the GPU cluster out to hundreds of nodes and tens of tenants, you start having a concentration of that state that is concentrated on the leaf that starts to be the limitation and the single point of failure. So with HBN we're shifting and distributing that state and essentially every DPU that connects into a switch becomes an independent VTEP and each DPU is running it's own tenant VRFs, the tenant Vlan terminating S V I will live off of your leaf and will live off the DPU off of that server port.
| Configuration | Non-HBN | HBN with Aviz ONES |
|---|---|---|
| VTEP Termination | Leaf switch | DPU (HBN container) |
| Tenant VRF | Leaf switch | DPU (HBN container) |
| SVI Interface (VLAN) | Leaf switch | DPU, server-facing port (pf0hpf_if) |
| BGP Routing | Leaf switch | FRR on DPU Arm cores |
| Tenant Isolation Boundary | Switch port | VF on DPU |
This change also has direct operational implications: Instead of configuring VRFs, VLANs, and VTEP entries on the leaf in traditional, non-HBN deployments, this activity is offloaded to the DPU with fully automated configuration performed by Aviz ONES. All an operator needs to do is add a tenant, associate a server with that tenant, and create a VF.
How Does Multi-Tenancy Work on a Shared Server?
We're able to bring many tenants up in a single hardware server at a time and leverage VF's in the DPU, at hardware-level to isolate each tenant. The DPU brings up 8 VFs per HGxx and 4 per GBxx node when it comes up, and these VFs remain in the DPU unconfigured on Day-0, until ONES gets ready and connects it to a tenant during the provisioning time of the cluster.
What Is a VF?
The Virtual Function (VF) is a piece of hardware (i.e. The DPU) that the DPU spits out of itself upon boot. Contrast this to a software vSwitch which simply multiplexes flows over a single NIC.
This VF is essentially a direct path into the DPU's built-in FRR router and maps directly out to an interface you can see on the host OS.
There's a one-to-one, opaque mapping across the DPU boundary. The FRR interface pf0vf2_if becomes VF 2, for example, on the server side. So if you are running a VM on your server, you bind that VM to VF 2, traffic leaves your VM, comes into the DPU, gets routed (via FRR) into the appropriate tenant VRF, wrapped inVXLAN with the right VNI, and exits the DPU onto the uplink to go to the rest of the fabric. All of that complexity is hidden from the server OS, which simply sees a standard network interface.
Figure 1: VFs provisioned at DPU startup. FRR-visible interfaces (pf0vfX_if, pf1vfX_if) on the DPU side map 1:1 to host-visible interfaces (vf0 to vf7) on the server side. No interface is assigned to a tenant at this stage.
How Does VF-Based Hardware Isolation Work?
We isolate VFs by enabling their isolation in hardware on the DPU's embedded switch. We don't do the isolation on the host CPU, and host software can't override it.
How Does ONES Provision a Tenant Onto a VF?
The provisioning process in Aviz ONES has four stages and each stage has a defined boundary and owner in terms of operators and the platform:
Step 1: How Is a Tenant Created?
The operator creates a tenant in ONES. ONES generates VNI, VRF, and subnet for the tenant. Then it returns the tenant details to the operator.
Step 2: How Is a Server Allocated?
The operator binds a server to the tenant.
If fabric is enabled with DPU, ONES will reprogram uplink leaf switches to fuse the VNI to DPU, the tenant's network is fabric-sided.
No, the VF-to-host connectivity is not established yet.
Step 3: How Are Available VFs Discovered?
The ONES system can retrieve all registered VFs available in the server provisioned for it and their current status, free, provisioned or error; ONS can only allocate and provide VFs that are free at the current point in time.
Step 4: How Is a VF Assigned to a Tenant?
- Operator picks an available VF and it's handed over to tenant.
- ONES configs the tenant VLAN on the VF port on the switch in the DPU and configures the new VTEP endpoint in the leaf switch EVPN config, also modifies the FRR VRF-to-VNI binding in DPU.
- VM connects to host-side VF and thus south-to-north end-to-end connectivity is provided.
Figure 2: Two tenants assigned on the same server. Tenant BLUE is wired to vf2 via pf0vf2_if and Tenant RED to vf6 via pf1vf2_if. The remaining 6 VFs stay free for future tenant assignments.
Steps 1 and 2 are the existing ONES tenant workflow. Steps 3 and 4 are the HBN-specific additions, both fully orchestrated by ONES.
The operator selects a VF and invokes the assignment. ONES handles the switch configuration, the DPU FRR update, and the EVPN synchronization atomically.
How Does ONES Configure HBN in the Network Setup Tab?
Your Aviz ONES HBN configuration starts in the Network Setup section, and in particular the N-S Network configuration:
Turning on DPU HBN requires one switch setting. Enabling HBN will change how and what gets provided by ONES throughout the entire fabric. The following 5 settings are coupled and the way they're configured defines the entire N-S network model.
Figure 3: N-S Network Setup: the five fields that define HBN behavior, tenant isolation, and encapsulation.
Disabling or Enabling the DPU HBN functionality on ONES lets ONES know about which hosts in the fabric has the BlueField-3 DPUs with HBN container. With that, ONES now models VTEP termination at the DPU itself and not at the leaf. It also flips the credential required at deploy time.
Instead, ONES directly programs the DPUs.
So it needs the DPU IP's and credentials rather than hosts entries.
What Does the Operating System Field Control?
The Operating system team decides the NOS of the NOS to be provided to the N-S switches, which can either be Cumulus Linux or SONiC. It will always be a lean version of Cumulus Linux for the DPU internally irrespective of the NOS decision of the switch.
What Does the Starting Subnet Define?
This represents the very first octet of the IP address space of the N-S network fabric. Every subnet will be anchored off of this.
What Does Tenant Aware Storage Control?
Tenant Aware (Storage) decides if ONES can setup N-S dedicated network segments pertenant used for traffic within storage-related traffic. If set to true, the traffic related to storage at the DPU are segmented with user's traffic. At HBN container, both the data and management plane use the similar model of VRF-per- tenant.
What Does Tenant Segmentation Define?
Tenant Segmentation identifies the encapsulation technology.
If and when DPU HBN is enabled,VXLAN alone can be used. It's by design. Since HBN relies on EVPN as the control plane and EVPN's control plane needs VXLAN as data plane encapsulation, VLAN-only segmentation can't be used.
Why Is VXLAN Mandatory With HBN?
In ONES,VXLAN is enforced on validation. This means that configuring a VLAN with DPU HBN enabled results in an error and preventsfabric from deploying.
The control plane for EVPN within FRR necessitate an encapsulation for VXLAN. This is not a choice rather it is a matter of design.
What Does ONES Automate at Deploy Time?
Deploytab is where the HBN fabric design fits physical infrastructure in. There's just one thing where we do it differently the day we turn on DPU HBN, and the deploy process for it requires import host entries along with the DPU IPs and its credentials in Import Management IPs as that enables ONES to connect to the HBN container running inside DPU Arm cores and applies the configuration.
Figure 4: Deploy tab: the pre-flight checklist that gates HBN fabric deployment.
How Does ONES Validate DPU Readiness?
One is that ONES performs an automated check when deploying against your infrastructure, before moving forward, that the necessary VFs are detected on each of the DPUs. Each HGxx DPUs expect to have 8 VFs. Each GBxx DPUs expect to have 4 VFs.
If either the expected or remaining count is missing from a given DPU, then a deploy command is rejected.
The Apply Design to Fabric operation is not visible to the user until the check has been successfully performed for all DPUs in the fabric design. In essence, we want to prevent partial deployments where the fabric is configured to use a DPUs VFs that do not exist.
How Does ONES Validate Switch and Server Inventory?
Check Switch Inventory and Check Server Inventory are the actions that ensure all detected devices are running with the appropriate NOS versions to match your fabric design. In the case of HBN fabric, we would be checking that leaf switches which are connected to DPU are operating the NOS from the Network Setup tab.
How Are Border Leaf Ports Configured?
Border Leaf Port Config represents the two uplink ports from the border leaf and connects to the external firewall.
On a HBN deployment, incoming traffic reaches the border leaf and is sent into the fabric underlay to finish at the DPU VTEP to represent the tenant. Without this being defined, the ONES is unable to determine how traffic would traverse into the fabric from your outside.
What Happens When Apply Design to Fabric Executes?
When Apply Design to Fabric executes, ONES orchestrates the full deployment simultaneously:
- Leaf switch EVPN configuration
- DPU Day-0 configuration through the HBN container
- BGP adjacency setup between DPUs and leaf switches
- VXLAN overlay instantiation
The fabric transitions from a design model to a live, operating infrastructure in a single coordinated push.
How Does ONES Make HBN Deployable at Scale?
HBN represents a fundamental transition in networking whereby routing, tenancy enforcement and VTEP termination is pushed from a core switch infrastructure to each server where DPUs are resident. This new distributed architecture allows for easier scaling, a reduction in leaf switch state, and hardware-enforced multitenancy without the burden of the hypervisor layer. However, this new HBN architecture, devoid of orchestration, simply distributes the complexity rather than reduce it.
At scale, configuring FRR, EVPN, VRF-to-VNI mapping, and V F configuration per-tenant onto each of the server resident DPUs manually is simply not feasible to operate.
The Aviz ONES platform has adapted HBN into a simplified workflow. Fabric design is entered just once. ONES is used to validate the environment, pushing the Day 0 configuration for the switches and DPUs.ON ES also controls the end to end lifecycle of a tenant including creating tenants, assigning VF s, and de- provisioning tenants. By combining the distributed intelligence of H BN with the central command of the ONES orchestration layer, we have created a production-ready A I network architecture.
Conclusion
Instead of running these crucial networking operations on leaf switches, Host-Based Networking leverages BlueField-3 DPU to distribute that intelligence, offering routing and tenant isolation, along with VTEP termination and VXLAN networking. This scalable architectural pattern also brings the benefits of hardware-driven multi-tenancy. The trick, however, is managing it all at scale - coordinating those functions distributed across a myriad of DPUs adds tremendous operational friction.
Aviz ONES eliminates that friction with its synchronized, top-to-bottom workflow across all networking switches, DPU cards, tenants, and their VFs.
From initial fabric design and Day-0 setup to VM VF-to-DPU VFs and EVPN data sync, ONES provides automation, orchestration, and pre-deploy validation. This ensures a repeatable process for deploying and operating HBN AI fabrics. In short, keeping the intelligence decentralized without the management nightmare.
Frequently Asked Questions
What is Host-Based Networking (HBN)?
HBN is a DOCA service implemented as a container on the BlueField-3 DPU Arm cores that turns the DPU into a server-side BGP router. It offloads VTEP termination, tenant VRF enforcement, and VXLAN encapsulation from the leaf switch to the DPU itself. required.
What role does Aviz ONES play in an HBN deployment?
Aviz ONES automates the full operational lifecycle of HBN, from Day-0 switch and DPU configuration to per-tenant VF allocation and de-provisioning. It provides a unified orchestration layer for switches, DPUs, tenants, and virtual functions.
How does multi-tenancy work on a shared server with HBN?
The DPU provisions up to 8 VFs per HGxx node and 4 per GBxx node at boot, and each VF provides hardware-level tenant isolation enforced on the DPU's embedded switch. ONES assigns these VFs to tenants during provisioning, configuring the appropriate VLAN, VRF-to-VNI binding, and EVPN settings automatically.
Why is VXLAN required when HBN is enabled?
HBN relies on EVPN as its control plane, and EVPN requires VXLAN as the data plane encapsulation by design. ONES enforces this at validation, returning an error if VLAN-only segmentation is configured alongside HBN.
How does ONES validate DPU readiness before deployment? Log?
ONES performs an automated pre-flight check to confirm that each DPU exposes the expected number of VFs (8 for HGxx, 4 for GBxx). The Apply Design to Fabric option is not available until all DPUs in the fabric design pass this check.
What does ONES configure when Apply Design to Fabric is executed?
ONES simultaneously pushes leaf switch EVPN configuration, DPU Day-0 configuration through the HBN container, BGP adjacency setup between DPUs and leaf switches, and VXLAN overlay instantiation in a single coordinated operation.
How does ONES provision a tenant onto a VF?
An operator creates a tenant in ONES, binds a server to that tenant, selects an available VF, and ONES handles the switch configuration, DPU FRR update, and EVPN synchronization atomically. The VM on the host then connects to the host-side VF to establish end-to-end connectivity.
What is the difference between HBN and a traditional non-HBN setup?
In a non-HBN setup, the leaf switch handles VTEP termination, tenant VRFs, SVI interfaces, and BGP routing, creating a concentration of state that becomes a scaling bottleneck. With HBN, each DPU acts as an independent VTEP running its own tenant VRFs, distributing that state across the rack.
Learn how Aviz ONES simplifies AI GPU fabric operations by combining centralized orchestration with the distributed networking capabilities of NVIDIA BlueField-3 DPUs.





