Slicing a Superchip: GPU Multitenancy on NVIDIA GB200 & GB300 with ONES Fabric Manager

TL;DR

NVIDIA GB200 and GB300 NVL72 systems pair 72 Blackwell GPUs via a low-latency NVLink fabric, which forms a highly potent rack-scale AI accelerator. Multi-tenant sharing necessitates both GPU and fabric layer isolation. The ONES Fabric Manager delivers a single control plane to autonomously manage GPU partitioning within a rack and coordinate cross-rack InfiniBand partitioning to accommodate a tenant in multiple racks.

The fabric manager also supports per-GPU/tray allocation, atomic provisioning and rollback, live inventory, and automated NVIDIA UFM coordination.

With this functionality, ONES Fabric Manager brings secure, hardware enforced GPU multi-tenancy to both single and multi-rack AI factory environments.

Introduction

AI factories are becoming ever-more rack-scale, with thousands of GPUs integrated into a unified compute environment through NVLink. These NVIDIA GB200 and GB300 NVL72 systems up the ante with 72 Blackwell GPUs clustered together with high bandwidth connectivity, ready for the most demanding AI training workloads. But when an organization brings these systems into a shared infrastructure model, a central question arises: how can different customers utilize the same GPU environment while maintaining complete isolation of their workloads?

Standard VLAN solutions used for Ethernet cannot enforce this isolation as NV Link is not a traditional Ethernet network.

Isolation needs to occur within the fabric itself. This issue is complicated even more when a tenant workload is sprawled across multiple racks, requiring both the NV Link scale-up fabric and the InfiniBand scale-out fabric to be synchronized. ONES Fabric Manager provides the answer by offering a unified control plane with GPU allocation and fabric level isolation. The solution identifies and parses the involved topology, dynamically configures appropriate NV Link partitions, and synchronizes InfiniBand isolation across racks as needed.

This results in a more streamlined process for managing sophisticated multi-tenant GPU environments for operators.

The Problem: How Can One Giant GPU Serve Multiple Customers?

The ONES Fabric Manager tackles this by building it end-to-end. When NVIDIA packs 72 of the GB200 NVL72 and newly announced GB300 NVL72 Blackwell GPUs into what act's as a single giant GPU when using a rack-scale NVLink fabric, that's what gives them their power and what makes them impossible to share in the traditional way. If every GPU can be instantly connected to every other GPU on a full scale basis, how can you share pieces of one large system with multiple customers with the guarantee there will never be any traffic mixing?

That requires fabric-level, hardware based solutions - no ethernet tricks or VLAN's.

We need an solution that keeps the isolation intact, even after a tenant grows beyond a single rack.

How Do NVLink and InfiniBand Work Together in a Multi-Tenant AI Factory?

To support the operation of a rack for a GB200 or GB300 cluster you need to isolate two different fabrics. In a rack, GPUs communicate over a NVLink-based scale-up tier. The scale-out tier carrying traffic between racks is an InfiniBand interconnect, and dual-plane fabrics ensure a pair of independent rails to the compute trays within a rack for band width and reliability.

NVIDIA offers an agent for controlling the fabric layer of each tier.

The ONES Fabric Manager acts as a shared management plane for both layers.

Figure 1: The two-tier interconnect. ONES FM orchestrates both the NVLink tier inside each rack and the InfiniBand tier between racks through one unified API.

This segmentation is an interesting point to remember for the operators. You just need NVLink-level partitioning for rack-contained workload and you need both levels of partitions on rack-wide boundaries. And what coordinates it in the right way - and gracefully recovers even in mid-failure situation - ONES Fabric Manager.

How Does ONES Fabric Manager Automatically Determine the Required Isolation?

The one simple design tenet is that the caller shouldn't need to know or care if a tenant exists on one rack or several.

When asking for GPUs for a tenant, you simply tell the platform WHICH machines and which GPUs you want to give to that tenant. The platformfigures out which racks are implied, portions the NVLink network within each implied rack, and-when those racks are in fact separate-confines all of the cross-rack communication.

It all commits in one atomic operation. If anything goes wrong at any step of the process, the platform cleanly rolls back and returns the fabric to exactly the state you started it in.

Figure 2: Tenant provisioning at a glance. The cross-rack step engages automatically, and only when needed.

Same for scale up, same for scale down, and tenant grew or shrink out to the next rack, they get rack-scale fence created as a "side-effect", and tenant shrinks, they have rack-scale fence removed, as single rack isolation would be adequate.

What Does Hardware-Enforced Tenant Isolation Look Like?

Tenants are allocated a hardware level GPU partition where GPUs operate with complete NVLink bandwidth. There is no interconnectivity between such partitions. In case of multi-rack tenants, they are provisioned a private inter-rack partition in addition to their intra-rack partition. The cross-rack partition is not accessible/joinable by any other tenant.

Figure 3: The tenant’s-eye view. Hard isolation walls inside each rack and a private, fenced partition between racks for tenants that span them.

The result is layered isolation by design:

• Intra-rack isolation is enforced by the NVLink fabric.
• Inter-rack isolation is enforced by the InfiniBand fabric.
• ONES Fabric Manager guarantees that the two remain in agreement.

How Does ONES Fabric Manager Handle Two Tenants Across Two Racks?

Ignore that for now, this is a picture of an ONES Fabric Manager showing exactly what we are describing.

The tenants don’t see or know about each other’s GPUs. The racks neither tenant has claimed remain entirely free.

Two tenants, Gold and Blue, have four GPUs each. They're isolated within a rack. Gold is onrack01. Blue is on rack00.

Since neither tenant is spanning rack boundaries, we isolate at the NV Link layer. No racks to wrangle. No UFM.

Figure 4: Gold tenant: four GPUs allocated on rack01 (node SA-ZT-NDR-02), configuration complete. The rack00 node remains fully available.

Figure 5: Blue tenant: four GPUs allocated on rack00 (node SA-ZT-NDR-01), again fully provisioned and isolated, with the rack01 node left free.

Each tenant occupies its own rack, with 4 GPUs a piece. There are clearly separated, fully insulated resources for each, and each has completed configuration status. Each tenant occupies only one NVLink domain and, as such, multitenancy is achieved in one workflow, avoiding the inter-rack fabric entirely.

What Happens When a Tenant Spans Multiple Racks?

For the above example, each tenant is housed in a single rack, so the entire isolation occurs at the scale-up tier. The second isolation tier becomes applicable when a tenant is stretched across more than one rack. ONES Fabric Manager enlists UFM to cordon off the cross-rack InfiniBand traffic of that tenant, so that isolation is ensured at the domain border as much as isolation is ensured within a domain.

In this image the tenant Blue is deployed in the two racks. ONES Fabric Manager slices GPU partitions for each domain in a rack as previously discussed and, as there is now one tenant spanning two domains, creates a private partition that runs across domains in UFM with a single atomic operation.

Figure 6: ONES FM: the Blue tenant spans both racks, holding three of four GPUs on rack00 (SA-ZT-NDR-01) and three of four on rack01 (SA-ZT-NDR-02), each rack reporting a completed configuration. Note the fabric is flagged externally managed and view-only. ONES FM and UFM stay in sync.

Figure 7: NVIDIA UFM: a dedicated partition per tenant (0x1 Blue, 0x2 Gold), alongside the default management partition. The Blue partition enrolls all six of the tenant’s HCAs, three on each rack, as its members. This is the inter-rack fence that keeps Blue’s cross-rack traffic private.

The two stories come together to form one reality: ONES Fabric Manager allocates all GPUs within a rack, and UFM enforces a corresponding partition so tenant traffic moving between racks cannot bleed into any other tenant's traffic. To both the operator and the consumer, UFM will remain a black box. ONES Fabric Manager is running underneath both tiers and the inter-rack partition manifests itself and de-manifests itself as tenants add and remove servers from rack boundaries.

How Is ONES Fabric Manager Built for Real-World Operations?

A control plane proves its worth when things go wrong, not when they go right.

Beyond provisioning, the integration is engineered for real-world operations:

• Fault containment – Each rack is self-contained. If a controller fails in a rack, the whole system (any other rack) will not be stopped or affected.
• Self-healing – The connections of a failed controller are restored with intelligent back-off. A failed rack is repaired without the intervention of a human operator.
• Fail fast, never guess – When something breaks in the chain that's required, we kill the requests immediately. If racks are down temporarily, requests will instantly and openly fail, and there will be no dangling queues or partially populated tenants.
• Live inventory – The platform monitors GPUs available and in what state in real-time using event streams, while reconciliation in the background ensures the platform state and the fabric state stay in sync.
• All-or-nothing changes – With multi-rack operations either succeeding and fully landing, or failing completely and rolling back, we never leave a tenant configuration half-baked, or compromise the current tenancy setup.

What Changes Between Single-Rack and Multi-Rack GPU Deployments?

Deployment	Isolation Applied	Operator Effort
Single rack	NVLink GPU partitions	One API call
Multiple racks	NVLink partitions + private cross-rack InfiniBand partition, coordinated	Same API call. Topology handled automatically

Operators enjoy one control plane and one mental model, and tenants are physically isolated from one another across both planes of the NVL72 interconnect, regardless of whether you're deploying GB200 or GB300, even when utilizing dual plane fabrics. All the coordination burden remains tucked neatly behind the platform's abstraction layer.

Conclusion

The challenges of achieving rack-scale multitenancy of GPUs are significant and require more than simply assigning them to users. For NVIDIA GB200 and GB300 NVL72 systems, even the NVLink scale-up fabric and InfiniBand scale-out fabric must be isolated in cases where workloads traverse rack boundaries.

ONES Fabric Manager consolidates the isolation, topology and orchestration of these two tiers within a unified control plane and API. It intelligently detects topology details, applies the correct isolation, interfaces with UFM for cross-rack use cases, and provides atomic provisioning and rollback capabilities. The outcome is a realistic service model that enables a rack-scale GPU infrastructure to be transformed into a multitenant service, with hardware-enforced isolation that removes dependency on scripts and ad-hoc tools.

ONES Fabric Manager can provision NVIDIA GB200 and GB300 rack-scale topologies such as NVL72, NVL36, dual-plane scale out fabrics, and allocations down to tray and GPU levels.