NVIDIA Vera Rubin NVL72 Posts Up to 3.7x Throughput Gain in MLPerf Inference Debut
NVIDIA’s next-generation Vera Rubin NVL72 AI system has made its first preview submission to MLPerf Inference, with results showing up to 3.7x higher throughput than the GB300 NVL72 on the Qwen3-VL benchmark.
The results were released September 16 as part of MLPerf Inference v6.1, the latest version of MLCommons’ industry benchmark for measuring AI inference performance. MLCommons said the new release includes submissions from a record 30 participating organizations and introduces tests designed around newer AI workloads, including retrieval-augmented generation and agentic inference.
For NVIDIA, the results provide an early look at how Vera Rubin is expected to handle increasingly demanding AI inference workloads as companies deploy larger models and AI agents.
Vera Rubin NVL72 reaches up to 3.7x higher throughput
NVIDIA submitted Vera Rubin NVL72 preview results for two demanding workloads in MLPerf Inference v6.1: DeepSeek-R1 and Qwen3-VL.
On Qwen3-VL, NVIDIA reported up to 3.7x higher throughput than GB300 NVL72 across offline, server and interactive scenarios. The submission used vLLM together with NVIDIA’s Dynamo inference framework.
On DeepSeek-R1, the company reported throughput of up to 2.5x higher than GB300 NVL72, using NVIDIA TensorRT-LLM.
The published NVIDIA developer benchmark data lists a Qwen3-VL result of 1,307 queries per second for a 72-GPU Vera Rubin NVL72 system, compared with 349 queries per second for a 72-GPU GB300 NVL72 configuration in the listed offline scenario.
Because the Vera Rubin figures are identified as preview submissions, they should be treated as early benchmark results rather than a complete picture of production performance.
What is driving the performance?
NVIDIA attributes the results to a combination of hardware and software changes rather than a single component.
The Vera Rubin platform uses enhanced Tensor Cores and Transformer Engine technology for inference operations, while NVFP4 precision reduces the memory footprint associated with model weights, attention and the KV cache.
NVIDIA also used disaggregated serving techniques that separate the prefill and decode stages of inference. Large-scale expert parallelism was used for mixture-of-experts models such as DeepSeek-R1 and Qwen3-VL.
The system’s rack-scale architecture is built around sixth-generation NVIDIA NVLink and NVLink Switch technology. NVIDIA says the interconnect provides higher packet rates and lower latency than conventional Ethernet, helping distribute inference workloads across the GPUs inside an NVL72 rack.
GB300 NVL72 shows how multi-rack scaling matters
The same MLPerf release also highlighted NVIDIA’s current GB300 NVL72 platform.
A DeepSeek-R1 submission expanded from one 72-GPU GB300 NVL72 rack to four racks containing 288 GPUs. NVIDIA reported 99% scaling efficiency in the offline scenario, meaning throughput increased nearly proportionally as additional GPU capacity was added.
That measurement is important for large AI deployments because simply adding GPUs does not guarantee equivalent increases in useful throughput.
Networking, workload scheduling, model parallelism and software all have to scale alongside the accelerator count. High scaling efficiency can therefore affect how effectively expensive AI infrastructure is used.
GB300 NVL72 also posted results on the WAN 2.2 text-to-video benchmark, where NVIDIA reported 0.65 720p videos per second at 5.7 seconds per video for the multi-GPU configuration. NVIDIA said that represented 9x higher throughput and 7.5x lower latency than a single node.
Software remains a major part of the gains
The MLPerf results also underline how much AI inference performance can change through software optimization.
NVIDIA said its GB300 NVL72 Qwen3-VL results improved by as much as 1.6x compared with MLPerf Inference v6.0. The company attributed those gains to changes including lower KV-cache precision, kernel fusion, improved kernels and disaggregated serving using vLLM and NVIDIA Dynamo.
NVIDIA also disclosed additional post-submission results for GPT-OSS-120B and DLRMv3. Those results have not yet been verified by MLCommons, so they should not be treated as official MLPerf results.
That distinction matters as AI hardware comparisons increasingly depend on the software stack running on top of the accelerators.
MLPerf is expanding toward agentic AI
The timing of the Vera Rubin results also reflects a broader change in how AI systems are being measured.
MLPerf Inference v6.1 adds an End-to-End RAG benchmark and an Edge Agentic Inference benchmark, both designed around workloads that involve multiple steps rather than a single model response. MLCommons also added support for speculative decoding in selected interactive inference scenarios.
NVIDIA separately said Vera Rubin NVL72 achieved substantially higher performance than GB300 NVL72 in preview testing with the SemiAnalysis AgentX benchmark. That result comes from a different benchmark and should therefore not be directly treated as an MLPerf result.
The shift toward agentic workloads could make measurements such as latency, multi-step execution and system-level throughput increasingly important alongside traditional token-generation benchmarks.
What the Vera Rubin results mean for AI infrastructure
The MLPerf v6.1 results give infrastructure buyers an early benchmark-based view of NVIDIA’s transition from Blackwell-era systems toward Vera Rubin.
MLCommons describes MLPerf Inference as an architecture-neutral and reproducible benchmark intended to provide comparable performance information for organizations evaluating AI systems. The latest release includes hardware ranging from accelerators and servers to edge systems, with Vera Rubin NVL72 listed among the preview platforms.
For NVIDIA, the combination of Vera Rubin’s early throughput results, GB300’s multi-rack scaling and continued software optimization highlights a broader strategy: improving the entire AI infrastructure stack rather than relying only on faster accelerators.
The Vera Rubin figures are still preview results, so future validated submissions and production deployments will provide additional data on how the platform performs across a wider range of models and workloads.
For now, the MLPerf Inference v6.1 debut establishes Vera Rubin NVL72 as one of the newest systems being measured as AI inference moves toward larger models, multimodal applications and increasingly complex AI agents.
