NVIDIA Pushes AI Factories Toward More Tokens Per Watt With Vera Rubin and DSX
NVIDIA is shifting the focus of AI infrastructure from peak computing performance toward a different measure: how much useful AI work can be produced from every megawatt of power.
At the AI Infra Summit in Santa Clara, California, NVIDIA executive Ian Buck outlined new developments around the company’s Vera Rubin platform, DSX power-management software, networking technologies and partnerships designed to make increasingly power-hungry AI factories more efficient. The event runs September 15–17 and has more than 8,000 attendees, according to NVIDIA.
The announcements come as agentic AI workloads become more demanding. AI systems that reason through multiple steps, call tools and maintain large contexts can consume substantially more compute than a conventional single-turn request.
NVIDIA’s argument is that the next generation of AI data centers cannot simply add more GPUs. Operators also need to get more useful output from the power and infrastructure they already have.
NVIDIA Targets More AI Work From the Same Power Budget
The centerpiece of NVIDIA’s approach is its full-stack AI factory platform, which combines Vera Rubin computing systems with networking, inference software and power-management technologies.
The company says DSX MaxLPS can dynamically redistribute available power across GPUs and racks based on workload requirements. Instead of reserving the maximum amount of power for every system, the software can recover unused electrical capacity and direct it toward workloads that can use it.
That approach could become increasingly important as utilities and data-center operators face limits on how much additional electricity can be supplied to AI facilities.
NVIDIA says DSX MaxLPS can provide up to 1.4 times more tokens per megawatt through factory-wide power optimization. For future Vera Rubin NVL72 deployments, the company says the technology can enable up to 40% more GPU capacity within the same megawatt budget in suitable environments.
Lambda Reports 23% Better Performance Per Watt
One of the new results presented at the summit came from AI cloud provider Lambda.
Lambda tested DSX MaxLPS on NVIDIA Blackwell GPU servers and ran 19 nodes within a power budget that would typically support 16 full-power nodes. NVIDIA said the configuration increased cluster-wide token throughput by 24%, from roughly 4 million to 5 million tokens per second.
Performance per watt improved by 23%.
The result illustrates why NVIDIA is increasingly emphasizing power efficiency as an AI infrastructure metric. More computing capacity can potentially be deployed without proportionally increasing the facility’s power envelope.
The actual gains, however, will depend on workload characteristics, deployment conditions and how effectively the available power can be redistributed.
Vera Rubin Targets Agentic AI Efficiency
NVIDIA is also positioning Vera Rubin NVL72 specifically for the next phase of AI inference.
The company says Vera Rubin can deliver up to 40% more GPUs within the same site-power envelope and up to 35% higher token throughput without requiring additional power lines. Rack-level power smoothing and energy buffering are designed to absorb short power spikes so systems can operate closer to sustained demand.
NVIDIA is pairing Vera Rubin with its Groq 3 LPX technology for workloads where low and predictable latency is particularly important.
For a 100,000-token-context Qwen 3.8 27B workload, NVIDIA reported 2,529 output tokens per second per user with Groq 3 LPX. The company also claims up to 35 times higher token throughput per megawatt than GB200 NVL72 for certain models with more than 2 trillion parameters and long contexts.
Those figures are NVIDIA-reported results and apply to specific workloads and configurations rather than representing a universal performance advantage across AI applications.
SemiAnalysis Reports Up to 30x Higher Throughput Per Megawatt
Another major data point comes from SemiAnalysis’ AgentX benchmark, which is designed around real-world agentic coding sessions rather than isolated single-request inference tests.
NVIDIA said Vera Rubin NVL72 delivered up to 30 times higher throughput per megawatt than NVIDIA GB300 NVL72 on the DeepSeek V4 Pro workload measured by AgentX. NVIDIA also cited up to 45 times lower cost per million tokens in those results.
SemiAnalysis separately published its own analysis of Vera Rubin’s AgentX performance and reported substantial efficiency improvements compared with previous-generation systems. Its analysis emphasizes that results vary depending on the model, inference engine and target response speed.
That distinction matters because AI benchmark numbers can change dramatically depending on workload design and software configuration.
Agentic AI is particularly challenging to benchmark because an agent can accumulate large amounts of context while making tool calls and spawning additional tasks. A benchmark that captures the complete trajectory therefore measures a different kind of performance than a conventional single-request test.
Emerald AI Demonstrates AI Data Centers Responding to the Grid
NVIDIA’s strategy also extends beyond the data center itself.
Emerald AI and NVIDIA demonstrated a flexible-load program with Silicon Valley Power in which an AI factory responded automatically to grid signals. According to NVIDIA, the system handled hundreds of demand signals while protecting higher-priority AI workloads.
Emerald AI plans to use NVIDIA DSX Flex with its Conductor software to dynamically adjust energy consumption based on electricity-grid conditions and available energy sources.
Under the system, lower-priority AI jobs can temporarily be throttled when the grid needs relief and then resume when conditions improve.
That creates a different model for AI infrastructure: instead of treating a data center as a fixed electricity consumer, operators can potentially make some computing workloads responsive to the grid.
NVIDIA Expands the Infrastructure Stack
The broader Vera Rubin platform combines computing, networking, inference and infrastructure-management technologies.
NVIDIA highlighted technologies including sixth-generation NVLink for scale-up computing, Spectrum-X Ethernet and ConnectX networking for large AI clusters, BlueField infrastructure technologies and Dynamo inference software.
The company also announced additional ecosystem activity, including work with Amazon’s Annapurna Labs on NVHBM custom high-bandwidth memory technology and an NVLink Fusion integration involving d-Matrix Raptor XPUs.
Pinterest is also using NVIDIA Blackwell and Dynamo for conversational AI features in visual discovery, showing how the infrastructure push extends beyond traditional model training into consumer-facing inference workloads.
Why Tokens Per Megawatt Are Becoming Important
The underlying problem is straightforward: AI models are becoming more capable, but the infrastructure required to run them is consuming enormous amounts of electricity.
Adding GPUs can increase capacity, but power availability can become the limiting factor for new AI factories.
That makes tokens per megawatt an increasingly useful way to think about AI infrastructure efficiency. If a system can generate more useful inference output from the same electrical capacity, an operator may be able to serve more workloads without expanding the facility’s power allocation at the same rate.
NVIDIA’s latest announcements therefore represent more than another GPU performance push. The company is trying to make compute, networking, software and electricity management operate as a single system.
The key question now is how these results translate from controlled benchmarks and early deployments into large commercial AI factories running diverse workloads.
With Vera Rubin, DSX and related networking and inference technologies moving into broader deployment, that real-world validation will become increasingly important.
