NVIDIA Vera Rubin NVL72: up to 30x more throughput per dollar with just 3% more HBM

NVIDIA published a technical analysis on November 5, 2026, on its official blog, explaining how it plans to deal with growing pressure on high-bandwidth memory (HBM) in AI factories. The company's thesis is that the answer is not simply to put more memory on each GPU, but to co-design GPU, CPU, networking, storage and software — so-called full-stack codesign — to extract more useful work from every byte on the Vera Rubin NVL72 platform.

According to NVIDIA, from the Blackwell Ultra platform to Vera Rubin, HBM capacity per GPU rose by only about 3%, while agentic throughput per dollar rose by up to 30x on SemiAnalysis's AgentX benchmark. At 125 tokens per second per user, the Vera Rubin NVL72 platform would deliver up to 30 times more throughput per dollar than the GB300 NVL72. On the same benchmark, the GB300 NVL72 would deliver up to 10 times more throughput per dollar than the H200 platform, at 100 tokens per second per user.

These figures are NVIDIA's own claims, based on SemiAnalysis's AgentX benchmark, with no independently published verification so far. What the company presents as fact is the strategy behind them.

Why memory became the bottleneck

Frontier models already have trillions of parameters, and the KV cache — the context a model must keep active during each request — is growing too. AI agent trajectories carry context across repeated model and tool calls, in some cases processing up to 3 million tokens. With HBM supply tight and prices rising, every byte of fast memory has become a contested resource.

NVFP4, Vera CPU and context storage

NVIDIA's strategy combines several layers. The NVFP4 format reduces the space needed for model weights compared with FP8, and models such as FLUX.2 [klein] from Black Forest Labs, Inkling from Thinking Machines Lab, Beam 1.0, Meta Muse Spark, Step 5 Preview and Nemotron 3.5 Super VL have already shipped with NVFP4 checkpoints. Within a Vera Rubin NVL72 system, weights can be distributed across up to 72 GPUs, leaving more HBM free for KV cache and concurrent users.

The Vera CPU helps in two ways: faster tool execution shortens the time KV cache sits idle during tool calls, and its LPDDR memory, connected to Rubin GPUs over NVLink-C2C, gives paused context a nearby place to wait and return quickly. When context is not in use, the Dynamo inference software moves it to CMX, a shared context-memory storage tier across servers — with partners including Cloudian, DDN, Dell, Everpure (formerly Pure Storage), HPE, IBM, NetApp, VAST and WEKA.

NVIDIA's message is direct: with memory expensive and scarce, efficiency gains on Vera Rubin NVL72 now come from the whole stack — not from stacking more HBM onto each GPU. It remains to be seen whether the 30x figures hold up outside benchmarks chosen by the manufacturer itself.

Also on Geekiki

Image credit: chart published on the official NVIDIA Blog, used as an illustration.

Sources

Transparency: this article was produced with the help of artificial intelligence, based on official announcements and public sources on the internet, and reviewed before publication. Please consult the original sources linked below to verify the details.

Muse