NVIDIA Vera Rubin NVL72 Sets New Standards for Agentic AI Inference

New MLPerf Endpoints v1.0 results, published by NVIDIA on November 16, 2026, highlight the Vera Rubin NVL72 platform's significant lead in agentic AI…

On November 16, 2026, the NVIDIA Developer Blog announced the provisional MLPerf Endpoints v1.0 benchmark results, showcasing the NVIDIA Vera Rubin NVL72 platform's remarkable performance in agentic AI inference. The new system demonstrated over 10 times the throughput per kilowatt compared to the NVIDIA GB300 NVL72, maintaining a shared minimum end-to-end interactivity of 50 tokens per second per user. These results, though provisional and pending MLCommons Association verification, position the Vera Rubin NVL72 as a critical advancement for AI workloads that demand both high capacity and low latency, especially in complex multi-turn interactions characteristic of AI agents.

Unprecedented Efficiency for Agentic Workloads

The MLPerf Endpoints v1.0 benchmarks are designed to evaluate the balance between serving capacity and responsiveness for agentic AI, specifically using coding and enterprise workflow scenarios via API endpoints. These workloads are particularly challenging as AI agents often carry extensive conversation histories and generate a mix of reasoning steps, tool calls, and final responses. The Vera Rubin NVL72 exhibited significant improvements across various models, including Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.6-35B-A3B. The platform achieved 5x to 10x higher throughput per kilowatt and 3x to 5x higher end-to-end interactivity when compared to the GB300 NVL72 at matched concurrency levels. This substantial increase in efficiency means that organizations can support a far greater number of concurrent agentic AI users or execute more complex operations within the same power envelope, directly translating into reduced operational costs and enhanced service delivery for scalable AI applications.

Hardware and Software Synergies Driving Performance

The superior performance of the Vera Rubin NVL72 is attributed to a combination of advanced hardware and sophisticated software optimizations. At its core, the NVIDIA Rubin GPUs integrate high-bandwidth HBM4 memory, advanced Tensor Cores for accelerating matrix operations, and sixth-generation NVLink for rapid inter-GPU communication. Building on this hardware foundation, NVIDIA has implemented several key software innovations. These include context-aware serving techniques like prefix caching and state reuse, NVIDIA Dynamo routing for optimized load balancing, and disaggregated serving to independently optimize prefill (prompt processing) and decode (token generation) stages. Furthermore, model-specific GPU execution optimizations, such as DeepSeek’s MegaMoE kernel and Qwen’s multi-token prediction with Tensor Core verification, reduce data movement and sequential decoding steps, tailor kernel operations, and improve overall execution efficiency. These synergistic improvements are crucial for handling the evolving and demanding nature of agentic inference.

Practical Implications and Current Limitations

For enterprises leveraging or planning to deploy agentic AI, the Vera Rubin NVL72's performance carries significant practical implications. It promises to unlock new levels of scalability and responsiveness, enabling more sophisticated AI assistants, advanced code generation tools, and intelligent enterprise automation without disproportionately increasing infrastructure footprints or power consumption. This efficiency is vital as context lengths grow in multi-turn interactions, making previous inference systems less viable. From a cost perspective, the improved throughput per kilowatt directly reduces the energy spend per AI transaction, an increasingly important factor for large-scale deployments. However, it is crucial to note that the Vera Rubin results are provisional and are currently pending verification by the MLCommons Association, meaning the final validated numbers could differ. Furthermore, achieving these performance gains necessitates investment in new Vera Rubin platform hardware, which represents a significant upgrade cycle. Adoption will also depend on the specific optimization needs for diverse agentic models beyond those benchmarked.

Conclusion

The provisional MLPerf Endpoints v1.0 results for the NVIDIA Vera Rubin NVL72 indicate a substantial leap in efficiency and performance for agentic AI inference. Organizations with workloads involving multi-turn AI agents should closely follow the final verification of these benchmarks. Planning for infrastructure upgrades to platforms like the Vera Rubin NVL72 could offer a competitive edge in delivering more capable and cost-effective AI services in the near future.

Source and news date

NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0 — 2026-11-16

Related ServerNet services

ServerNet Support

ServerNet engineering & editorial team — specialists in infrastructure, networking and web hosting.

Linux Hosting
Share:

Comments 0

No comments yet — be the first!

Leave a comment

Related service

Linux Hosting

PHP & MySQL hosting on NVMe RAID-10 with LiteSpeed — the solid base for any website, from personal blogs to enterprise Laravel apps. At a price competitors can't explain.