Artificial Intelligence1 min read

NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0

By · Published by Everything Blog

In short

The MLPerf Endpoints v1.0 suite assesses hardware efficiency during multi-turn agentic workflows by balancing serving capacity with interactive responsiveness on coding and enterprise workloads. While older generation systems have achieved official verification from the MLCommons Association, the newer architecture achieved these provisional metrics while maintaining a shared interactivity floor of 50 tokens per second per user. The test framework systematically verifies workflow intent and tool calls against reference standards to ensure that rapid token generation does not undermine overall task-solving accuracy.

Key points

  • The benchmark requires systems to maintain a minimum interactivity baseline of 50 tokens…: The benchmark requires systems to maintain a minimum interactivity baseline of 50 tokens per second for every individual user.
  • MLPerf Endpoints v1.0 validates tool calling, reasoning, and workflow intent to ensure ac…: MLPerf Endpoints v1.0 validates tool calling, reasoning, and workflow intent to ensure accelerated serving speeds preserve model output quality.
  • The reported Vera Rubin performance metrics are currently provisional and remain subject…: The reported Vera Rubin performance metrics are currently provisional and remain subject to peer review prior to official MLCommons Association verification.

Sources checked by our analysis pipeline: 1

Agentic workloads are demanding more from inference systems. AI agents carry growing conversation histories across turns and generate a mix of reasoning, tool calls, and responses. Inference systems must support users at scale while remaining responsive.

MLPerf Endpoints v1.0 evaluates this balance between serving capacity and responsiveness using coding and enterprise workflows through API endpoints. During performance runs, generated tool calls and workflow intent are checked against reference values. Separate SWE-bench Verified evaluations assess end-to-end task-solving capability. These checks evaluate whether faster serving preserves model quality.

In MLPerf Endpoints v1.0, provisional NVIDIA Vera Rubin NVL72 results show over 10x the throughput per kilowatt of provisioned power of NVIDIA GB300 NVL72 at a shared minimum end-to-end interactivity of 50 tokens per second per user.1

This post presents the benchmark results and outlines the hardware and software optimizations used in the submissions.

1MLPerf® Endpoints v1.0, Kimi K3, Standardized division, Available category, published November 9, 2026. NVIDIA Vera Rubin NVL72, 32 accelerators, result 1.0.1.kimi-k3.agentic_combined.1p, run 9: concurrency = 1,024 queries, TTFT P90 = 4,673.9 ms, system TPS = 41,414.2 tokens/sec, E2E average interactivity…

Artificial Intelligence