Skip to main content

2 posts tagged with "MLPerf"

MLPerf benchmark results and analysis

View all tags

AMD创MLPerf推理纪录：512卡MI350实现575万tokens/秒，NVIDIA垄断首次实质性松动

· 5 min read
AI Compute Cards Wiki Editorial
Industry Research Team

2026年9月16日，MLPerf Inference v6.1结果出炉：AMD以Instinct MI350系列在512卡规模创下575万tokens/秒的MLPerf纪录，而NVIDIA Vera Rubin NVL72也在本轮完成首秀。分析师称，这是NVIDIA在AI训练与推理领域历史性垄断的"实质性侵蚀"。

MLPerf v6.1：本轮关键结果​

MLPerf Inference是业界最具公信力的AI推理基准测试，v6.1这一轮的格局变化堪称历史性：

提交方系统关键结果
AMDInstinct MI350系列，512卡575万tokens/秒（5.75M tokens/sec）MLPerf纪录
AMD合作方多家系统厂商结果与AMD平均差距在4%以内
NVIDIAVera Rubin NVL72（首秀）Qwen3-VL吞吐较GB300 NVL72最高提升3.7倍
NVIDIAVera Rubin NVL72DeepSeek-R1吞吐较GB300 NVL72最高提升2.5倍
NVIDIAVera Rubin NVL72，288卡演示99%扩展效率
Intel软件优化以软件优化收获性能增益

AMD的512卡提交以575万tokens/秒创下MLPerf纪录，更值得注意的是，合作方的独立提交与AMD官方结果平均差距在4%以内——第三方可复现性意味着这个纪录不是实验室里的孤立数字。

NVIDIA的回应：Vera Rubin NVL72首秀​

NVIDIA并未坐视。Vera Rubin NVL72在本轮MLPerf完成首秀，成绩同样亮眼：Qwen3-VL吞吐较上一代GB300 NVL72最高提升3.7倍，DeepSeek-R1最高提升2.5倍，288卡规模演示了99%的扩展效率。

这里存在一个口径差异需要读者注意：AMD刷新的是tokens/秒的绝对纪录（512卡），NVIDIA展示的是代际提升幅度（相对自家GB300）。两者并不矛盾，但说明竞争已从"单点性能"转向"规模效率与代际节奏"的全方位对垒。此外，Intel虽无新硬件登场，但证明了在成熟硬件上软件优化仍能压榨出可观的性能增益——软件栈的权重在推理竞争中持续上升。

MLPerf看什么：吞吐、延迟与规模效率​

对于习惯只看 headlines 的读者，读MLPerf结果建议关注三个维度：

  1. 吞吐（Throughput）：单位时间token产出，反映系统在大批量服务场景的容量上限。AMD本次575万tokens/秒就是吞吐口径的纪录。
  2. 延迟（Latency）：单请求响应速度，决定交互式应用的用户体验。吞吐成绩往往在放宽延迟约束时更高，需对照延迟约束看吞吐。
  3. 规模效率（Scaling Efficiency）：从几十卡扩展到数百卡时性能保留多少。NVIDIA的288卡99%扩展效率正是这个口径——硬件再强，扩展效率低下就无法支撑真实的超大规模部署。

三者组合才能还原一个系统的真实战力：吞吐高说明"能扛量"，延迟低说明"反应快"，规模效率高说明"能放大"。

AMD的部署进展：纪录之外的商业验证​

MLPerf成绩需要落地验证才有意义。AMD在2026年的部署进展已覆盖超算、国家级基础设施与商业训练：

  • LUMI-AI超算（欧洲）：将由MI430X与第六代EPYC驱动，延续AMD在欧洲超算领域的影响力；
  • HUMAIN（沙特）与Cisco等大型基础设施项目：MI355X系统参与其中，进入国家级AI基础设施采购清单；
  • Zyphra的商业训练：8月19日，Zyphra用1536张MI300X从零预训练ZAYA1-8B模型，证明AMD平台不仅能推理，也能承担大规模预训练。

从超算到主权AI项目再到独立AI公司的预训练，AMD正在补齐"性能证明→商业信任"的最后一环。

机架级竞争下的benchmark口径变化​

MLPerf v6.1还有一层容易被忽视的行业信号：benchmark口径正在变化。随着NVL72、Helios等机架级（rack-scale）系统成为竞争主战场，提交单位的颗粒度从"单卡/单机"演进到"整个机架"，互联、内存、网络全部被打包进成绩。

这带来两个影响：其一，比较口径更接近真实部署（用户买的就是整机架），成绩参考价值更高；其二，小玩家更难参与——机架级提交的门槛将绝大多数中小厂商挡在门外。未来MLPerf榜单可能进一步分化为"单节点榜"与"机架级/集群榜"两个层次。

国产芯片何时加入MLPerf提交​

另一个值得国内读者思考的问题：何时能看到国产AI芯片的MLPerf正式提交？MLPerf的公信力来自开放提交与第三方可复现，国产芯片若要在国际市场建立性能信任，参与MLPerf是绕不开的一步。目前MLPerf v6.1榜单上尚未出现国产芯片的正式提交，国产芯片何时加入、以什么规模加入，将成为观察其国际化进程的重要风向标。

小结​

MLPerf Inference v6.1是分水岭式的一轮：AMD以512卡MI350创下575万tokens/秒的MLPerf纪录、合作方结果差距在4%以内，Tom's Hardware口径的判断是NVIDIA在AI训练与推理领域的历史性垄断出现"实质性侵蚀"；NVIDIA则以Vera Rubin NVL72首秀回应——Qwen3-VL吞吐最高3.7倍、DeepSeek-R1最高2.5倍、288卡99%扩展效率。加上AMD在LUMI-AI超算、HUMAIN、Cisco与Zyphra等项目的落地，AI推理市场的竞争格局正在被实质性改写。对采购者而言，现在是重新评估"唯NVIDIA论"的时候了；对行业而言，benchmark口径的机架级演化与国产芯片的缺席，同样是下一阶段值得持续跟踪的主线。

Vera Rubin NVL72 MLPerf v6.1 Debut: Qwen3-VL Throughput 3.7x GB300, CoreWeave Brings Multi-Rack Cluster Online Same Day

· 5 min read
AI Compute Cards Wiki Editorial
Industry Research Team

On September 16, MLCommons released the MLPerf Inference v6.1 Closed Division results, and NVIDIA's Vera Rubin NVL72 completed its benchmark debut. The same day, CoreWeave announced that a multi-rack Vera Rubin cluster was live on its cloud — a "rent the new hardware on launch day" cadence that is compressing the cycle from next-generation compute announcement to billable output down to a quarter.

1. Debut Results: 3.7x and 2.5x​

NVIDIA, via preview submissions (entries 6.1-0106 / 6.1-0074), provided an apples-to-apples comparison of its two flagship generations:

Benchmark ModelVera Rubin NVL72 vs GB300 NVL72Software Stack
Qwen3-VL-235B-A22B (235B multimodal MoE)Throughput up to 3.7x (across offline / server / interactive scenarios)vLLM + NVIDIA Dynamo
DeepSeek-R1-671B (671B inference model)Throughput up to 2.5xTensorRT-LLM

The three technologies behind these numbers:

  • NVFP4 precision: compresses the memory footprint of model weights, attention tensors, and the KV Cache, enabling significantly larger effective batch sizes;
  • Disaggregated serving: prefill (compute-intensive) and decode (bandwidth-intensive) are split across separate GPU pools, each independently optimized;
  • Expert parallelism: MoE expert layers route across chips in a distributed fashion, paired with sixth-generation NVLink / NVLink Switch (NVIDIA claims 10x the packet rate of commodity Ethernet at 3x lower latency).

2. Same-Session Highlights: Scaling Efficiency and the Software Dividend​

  • GB300 four racks, 288 chips, 99% scaling efficiency: in the DeepSeek-R1 offline scenario, scaling from a single rack of 72 chips to 4 racks of 288 chips shows almost no loss (entries 6.1-0073 / 6.1-0074) — the first time rack-scale interconnect scaling efficiency has been validated at the 288-chip level;
  • The software dividend is still being paid out: on the same GB300 hardware, v6.1 software optimizations deliver up to 1.6x Qwen3-VL gains over v6.0; NVIDIA also reported post-submission gains for GPT-OSS-120B and DLRMv3 (not verified by MLCommons);
  • 19 partners submitted, 8 of them using multi-node Blackwell NVL72 configurations (ASUS, Azure, Cisco, CoreWeave, Crusoe, Dell, Fujitsu, HPE, Lambda, Nebius, OCI, Supermicro, and others); Nebius also submitted Vera Rubin preview results.

3. CoreWeave Online the Same Day: Rentable at Launch​

On the same day the MLPerf results were published, CoreWeave announced that its multi-rack Vera Rubin NVL72 cluster was live on its cloud:

  • Spectrum-X networking links hundreds of Rubin GPUs into a single scale-out cluster, with each GPU paired with 2 ConnectX-9 SuperNICs (1.6Tb/s scale-out bandwidth);
  • The AI object store LOTA cuts read latency by 8x;
  • On the operations side: Valvey software-defined liquid cooling (NVIDIA spec: 45°C liquid inlet), the Racky rack control layer, and Rack LifeCycle Controller manage an entire NVL72 rack as a single programmable entity;
  • Back in July, CoreWeave's own testing put Vera Rubin at 10x tokens/s/MW versus GB200 NVL72 on DeepSeek R1 (company figures, not MLCommons-verified).

4. The Competitive Picture​

  • AMD + Crusoe: set MLPerf's all-time high aggregate throughput of 5.75M tokens/s on 512 chips (AMD had the broadest submission coverage this round);
  • SemiAnalysis AgentX preview: Vera Rubin up to 30x over GB300 on agentic workloads (preview figures); the MLPerf Endpoints benchmark is forthcoming and will bring agentic inference into standardized measurement;
  • Pending validation: Vera Rubin results are preview submissions awaiting independent reproduction; no quantified quality metrics were given for NVFP4's "virtually lossless" claim; per-token cost and power-consumption scaling comparisons have not been published. Matching submissions from Google TPU v7 and AMD MI455X UALoE72 are expected within 2026.

On-site spec comparisons: Rubin R200 (288GB / 22TB/s), B300 Ultra (GB300-class core).

5. Takeaways​

  • Vera Rubin NVL72 debut: Qwen3-VL 3.7x, DeepSeek-R1 2.5x over GB300 NVL72 (preview figures);
  • GB300 across four racks of 288 chips at 99% scaling efficiency — rack-scale interconnects have entered the usable range;
  • CoreWeave brought a multi-rack cluster online the same day — the cycle from next-gen compute launch to billable output is now measured in quarters;
  • What to watch: independent reproductions and formal (non-preview) submissions, TPU v7 / MI455X comparison results, and measured per-MW token throughput.

Further Reading​

References​

  • NVIDIA Blog: Vera Rubin NVL72 Makes Its First Appearance in MLPerf Inference v6.1 (2026-09-16)
  • MLCommons: MLPerf Inference v6.1 Closed Division results (entries 6.1-0073 / 6.1-0074 / 6.1-0106)
  • CoreWeave Investor Relations: multi-rack Vera Rubin NVL72 availability announcement (2026-09-16)
  • AMD Blogs: MLPerf Inference v6.1 submission results

This article is compiled from public MLCommons results and vendor official statements. Vera Rubin results are preview submissions; some gain figures are not MLCommons-verified and are flagged item by item.