15 posts tagged with "Benchmarks"

快科技10月10日消息，AI 代码执行平台 Daytona 公布了最新一轮智能体 AI 基准测试结果：NVIDIA Vera 处理器在沙盒启动速度和任务吞吐量两项核心指标上，均明显优于采用 AMD EPYC "Zen 5" 架构的产品。对正处爆发期的 Agentic AI 行业而言，这是第一次有一组公开数据，把"多智能体场景下的 CPU 选型"这个新命题摆上了台面。

2026年9月16日，MLPerf Inference v6.1结果出炉：AMD以Instinct MI350系列在512卡规模创下575万tokens/秒的MLPerf纪录，而NVIDIA Vera Rubin NVL72也在本轮完成首秀。分析师称，这是NVIDIA在AI训练与推理领域历史性垄断的"实质性侵蚀"。

MLPerf v6.1：本轮关键结果​

MLPerf Inference是业界最具公信力的AI推理基准测试，v6.1这一轮的格局变化堪称历史性：

AMD的512卡提交以575万tokens/秒创下MLPerf纪录，更值得注意的是，合作方的独立提交与AMD官方结果平均差距在4%以内——第三方可复现性意味着这个纪录不是实验室里的孤立数字。

NVIDIA的回应：Vera Rubin NVL72首秀​

NVIDIA并未坐视。Vera Rubin NVL72在本轮MLPerf完成首秀，成绩同样亮眼：Qwen3-VL吞吐较上一代GB300 NVL72最高提升3.7倍，DeepSeek-R1最高提升2.5倍，288卡规模演示了99%的扩展效率。

这里存在一个口径差异需要读者注意：AMD刷新的是tokens/秒的绝对纪录（512卡），NVIDIA展示的是代际提升幅度（相对自家GB300）。两者并不矛盾，但说明竞争已从"单点性能"转向"规模效率与代际节奏"的全方位对垒。此外，Intel虽无新硬件登场，但证明了在成熟硬件上软件优化仍能压榨出可观的性能增益——软件栈的权重在推理竞争中持续上升。

MLPerf看什么：吞吐、延迟与规模效率​

对于习惯只看 headlines 的读者，读MLPerf结果建议关注三个维度：

1. 吞吐（Throughput）：单位时间token产出，反映系统在大批量服务场景的容量上限。AMD本次575万tokens/秒就是吞吐口径的纪录。
2. 延迟（Latency）：单请求响应速度，决定交互式应用的用户体验。吞吐成绩往往在放宽延迟约束时更高，需对照延迟约束看吞吐。
3. 规模效率（Scaling Efficiency）：从几十卡扩展到数百卡时性能保留多少。NVIDIA的288卡99%扩展效率正是这个口径——硬件再强，扩展效率低下就无法支撑真实的超大规模部署。

三者组合才能还原一个系统的真实战力：吞吐高说明"能扛量"，延迟低说明"反应快"，规模效率高说明"能放大"。

AMD的部署进展：纪录之外的商业验证​

MLPerf成绩需要落地验证才有意义。AMD在2026年的部署进展已覆盖超算、国家级基础设施与商业训练：

• LUMI-AI超算（欧洲）：将由MI430X与第六代EPYC驱动，延续AMD在欧洲超算领域的影响力；
• HUMAIN（沙特）与Cisco等大型基础设施项目：MI355X系统参与其中，进入国家级AI基础设施采购清单；
• Zyphra的商业训练：8月19日，Zyphra用1536张MI300X从零预训练ZAYA1-8B模型，证明AMD平台不仅能推理，也能承担大规模预训练。

从超算到主权AI项目再到独立AI公司的预训练，AMD正在补齐"性能证明→商业信任"的最后一环。

机架级竞争下的benchmark口径变化​

MLPerf v6.1还有一层容易被忽视的行业信号：benchmark口径正在变化。随着NVL72、Helios等机架级（rack-scale）系统成为竞争主战场，提交单位的颗粒度从"单卡/单机"演进到"整个机架"，互联、内存、网络全部被打包进成绩。

这带来两个影响：其一，比较口径更接近真实部署（用户买的就是整机架），成绩参考价值更高；其二，小玩家更难参与——机架级提交的门槛将绝大多数中小厂商挡在门外。未来MLPerf榜单可能进一步分化为"单节点榜"与"机架级/集群榜"两个层次。

国产芯片何时加入MLPerf提交​

另一个值得国内读者思考的问题：何时能看到国产AI芯片的MLPerf正式提交？MLPerf的公信力来自开放提交与第三方可复现，国产芯片若要在国际市场建立性能信任，参与MLPerf是绕不开的一步。目前MLPerf v6.1榜单上尚未出现国产芯片的正式提交，国产芯片何时加入、以什么规模加入，将成为观察其国际化进程的重要风向标。

小结​

MLPerf Inference v6.1是分水岭式的一轮：AMD以512卡MI350创下575万tokens/秒的MLPerf纪录、合作方结果差距在4%以内，Tom's Hardware口径的判断是NVIDIA在AI训练与推理领域的历史性垄断出现"实质性侵蚀"；NVIDIA则以Vera Rubin NVL72首秀回应——Qwen3-VL吞吐最高3.7倍、DeepSeek-R1最高2.5倍、288卡99%扩展效率。加上AMD在LUMI-AI超算、HUMAIN、Cisco与Zyphra等项目的落地，AI推理市场的竞争格局正在被实质性改写。对采购者而言，现在是重新评估"唯NVIDIA论"的时候了；对行业而言，benchmark口径的机架级演化与国产芯片的缺席，同样是下一阶段值得持续跟踪的主线。

On September 16, MLCommons released the MLPerf Inference v6.1 Closed Division results, and NVIDIA's Vera Rubin NVL72 completed its benchmark debut. The same day, CoreWeave announced that a multi-rack Vera Rubin cluster was live on its cloud — a "rent the new hardware on launch day" cadence that is compressing the cycle from next-generation compute announcement to billable output down to a quarter.

1. Debut Results: 3.7x and 2.5x​

NVIDIA, via preview submissions (entries 6.1-0106 / 6.1-0074), provided an apples-to-apples comparison of its two flagship generations:

The three technologies behind these numbers:

• NVFP4 precision: compresses the memory footprint of model weights, attention tensors, and the KV Cache, enabling significantly larger effective batch sizes;
• Disaggregated serving: prefill (compute-intensive) and decode (bandwidth-intensive) are split across separate GPU pools, each independently optimized;
• Expert parallelism: MoE expert layers route across chips in a distributed fashion, paired with sixth-generation NVLink / NVLink Switch (NVIDIA claims 10x the packet rate of commodity Ethernet at 3x lower latency).

2. Same-Session Highlights: Scaling Efficiency and the Software Dividend​

• GB300 four racks, 288 chips, 99% scaling efficiency: in the DeepSeek-R1 offline scenario, scaling from a single rack of 72 chips to 4 racks of 288 chips shows almost no loss (entries 6.1-0073 / 6.1-0074) — the first time rack-scale interconnect scaling efficiency has been validated at the 288-chip level;
• The software dividend is still being paid out: on the same GB300 hardware, v6.1 software optimizations deliver up to 1.6x Qwen3-VL gains over v6.0; NVIDIA also reported post-submission gains for GPT-OSS-120B and DLRMv3 (not verified by MLCommons);
• 19 partners submitted, 8 of them using multi-node Blackwell NVL72 configurations (ASUS, Azure, Cisco, CoreWeave, Crusoe, Dell, Fujitsu, HPE, Lambda, Nebius, OCI, Supermicro, and others); Nebius also submitted Vera Rubin preview results.

3. CoreWeave Online the Same Day: Rentable at Launch​

On the same day the MLPerf results were published, CoreWeave announced that its multi-rack Vera Rubin NVL72 cluster was live on its cloud:

• Spectrum-X networking links hundreds of Rubin GPUs into a single scale-out cluster, with each GPU paired with 2 ConnectX-9 SuperNICs (1.6Tb/s scale-out bandwidth);
• The AI object store LOTA cuts read latency by 8x;
• On the operations side: Valvey software-defined liquid cooling (NVIDIA spec: 45°C liquid inlet), the Racky rack control layer, and Rack LifeCycle Controller manage an entire NVL72 rack as a single programmable entity;
• Back in July, CoreWeave's own testing put Vera Rubin at 10x tokens/s/MW versus GB200 NVL72 on DeepSeek R1 (company figures, not MLCommons-verified).

4. The Competitive Picture​

• AMD + Crusoe: set MLPerf's all-time high aggregate throughput of 5.75M tokens/s on 512 chips (AMD had the broadest submission coverage this round);
• SemiAnalysis AgentX preview: Vera Rubin up to 30x over GB300 on agentic workloads (preview figures); the MLPerf Endpoints benchmark is forthcoming and will bring agentic inference into standardized measurement;
• Pending validation: Vera Rubin results are preview submissions awaiting independent reproduction; no quantified quality metrics were given for NVFP4's "virtually lossless" claim; per-token cost and power-consumption scaling comparisons have not been published. Matching submissions from Google TPU v7 and AMD MI455X UALoE72 are expected within 2026.

On-site spec comparisons: Rubin R200 (288GB / 22TB/s), B300 Ultra (GB300-class core).

5. Takeaways​

• Vera Rubin NVL72 debut: Qwen3-VL 3.7x, DeepSeek-R1 2.5x over GB300 NVL72 (preview figures);
• GB300 across four racks of 288 chips at 99% scaling efficiency — rack-scale interconnects have entered the usable range;
• CoreWeave brought a multi-rack cluster online the same day — the cycle from next-gen compute launch to billable output is now measured in quarters;
• What to watch: independent reproductions and formal (non-preview) submissions, TPU v7 / MI455X comparison results, and measured per-MW token throughput.

Further Reading​

• Rubin R200 Spec Page / B300 Ultra Spec Page (this site)
• NVIDIA Vera Rubin Mass Production and Liquid Cooling Deployment
• Hot Chips 2026 Full Recap: Rubin, MI455X, Crescent Island on the Same Stage

References​

• NVIDIA Blog: Vera Rubin NVL72 Makes Its First Appearance in MLPerf Inference v6.1 (2026-09-16)
• MLCommons: MLPerf Inference v6.1 Closed Division results (entries 6.1-0073 / 6.1-0074 / 6.1-0106)
• CoreWeave Investor Relations: multi-rack Vera Rubin NVL72 availability announcement (2026-09-16)
• AMD Blogs: MLPerf Inference v6.1 submission results

This article is compiled from public MLCommons results and vendor official statements. Vera Rubin results are preview submissions; some gain figures are not MLCommons-verified and are flagged item by item.

2026 H2 is the richest era for the AI compute market: NVIDIA Rubin R200, AMD MI400, Trainium 3, TPU 8t/8i, Ascend 920, and Groq 3 LPX are all in place. This article provides a complete selection tree to help you choose the most suitable product based on model size, training/inference, latency requirements, budget, and region.

LLM inference performance = Algorithm + Software + Hardware. Hardware (H100, B300, Rubin) only determines the theoretical ceiling. Actual inference performance can be improved 5-30× through algorithmic optimization. This article provides a deep analysis of the three major inference optimization technologies: PagedAttention, FlashAttention, and Speculative Decoding.