Skip to main content

Inference Accelerator Market 2026: 60%–70% of the Accelerator Market, GPU vs ASIC Share Inverts, Five Schools Clash

· 5 min read
Industry Research Team

For the past three years, the entire AI hardware story was "training": who had the most H100s, who could connect a hundred thousand GPUs into a cluster. That race is essentially settled — NVIDIA won. But the next battlefield, "inference," is being fought under completely different rules: the measure is no longer peak FLOPS, but cost-per-token, latency, and power. In 2026, inference chips overtake training in scale for the first time, becoming the main battlefield of AI accelerators.


1. Inference Becomes the Main Battlefield: 80%–90% of Compute Spent on Inference

Training a large model costs hundreds of millions of dollars — once. But once the model goes live, it must answer billions of queries day after day. A popular consumer model may need tens of thousands of accelerators running 7×24 to keep up with demand. Therefore:

  • Inference accounts for roughly 80%–90% of a model's lifecycle compute;
  • Inference chips will make up about 60%–70% of the ~$400B AI accelerator market in 2026, up from only ~40% in 2023;
  • Inference chip growth (estimated +52.7% YoY) significantly outpaces training chips (+28.4%); the share of inference-side compute demand exceeded training-side for the first time in 2026, reaching 54% (~$1010B).

The economics of inference are straightforward: training cost is amortized to near-zero, while inference cost becomes the entire bill. Every 1% cut in inference cost flows directly to profit — for a company whose inference traffic reaches hyperscale like OpenAI, the half of the bill is a number followed by a string of zeros.


2. Market Size: Structural Growth Inflection Point Has Arrived

Market2026 SizeGrowthNotes
Global dedicated inference chips$412.7B+38.4%14.2 pct higher growth than training chips
China dedicated inference chips$118.6B (28.7% of global)+44.1%Strongest single market in APAC by growth
Global AI training/inference chips (incl. GPU/NPU)exceeds $1850B+40.2%GPU ~62%

China's domestic substitution is accelerating, with domestic inference chips reaching 34.6% of shipments, up 9.8 pct from 2025.


3. Technology-Axis Share Inverts: GPU Slows, ASIC Soars

Axis2026 Shipment ShareTrend
GPU52.6%Still leads, but growth slows to 22.7%
ASIC custom chips41.3%Up sharply from 17.8% in 2022
FPGAStableSpecific low-latency scenarios

Thanks to ecosystem maturity, GPU remains the mainstay, but NPU/ASIC already holds a 1.8× advantage over same-generation GPUs in energy efficiency, driving rapid adoption at the edge and on-device. Shipments of inference-optimized ASICs are expected to reach 11.5 million units, with unit cost about 35% lower than GPUs.


4. Five Schools Clash

SchoolRepresentative ProductsCore StrengthUse Cases
General-purpose GPUNVIDIA Rubin / B200 / H200Mature ecosystem, train+infer unifiedFrontier training + highly interactive inference
LPU (Language Processing Unit)Groq LPUUltra-low latency, deterministic throughputReal-time dialogue, high-concurrency inference
TPU (inference-specific)Google TPU 8i (Zebrafish)288GB HBM, 384MB on-chip SRAM, 19.2 Tb/s ICIGoogle's scaled inference
Custom ASICOpenAI Jalapeno, Microsoft Maia 200, Meta MTIAStrip generality tax for own models, ~50% lower cost/tokenHyperscaler's own workloads
Air-cooled inference cardIntel Crescent Island350W air-cooled, 480GB LPDDR5X, tokens/wattCost-sensitive mid/long-tail inference

OpenAI's Jalapeno, co-developed with Broadcom, aims to cut inference token cost by roughly 50% versus a general-purpose GPU stack — the fifth member to join the "custom inference chip club" (after Google TPU, Amazon Inferentia/Trainium, Microsoft Maia, and Meta MTIA).


5. Core Metric Shifts: cost-per-token and tokens/watt

The fundamental difference between the inference race and the training race is the low switching cost:

  • Training requires a 100k-GPU cluster + NVLink + CUDA, with extremely high migration cost;
  • Inference is "embarrassingly parallel" at the endpoint level — no million-GPU cluster needed; a node that produces tokens fast and cheaply suffices, and is replaceable per endpoint.

This means NVIDIA's three moats (fastest silicon, NVLink scale-out, CUDA) are no longer absolute on the inference side. When the largest AI buyer (OpenAI) starts treating GPUs as "one of the options," the GPU premium begins to erode — pricing power relies on scarcity, and custom chips attack that scarcity from two directions at once: both reducing merchant-chip demand and giving buyers a credible external negotiation option.


6. Edge and On-Device Explosion: Long-Tail Signal

Demand shows significant long-tail and fragmentation:

Scenario2026 Demand SizeGrowth
Cloud inference$198.2B (48%)+24.5% (slowing)
Edge inference$126.5B (30.7%)+52.3%
On-device inference$88.0B (21.3%)+68.9%
Autonomous-driving inference$67.3B+58.2%
Industrial QA / robotics inference$42.1B+63.7%

The latency sensitivity and power constraints of inference workloads are reshaping chip architecture design priorities — which also explains why "air-cooled, large-memory" solutions like Crescent Island can find a niche.

References


This article is compiled from publicly available 2026 market research, brokerage reports, and industry analysis. Market sizes and shares are third-party estimates with inconsistent methodologies and are for reference only.

WAIC 2026 Recap: Huawei Atlas 950 SuperPoD Live Hardware Wins SAIL Grand Award, Domestic Compute Enters the "System-Level" Showdown

· 5 min read
Industry Research Team

The 2026 World Artificial Intelligence Conference (WAIC) was held July 17-20, 2026 at the Shanghai World Expo Center, themed "Intelligent Partners, Creating the Future Together." Over 1,100 companies showcased 3,000+ exhibits, with 300+ products debuting globally. For the compute-card industry, this concentrated review of domestic compute sent a clear signal: the competitive main line is shifting from "single-chip peak compute" to "SuperNode system-level effective compute."

1. Huawei Atlas 950 SuperPoD: live debut, wins SAIL grand award

Huawei's Atlas 950 SuperPoD live hardware made its first public appearance at WAIC 2026, on-site carrying 16 compute cabinets with 1,024 Ascend cards total. With three system-level innovations — "ultra-wide bandwidth, ultra-low latency, unified memory addressing" — it stood out from hundreds of domestic and international entries to win the conference's top honor, the SAIL (Super AI Leader) Award.

Core parameters (confirmed on-site at WAIC)

MetricAtlas 950 SuperPoD
Exhibited scale16 compute cabinets / 1,024 Ascend cards
Max interconnect scale8,192 Ascend NPU cards fully interconnected (full config)
Interconnect protocolHuawei in-house "Lingqu" (UnifiedBus) 2.0
Total compute1 EFLOPS FP8 / 2 EFLOPS FP4 (1,024 cards); full 8,192-card ~8 EFLOPS FP8
Unified memory256 TB globally unified memory address space
Interconnect latency3 μs ultra-low RTT; TB-level NPU interconnect bandwidth
Full config128 compute cabinets + 32 interconnect cabinets = 160 cabinets, ~1000㎡, carrying 8,192 Ascend 950DT
LaunchFull config planned for Q4 2026
CoolingFully liquid-cooled blind-plug architecture

Huawei disclosed for the first time: the previous-gen Ascend 384 SuperNode has cumulatively shipped 750+ units commercially, deployed across 20+ industries including internet, operators, finance, education, healthcare, transportation, and manufacturing, calling it "the only domestic SuperNode that has trained SOTA models."

2. Software ecosystem: CANN fully open-sourced, developers at scale

Beyond hardware, Huawei highlighted open-source software ecosystem progress:

  • CANN heterogeneous compute architecture and MindSeries base software suite were fully open-sourced end of 2025;
  • The CANN open-source community has incubated 67 projects, 12.44M+ lines of code, with 3,500+ monthly active developers;
  • Huawei has co-developed 7,000+ solutions with 3,000+ industry partners, serving 2,000+ core government/enterprise customers;
  • WAIC showcased 60+ real business scenarios, 20+ benchmark cases, covering the full chain from technology breakthrough to scaled commercial deployment.

3. Domestic chips' Day-0 adaptation becomes routine

On July 6, 2026, Tencent released the MoE model Hunyuan T3 (295B parameters, 256K context); domestic chips rapidly completed Day-0 adaptation:

VendorChipAdaptation status
Moore ThreadsMTT S5000Completed rapid Hunyuan T3 adaptation (previously adapted DeepSeek-V4, GLM-5.2)
MetaXXiyun C seriesIn-house MXMACA stack first to full-chain Day-0 adaptation, zero-code deployment

Moore Threads also showcased the MTT C256 SuperNode (first-of-its-kind single-layer Scale-up 256-card full interconnect, sub-microsecond latency) and three AI-factory solutions — "model training factory / token production factory / agent production factory."

4. More domestic compute debut highlights

Vendor / productHighlight
Orient AlphaChip DF1000World's first "software-defined + near-memory computing" 3D chip, interconnect pitch compressed to sub-micron
ZhongHao XinYing "Xuyu"Fully in-house next-gen TPU-architecture AI-specific chip, with Taize 2.0 server
Enflame × IluvatarDomestic high-performance Matrix SuperNode based on OEX+dOCS architecture, shortlisted for the conference "Excellent AI Leader Award"
Rongming MicroelectronicsAdvancing next-gen VPU, evolving from video processing to "visual-agent compute base"

The domestic AI chip lineup also included Moore Threads, MetaX, Enflame, Houmo, Cixiong, Suaneng, SemiDrive, Phytium, Aixin, Iluvatar, and others.

Industry interpretation: from "can it be built" to "is it used well"

WAIC 2026 reflects a fundamental shift in the competitive stage of domestic AI chips:

  1. SuperNode becomes the main battlefield: beyond single-chip performance, system-level capabilities — "inter-chip interconnect + cluster scale + cooling" — become the breakthrough key. Huawei Lingqu and Enflame/Iluvatar OEX are both pushing here. Huatai Securities defines 2026 as the "first year of domestic SuperNodes," estimating China's SuperNode architecture market could reach ¥341.4B by 2028, with 2026-2028 CAGR of 194%.
  2. Software ecosystem delivers: Day-0 adaptation has gone from slogan to routine; the "launch-and-adapt" closed loop between domestic large models (DeepSeek-V4, GLM-5.2, Hunyuan T3) and domestic chips is essentially formed.
  3. Demand-side endorsement: China Mobile earlier released its 2026-2027 AI SuperNode centralized procurement announcement — about 6,208 cards, over ¥2B — accelerating domestic SuperNode scaled commercialization.

References


This article is compiled from WAIC 2026 (July 17-20) on-site and official disclosures, and will continuously track the 950 SuperNode Q4 launch.

AMD Advancing AI 2026 Opens Tomorrow: Three CDNA5 MI400 Models, Helios Rack Hits 3 exaFLOPS, OpenAI + Meta Lock 12GW Deal

· 4 min read
AI Hardware Analyst

AMD has confirmed its flagship AI event Advancing AI 2026 will be held July 22-23, 2026 at the Moscone Center in San Francisco, with the keynote on July 23 hosted by Chair and CEO Lisa Su. The event will complete the Instinct MI400 series availability timeline, pricing, and independent benchmark data.

1. Instinct MI400 family: three CDNA 5 accelerators

AMD fully revealed the MI400 matrix at CES 2026; all three accelerators use CDNA 5 architecture, TSMC 2nm process, differentiated by precision and scenario:

ModelPositioningKey specs
MI455X (flagship)Large-scale train/inference (rack-scale)320B transistors, 12 chiplets, 432 GB HBM4 (12×36GB), 19.6 TB/s, FP4 40 PFLOPS / FP8 20 PFLOPS
MI440X (enterprise)Local enterprise AI (8-card node)Low-precision AI (FP4/FP8/BF16), direct MI300/MI350 replacement, compatible with existing power/cooling
MI430X (HPC/sovereign AI)High-precision scientific computing + AIFull FP32/FP64, already deployed at Oak Ridge Discovery and France's first exascale Alice Recoque

MI455X and MI440X target low-precision AI (FP4/FP8/BF16); MI430X fills traditional HPC high-precision needs — improving energy efficiency and cost-performance by "trimming execution units by precision." All three support UALink (among the first accelerators compatible with the standard) and Infinity Fabric die-to-die interconnect; rack scaling uses Ultra Ethernet.

Lisa Su confirmed on the Q1 2026 earnings call: MI455X samples have been sent to core customers, with demand "exceeding the company's internal expectations for 2027."

2. Helios rack: 3 exaFLOPS per cabinet

AMD enters the hyperscale market with the Helios rack-scale platform:

MetricHelios rack
Accelerators72 × MI455X
Aggregate HBM431 TB
Total memory bandwidth1.4 PB/s
Per-cabinet computeUp to 3 AI exaFLOPS (Q3 delivery target)
Target customersHyperscale train/inference clusters

Helios uses AMD's in-house Zen 6 EPYC Venice CPU (18 per rack) + Pensando Vulcano 800G NIC, integrated via the open ROCm software stack; AMD also plans a double-width 128-card Helios variant, pushing per-cabinet compute to the 3 AI exaFLOPS ceiling. Further out, the MI500 series (CDNA 6, 2nm, HBM4E) is planned for 2027, with official claims of up to 1000× AI performance vs MI300X.

3. 12GW deal: OpenAI + Meta dual endorsement

AMD holds two historic-scale compute agreements totaling about 12 GW, with lifetime potential revenue possibly reaching $100B:

CustomerScaleFirst deploymentStructure
OpenAI6 GW (multi-gen products)First 1 GW, H2 2026 on MI450"compute-for-upside": up to 160M warrants, vesting by milestone and stock-price targets
Meta6 GWCustom MI450 chips, from H2 2026Deployed in next-gen data centers

Financial expectations

Metric2026 forecast
MI400 series revenue~$7.2B (about 25% of data-center sales)
Data-center GPU revenue~$15B (up +114% YoY)
Total data-center revenuePossibly $28.7B (up +73% YoY)

⚠️ Execution risk: AMD has flagged that MI450's Q3 mass production will weigh on gross margin (new products below company average); advanced process and advanced packaging (TSMC CoWoS) capacity remain the main constraint.

Industry interpretation

  1. CUDA moat being pried open: when companies building the world's largest training clusters — Meta, OpenAI — bet on AMD silicon, AMD's long-standing 5-7% GPU share ceiling is being broken.
  2. Memory advantage as differentiation: 432 GB HBM4 / 19.6 TB/s vs NVIDIA Rubin's 288 GB offers capacity advantage, critical for large-model inference (KV Cache-constrained scenarios).
  3. Tight benchmarking pace: MI450 and NVIDIA Vera Rubin both ramp in H2 2026, with the two giants competing head-on over HBM4 supply and CoWoS capacity.

References


This article was written on the eve of Advancing AI 2026 (July 22-23, opening tomorrow); the keynote is July 23 hosted by Lisa Su, where MI400's final availability, pricing, and independent benchmarks will be revealed — we will update in sync.

Domestic GPU IPO Wave: The "Four Little Dragons" Assemble on Capital Markets, Moore Threads MTT S5000 Benchmarks Against H100

· 5 min read
Industry Research Team

From December 2025 to July 2026 — just half a year — at least 6 AI chip companies have listed or are about to list on capital markets. Together with already-listed Cambricon, Hygon, and Iluvatar, the domestic GPU corps' total market cap is approaching ¥2 trillion. This marks the critical climb from domestic GPUs being "usable" to "useful."

1. The "Four Little Dragons" assemble on capital markets

CompanyListing statusRaise / issue priceSponsor
Moore ThreadsListed (STAR Market sh688795, 2025-12-05)Issue price ¥114.28, raised ¥8BCITIC Securities
MetaXIPO accepted (2026-06-30)¥3.904B (total investment ¥5B)Huatai United
EnflamePassed review (2026-06-15)¥6B
BirenHKEX / sprinting

Already-listed camp: Cambricon (sh688256, STAR Market 2020-07-20), Hygon, Iluvatar (HKEX). Moore Threads turned a book profit of ¥29.35M in Q1; MetaX narrowed losses 57.7% and gave a 2026 breakeven timeline.

2. Moore Threads MTT S5000: benchmarking against H100

Moore Threads announced its flagship AI train+inference GPU MTT S5000 successfully completed full-pipeline adaptation validation of Zhipu's new-generation large model GLM-5 — measured performance "breaks the domestic compute ceiling":

MetricMTT S5000
Architecture4th-gen "Pinghu" architecture
FP8 compute1 PFLOPS (1,000 TFLOPS)
Memory bandwidth1.6 TB/s
PositioningFull-function train+inference GPU, benchmarks against NVIDIA H100
ProductionMass-produced; clusters online supporting trillion-parameter training

Deployment validation: jointly completed full-pipeline training of embodied-brain model RoboBrain 2.5 with BAAI; partnered with SiliconFlow for high-performance DeepSeek-V3 inference, single-card speed near international top products. IPO funds go to three directions: next-gen AI train+inference chip, next-gen graphics chip, next-gen AI SoC chip.

WAIC 2026 new progress: Moore Threads showcased the MTT C256 SuperNode (first-of-its-kind single-layer Scale-up 256-card full interconnect, sub-microsecond latency) and three AI-factory solutions — "model training factory / token production factory / agent production factory"; the company pre-announced H1 2026 revenue of ¥1.65B-1.75B, up 135%-149% YoY.

3. Cambricon: dual flagships MLU590/690

ChipProcessComputeMemoryCustomer / status
MLU590 (思元590)7nm ChipletINT8 512 TOPS / FP16 345 TFLOPS96 GB HBM2eByteDance inference mainstay, ~80% of A100 overall, mass shipments early 2026
MLU690 (思元690)5nm-class (SMIC N+2)FP16 700+ TFLOPS / INT8 2800+ TOPS196 GB HBM3 (3.35 TB/s)Dual-die packaging, MLU-Link 890 Gbps; ~70% of H100 (80-90% pure inference); ByteDance largest customer, mass production early 2026

Cambricon is the only domestic AI chip vendor with a "unified edge-cloud architecture" — one MLU instruction set spans 思元 220 (edge) → 370 (border) → 590/690 (cloud), with one NeuWare toolchain across compute tiers.

Capital and performance double explosion: Cambricon's total market cap exceeded ¥1 trillion on June 30, 2026, becoming the STAR Market's first "trillion-yuan stock," up 75%+ YTD. On performance, Q1 2026 revenue ¥2.885B (+160% YoY), deducted net profit ¥934M; full-year 2025 revenue ¥6.497B (+453% YoY), net profit attributable to parent ¥2.059B, ending long-term losses. ByteDance has cumulatively deployed over 100k 思元 590/690, its largest customer.

4. DeepSeek-V4 effect: changing the expectation coordinate system

On April 24, 2026, DeepSeek released the trillion-parameter flagship DeepSeek-V4. Unlike a year earlier when V3's launch sparked debate over "can domestic chips even run large models," this time multiple domestic chips — Huawei Ascend, Cambricon, Hygon, MetaX, Moore Threads, Kunlun, T-Head, Iluvatar — completed adaptation on launch day.

The evaluation coordinate system is shifting: from "what percentage of NVIDIA's same-generation product performance" to "can it carry the real workloads of top-tier large models."

Industry interpretation

  1. Capital ammunition in place: dense IPOs provide ample funding for domestic GPU R&D iteration and capacity expansion, moving from "technology breakthrough" to "commercial virtuous cycle."
  2. Train+inference becomes the mainstream route: Moore Threads takes the full-function GPU route (graphics+AI+general compute), differentiating from Huawei Ascend's "AI-focused."
  3. Software ecosystem is the decider: Day-0 adaptation and the maturity of unified software stacks (MUSA / NeuWare / MXMACA) are replacing raw peak compute as the core yardstick of domestic GPU "usability."

References


This article continuously tracks the domestic GPU listing process and product iteration.

Huawei Ascend 950 Series Capacity & Orders Deep Dive: 950PR Monthly Capacity Jumps 10×, ByteDance Locks In 350k Units for $5.6B

· 4 min read
Industry Research Team

The Ascend 950 series (950PR inference / 950DT training) has become the core supply of domestic AI compute. Per multiple brokerages and industry research, 950 series capacity is 100% booked with scarce spot supply; the full-year 1.2M-unit target is "100% certain," with expectations of an upward revision to 1.5M. This article summarizes capacity and order data as of July 2026.

1. Capacity pace: ~10× MoM jump in June

Time950PR monthly capacityNotes
May 202650k-60k unitsNear full production
June 2026500k-600k units~10× MoM; SMIC, Hua Hong tier-1 suppliers on overtime
Q3 2026 (est.)700k-800k unitsPer month
Full-year 2026 target1.2M unitsUpward revision to 1.5M expected

Supply chain delivery is tight: high-speed backplanes and liquid-cooling connectors' lead time stretched from 2 weeks to 6-8 weeks; orders are booked into 2027.

2. Order structure: top cloud providers + operators + overseas

CustomerLocked volumeAmount / Notes
ByteDance350k 950PR$5.6B, concentrated delivery from Q3 2026
Tencent / Alibaba / Baidu~250k 950PR + 150k 950DTCombined ~400k units
Three major operators200k+ unitsCentralized procurement, for intelligent compute centers and AI private networks
OverseasSouth Korea 2,000 units, Malaysia 3,000 servers, Russia ten-thousand-card clusterFrom pilot to commercial

3. Shipment forecast: firmly #1 domestic

Per CCA (Kezhi) Consulting estimates:

Metric20252026 (forecast)
Huawei Ascend total shipments812k cards1.026M cards
Of which 950PR~800k units
Of which 950DT~100k-200k units

Huawei has completed the product transition from the 910 series to the 950 series. The internet industry has become Ascend's largest application market; competitive advantage is extending from single-hardware performance to software ecosystem and system capabilities.

4. Going overseas: formal South Korea entry in Q4

Per Korean media ETNews, Huawei plans Q4 2026 to formally enter the South Korean market with the Ascend series and Atlas 950 SuperPod:

  • Local distributor agreements signed; two channel partners including SK Shieldus selected
  • Main products: 950PR (mass-produced and delivered since April) and 950DT (launched Q4)
  • Official line: 950PR inference performance is 2.87× that of H20, priced at about 1/4 of it

5. WAIC 2026: 1024-card live debut confirmed

At WAIC 2026 (July 17-20, Shanghai), Huawei's Atlas 950 SuperPoD live hardware made its first public appearance — a 16 compute-cabinet, 1,024 Ascend-card scale — and won the conference's top honor, the SAIL Award:

  • Core metrics: total compute 1 EFLOPS FP8 / 2 EFLOPS FP4, 256 TB globally unified memory addressing, Lingqu 2.0 interconnect, 3 μs ultra-low RTT latency
  • Full configuration: 128 compute cabinets + 32 interconnect cabinets = 160 cabinets, ~1000㎡ footprint, carrying 8,192 Ascend 950DT, planned Q4 2026 launch
  • Commercial foundation: previous-gen 384 SuperNode has cumulatively shipped 750+ units, deployed in 20+ industries
  • Software ecosystem: CANN fully open-sourced end of 2025; community incubated 67 projects, 12.44M+ lines of code, 3,500+ monthly active developers

WAIC's debut confirmed the 950 series' "SuperNode-first" product logic: beyond single-card compute, system-level effective compute (interconnect bandwidth + unified memory + low latency) is the key dimension for domestic compute to benchmark against international flagships.

Ascend roadmap recap

ProductPositioningKey metrics (official roadmap)
950PRInference1 PFLOPS (FP8) / 2 PFLOPS (FP4), 2 TB/s interconnect
950DTTrainingSuperNode core, launched Q4
960Train/inference2 PFLOPS (FP8) / 4 PFLOPS
970Next-genIn planning

Industry interpretation

  1. Domestic substitution moves from inference to training: 950PR (inference) ramps first, 950DT (training) follows in Q4, combined with the Atlas 950 SuperPoD ten-thousand-card interconnect — domestic compute now has the complete "training substitution" puzzle for the first time.
  2. Capacity is the biggest variable: order certainty is extremely high, but SMIC/Hua Hong advanced-process capacity, HBM supply, and advanced packaging remain ramp bottlenecks — the root of "scarce spot supply."
  3. Going overseas opens a second growth curve: bulk procurement from South Korea, Malaysia, Russia, and Latin America marks domestic compute's shift from "internal circulation" to "external circulation."

References


Data in this article is based on official and major brokerage research; capacity/orders are dynamic figures and will be continuously updated.

NVIDIA Vera Rubin Officially Ships: First VR200 NVL72 Delivered, Samsung HBM4 Mass Production, Rubin Ultra Cabinet Sky-High Price

· 5 min read
Industry Research Team

July 2026, NVIDIA's next-gen AI compute platform Vera Rubin officially began its first shipments, succeeding the Blackwell architecture, with large-scale mass production planned for H2 2026. First customers include Microsoft, Google, Amazon, Meta, Oracle, and other large cloud providers.

1. World's First VR200 NVL72 Delivered (Milestone)

CoreWeave jointly with Dell announced that the world's first NVIDIA Vera Rubin VR200 NVL72 cabinet has been officially delivered and passed the L11 full-cabinet hardware diagnostics on the first try. This marks Rubin's move from roadmap to physical product, with no major bottlenecks in core supply-chain links (HBM4, advanced packaging, liquid cooling, ultra-high-power power supply).

VR200 NVL72 Core Configuration

MetricVera Rubin VR200 NVL72
Cabinet codenameOberon
GPU72 Rubin GPUs
CPU36 Vera CPUs
Per-GPU memory288 GB HBM4
Per-CPU memory1.5 TB LPDDR5X
Total cabinet HBM420.7 TB (20,736 GB)
Total cabinet LPDDR5X54 TB
InterconnectNVLink 6 full mesh
Inference performance~3.6 exaFLOPS class
CoolingLiquid cooling
Generational improvement~3.5× per-GPU compute, ~2.8× memory bandwidth (vs Blackwell)

Vera CPU integrates 88 custom Olympus ARM cores, with 1.8 TB/s interconnect to the GPU, usable as a GPU memory expansion pool. NVIDIA completed its first Vera CPU deliveries to Anthropic, OpenAI, xAI, and Oracle Cloud in May.

2. Samsung HBM4 Mass Production: Key Bottleneck Eases

July 8, 2026, Samsung Electronics officially started HBM4 mass production for the Vera Rubin platform, with reported HBM4 mass-production yield reaching 70% (above the initial 60-65% expectation). Confirmation of this key supply-chain link clears obstacles for Rubin's large-scale deployment.

HBM Supply Landscape (2026 Q1)Share
SK hynix45%
Samsung40%
Micron15%

HBM4 uses 8-layer stacking (12-layer design planned for 2028), priced at about 2.8× HBM3e. TrendForce predicts HBM supply will grow 65% annually, with HBM4 reaching 35% of total output by 2027 Q4.

3. Rubin Ultra Sky-High Price: HBM Cost Dominates

Per BofA Global Research estimates, the Rubin generation will push single-server cost to a new high:

Cost ItemRubin VR200 (Oberon)Comparison
Cabinet HBM4 usage20,736 GB
HBM4 unit price~$18.40 / GBBlackwell (HBM3e) ~$11.26 / GB
HBM4 cost alone~$382KExcluding LPDDR5X
Rubin Ultra cabinet estimated price~$21MITHome / BofA estimate

4. Rubin Ultra Design Change: Original 4-die Cancelled (per SemiAnalysis)

Semiconductor research firm SemiAnalysis (2026-06-30) disclosed that the original 4-die Rubin Ultra GPU unveiled at GTC 2026 has been cancelled; the version actually shipping in 2027 is roughly halved in scale and performance:

  • Reason for cancellation: The original integrated 4 compute dies + 16 HBM4E in a single CoWoS-L package; the substrate warped under the 4-die config, causing compute-die-to-substrate contact failure and yield collapse; the alternative CoPoS won't reach mass production until after late 2028, missing the 2027 node.
  • New approach: Changed to dual-die (same construction as standard Rubin) + HBM4E, ~384 GB HBM4E per GPU (higher than standard Rubin's 288 GB), but total compute and bandwidth only half the original; to approach the original's aggregate compute, NVIDIA plans to assemble "2+2" board-level configs within the Kyber rack to reach four-die equivalent scale.
  • Kyber rack delay: The companion Kyber NVL144 rack is delayed 12+ months to 2028 due to midplane PCB manufacturing difficulties; the 800V DC power scheme is likewise delayed to 2028.

⚠️ Note: NVIDIA has not commented officially on the above design change; some on X argue "the chip count hasn't changed, it's old news reheated." This section is compiled from SemiAnalysis public reports, subject to final NVIDIA disclosure. We have marked "specs pending official confirmation" on the Rubin Ultra preview card.

Industry Interpretation

  1. "Never doubt" moment realized: Rubin's first delivery passed L11 on the first try, dispelling market doubts about "Rubin delay," locking in H2 2026 AI compute supply certainty ahead of time.
  2. Designed for Agentic AI: Rubin targets agentic workflows and ultra-long-context inference, further lowering the training/inference cost curve for trillion-parameter models.
  3. HBM is the full-chain winner: 20.7 TB HBM4 per cabinet is enormous usage; SK hynix, Samsung, Micron, advanced packaging (CoWoS-L), liquid cooling, and power retrofitting all benefit across the chain, while also becoming the biggest cost and capacity constraint.

References


This article continuously tracks Vera Rubin mass-production ramp and HBM4 supply-chain dynamics.

2026 H1 AI Chip Industry Review: Blackwell Ultra, the Domestic Big Three, and the Inference Era

· 11 min read
Industry Research Team

In the first half of 2026, the AI chip industry underwent a historic turning point — the center of gravity shifted from the "training race" to "inference efficiency," domestic chip market share broke 40% for the first time, NVIDIA built higher barriers with Blackwell Ultra, and the inference-specific chip track bloomed in diversity.


I. Compute Doubles Again: NVIDIA Blackwell Ultra Launch (June 1)

On June 1, 2026, NVIDIA CEO Jensen Huang unveiled the new-generation AI chip Blackwell Ultra at Computex 2026 (Taipei), setting a new starting line for the AI infrastructure race over the next two years.

Key Specs

MetricBlackwell UltraB200Improvement
FP8 compute20 petaFLOPS~10 petaFLOPS100%
ArchitectureBlackwell UltraBlackwellUpgrade
Expected delivery2027 Q12026 Q1
PositioningHyperscale training + inferenceTraining + inferenceFlagship

Industry Significance

  1. Direct impact of doubled compute: 20 petaFLOPS FP8 means training time for hundred-billion-parameter models drops sharply; trillion-parameter model training moves from "scientific experiment" to "engineering routine"
  2. System-level balance: Blackwell Ultra is not just a chip but a system-level engineering breakthrough across NVLink, HBM, cooling, and power delivery
  3. Roadmap certainty: The Q1 2027 delivery timeline lets cloud vendors and AI labs plan infrastructure budgets 18 months ahead

Challenges

  • Energy crisis: Doubled performance comes with sharply higher power; datacenter power and cooling design face extreme challenges
  • Accessibility: Top-tier compute goes first to top cloud vendors; how smaller developers and research institutes reach compute at reasonable cost via cloud services
  • Software stack adaptation: New hardware needs matching CUDA versions and framework support; software ecosystem maturity becomes the key bottleneck for compute conversion

II. Domestic AI Chips: The Tipping Point from "Usable" to "Good"

On June 16, 2026, Xinchuang World published "2026 China Domestic AI Chip Vendor Capability Quadrant", clearly outlining the overall domestic landscape.

2.1 Capability Quadrant Ranking

QuadrantRepresentative Vendors
Leader quadrantHuawei Ascend, Hygon, Cambricon, Alibaba T-Head, Moore Threads
Visionary quadrantBaidu Kunlunxin, Biren, Enflame, Iluvatar, HardyVision
Contender quadrantTSINGMICRO, Black Sesame, SemiDrive, Lisuan, Houmo
Challenger quadrantDenglin, Zhicun, VeriSilicon, Rockchip, Intellifusion

2.2 Huawei Ascend: The Anchor of Domestic Compute

Market Position

  • In 2025, Ascend series shipped 812,000 units, capturing 49% of the domestic AI accelerator card share, firmly No.1 domestically
  • Ascend 950PR single-card FP8 compute reaches 1P (PetaFLOPS), FP4 compute reaches 2P
  • Inference performance is about 2.87x that of NVIDIA H20, priced at only 72,000-75,000 RMB, a significant price/performance advantage

Full-Stack Advantage

Huawei's "device-network-cloud-chip" integrated strategy is Ascend's core moat:

  • Chip design: Da Vinci 3.0 architecture iterating continuously
  • OS: HarmonyOS/Euler OS deeply optimized
  • Networking: Euler network protocol stack
  • Cloud: Huawei Cloud ModelArts platform seamlessly integrated

Latest Progress

  • On June 5, 2026, Shenzhen Hetao College, together with HIT (Shenzhen) and Huawei, completed full-parameter post-training of a 1.6-trillion-parameter DeepSeek V4 Pro model on an Ascend 910C cluster
  • This is the first time domestic AI chips completed trillion-parameter-level model training, marking "domestic substitution" moving from inference to training

2.3 Cambricon: The First Profitable Domestic AI Chip Benchmark

Performance Explosion

MetricFull-year 20252026 Q1YoY Growth
Revenue6.497B RMB2.885B RMB+453% / +160%
Net profit2.059B RMB (first annual profit)1.013B RMB— / +185%

Core Product: Siyuan 590

  • In DeepSeek R1 inference scenarios, TPS reaches 942, about 50% higher than H20
  • Years of joint optimization with ByteDance; strongest short-term cloud inference deployment capability
  • Of 2.885B RMB Q1 2026 revenue, Siyuan 590 contributed over 70%

Potential Risks

Absent from the 2nd 2026 "Safe and Reliable Evaluation Results Announcement"; the reason is unclear and will affect its domestic government/enterprise market performance.

2.4 TSINGMICRO: The "Third Route" of Reconfigurable Chips

Technical Route

TSINGMICRO adopts a reconfigurable dataflow architecture同源 with Groq LPU, finding a balance between GPU generality and ASIC extreme efficiency.

MetricTSINGMICRO TX81Traditional GPUAdvantage
Inference costBaseline+100%Reduced 50%
Energy efficiencyBaselineBaseline3x improvement
ArchitectureReconfigurable dataflowSIMT/SIMDBetter for inference

Deployment Progress

  • Cumulative shipments of reconfigurable chips exceed 30 million units
  • Scaled deployment in a dozen-plus thousand-card-scale intelligent computing centers nationwide
  • Has begun A-share IPO tutoring; likely to become the "first reconfigurable chip stock"

III. The Inference Chip Track: Core Signal of the Industry Shift

On June 4, 2026, TrendForce published a deep report "The Era of Inference Economy: The Rules of AI Chips Are Being Rewritten," pointing out that the compute competition center of gravity is shifting from training to inference.

3.1 Why Now?

Cost Structure Changed

  • Training is a one-time cost: Once a model is trained, marginal cost approaches zero
  • Inference is a recurring cost: Every API call, every generated token represents compute consumption and gross-margin pressure
  • Per-unit inference cost and energy efficiency directly affect gross margin and scale-expansion capability

Model Compression Tech Matured

  • 1.58-bit quantization and weight pruning let models maintain inference accuracy at extremely low memory footprint
  • MoE (Mixture of Experts) architecture activates only a few expert sub-networks per inference via "partial wake-up," greatly reducing actual computation
  • The rise of slimmed models provides commercial viability for hard-wired inference chips

3.2 NVIDIA's $20B Bet: Acquiring Groq (December 2025)

On December 24, 2025, NVIDIA acquired Groq's inference technology license and core team for $20 billion, one of NVIDIA's largest M&A/tech acquisitions ever.

Strategic intent:

  1. Fill the inference gap: NVIDIA GPU is unshakable in training, but inference efficiency was never its strongest suit
  2. Counter specialized inference chips: Cerebras, Taalas, SambaNova and other startups are eroding the inference market
  3. Position for Agentic AI: Agentic AI needs extremely low-latency, high-throughput inference

3.3 Taalas HC1: Proof of Concept for Hard-Wired Inference

On February 20, 2026, Canadian AI chip startup Taalas launched inference chip Taalas HC1, directly etching Meta's open-source AI model Llama 3.1 8B into the chip.

Key Metrics

MetricTaalas HC1NVIDIA B200 (throughput optimized)Advantage
Inference rate16,960 tokens/s/userBaseline~4-5x
Cost per million tokens0.75 cents3.79 centsReduced 80%
Power~250W~700WReduced 64%
ProcessTSMC N6TSMC 4nmMore mature
HBM❌ Not used✅ HBM3eLower cost

Technical Principle

Taalas HC1 uses an aggressive Computing-in-Memory (CIM) implementation:

  • Model weights directly固化 in Mask ROM (fully hardware-defined)
  • On-chip SRAM handles dynamic data (KV cache and LoRA fine-tuning weights)
  • Only 2 mask layers need modification to produce a dedicated chip for another AI model; turning an AI model into a physical chip takes only 2 months

Limitations

  • Lack of flexibility: Hard-wiring cannot cope with rapidly iterating model updates
  • Ecosystem barrier: The current cloud market still relies on general-purpose platforms; customers may prefer flexible solutions that upgrade with models
  • NRE cost: High one-time engineering cost, requiring sufficient deployment scale to amortize

3.4 Cerebras: The IPO Path of Wafer-Scale Integration

On May 14, 2026, Cerebras Systems officially listed on NASDAQ, becoming the first wafer-scale AI chip company to go public.

Core Technology: Wafer-Scale Integration (WSI)

  • WSE-3 (third-gen wafer-scale engine): An entire 12-inch wafer as a single chip
  • 44GB on-chip SRAM: No external HBM, eliminating the memory bandwidth bottleneck
  • 21 PB/s bandwidth: On-chip communication bandwidth, thousands of times that of GPUs
  • Partnership with OpenAI: Signed a 3-year, 750MW, $20B+ compute cooperation agreement

IPO Significance

Cerebras's listing marks the maturation of the inference-specific chip track:

  1. Capital markets begin pricing such companies
  2. Proves "non-GPU" technical routes have commercial viability
  3. Provides valuation references for other inference chip startups (Groq, SambaNova, Taalas, etc.)

3.5 Inference Chip Landscape: Multiple Technical Routes Coexist

CompanyTechnical RouteCore AdvantageRepresentative Product
TaalasHard-wired (Mask ROM)Extreme inference efficiency, low costHC1
CerebrasWafer-scale integration (WSI)Ultra-high bandwidth, large-model inferenceWSE-3
GroqSRAM-first architectureDeterministic latency, high throughputLPU (acquired by NVIDIA)
d-MatrixDigital in-memory compute (DIMC)More flexible than hard-wiringCorsair
EtchedHard-wired TransformerWorks for all Transformer modelsSohu
Axelera AIDigital in-memory compute (D-IMC) + RISC-VHigh energy efficiencyMetis AIPU

TrendForce predicts:

  • General-purpose GPUs still dominate training and multi-model environments
  • But in mature, predictable scenarios, general-purpose GPU profit margins will be compressed
  • The industry shifts from general compute monopoly to a dual-track structure of general + specialized coexistence

IV. Overall Domestic AI Chip Landscape in H1 2026

4.1 Industry Enters Scale-Up Phase

Metric20252026 Q1Trend
Domestic AI accelerator shipments1.65M units (41% share)Rising
Total China AI accelerator shipments~4M units
Hygon revenue growthDoubled
Cambricon revenue growth+160%
Moore Threads revenue growthDoubled

Leading vendors collectively entered the revenue realization channel, moving from "technical validation" to "scale commercialization."

Trend 1: Capitalization Wave Reshapes the Landscape

  • Late 2025 to early 2026: Moore Threads, Iluvatar listed on the STAR Market
  • Biren listed on the Hong Kong stock exchange
  • Enflame STAR Market IPO accepted
  • Kunlunxin, T-Head initiated listing processes
  • TSINGMICRO, HardyVision and others advancing IPOs

Capitalization brings dual effects:

  • Positive: Supports R&D and ecosystem building
  • ⚠️ Negative: Valuation bubbles and revenue realization pressure

Trend 2: Capacity Becomes the Biggest Constraint Variable

The contradiction between explosive domestic AI chip demand and limited advanced-process capacity is sharpening:

VendorAdvanced-process capacity needActually obtained
Huawei Ascend15K wafers/month (7nm-class)Priority guaranteed
SMIC total capacity~20K wafers/month (7nm-class)
Other vendors~5K wafers/month combinedExtremely tight

Whether stable wafer capacity can be secured directly determines vendor survival. Cambricon's 75.4% inventory-to-revenue ratio is essentially a lock on capacity.

Trend 3: Competition Shifts from "Usable" to "Good"

Early competition focused on "can it run the model"; now it's about "runtime efficiency, deployment cost":

Dimension"Usable" era"Good" era
Hardware performanceCan it run the modelRuntime efficiency, energy efficiency
Software stackBasic adaptationMaturity, framework breadth
EcosystemExistenceDeveloper community activity
Deployment costInsensitiveCore competitive factor

V. H2 2026 Outlook

5.1 Upcoming Key Events

TimeEventImpact
2026 Q3NVIDIA Rubin architecture details revealedNext-gen flagship specs unveiled
2026 Q3Huawei Ascend 950PR/950DT formally launchedNew benchmark for domestic inference chips
2026 Q4AMD MI350X scaled deliveryNVIDIA Blackwell competitor
2026 Q4Cambricon Siyuan 690 launch (est.)New-gen training chip
2027 Q1NVIDIA Blackwell Ultra deliveryNew compute benchmark lands

5.2 Key Competitive Factors Over the Next Three Years

  1. Wafer capacity access: Advanced-process capacity is a scarce resource; vendors tied to SMIC and TSMC have inherent advantages
  2. Capital operation efficiency: The IPO window is limited; raising enough capital on the market determines R&D sustainability
  3. Software ecosystem depth: Hardware performance is only the entry ticket; software stack maturity, framework adaptation breadth, and developer community activity are the core moat

VI. Conclusion: A Diverse Ecosystem Will Eventually Form

In H1 2026, the AI chip industry is undergoing a historic transition from "one dominant player" to "pluralistic coexistence."

  • NVIDIA builds higher training barriers with Blackwell Ultra while laying out inference efficiency via the Groq acquisition
  • Huawei Ascend holds the domestic compute baseline with full-stack capability; 950PR begins to surpass H20 in inference
  • Cambricon proves the commercial viability of domestic AI chips by turning profitable first; Siyuan 590 surpasses international rivals in specific scenarios
  • Cerebras, Taalas and other inference-specific chip companies opened a "non-GPU" third route
  • TSINGMICRO's reconfigurable architecture provides a diversified technical route choice for China's AI chips

Over the next three years, the domestic AI chip endgame will form a pluralistic ecosystem where GPU, ASIC, and reconfigurable computing three technical routes coexist, with cloud and edge developing in coordination. "Domestic substitution" is no longer a slogan, but an industrial reality happening now.


Data sources:

  • Xinchuang World "2026 China Domestic AI Chip Vendor Capability Quadrant" (2026-06-16)
  • TrendForce "The Era of Inference Economy: The Rules of AI Chips Are Being Rewritten" (2026-06-04)
  • RayByte "Compute Doubles! NVIDIA Blackwell Ultra Chip Launched" (2026-06-02)
  • Official financial reports and announcements of each company

Related reading:


June 2026 AI Chip Major Events Roundup: Ascend 910C Trains Trillion-Parameter Model, OpenAI Custom Chip, RTX Spark Launch

· 6 min read
Industry Research Team

June 2026 saw multiple milestone events in the AI chip field, marking acceleration of two major trends: "domestic substitution" and "de-NVIDIA-ization."

1. Huawei Ascend 910C Completes 1.6-Trillion-Parameter DeepSeek V4 Pro Training (2026-06-05)

Event Overview

June 5, 2026, Shenzhen Hetao College, together with Harbin Institute of Technology (Shenzhen), Shenzhen Big Data Research Institute, Huawei, and other teams, relied on an Ascend 910C domestic AI compute cluster to successfully complete full-parameter post-training of the 1.6-trillion-parameter DeepSeek V4 Pro large model.

Technical Significance

MetricValue
Model parameters1.6 trillion
Training chipAscend 910C cluster
Training typeFull Parameter Post-Training
SignificanceFirst time domestic AI chips complete trillion-parameter-level model training

Industry Impact

  1. Breaks technology blockade: Proves domestic AI chips can train trillion-parameter models
  2. Accelerates "farewell to NVIDIA": DeepSeek fully switches to Huawei Ascend, reducing dependence on H100
  3. Domestic substitution inflection point: From "inference substitution" to "training substitution"

2. OpenAI Launches First Custom AI Inference Chip Jalapeño (2026-06-24)

Event Overview

June 24, 2026, OpenAI and Broadcom jointly launched the first custom AI inference chip Jalapeño, with a design cycle of only 9 months (industry average 18 months), using TSMC 3nm process.

Key Metrics

MetricJalapeñoComparison (Blackwell)
ProcessTSMC 3nmTSMC 4nm
ArchitectureSystolic ArrayBlackwell GPU
Design cycle9 months~18 months
Inference cost-50%Baseline
AI-assisted design✅ First❌ No
DeploymentEnd of 2026Shipped

Strategic Significance

  1. First AI chip with AI-assisted design: OpenAI used models like GPT-5.3-Codex-Spark to assist architecture exploration
  2. Accelerates "de-NVIDIA-ization": Tech giants (Google, Amazon, Microsoft, Meta, OpenAI) collectively develop custom chips
  3. Inference cost revolution: For OpenAI processing hundreds of millions of API calls daily, a 50% cost reduction is significant

3. NVIDIA Launches RTX Spark AI PC Superchip at Computex 2026 (2026-06-01)

Event Overview

June 1, 2026, NVIDIA CEO Jensen Huang launched the RTX Spark AI PC superchip at Computex 2026 / GTC Taipei, in collaboration with MediaTek, using an Arm CPU + Blackwell GPU unified-memory architecture.

Key Metrics

MetricRTX Spark
CPUUp to 20-core Arm (with MediaTek)
GPU6,144 CUDA cores (Blackwell)
Unified memory128GB LPDDR5X (shared CPU+GPU)
Memory bandwidth300 GB/s
AI compute~1 PFLOPS (est.)
Model capacityCan run 120B-parameter models
ContextUp to 1 million tokens
TDP~100W (est.)
AvailabilityFall 2026

Industry Impact

  1. NVIDIA enters PC chip market: Challenges Intel's dominance in personal computers
  2. New AI PC standard: Run 120B-parameter models locally, 1M-token context
  3. Windows transforms into AI Agent platform: Deep collaboration with Microsoft OpenShell framework

4. MIIT Publishes "2026 AI Chip Industry Development White Paper" (2026-06-09)

Event Overview

June 9, 2026, China's Ministry of Industry and Information Technology published the "2026 AI Chip Industry Development White Paper," predicting the domestic AI chip market will exceed 200 billion RMB in 2026.

Key Predictions

Metric2026 Prediction
Market sizeExceed 200 billion RMB
Domestic chip share>50% (41% in 2025)
Edge inference chipsSignificant progress
Shipment growthMore than double (vs 2025)

Industry Significance

  1. Domestic AI chip capitalization accelerates: Cambricon, Enflame, Moore Threads, etc. accelerate IPOs
  2. Edge inference becomes the breakthrough: Easier to achieve domestic substitution than training chips
  3. Policy dividend continues: Domestic substitution upgraded from "market behavior" to "national strategy"

5. ByteDance in Talks to Procure 50K Iluvatar Inference Chips (2026-06-17)

Event Overview

June 17, 2026, Reuters reported that ByteDance is in talks with Shanghai AI chip firm Iluvatar to procure at least 50,000 AI chips, mainly for inference tasks.

Deal Details

ItemContent
BuyerByteDance
SupplierIluvatar
Chip modelZhiKai series (inference GPU)
QuantityAt least 50,000
UseInference workloads
Training chipTianTai series

Industry Significance

  1. Domestic GPU top player "adds a member": Iluvatar enters a top internet company's supply chain for the first time
  2. ByteDance 2026 capex raised over 200B RMB: Mainly for AI compute and datacenters
  3. "Domestic substitution" extends from government/SOEs to private tech giants

Trend 1: "Domestic Substitution" Moves from Inference to Training

  • Ascend 910C completes 1.6-trillion-parameter model training → Proves domestic chips have training capability
  • DeepSeek fully switches to Ascend → Leading AI companies first to "farewell to NVIDIA"
  • ByteDance procures Iluvatar → Private tech giants follow

Trend 2: "De-NVIDIA-ization" from Slogan to Action

  • OpenAI Jalapeño → First custom chip, inference cost -50%
  • Google TPU, Amazon Trainium, Microsoft Maia → Continuous iteration
  • Meta MTIA, Apple M5 Ultra → Increased investment

Trend 3: AI PC and Edge Inference Become New Battlefield

  • NVIDIA RTX Spark → New AI PC standard, launches Fall 2026
  • Edge inference chip localization accelerates → Key mention in MIIT white paper
  • "Local trillion-parameter model execution" → New consumer market selling point

Looking Ahead (2026 H2)

  1. Ascend 950DT full scale-up (2026 Q4) → Huawei's latest-gen training chip
  2. NVIDIA Rubin R200 shipment (2026 H2) → Next-gen flagship
  3. AMD MI400 Helios rack (2026 H2) → Targets NVIDIA GB200
  4. OpenAI Jalapeño deployment (end of 2026) → Gigawatt-scale datacenters
  5. Domestic AI chip shipments more than double → CITIC Securities prediction

References


This article is continuously updated. Please provide the latest developments.

OpenAI's In-House AI Chip Jalapeño Deep Dive: Taped Out in 9 Months, Inference Cost Cut 50%

· 9 min read
AI Hardware Analyst

On June 24, 2026, OpenAI and Broadcom jointly announced their first in-house AI inference chip, Jalapeño. This ASIC designed specifically for large language model inference went from design to tape-out in just 9 months and cuts inference cost by roughly 50%, marking OpenAI's transformation from a pure model company into a full-stack AI infrastructure provider.


1. Core conclusions (read this first)

DimensionJalapeñoCurrent GPU solutionAdvantage
Inference cost-50%Baseline✅ Half the cost
Performance per wattClearly superiorMost advanced accelerator✅ Energy-efficiency lead
Design cycle9 months~18 months✅ 2× faster
PositioningInference ASICTrain+inference GPUDedicated optimization
SupplyInternal onlyMarket purchase⚠️ Not for sale

One-line summary: Jalapeño is a key step in OpenAI's full-stack AI strategy, using in-house silicon to cut inference cost 50% while opening a new paradigm of "AI-assisted design of AI chips."


2. What is Jalapeño?

2.1 Basic information

ItemDetail
NameJalapeño (a chili pepper)
TypeApplication-specific integrated circuit (ASIC)
PositioningLarge language model inference
Announced2026-06-24
Taped outSep 2025 (est., 9-month rapid tape-out)
DeploymentEnd of 2026 (gigawatt-scale data centers)
PartnersBroadcom, TSMC, Celestica
ProcessTSMC 3nm
ArchitectureSystolic Array
HBM8 stacks (est. HBM3E or HBM4)

2.2 Why "Jalapeño"?

Jalapeño is a Mexican chili known for "medium heat, strong flavor." OpenAI's naming hints that the chip:

  • Medium heat: not the most aggressive architecture (vs Cerebras WSE), but effective enough
  • Strong flavor: strong presence in inference scenarios (50% cost reduction)
  • Appetizer: just "the first step of a multi-generation roadmap" (Broadcom CEO Hock Tan)

3. Deep technical analysis

3.1 9-month rapid tape-out: the new paradigm of AI-assisted chip design

Normally, designing an ASIC from scratch takes 1.5 to 2 years. Jalapeño went from initial design to manufacturing tape-out in just 9 months.

Key reason: deep software-hardware co-development

TechniqueDescription
AI-assisted architecture explorationOpenAI used its own frontier models (GPT-5.3-Codex-Spark) to explore chip architecture design space
AI power simulationAI models for power simulation and optimization
RL optimizationRL to optimize chip placement and routing
Broadcom silicon implementationBroadcom provides top-tier ASIC implementation (network, switch chip experience)

OpenAI President Greg Brockman said:

"We use the frontier models that serve our users to optimize the infrastructure that runs the models of the future."

3.2 Architecture optimized for inference

Unlike general-purpose GPUs, Jalapeño is an ASIC built from scratch around OpenAI's deep understanding of LLM inference workloads:

Architecture featureDescription
Reduce data movementCore principle is minimizing data movement (the main bottleneck in inference)
Balanced compute-memory-networkResource allocation optimized for inference, bringing real utilization closer to theoretical peak
High throughput + low latencyAims to combine the throughput of leading accelerators with the low latency of the fastest dedicated inference systems
Future model supportSupports not only current models (GPT-5, GPT-5.3) but adapts to next-gen inference needs

3.3 Full-stack platform: more than a chip

Jalapeño is a multi-generation compute platform, not just a chip:

ComponentSupplierDescription
Accelerator chipOpenAI design, TSMC fabTSMC 3nm, 8-stack HBM
Network switch chipBroadcom TomahawkHigh-speed interconnect (competes with NVIDIA NVLink)
Board, rack, systemCelesticaFull-rack solution
Software stackOpenAIDeep adaptation for GPT, Codex, Agent products

Deployment target: gigawatt-scale data centers

Broadcom CEO Hock Tan said:

"Jalapeño will begin deployment this year in gigawatt-scale data centers with Microsoft and other partners."


4. Performance and cost analysis

4.1 Inference cost cut 50%

Although OpenAI's official release was conservative on Jalapeño's cost savings — only stating its "performance per watt is substantially better than today's state of the art" without a specific percentage — per Bloomberg, Broadcom CEO Hock Tan revealed:

Early internal tests show Jalapeño achieves roughly 50% inference cost savings versus today's mainstream AI GPUs.

Significance for OpenAI:

ItemCurrent (GPU)JalapeñoSavings
Daily API callsHundreds of millionsHundreds of millions
Inference cost share~60-70% of operating cost~30-35%-50%
Annual compute spendBillions of dollarsHundreds of millionsSaves billions

4.2 Performance per watt clearly better than state of the art

OpenAI's announcement states:

"Jalapeño engineering samples have successfully run complex reinforcement-learning tasks such as GPT-5.3-Codex-Spark at target frequency and power; early tests show performance per watt substantially better than today's most advanced AI accelerators."

Comparison target: NVIDIA Blackwell (today's most advanced AI accelerator)

MetricJalapeñoNVIDIA BlackwellNote
Performance per wattClearly superiorBaselineOpenAI official statement
Inference latencyOn par with fastest dedicated inference systemsBaselineTarget
ThroughputOn par with leading acceleratorsBaselineTarget
TDPNot disclosed (est. 400-700W)700-1000WJalapeño possibly lower

5. Impact on the AI chip market

5.1 "De-NVIDIA-ification" accelerates

Jalapeño's launch is another footnote in big-tech's collective challenge to NVIDIA's market dominance:

VendorIn-house chipTypeStatusRelation to OpenAI
GoogleTPU v6e / IronwoodTrain+inference✅ CommercialGoogle Cloud supplies OpenAI
AmazonTrainium 3Training✅ LaunchedAWS supplies OpenAI
MicrosoftMaia 100Train+inference✅ LaunchedOpenAI exclusive partner
MetaMTIATrain+inference✅ Launched
AppleNeural EngineOn-device inference✅ Commercial
OpenAIJalapeñoInference🚧 Deploy end of 2026Internal + possibly sold to third parties

5.2 OpenAI is not about to fully "abandon" NVIDIA

Brockman admitted:

"We simply cannot get compute fast enough."

Currently OpenAI is simultaneously procuring chips from NVIDIA, AWS, AMD, and Cerebras; Jalapeño is a structural supplement to its explosive compute demand, not a replacement.

5.3 Possibly sold to third parties

Broadcom CEO Hock Tan specifically emphasized:

"This is just 'the start of a multi-generation roadmap'; OpenAI and Broadcom aim to jointly build gigawatt-scale compute clusters."

This means OpenAI may sell its hardware to third parties, provided it can secure enough supply from Broadcom and TSMC.


6. Jalapeño vs other in-house chips

MetricJalapeño (OpenAI)TPU v6e (Google)Trainium 3 (Amazon)Maia 100 (Microsoft)
Announced2026-06-242024Q4 20252023
TypeInference ASICTrain+inference TPUTraining ASICTrain+inference
ProcessTSMC 3nmTSMC 4nmTSMC 5nm (est.)TSMC 5nm (est.)
For sale❌ Internal only (maybe later)✅ GCP✅ AWS❌ Internal only
Design cycle9 months~18 months~18 months~18 months
AI-assisted design✅ First❌ No❌ No❌ No
Cost advantage-50% inference costOptimizedOptimizedOptimized

Key differences:

  • ✅ Jalapeño is the first AI chip designed with AI assistance
  • ✅ Jalapeño design cycle only 9 months (industry average 18 months)
  • ⚠️ Jalapeño not for sale (at least for now)

7. Future roadmap

7.1 Multi-generation chip platform

Jalapeño is just "the start of a multi-generation roadmap":

TimeEvent
End of 2026Jalapeño initial deployment (gigawatt-scale data centers)
2027Jalapeño v2 (est., architecture optimization)
2027-2028Jalapeño training version (est., challenging TPU/Trainium)
2028 and beyondGigawatt-scale compute cluster fully built

7.2 OpenAI full-stack AI infrastructure strategy

LayerOpenAI in-houseOutsourced/procured
Models✅ GPT-5, GPT-5.3, Codex
Chips✅ Jalapeño (inference)NVIDIA GPU, AWS Trainium, AMD GPU
Systems✅ With CelesticaMicrosoft Azure data centers
Network✅ Broadcom TomahawkMicrosoft Azure network
Cloud platform❌ NoneMicrosoft Azure (exclusive partner)

8. Industry reaction and expert views

8.1 Supportive views

ExpertView
Broadcom CEO Hock Tan"Jalapeño is just the start of a multi-generation roadmap; the goal is to jointly build gigawatt-scale compute clusters."
OpenAI President Greg Brockman"We use the frontier models that serve our users to optimize the infrastructure that runs the models of the future."
Industry insiders"Jalapeño's launch is another footnote in big-tech's collective challenge to NVIDIA's market dominance."

8.2 Skeptical views

ConcernDescription
Not for saleCurrently internal only; third parties cannot purchase, limited impact on NVIDIA's market share
Software ecosystemOpenAI must build its own software stack; competing with CUDA is hard
Supply capacityTSMC capacity is limited; can it meet OpenAI + Broadcom + other customers' demand?
Opaque performance dataOpenAI has not released specs (compute, memory, bandwidth, TDP), hard to assess objectively

9. Significance for developers

9.1 If OpenAI sells Jalapeño to developers in the future...

ScenarioCurrent (NVIDIA GPU)Future (Jalapeño)
Inference costBaseline-50%
Inference latencyBaselinePossibly lower
Software stackCUDA + TensorRTOpenAI API (possibly open-source stack)
Procurement difficultyHigh (export controls, supply shortage)Low (OpenAI direct supply)

9.2 Worth watching even if not sold

  • 50% inference cost cut forces NVIDIA, AMD, Intel to lower GPU prices
  • ✅ The AI-assisted chip design paradigm will be rapidly copied by the industry
  • ✅ The 9-month tape-out cycle becomes a new industry benchmark

10. Summary

DimensionAssessment
Technology innovation⭐⭐⭐⭐⭐ First AI chip designed with AI assistance, 9-month tape-out
Cost advantage⭐⭐⭐⭐⭐ 50% inference cost cut, billions saved annually
Strategic significance⭐⭐⭐⭐⭐ OpenAI transforms from pure model company to full-stack AI infrastructure provider
Market impact⭐⭐⭐⭐ "De-NVIDIA-ification" accelerates, big-tech in-house chip camp grows
Openness⭐⭐ Internal only for now, possibly sold to third parties later

Final recommendations:

  • 🇨🇳 China market: Keep watching Huawei Ascend, Cambricon MLU, Moore Threads MTT (Jalapeño not sold to China)
  • 🌍 International market: Watch whether Jalapeño is eventually sold externally and its impact on NVIDIA's market share
  • 💡 Developers: Watch for possible OpenAI API price cuts (50% inference cost cut may partially pass through)

References


Disclaimer: Some specs in this article are estimates, subject to OpenAI's official technical white paper. OpenAI will release a detailed performance white paper in the coming months.

Last updated: June 26, 2026

China's Domestic AI Chip Triopoly (2026): Ascend, Cambricon, Moore Threads — Who Is the "China H100"?

· 7 min read
AI Hardware Analyst

Against the backdrop of U.S. export controls, China's AI chip market is forming a "three-way standoff." This article compares the technical routes, product specs, software ecosystems, and commercial progress of the three major domestic AI chip vendors: Huawei Ascend, Cambricon MLU, and Moore Threads MTT.


Key Points

  • Huawei Ascend: leader in domestic AI training chips; Ascend 950 in mass production; most mature software ecosystem
  • Cambricon MLU690: the "China H100," compute close to H200, clear efficiency advantage
  • Moore Threads MTT S5000: full-function GPU route; achieved Day-0 support for Qwen3.5 and GLM-5.2 in June 2026
  • Shared challenge: affected by U.S. export controls, primarily aimed at the Chinese market, limited internationally

I. Vendor Overview

VendorFoundedFounderListed2025 RevenueMain Customers
Huawei Ascend2018 (division)Ren Zhengfeiprivate (wholly owned by Huawei)~¥20B (est.)Chinese gov, SOEs, military
Cambricon2016Chen Tianshi (CAS)2020-07 (STAR Market 688256)~¥5.2BByteDance, Alibaba, Baidu
Moore Threads2020Zhang Jianzhong (ex-NVIDIA China)2023-12 (STAR Market 688495)~¥1.5B (est.)gov, SOEs, gaming cos.

Strategic Positioning

VendorTech routeCore strengthMain challenge
Huawei AscendAI-training-specific (Da Vinci)co-optimized HW/SW, carrier channelssanctions, process limits
CambriconAI-training-specific (MLUarch)high efficiency, competitive priceimmature ecosystem
Moore ThreadsFull-function GPU (MUSA)graphics + AI + general compute, Day-0 supportcompute below dedicated AI chips

II. Flagship Product Comparison

1. Huawei Ascend 950DT (2026 flagship)

ItemSpec
BF16 compute1,000 TFLOPS
Memory144GB HiZQ 2.0 (in-house HBM)
Memory bandwidth4 TB/s
TDP400W
ProcessN+2 (improved 7nm)
Released2026-04
Mass production2026-Q2
Unit price~¥80,000 (est.)

Strengths:

  • High large-model inference throughput: 144GB memory friendly to DeepSeek R1 (671B MoE)
  • Most mature ecosystem: CANN ~85% operator coverage, supports PyTorch, TensorFlow
  • Strong carrier channel: China Mobile, China Telecom large purchases

Weaknesses:

  • Process limited: N+2 below TSMC 4nm
  • Mediocre efficiency: 400W TDP, 2.5 TFLOPS/W

2. Cambricon MLU690 (2026 flagship)

ItemSpec
BF16 compute600 TFLOPS
Memory64GB HBM3
Memory bandwidth2 TB/s
TDP280W
ProcessTSMC 7nm
Released2025-Q4
Mass production2026-Q1
Unit price~¥140,000 (est.)

Strengths:

  • Best efficiency: 280W TDP, 2.14 TFLOPS/W (1.5x H100)
  • Competitive price: ~$20,000, 33% cheaper than H100
  • Top-tier customer orders: ByteDance, Alibaba, Baidu

Weaknesses:

  • Small memory: 64GB limits large-model training scale
  • Immature ecosystem: NeuWare ~75–85% coverage; complex LLMs need manual tuning

3. Moore Threads MTT S5000 (2025 flagship)

ItemSpec
FP16 compute~1,000 TFLOPS (est.)
Memory80GB GDDR6X
Memory bandwidth1.6 TB/s
TDP~350W
ProcessTSMC 4nm (est.)
Released2025-02
Mass production2025-Q2
Unit price~¥50,000 (est.)

Strengths:

  • Full-function GPU: graphics + AI + general compute, broader scenarios
  • Strong Day-0 support: June 2026 Day-0 support for Qwen3.5, GLM-5.2, MiniMax M3
  • Lowest price: ~¥50,000, high cost-performance

Weaknesses:

  • Compute below dedicated AI chips: FP16 ~50% of H100
  • Low memory bandwidth: 1.6 TB/s (48% of H100), limits large-model training

III. Compute Comparison (BF16/FP16)

ChipBF16 computeMemoryBandwidthTDPEfficiency
Huawei Ascend 950DT1,000 TFLOPS144GB4 TB/s400W2.5 TFLOPS/W
Cambricon MLU690600 TFLOPS64GB2 TB/s280W2.14 TFLOPS/W
Moore Threads MTT S5000~1,000 TFLOPS80GB1.6 TB/s~350W~2.86 TFLOPS/W
NVIDIA H100989 TFLOPS80GB3.35 TB/s700W1.41 TFLOPS/W
NVIDIA H200989 TFLOPS141GB4.8 TB/s700W1.41 TFLOPS/W

Key insights:

  1. Ascend 950DT has the highest compute (1,000 TFLOPS) but mediocre efficiency
  2. Cambricon MLU690 has the best efficiency (2.14 TFLOPS/W), TDP only 280W
  3. Moore Threads MTT S5000 wins on full-function versatility but low bandwidth

IV. Software Ecosystem

VendorStackFramework supportCoverageMaturity
Huawei AscendCANNPyTorch, TensorFlow, MindSpore~85%⭐⭐⭐⭐ (4/5)
CambriconNeuWarePyTorch-Cambricon, TensorFlow-Cambricon~75–85%⭐⭐⭐ (3/5)
Moore ThreadsMUSIFYPyTorch, TensorFlow, ONNX~70%⭐⭐⭐ (3/5)
NVIDIACUDAall~99%⭐⭐⭐⭐⭐ (5/5)

Ecosystem Maturity Assessment

Huawei Ascend CANN:

  • ✅ Strength: highest operator coverage, supports MindSpore (in-house framework)
  • ❌ Weakness: steep learning curve, incomplete docs

Cambricon NeuWare:

  • ✅ Strength: PyTorch/TensorFlow compatible, low migration cost
  • ❌ Weakness: complex LLMs need manual tuning

Moore Threads MUSIFY:

  • ✅ Strength: strong Day-0 support, ONNX support
  • ❌ Weakness: lowest operator coverage, dual graphics+AI engine complexity

V. Commercial Progress

Vendor2026 commercial progressMain customersShipments
Huawei AscendAscend 950 mass production; China Mobile large purchaseChina Mobile, China Telecom, gov~100K/yr (est.)
CambriconMLU690 mass production; ByteDance, Alibaba ordersByteDance, Alibaba, Baidu~50K/yr (est.)
Moore ThreadsMTT S5000 mass production; Day-0 Qwen3.5gov, SOEs, gaming cos.~30K/yr (est.)

Latest as of June 2026

Huawei Ascend:

  • ✅ Ascend 950DT fully ramping
  • ✅ ¥1B procurement agreement with China Mobile

Cambricon:

  • ✅ MLU690 in volume shipment
  • ✅ ByteDance order ~20K units

Moore Threads:

  • ✅ Day-0 support for Qwen3.5, GLM-5.2, MiniMax M3
  • ✅ MTT S5000 2nd-gen released

VI. Selection Advice

Scenario 1: Trillion-parameter training (GPT-4 class)

Recommended: Huawei Ascend 950DT

  • ✅ 144GB large memory supports super-large models
  • ✅ Most mature ecosystem (~85% coverage)
  • ✅ Strong carrier channel, Chinese government backing

Alternative: Cambricon MLU690 (high efficiency, but small memory)

Scenario 2: Tens-to-hundreds-of-billions parameter training

Recommended: Cambricon MLU690

  • ✅ Best efficiency (2.14 TFLOPS/W), low TCO
  • ✅ Competitive price (~$20,000)
  • ✅ Validated by top customers (ByteDance, Alibaba)

Alternative: Huawei Ascend 920 (more compute, mediocre efficiency)

Scenario 3: Cloud AI inference

Recommended: Huawei Ascend 950PR (inference-specific)

  • ✅ Well-optimized inference throughput
  • ✅ 128GB memory friendly to MoE models
  • ✅ Mature stack, low deployment cost

Alternative: Moore Threads MTT S5000 (full-function GPU, inference + graphics)

Scenario 4: Edge AI / on-device inference

Recommended: Moore Threads MTT S5000

  • ✅ Full-function GPU, graphics + AI
  • ✅ Lowest price (~¥50,000)
  • ✅ Strong Day-0 support

Alternative: Huawei Ascend 310 (low power, 8W TDP)

Scenario 5: Domestic substitution (gov, SOEs)

Recommended: Huawei Ascend 950DT

  • ✅ Chinese government first choice, carrier bulk buys
  • ✅ Co-optimized HW/SW, stable performance
  • ✅ Supported by national semiconductor fund

Alternative: Cambricon MLU690 (high efficiency, competitive price)


VII. Future Roadmap

Vendor2026 H220272028
Huawei Ascend950DT ramp960 (FP8 ~2 PFLOPS)970 (N+3 process)
CambriconMLU690 rampMLU790 (5nm, BF16 ~1,000 TFLOPS)MLU890 (3nm)
Moore ThreadsMTT S5000 2nd-genMTT S6000 (HBM3, FP16 ~1,500 TFLOPS)MTT S7000

VIII. Summary: Who Is the "China H100"?

DimensionAscend 950DTMLU690MTT S5000
Compute⭐⭐⭐⭐⭐ (5/5)⭐⭐⭐ (3/5)⭐⭐⭐ (3/5)
Memory⭐⭐⭐⭐⭐ (5/5)⭐⭐ (2/5)⭐⭐⭐ (3/5)
Efficiency⭐⭐⭐ (3/5)⭐⭐⭐⭐⭐ (5/5)⭐⭐⭐⭐ (4/5)
Ecosystem⭐⭐⭐⭐ (4/5)⭐⭐⭐ (3/5)⭐⭐⭐ (3/5)
Price⭐⭐⭐ (3/5)⭐⭐⭐⭐ (4/5)⭐⭐⭐⭐⭐ (5/5)
Overall⭐⭐⭐⭐ (4/5)⭐⭐⭐ (3/5)⭐⭐⭐ (3/5)

Final conclusion:

  • Huawei Ascend 950DT is the domestic AI training chip closest to H100, strongest overall
  • Cambricon MLU690 is the most efficient domestic AI chip, lowest TCO
  • Moore Threads MTT S5000 is the cheapest full-function GPU, suited to edge AI and graphics+AI

References


Disclaimer: Data based on public sources; actual specs per vendor official. MirrorFrog continuously updates domestic AI chip data — corrections welcome.

Changelog: 2026-06-23 initial release