The Script · HBM4 Splits the Rack · 2026-07-26

Deep Read: AMD's MI455X/Helios and NVIDIA Rubin ship HBM4 at rack scale — and the 2,048-bit interposer locks the build.

Opening Script

Two production HBM4 accelerators landed in one week. AMD's MI455X carries 432GB at 19.6 TB/s per GPU (vendor-stated); NVIDIA's Rubin GPU carries 384GB at 22 TB/s (vendor-stated). The mechanical change is the interposer: HBM4's 2,048-bit width doubles HBM3e's 1,024-bit. That physical difference means MI455X cards cannot migrate into NVIDIA HBM3e infrastructure, and vice versa. Rack-scale inference just got denser — and the substrate now dictates the vendor.

Architecture Watch

AMD moves MI400 to volume production. Per tech-insider.org (tier-2) coverage of Advancing AI 2026, July 23: MI455X ships 432GB HBM4 across 16 stacks at 19.6 TB/s — a 145% bandwidth jump over MI350's 288GB HBM3e at 8 TB/s. Helios racks 72 MI455X for 2.9 EFLOPS FP4 inference (vendor-stated). ROCm 7 is reported at 90–95% maturity. Self-hosting teams should demand independent MLPerf runs before committing.

NVIDIA Rubin enters full production; partner products H2 2026. Per zylos.ai (tier-2): 3.6 EFLOPS dense FP4 versus 1.1 for B300, 384GB HBM4 (8 stacks) at 22 TB/s — all vendor-stated. Vendor claims 10x inference token-cost reduction and 4x fewer GPUs for MoE training versus Blackwell. Rubin is in production; the 10x figure is unverified. Micron's 2026 HBM4 output is reported sold out.

Frontier token prices down ~94.5% since March 2023. Per BenchLM's Token Price Index (updated July 2, 2026), the frontier output index fell from 100 to 5.5; DeepSeek V3 sits at $1.10/1M output tokens. The counterweight: reasoning models bill internal chain-of-thought as output tokens, raising effective cost per useful output. Cheap tokens do not equal cheap answers for agentic loads.

OpenAI Frontier Evals drops SWE-bench Verified. Per buildmvpfast.com (tier-2), the team cited contamination making the 500-issue score partly a memorization measure. Opus 4.8 fell ~14% under restricted-internet conditions (per coverage), exposing retrieval-based inflation. LiveCodeBench is cited as a partial defense — it date-stamps problems for post-cutoff-only scoring. Engineers picking a coding scoreboard should require post-cutoff evaluation.

Intellia posts one-time in vivo CRISPR Phase 3 data. Per FierceBiotech (tier-1), HAELO data (June 13, 2026, published in NEJM) showed an 89% reduction in monthly attacks needing on-demand treatment (weeks 5–28) and a 91% reduction in moderate-to-severe attacks. A single IV LNP-delivered Cas9 knocks out KLKB1 in the liver. Status: pilot; rolling BLA underway, anticipated U.S. launch H1 2027.

The Deep Read

The system: HBM4-based rack-scale inference silicon, shipping this week from two vendors. AMD's Helios packs 72 MI455X accelerators with EPYC Venice CPUs; NVIDIA's Rubin platform pairs eight-stack HBM4 GPUs into partner systems. The mechanism that matters is memory bandwidth per accelerator. MI455X moves 19.6 TB/s across 432GB (vendor-stated); Rubin moves 22 TB/s across 384GB (vendor-stated).

Inference throughput on large models is bandwidth-bound, not FLOP-bound, once weights exceed on-die cache. HBM4's 2,048-bit interposer — double HBM3e's 1,024-bit — is what unlocks that bandwidth. It is also what fractures the ecosystem.

Who builds on it: Teams serving frontier-scale models at sustained high concurrency — workloads where memory bandwidth and per-rack density set the token cost. OpenAI and Meta's reported 12GW commitment to AMD fits that profile. Teams running sub-100B models, batch workloads, or intermittent inference do not need HBM4 density; HBM3e and prior-gen racks remain the rational floor. The interposer split makes the build decision a multi-year lock, not a per-node swap.

What it costs: No public per-rack capex figure exists for either Helios or Rubin partner systems. Treat vendor throughput as the only hard number and derive $/token from your own concurrency. The supply constraint is concrete: Micron's 2026 HBM4 output is reported sold out (per zylos.ai). On the demand side, BenchLM's index shows frontier output pricing down ~94.5% since March 2023, with DeepSeek V3 at $1.10/1M output tokens. Cheaper silicon and cheaper tokens push the same direction; the effective-cost caveat is reasoning-token inflation.

The complexity math: The levers are bandwidth per accelerator (19.6 vs 22 TB/s, vendor-stated), memory capacity per GPU (432 vs 384GB), and rack density (72 GPUs for Helios). For inference, capacity determines how much of a model — and how many concurrent KV caches — fit without cross-node traffic. Bandwidth determines token rate once resident. Precision matters: the 2.9 EFLOPS Helios figure is FP4 inference; the 1.4 EFLOPS figure is FP8 training. Do not compare FP4 inference numbers against FP8 training numbers. Software maturity is the hidden lever — ROCm 7 at 90–95% (per coverage) is not parity until proven on your kernels.

The failure modes: First, benchmark-to-production gap. Every throughput figure here is vendor-stated; no independent MLPerf-style number exists yet. The AI-systems items this week show the pattern directly — OpenAI dropped SWE-bench Verified for contamination, and Opus 4.8 lost ~14% under restricted internet (per coverage). Silicon claims deserve the same skepticism as model claims. Second, interposer lock-in: the 2,048-bit width makes cross-vendor migration a hardware impossibility, not a config change. Third, sold-out HBM4 can stall rollout regardless of order book.

The engineer's list:

• Demand independent MLPerf-style inference throughput for your model class and precision — reject FP4-vs-FP8 comparisons and vendor-only figures.
• Model the interposer lock as a multi-year commitment: price the switching cost of being unable to migrate cards across HBM generations.
• Test ROCm 7 (or the Rubin stack) on your actual kernels before assuming the 90–95% maturity figure covers your workload.

The Tool Box

LiveCodeBench — an evaluation harness for coding models. Its mechanic is date-stamping: every competitive-programming problem carries a release date, so evaluators can score a model only on problems published after its training cutoff. That directly counters the contamination that led OpenAI's Frontier Evals to drop SWE-bench Verified (per buildmvpfast.com). It is for ML engineers who need a coding scoreboard resistant to memorization inflation. Pricing model: not publicly listed. Limitation: post-cutoff-only scoring shrinks the usable problem set as a model ages, and competitive-programming tasks do not test production judgment or repository-scale context.

The Scoreboard

Independent technical analysis — not investment advice or engineering counsel. Production decisions warrant system-specific diligence.