# AMD MI455X and Helios: 432GB HBM4, 72-GPU Racks, and a Real Answer to Vera Rubin > Source: > Published: 2026-07-23 18:30:00+00:00 AMD hosted its largest Advancing AI event to date, emphasizing scale as a central theme. The company introduced the Instinct MI455X GPU, the 72-GPU Helios rack, and the 6th Generation EPYC Venice CPUs. The MI455X features 432GB of HBM4, a 50% increase over NVIDIA’s B300 or Rubin, with 23.3TB/s of memory bandwidth and up to 40.26 PFLOPS of MXFP4 compute. A complete Helios rack scales these figures by 72, achieving 2.9 exaFLOPS of FP4, 31TB of HBM4, 1.7PB/s of memory bandwidth, 260TB/s of scale-up, and 43TB/s of scale-out bandwidth to the datacenter. Across all metrics, AMD aims to lead the industry. This article examines the MI455X and Helios; the Venice launch is addressed in a separate analysis. The other theme is openness, reaching every layer, starting with the interconnect. Inside the rack, all 72 GPUs share memory over UALink, an open consortium fabric that AMD runs on Ethernet as UALink-over-Ethernet (UALoE). Once traffic leaves the rack, it moves on Ultra Ethernet, the open scale-out standard from the Ultra Ethernet Consortium. The same Instinct runs up the stack. The low-precision math uses OCP’s open MXFP4, MXFP6, and MXFP8 data formats, and the rack that houses it all is built to the Open Compute Project’s Open Rack Wide design. Even the software is developed in the open, with ROCm’s compiler, runtime, and libraries all available in source. Because every specification in the stack is published and downloadable today, a hyperscaler can treat Helios as a blueprint and build a bespoke version tuned to its own facilities and workloads, swapping networking, power delivery, or management to fit. This means everything we walk through in this write-up is the reference design AMD presented; the units customers deploy can differ significantly. The takers are already lined up: AMD says OpenAI, Meta, Anthropic, [Microsoft](https://www.storagereview.com/news/microsoft-will-ramp-amds-helios-rack-scale-ai-platform-at-scale-on-azure), Oracle, and more are adopting Helios. # AMD Instinct MI455X: the CDNA 5 flagship MI455X is the first CDNA 5 accelerator, comprising 320 billion transistors. The MI455X features eight Accelerator Complex Dies (XCDs) built on TSMC’s N2 node, plus two I/O Dies and two Fabric and Cache Dies on the N3 node, and twelve HBM4 stacks. It is the largest chip ever built on TSMC’s CoWoS-L packaging. Memory is one of the headlines. Those twelve HBM4 stacks total 432GB at 23.3TB/s, with HBM4 doubling the per-stack interface to 2,048 bits, and the two Fabric and Cache dies add a 192MB L2 running at 54TB/s. The MI455X does not hold back on I/O either. It carries 72 lanes of UALoE for 3.6TB/s of bidirectional scale-up bandwidth to the rest of a rack, 256GB/s of bidirectional Infinity Fabric to its host CPU, and a choice of two PCIe Gen6 x16 links or three AMD AI-NICs for scale-out. Lined up against the chip it replaces and NVIDIA’s offerings, the MI455X leads in every metric on the table. Specification | AMD MI455X | NVIDIA Rubin | AMD MI355X | NVIDIA B300 | |---|---|---|---|---| | Architecture | CDNA 5 | Rubin | CDNA 4 | Blackwell Ultra | | Transistors | 320B | 336B | 185B | 208B | | HBM capacity | 432GB HBM4 | 288GB HBM4 | 288GB HBM3E | 288GB HBM3E | | HBM bandwidth | 23.3TB/s | 22TB/s | 8TB/s | 8TB/s | | Scale-up per GPU | 3.6TB/s | 3.6TB/s | 1.08TB/s | 1.8TB/s | | Scale-out per GPU | 2,400 Gb/s | 1,600 Gb/s | 400 Gb/s | 800 Gb/s | | CPU-GPU link | 256GB/s Infinity Fabric | 1.8TB/s C2C (1:2) | PCIe 5 | 900GB/s C2C (1:2) | Let’s start with where AMD leads. At 432GB, the MI455X carries 50% more HBM than the MI355X, B300, or Rubin, all topping out at 288GB. Its 23.3TB/s of memory bandwidth is also the highest in the group. The scale-out row tilts the same way: 2,400 Gbit/s per GPU against 1,600 for Rubin and 800 for B300. Every MI455X leaves the rack with 50% more network bandwidth than its closest competitor. AMD has finally caught up in scale-up as well. NVLink has been the leading GPU fabric for years and, for two generations, was the only way to get top performance on MoE models with WideEP. UALoE closes that gap in a single generation: at 3.6TB/s, the MI455X matches Rubin’s NVLink 6. NVIDIA still holds a clear lead in the host link. One Vera CPU feeds two Rubin GPUs over 1.8TB/s of C2C, while each MI455X talks to its Venice host over a 256GB/s Infinity Fabric link. That difference shapes how the two rack architectures diverge later in this piece. On raw compute, the MI455X leads across the board, with one footnote: AMD’s OCP MX formats and NVIDIA’s NVFP4 scale differently, so treat these as advertised peaks; delivered performance is a separate question. Format | AMD MI455X | NVIDIA Rubin | AMD MI355X | NVIDIA B300 | |---|---|---|---|---| | MXFP4 / NVFP4 | 40.26 PF | 35 PF | 10.1 PF | 15 PF | | MXFP6 / FP6 | 20.13 PF | 17.5 PF | 10.1 PF | 5 PF | | MXFP8 / FP8 | 20.13 PF | 17.5 PF | 5 PF | 5 PF | | FP16 / BF16 | 5.03 PF | 4 PF | 2.5 PF | 2.5 PF | | FP32 | 315 TF | 130 TF | 157.3 TF | 75 TF | Relative to the MI355X, the MI455X delivers four times the MXFP4 and MXFP8 throughput and twice the FP16/BF16 and FP32 rates. The comparison against NVIDIA divides into two parts. Against the B300, the MI455X delivers 2.7× the FP4 throughput and four times the FP6 and FP8 rates. Rubin is the meaningful benchmark, and against it the MI455X holds a consistent advantage: 15% at FP4, 15% at FP6 and FP8, and 26% at FP16/BF16. The widest gap appears at FP32, where the MI455X’s 315 TF is roughly 2.4× Rubin’s 130 TF and more than four times B300’s 75 TF. That figure comes from Instinct’s HPC lineage and still matters for AI work, since master weights, high-precision accumulation, and scientific workloads continue to run above the low-bit formats. ## Inside CDNA 5 Let’s double-click into the architecture and see what actually powers this class-leading performance. ### From XCD to SIMD Walking the hierarchy from the package down shows how much was rebuilt because CDNA 4 organized the compute die very differently. On the MI355X, each XCD carried 32 active Compute Units and a private 4MB L2 cache that pooled the die’s traffic before it reached the Infinity Fabric. CDNA 5 keeps the eight XCDs but rebuilds what sits inside them, borrowing structure and terminology from AMD’s RDNA graphics line. Each MI455X XCD now splits into two Shader Engines. Each Shader Engine physically holds 17 Work Group Processors, with 16 enabled, one spare for yield. The per-XCD L2 is gone entirely, lifted off the compute die and down into the base dies below, which the memory section returns to. The arithmetic that matters is what did not change. An XCD still contributes 32 active units, and the GPU still totals 256, the same count the MI355X carried as Compute Units. None of the generational 4× in low-precision throughput comes from adding execution units; all of it comes from each WGP doing more work per cycle, and the WGP is where the redesign concentrates. A WGP is built from four 32-lane SIMD units and four scalar units sharing a constant cache. The biggest change is how threads flow through it: the move from Wave64 to Wave32. A wave is the bundle of threads a SIMD runs in lockstep. CDNA 4 used Wave64, pushing each 64-thread wave through a 16-lane SIMD over four clock cycles. CDNA 5 drops Wave64 support entirely, the first Instinct architecture to do so, and runs Wave32 natively. A 32-thread wave maps one-to-one onto each of the WGP’s four 32-lane SIMD units, issues in a single cycle, and lets every SIMD start a fresh instruction every clock. Narrower, faster waves change how work moves through the machine. Instruction latency drops because a wave finishes sooner. Branch divergence costs less since a taken-or-not split now stalls at most 32 threads instead of 64. Register pressure eases, so more waves stay resident, up to 64 per WGP against half that before. This gives the scheduler more small, independent pieces of work to hide memory latency behind. Wave32 also makes it easier to map different tile sizes for tensor operations onto the hardware, simplifying kernel development. Single-cycle issue is only the start of the throughput story. The SIMDs co-execute, starting new instructions while earlier multi-cycle operations drain underneath, and packed vector instructions carry 64 threads’ worth of work in a single issue, details AMD’s architects confirmed in the post-briefing Q&A. The vector pipeline also gains native BF16 support and a set of new data-conversion instructions for moving tensors between formats. The transcendental units double their throughput over the MI355X and add a native tanh instruction, so the softmax and activation math inside attention keeps pace with the tensor hardware around it. That path is becoming a habit: CDNA 4 doubled the transcendental rates to accelerate attention, and CDNA 5 doubles them again. ### The Memory Hierarchy Behind the execution units sits a hierarchy rebuilt from top to bottom, and the clearest way to see it is level by level against the MI355X. Level | MI455X (CDNA 5) | MI355X (CDNA 4) | |---|---|---| | Vector registers | 128KB per SIMD; 1,024 per thread; 2× bandwidth | 128KB per SIMD; 256 per thread | | WGP / CU local store | 384KB (320KB LDS + 64KB vector cache); 2× bandwidth | 192KB (160KB LDS + 32KB L1) | | Instruction / constant cache | 64KB + 16KB per WGP | 64KB shared per two CUs + 16KB | | L2 | 2 × 96MB on the FCDs; 54TB/s | 8 × 4MB, one per XCD | | Memory-side cache | Eliminated | 256MB Infinity Cache | | HBM | 432GB HBM4; 12 × 2,048-bit stacks; 23.3TB/s | 288GB HBM3E; 8 × 1,024-bit stacks; 8TB/s | The cache rows are where the architecture changed shape. CDNA 4 ran a three-level design: each XCD’s private 4MB L2 coalesced that die’s traffic before it reached the Infinity Fabric, and a shared 256MB Infinity Cache in the I/O dies sat on the memory side in front of the HBM controllers. CDNA 5 deletes both layers and replaces them with two independent 96MB L2 caches, one per Fabric and Cache Die, each built as 96 one-megabyte blocks. The layout is vertical: four XCDs, or eight Shader Engines, are hybrid-bonded on top of each FCD, which also holds six of the twelve HBM4 sites, and the two FCDs meet at a central Infinity Fabric down the middle of the package with the I/O dies capping either end. Either L2 can hold any address in the GPU’s memory, and Infinity Fabric keeps the pair coherent. AMD’s stated reason is bandwidth: one of these caches alone delivers 1.5 times the aggregate bandwidth of the MI355X’s entire Infinity Cache, the pair delivers three times, and none of that traffic has to cross the die-to-die bisection that capped the old layout. The cache picks up new duties as well. Device-scope atomics, which previously executed out in the fabric, now run inside the L2 at far higher rates, while system-scope atomics stay in the Infinity Fabric as before. A new broadcast arbiter rounds it out, multicasting tensor tiles to every WGP cooperating on the same matrix. Therefore, a weight fetched once serves all of them, which amplifies effective read bandwidth by up to 4×. The levels above scale to match. Per-WGP local storage doubles to 384KB, split as 320KB of LDS and a 64KB vector data cache, with twice the read bandwidth. That is room for FlashAttention to hold queries, keys, values, and partial reductions on chip instead of writing out the full attention matrix. It also supports fused MoE kernels to keep routing state and accumulators resident. The vector register file keeps its 128KB-per-SIMD capacity but is reorganized for Wave32. This results in twice as many waves, allows a single thread to address 1,024 registers instead of 256, and doubles register bandwidth to feed the wider SIMDs and their co-execution units. The scalar side is rebuilt to match, at 128 scalar registers per wave and 32KB per WGP. At the base, HBM4 moves from eight 1,024-bit stacks to twelve 2,048-bit stacks, lifting capacity 50% to 432GB and bandwidth 2.9× to 23.3TB/s across a 192-channel interface. Feeding all of that is a new Tensor Data Mover, one per WGP, which understands tensor tiling schemes up to five dimensions and streams tiles asynchronously between DRAM and the local store with no intermediate register staging. Transfers are described by descriptors loaded from the scalar registers and bounds-checked in hardware for security. Multicast loads are supported, so the SIMD units never stall waiting on a copy or burn registers staging one. It is CDNA 5’s answer to the tensor-memory accelerators on recent NVIDIA parts. A set of utilization features rounds out the front of the machine: workgroup clusters give kernels explicit control over placement and concurrency for data-sharing workloads, split and named barriers let a producer signal completion and move on without waiting for the consumer to answer, prefetchers at each level of the hierarchy stage data toward its point of consumption, and a reworked command front end cuts kernel launch and dispatch latency for the short kernels that dominate inference. The DMA system was rebuilt on the same philosophy. Software schedules transfers against DMA front ends, while physically aware back ends sitting beside the UALoE links split each work item, load-balance it across every available link, and pull buffers from memory straight out the door instead of hauling data across the chip to a distant engine. The back ends also react to congestion back-pressure from the scale-up network and steer around loaded paths, so communication libraries get well-balanced fabric traffic without ever understanding the topology underneath. ### Slicing the GPU: NPS and SR-IOV The two-L2 physical layout pays a second dividend in how the GPU partitions. In NPS1, the whole chip is one NUMA domain: addresses interleave across all twelve HBM stacks and both halves for uniform bandwidth, the easy mode for porting and for evenly spread access patterns. NPS2 splits the GPU into two NUMA domains, each owning six HBM stacks, one Fabric and Cache Die, and the XCDs stacked on it. Every memory reference then stays inside its own half, and each domain effectively gets a private 96MB L2. That does more than shorten the physical path. With no cache lines shared between the halves, the Infinity Fabric coherency traffic between the two L2s largely disappears, and AMD says the result is lower latency and better efficiency for NUMA-aware applications. CDNA 4 offered the same broad trade, with NPS2 keeping traffic inside one I/O die, but CDNA 5 sharpens it because the thing being localized is now the full L2 cache rather than a slice of a memory-side buffer. Compute partitioning stacks on top. The eight XCDs let the GPU boot as one, two, four, or eight spatial partitions, dividing the 432GB of HBM into even slices of 432, 216, 108, or 54GB backed by eight down to one XCD each. Pairing partitions with the NUMA domains lets the runtime dispatch work and place allocations spatially, so a job lands on the XCDs closest to its memory. SR-IOV then virtualizes the partitions into as many as eight hardware-isolated virtual machines, with the isolation enforced in the memory system itself, independent of which NUMA mode is running. The MI355X offered the same one-through-eight partition options, so the granularity is not new; what CDNA 5 adds underneath is the private-L2 behavior, and above it the rack-level Virtual Pods that the Helios section covers. # AMD Helios A single MI455X is fast. But with this launch, AMD joins the rack-scale and large-scale-up domain club. Physically, Helios drops the traditional 19-inch and 21-inch racks for Open Rack Wide, a format AMD helped develop with Meta at OCP: a cabinet 1.2 meters wide and 1.3 meters deep with 44 OU of vertical space. Inside, the 72 GPUs sit in two banks of nine compute trays with the six switch trays stacked between them, and every GPU-to-switch link is copper run through four blind-mate cable cartridges at the rear, so trays slide out for service with no cables to unplug by hand. The whole rack draws 225 to 245kW depending on workload, delivered over a 50V liquid-cooled bus bar, with rear manifolds pushing roughly 385 liters of coolant per minute from the facility loop. The trays themselves are serious hardware: each weighs around 170 pounds, and seating a switch tray’s 1,728 differential-pair connections takes about 690 pounds of insertion force, which is why its cam handles run nearly the full width of the tray. ## The building blocks ### Compute Tray In the reference design, each compute tray is a self-contained node built around 4x MI455X modules and a high-frequency 96-core Venice SP7 CPU that boosts to 5GHz. Its 16 DIMM sockets carry 1TB of DRAM as 16 × 64GB DDR5 ECC RDIMMs, with 5 E1.S NVMe slots hanging off the CPU. The platform is rated for far more: Venice’s 16 memory channels support up to 1.6TB/s of bandwidth, and with the 256GB RDIMMs at the top of the DDR5 range today, a 16-channel socket at 1 DIMM per channel tops out at 4TB. Following in NVIDIA’s footsteps, the CPU joins the coherent memory domain over Infinity Fabric instead of sitting behind the GPUs as a plain PCIe host, and AMD argues the 1:4 CPU-to-GPU ratio is deliberate: the core itself outruns the competition, with AMD’s apples-to-apples estimates putting the 5GHz Zen 6 core about 20% ahead of NVIDIA’s Vera in per-core performance, and because the socket is a standard SP7, customers who want more host compute can fit any Venice SKU up to the 256-core flagship. One Venice socket also carries far more DDR5 capacity than an LPDDR host design, and its memory bandwidth saturates out to all 4 GPUs across the Infinity Fabric links. That Infinity Fabric link is worth a closer look. Talking through the Venice-to-MI455X connection with George Cozma of [Chips and Cheese](https://chipsandcheese.com), he suggested the coherent link rides on the CPU’s PCIe lanes, the way EPYC has carried its xGMI socket links over PCIe PHYs for years. The numbers back that theory. PCIe Gen 6 signals at 64 Gb/s per lane, and an x16 link at that rate works out to 128GB/s each way, exactly the 256GB/s bidirectional figure AMD quotes per GPU. The CDNA 5 whitepaper’s own block diagram labels the host Infinity Fabric interface at 64 Gb/s per lane, the exact Gen 6 signaling rate. The theory also explains why any Venice SKU drops in: the 4 GPUs consume 64 of the CPU’s 128 Gen 6 lanes, leaving the rest free for DPUs, storage, and other system needs. Three separate networks pass through each compute tray, and each exists for a different job. The most conventional is the front end: a single Pensando Salina 400G DPU connects the node to the regular datacenter network, which we will explore in more detail later. The second is scale-out, the network that joins racks into clusters, and the cleanest way to understand it is to countSerDess. The MI455X’s scale-out can use either PCIe Gen 6 at 64 Gb/s per lane or UALink128 at 128 Gb/s, and a Vulcano 800 NIC needs roughly 128 Gb/s of attachment each way to keep its 800 GbE port fed. At Gen 6 rates, that takes a full x16 link per NIC, so the GPU carries 2 NICs; at UALink128’s doubled signaling rate, an x8 link does the same job on half the SerDes, so the GPU carries 3, which is the configuration Helios ships. Either way, the UALink128 hop is nothing more than a private wire between GPU and NIC; the network itself begins at the Vulcano. Each NIC drives an 800GbE port running UEC-compliant transports, including MRC, the multipath protocol OpenAI developed with AMD and other partners. Physically, the NICs sit on 2 custom boards per tray carrying 4 or 6 Vulcano ASICs each, matching the 2-per-GPU and 3-per-GPU configurations. In the full fit, that is 12 NICs per tray and 2,400 Gb/s of scale-out bandwidth per GPU. And because the NICs are attached to the GPUs, with the CPU nowhere in the path, rack-to-rack traffic never touches the host link. The third is scale-up, the fabric that makes Helios a true rack-scale system. Each GPU carries 36 UALoE links that run UALink’s memory semantics over ESUN Ethernet, each link good for 400 Gb/s, adding up to 3.6TB/s of bidirectional bandwidth per GPU. Those links exit the rear of the tray toward the switch trays, carrying the load-store traffic that fuses the 72 GPUs into one shared-memory pod. ### Switch Tray Next up, the switch trays, and the most striking thing about them is how ordinary their silicon is. Each of the 6 trays holds 2 Broadcom Tomahawk 6 ASICs, the same merchant Ethernet switch chips hyperscalers deploy in their leaf-spine networks, each carrying 512 lanes of 200G. Every GPU sends 3 UALoE links (each UALoE link is 2x 200G lanes) to every one of the 12 switches, with 144 links leaving each compute tray through the rear cable cartridges. Each Tomahawk therefore terminates 216 links at 400 Gb/s, moving 21.6TB/s of bidirectional bandwidth, while every GPU keeps its full 36 links (72x 200G lanes) and 3.6TB/s. The switches need nothing exotic to pull this off: UALoE’s encapsulation is a plain L2 protocol, forwarding relies on static MAC programming that Ethernet silicon has offered for two decades, and flow control is standard priority flow control. With a single tier, whole classes of datacenter congestion problems never arise: there is no multi-tier incast, and every GPU sits exactly one fixed-latency hop from every other. Compared to a direct mesh, the switched approach also lets a single flow claim an entire path’s bandwidth when a workload needs it, and keeps every GPU at equal distance. Hence, scheduling never has to think about locality, and gives every link the same fault protection. ## Fault tolerance Helios treats hardware failure as a design input. At this scale, something is always breaking: a flaky cable, a dropped packet, a switch pulled for a firmware update, a compute tray that dies outright. The fabric is built so that none of those events kills a job. Dropped packets are recovered by retransmission, and when a link, cable, or switch fails, traffic reroutes around it automatically after a brief pause, with the workload continuing on the bandwidth that remains instead of restarting from a checkpoint. The 12-plane topology is what makes the degradation graceful, and the 3-way striping sets the step size. Lose 1 of the 3 links a GPU runs to a switch, and that plane keeps two-thirds of its bandwidth. Lose an entire Tomahawk, and every GPU gives up 1/12 of its scale-up bandwidth while the all-to-all keeps working across the other 11 planes. Even losing a whole switch tray, 2 of the 12 switches, costs each GPU a sixth of its bandwidth without breaking connectivity, because no GPU depends on any single switch to reach another. For comparison, Vera Rubin NVL72 spreads each GPU across 36 NVSwitch 6 ASICs in 9 trays, so a switch-tray failure there costs closer to a ninth. NVIDIA buys smaller degradation steps with 3× as many switch ASICs; AMD counters that 12 higher-radix switches mean fewer components, cables, and connectors to fail in the first place. For a training run measured in weeks, the difference between losing a sixth of fabric bandwidth and losing the job is the entire economics of the rack. ## Virtual Pods The same machinery that partitions the fabric around failures can partition it on purpose. AMD calls the construct Virtual Pods, or vPods, and the unit is the compute node: any combination of the rack’s 18 4-GPU nodes can be fenced into an isolated pod, from 1 node for a small tenant to most of the rack for a large training job. The isolation is enforced down in the fabric hardware, below anything a scheduler decides. A vPod is tied to its tenant; other pods have no access to its memory or its traffic, and the line-rate AES-256-GCM encryption on every UALoE link, with support for customer-owned cluster keys, keeps one tenant’s tensors opaque to the next. A guest VM that spans several GPUs has its security domain extended transparently across them, with no requirement to trust the host OS. NVIDIA solves the same problem on its NVL72 racks by splitting the NVLink domain into partitions, with its IMEX service brokering which nodes may export and import memory to one another; vPods are the UALoE world’s equivalent, so operators coming from GB200 or GB300 fleets will find the concept familiar. If a compute tray crashes, the blast radius stops at its vPod: that workload restarts from checkpoint while every other pod runs on untouched, the tenant boundary doubling as a failure boundary. The partitioning story also nests all the way down, since a single MI455X can split into as many as 8 SR-IOV virtual machines so that the same rack can serve 1 customer running all 72 GPUs as one pod or as many as 576 GPU-slice tenants at the extreme, with hardware isolation at every level of that hierarchy. ## The management plane Running all of this is a dedicated software stack that follows the same openness thesis as the hardware. AMD Fabric Manager (AFM) is the control plane: it discovers and provisions the 72-GPU fabric with zero-touch bring-up, so powering the rack on is enough for all 72 GPUs to come up, then validates the cable-cartridge wiring against assembly mistakes, carves the rack into vPods, and coordinates the rerouting and recovery described above. There is no dedicated management tray. AFM runs on the switch trays’ own management processors as 3 redundant instances spread across the 6 trays with a distributed database between them, so losing a switch tray does nothing to the control plane, and a northbound REST API exposes the fabric to cluster controllers managing many racks. Under the hood, AFM borrows its plumbing from the cloud-native world, built on standard Kubernetes-style controllers with agents on each tray, and it handles the fabric details users never want to see, down to assigning the accelerator IDs that UALink uses to address each GPU. It is also the rack’s observability layer. A single dashboard tracks GPU and fabric utilization, link health, and failure events; when something breaks, it shows the remediation in progress and raises alerts that operators can wire into their own tooling. The screenshot above is AFM watching a Helios cluster in AMD’s own labs. Management works in-band or out-of-band, so diagnostics and configuration never disturb running workloads. The switches beneath AFM run a network OS built on SONiC, the open-source NOS, and AMD says its UALoE additions will be upstreamed and exposed through standard gNMI APIs. Above the rack, a Rack Infrastructure Manager covers node and switch lifecycle, power, and leak detection, and a Cluster Controller plugs Helios into Kubernetes and Slurm for scheduling. ## Helios vs. NVIDIA Vera Rubin NVL72 So let’s look at how this compares to the NVIDIA offering Helios will actually meet in the market: the Vera Rubin NVL72. Rack metric | AMD Helios | Vera Rubin NVL72 | |---|---|---| | GPUs | 72 MI455X | 72 Rubin | | CPUs | 18 Venice | 36 Vera | | HBM capacity | 31TB | 20.7TB | | HBM bandwidth | 1.7PB/s | 1.58PB/s | | Scale-up per GPU | 3.6TB/s | 3.6TB/s | | Rack scale-up | 260TB/s | 260TB/s | | Scale-out per GPU | 2,400 Gb/s | 1,600 Gb/s | | Scale-up switches | 12 Tomahawk 6 | 36 NVSwitch 6 | | Rack format | Double-wide ORW | Single-wide MGX | On paper, the scorecard tilts AMD’s way: 50% more HBM, the same 3.6TB/s of scale-up per GPU from a third as many switch ASICs, and 50% more scale-out bandwidth per GPU. AMD’s internal tests turn those specs into a performance claim, achieving 10 to 15% more tokens per second per GPU on Kimi K2 Thinking and up to 30% more tokens per dollar. Those are AMD’s numbers against NVIDIA’s published numbers, not independent measurements, but they set the bar AMD expects to be judged by. The more interesting differences hide in how each design connects its GPUs to the outside world. Start with scale-out. The MI455X’s NICs hang directly off the GPU. [According to SemiAnalysis,](https://newsletter.semianalysis.com/p/vera-rubin-extreme-co-design-an-evolution) Rubin’s do not: per SemiAnalysis, the package lacks the PCIe to feed both ConnectX-9 NICs, so they hang off the Vera CPU instead, and GPU traffic takes the long way around: Rubin to NVLink-C2C to Vera to PCIe to ConnectX-9. The detour costs a hop of latency and puts the C2C link on double duty. With compute, host traffic, and network all pegged at once, part of Vera’s C2C bandwidth goes to carrying NIC payload, and the effective host bandwidth a GPU sees drops below the headline 1.8TB/s. The bandwidth math compounds it. Each MI455X pushes 2,400 Gbit/s of scale-out to Rubin’s 1,600, so Helios carries more network per FLOP. AMD’s simulations of an 8,000-GPU training run credit the third NIC with about 13% faster job completion. Rubin punches back on storage, and the reason is again where the NIC sits. ConnectX-9 has a built-in PCIe switch, so NVMe can hang directly off the NIC and a GPU can pull data over GPUDirect Storage without touching the CPU. The MI455X has no equivalent: its storage hangs off the Venice host, so anything GPUDirect-shaped must cross the CPU and come back over the Infinity Fabric link. AMD optimized the network path and paid for it on the storage path; NVIDIA made the opposite trade. Which matters more depends on whether a workload is bound to moving activations between GPUs or streaming data off disk. ## What customers can change In short, everything above describes AMD’s reference design, and several of the numbers are floors customers can build past. The most obvious case is the host CPU. Rubin’s Vera arrives in one fixed configuration; the Venice in a Helios tray is a standard socketed SP7 part, and AMD confirmed any Venice SKU drops in with no Helios-specific customization. The reference tray uses the 96-core 5GHz part because single-threaded speed keeps GPUs fed. Still, nothing stops a customer from configuring their version with the 256-core flagship, or Venice-X with its 1,152MB of stacked L3 for cache-hungry preprocessing. Memory and networking follow the same socket-and-slot logic. The reference 1TB of DRAM is 16 modest 64GB RDIMMs; denser DIMMs take a tray to 4TB, and MRDIMM-12800 unlocks Venice’s full 1.6TB/s. On the network side, a build can drop from 3 NICs per GPU to 2 over plain PCIe Gen 6; each Vulcano port can run as 1x800G, 2x400G, 4x200G, or 8x100G against Tomahawk 5 or Tomahawk 6 fabrics, and the P4 pipeline leaves the transport, RoCEv2, MRC, or something proprietary, as the operator’s call. Even the management plane is swappable, since the switch NOS is open-source SONiC and AFM exposes the whole fabric through its northbound API. The power budget follows the socket too. NVIDIA’s superchips share one envelope: Vera is a 450W part with a capped slice, and recent generations slosh power toward the GPUs under load. AMD has not said whether the reference design caps or shifts host power, but with AMD’s design the question belongs to the customer, and they can customize the system with higher power usage with no power sloshing. The host link’s PCIe underpinnings, unpacked back in the compute tray section, open one last door, this one openly speculative. Venice supports 2P configurations, and AMD notes that select AI host platforms run 2P with up to 160 usable PCIe lanes by trading inter-socket xGMI width for I/O. A customer could conceivably build a two-socket tray to match NVIDIA’s 1:2 CPU-to-GPU ratio, or retune the xGMI links to raise effective CPU-to-GPU bandwidth. Nothing suggests anyone is building that today, and none of it closes the raw gap to NVLink-C2C at 1.8TB/s. The real point is who holds the pen: on Helios the host, its memory, its power, and potentially its topology are the customer’s decisions, and NVIDIA’s superchip hands the customer no pen at all. ## The Salina DPU Now back to the front-end network we deferred earlier. Salina, AMD’s 3rd-generation Pensando DPU, is a 400G card with a fully P4-programmable data path, which means a new encapsulation, telemetry hook, or transport is a firmware update, applied live without dropping traffic. The shipping services already cover the front-end checklist: SDN with VXLAN or NVGRE, a stateful firewall scaling to millions of rules, line-rate IPsec, PSP, DTLS, or custom encryption, NAT, and load balancing. It is also the most battle-tested silicon in the rack. Pensando DPUs have run hyperscalers since 2019; Salina fronts deployments at Microsoft, Oracle, and IBM today; Oracle credits the line with a 5× SDN gain, and one hyperscaler reclaimed 22 CPU cores per server by offloading I/O to it. Storage is the second act. Salina exposes NVMe-over-Fabrics devices to the host, virtualizing remote SSD pools over TCP or RDMA with encryption, digests, and compression done on the card. On Helios, it adds an agentic-era trick: a context-memory engine presents an emulated KV device, so overflowing KV cache spills to CPU DRAM, local SSD, or remote storage and streams back into HBM at line rate instead of being recomputed. As noted in the Rubin comparison, the MI455X lacks GPUDirect Storage; this KV offload is AMD’s partial answer for the traffic serving cares about most. It is also where our reservations sit. The bandwidth gap is plain: Salina is a 400G card, and the BlueField-4 shipping into Vera Rubin racks doubles that to 800G with a 64-core Grace CPU and a co-packaged ConnectX-9. The software gap is more debatable but real. NVIDIA’s DOCA hands developers containerized, prebuilt services programmable in ordinary C and C++; P4 is a specialized dataplane language most teams have never touched. The comparison is not “DOCA’s catalog versus bare P4,” since Salina ships its major services complete, and the hyperscalers that deploy it chose it partly because P4 lets new protocols like MRC land in firmware ahead of anyone’s silicon cycle. The real distinction is who the programmability serves. Salina’s flexibility is a weapon for AMD and P4-fluent hyperscale teams; DOCA is a toolkit an ordinary enterprise developer can pick up. For the broad market, NVIDIA’s software on-ramp is easier, and AMD knows it. ## ROCm.AI Speaking of software, AMD saved one of its bigger announcements for the stack itself. ROCm.AI, arriving in August, is AMD’s attempt to make the GPU platform agentic from the ground up. AI Skills plug ROCm into the coding agents developers already use, Claude, Codex, Cursor, and Gemini, so installing, serving, and debugging on Instinct happens in plain English. Hyperloom is the bolder piece: a no-human-in-the-loop optimizer that profiles a workload, tunes its serving configuration, rewrites GPU kernels, and validates the results while the operator sleeps. AMD says it is continuously optimizing some 14,000 models today, and a live demo squeezed 38% more throughput out of MiniMax M3. Beneath the agents, FlyDSL brings near-assembly control to Python, ROCm moves to a fixed 6-week release cadence, and AMD claims ROCm.AI delivers an average 3.3× inference and 2.4× training gain over ROCm 7 on identical hardware. ROCm 7 already marked real improvement; now AMD is betting on AI to accelerate the pace. Arguably the most important slide of the software session was about hardware. AMD was emphatic that every number on it was measured, the subtext being that MI455X silicon is up, running, and fast under ROCm today. The figures: 20TB/s in FP8 MLA decode, 20 PFLOPS of FP4 compute, 3.2TB/s of scale-up bandwidth, and 190GB/s of scale-out. In the Q&A, AMD acknowledged the FP4 result is a max-achievable-matmul-FLOPS (MAMF) measurement, run at the matrix shape that flatters the device most, which is standard practice for this class of benchmark. It is also a ballsy disclosure: AMD is openly admitting the MI455X sustains about 50% of its 40.26 PFLOPS peak MXFP4 rating, a number most vendors would bury. AMD calls this the highest demonstrated compute of any accelerator on the market, and that is where the grain of salt comes in. AMD’s FP4 is OCP MXFP4; NVIDIA’s is NVFP4. They are different recipes: NVFP4 applies a fractional FP8 scale to every 16-element block plus a tensor-level scale on top, while baseline MXFP4 uses a coarser power-of-two scale per 32 elements, so an NVFP4 FLOP carries more work than an MXFP4 FLOP. CDNA 5 can apply fractional scaling to MXFP4 too, but AMD did not say which recipe the measurement used. A Rubin MAMF run and an MI455X MAMF run are not measuring the same math, so cross-vendor FP4 comparisons only settle at the application level: tokens per second at matched accuracy. Measured beats projected, but these figures read most honestly against AMD’s own previous generation, where the 3× to 4× gains are unambiguous. There is a counterweight in AMD’s favor too. These are early ROCm.AI results on brand-new silicon, so if anything they understate what a hand-tuned production deployment will reach. The real verdict will arrive when these racks hit hyperscaler floors. # Closing Thoughts Helios is the most complete system AMD has ever shipped, and the first that meets NVIDIA head-on at rack scale instead of chip by chip. The scorecard reads AMD’s way in the places that decide AI capacity today: 50% more HBM per GPU, scale-up parity with Rubin, 50% more scale-out bandwidth, and, by AMD’s own modeling, up to 30% more tokens per dollar. Just as important is how it got there: merchant Tomahawk switches, open standards from the number formats to the cabinet, and a socketed host that leaves the final configuration in the customer’s hands. NVIDIA keeps genuine advantages in the C2C host link, the DPU, and its software on-ramp, but for the first time, the overall hardware argument on paper favors AMD. And the buyers agree. OpenAI, Meta, Anthropic, Microsoft, and Oracle are among the companies AMD says are adopting Helios, and AMD highlights that the racks are in production today. Following in NVIDIA’s footsteps, the roadmap is now an annual cadence: the CDNA 6-based MI500 series arrives in 2027 with next-generation HBM plus copper and optical interconnect, and the MI600 series is already in development for 2028. Which leaves software, and for the first time in years, we are not ending an AMD GPU story on that caveat. ROCm 7 closed real gaps, ROCm.AI arrives in August with measured gains on top, and the release cadence is now a fixed six weeks. It also matters who is buying. The labs and hyperscalers signing these deals co-design with AMD and employ enough engineers to fix whatever issues they hit. Enterprises that need a turnkey stack are a different story, and that market stays NVIDIA’s for now. But Helios was built for the hyperscalers and AI labs, and for them, the hardware is ready, the software keeps pace, and the racks are shipping. AMD has never been in a stronger position.