AMD Helios MI400 System Architecture at Hot Chips 2026

AMD detailed the system architecture of its MI400 series Instinct accelerators at Hot Chips 2026, centering on the Helios rack-scale platform built from the 96-core EPYC Venice CPU, Instinct MI455X GPU, and Pensando Vulcano AI NIC. Each Helios rack holds 72 GPUs, 31 TB of HBM4, and delivers 2.9 exaflops of AI compute, with 1.7 PB/s of HBM4 bandwidth, 260 TB/s of scale-up bandwidth, and 43 TB/s of scale-out bandwidth. The design uses a switched scale-up fabric with UALoE for shared memory across the pod, aiming to co-design AI infrastructure across CPUs, GPUs, networking, and software.

AMD is on stage again at Hot Chips 2026, detailing the system architecture behind its MI400 series Instinct accelerators. This is really a rack-scale story, and the talk centers on the Helios platform built from the EPYC Venice CPU, the Instinct MI455X GPU, and the Pensando Vulcano AI NIC. We are covering this live, so please excuse any typos. AMD Helios MI400 System Architecture at Hot Chips 2026 AMD opens by framing itself as a full-system company spanning CPUs, GPUs, software, networking, FPGAs and SoCs, and finished systems. That framing matters because the rest of the talk argues that AI infrastructure now has to be co-designed across those pieces instead of pushed together as separate parts. Now AMD lays out the three co-designed silicon blocks behind rack-scale AI. There is a 96 core AMD EPYC Venice, an Instinct MI455X and a Pensando Vulcano AI NIC. For a closer look at the Instinct silicon and Helios, check out Ryan’s awesome AMD Instinct MI455X Deep Dive, CDNA 5 Marks The Next Era of Instinct https://www.servethehome.com/amd-instinct-mi455x-deep-dive-cdna-5-marks-the-next-era-of-instinct/ goes through the architecture in more detail. Next comes the scale-up pod, built as a switched topology for maximum communication flexibility. Each MI455X gets 1.8 TB/s/dir across 72 IFoE links at 200G, while 800G NICs hang scale-out Ethernet off the same pod with up to 2.4 Tb/s/dir per GPU. AMD notes the scale-up hop is currently electrical and approaching an optical transition. They said something just now on stage that the already announced next-gen will start to transition to optical as well. Each Helios rack holds 72 GPUs, 31 TB of HBM4, and 2.9 exaflops of claimed AI compute, with 1.7 PB/s of HBM4 bandwidth, 260 TB/s of scale-up bandwidth, and 43 TB/s of scale-out bandwidth. This is the scale AMD is now aiming system design at, a far cry from the single-accelerator era. AMD then shows how the Helios rack is physically constructed. A 44OU ORW-HPR chassis holds 18 compute trays and six switch trays in the 72-GPU configuration, with blind-mate QD liquid cooling, redundant power supplies, and a 50V DC LC busbar. We have seen Helios racks in a couple of forms before, including variants shown at the OCP Summit https://www.servethehome.com/amd-helios-mi450-rack-at-ocp-summit-with-a-different-version-from-meta/ . Inside each compute tray, AMD packs four Instinct MI455X modules with a single Venice SP7 CPU host. Each GPU node gets 1.8 TB/s/dir of UALoE scale-up bandwidth across 12 3×2 links, coherent Infinity Fabric at 128 GB/s/dir to the CPU, and up to three Vulcano 800 AI NICs per EAM via UALink. At the start of the article, there is an example of the two Vulcano 800 AI NICs per GPU, which is an option. Next comes the switch tray, home of the scale-up fabric. Two 512-port 200G UALoE switch ASICs deliver 10.8 TB/s/dir and 72 active links per switch, arranged in a multi-plane architecture for all-to-all GPU connectivity. A Ryzen-based Run BMC handles management and power for this roughly 7kW liquid-cooled tray. Pulling back, AMD shows the full 72-GPU Helios pod, 12 switches and 18 compute trays wired into that switched fabric. This topology provides every GPU with shared load/store access across the entire pod rather than within a local rack segment. AMD walks through its software stack and UALoE here. This programming model sits on a shared memory fabric, managed by the AMD Instinct Fabric Manager, with the UALoE transport running over Ethernet and ESUN. Now AMD is showing the distributed shared memory model at the heart of UALoE. Importer and exporter applications map buffers in GPU HBM through mmap, so remote and local nodes exchange data over the shared memory fabric with DMA doing the actual movement. UALoE extends that load/store view across the entire Helios rack. Topology-aware DMA engines offload data movement from the WGPs and run in parallel with compute, using a frontend-backend architecture that dynamically distributes load and keeps backends adjacent to the UALoE links. AMD stresses that UALoE is built on open Ethernet and ESUN standards, and on often Broadcom switches, rather than on a closed fabric. That open-system claim is a deliberate contrast to proprietary scale-up interconnects from rivals. This figure digs into the transport itself. Each MI455X integrates 18x 800 Gbps UALoE adapters, and the fabric uses a lightweight, reliable protocol with dynamic packet packing to trade off bandwidth and latency. Packet loss is recovered through link-layer replay or end-to-end retransmission, and a failed link or switch can fall back to an alternate network plane before being brought back online. AMD is now laying out rack-scale confidential computing across EPYC, the MI455X, and the AI NIC, with secure boot, a DICE-based root of trust, universal link encryption with fast key rotation, and SEV-SNP external memory encryption. A footnote cautions that a malicious hypervisor can undermine some integrity guarantees on the current MI455X, though data and model confidentiality still hold. Virtual pods give operators flexible partitioning. A subset of nodes can be combined into a VPod. Pods are isolated from one another, and node-level failures remain contained to the affected VPod instead of taking down the whole rack. AMD then runs through scale-up fault resilience scenarios. Soft errors are caught by the reliable link layer in hardware, and the figures walk through single-link, single-switch, and whole-switch-tray failures. What about with a whole switch tray loss? DMA rebalances over the five remaining trays of six, and WGP references rebalance under software control, keeping the fabric alive even as a full tray disappears. Above that hardware, AMD Fabric Manager AFM runs as a three-node cluster on the scale-up switches, giving a coherent control plane with auto-discovery, RAS coordination, and policy. This cluster survives a single failure, drops to read-only on a double failure, and manages the scale-up network and compute from a single pane of glass without depending on the host stack. Next, AMD is talking about AI NIC side with the Pensando Vulcano 800. This programmable NIC runs SDN, security, RDMA in several flavors, and NVMeoF, and it can be deployed as a front-end Ethernet NIC, a scale-out RDMA NIC, or a storage initiator and target. Here is the Vulcano 800 block diagram. It is a P4-based design with 192 MPUs, P4DMA engines for RDMA and storage, PCIe Gen6 x16 and UAL128 host interfaces, and a single 800G network port, all tuned for low power and latency per unit of throughput. I asked Soni at AAI, and AMD thinks P4 is the way to go over a sea of Arm cores for the programmable part. AMD positions the Vulcano P4DMA engines as programmable transport engines. Across the evolution from legacy RoCEv2 toward MRC’s source routing, multipathing, and selective ACKs, the NIC carries custom protocol logic for collectives and adaptable telemetry. Congestion control gets the same programmability treatment. That P4 engine can carry and interpret custom CC signals and maintain state for newer schemes like MRC and UEC’s framework, while still supporting classic DCQCN. Vulcano 800 also pushes telemetry. A programmable P4 pipeline transfers statistics from the AI NIC to host memory at high frequency, with multi-level coverage spanning transport, congestion control, and filtering to select which stats to watch. Here AMD shows Vulcano MRC https://www.servethehome.com/nvidia-spectrum-x-ethernet-mrc-is-the-custom-rdma-transport-protocol-for-gigascale-ai/ performance across lane configurations. An 800G port can present as 1x800G, 2x400G, 4x200G, or 8x100G for message sizes of 64 KB or larger. This figure compares MRC against RoCEv2 in an all-reduce workload. MRC offers a throughput advantage with a single queue pair compared to RoCEv2’s escalating QP counts, and AMD credits packet spray and selective acknowledgment drop convergence at the edge for larger message sizes. Summary slide time. Helios runs 72 GPUs as a single unified compute engine, with shared load/store access to more than 30 TB of HBM4 at 1.8 TB/s/dir per GPU, flexible partitioning, and the Pensando NICs handling the scale-out side with protocol flexibility, load balancing, and telemetry. That wraps the MI400 system architecture walkthrough. Final Words It is quite remarkable that AMD has gone from “gluing” four 8-core CCDs together in the 2017 AMD EPYC to building rack-scale infrastructure in the span of 9 years. NVIDIA is still going to sell a lot of Vera Rubin NVL72 racks, but it is always good to have a second GPU vendor operating at this scale.