Breaking the KV cache memory wall using Dynamo and Liqid CXL shared memory A Micron team led by Dr. Skyler Windh built cross-host KV cache sharing into Nvidia Dynamo and ran it on the Great Abaco cluster, a CXL 2.0 shared-memory system designed with Liqid for the US Department of Energy's Pacific Northwest National Laboratory, reporting that a 32 GB CXL cache per server delivered 2.8× the throughput and 2.9× faster time to first token than DRAM of the same size on Llama3-70B with 90 percent shared prefixes. Hosts reusing each other's KV blocks got 3× the throughput with half the memory, and CXL fabric-attached memory ran within 99 percent of host DRAM. Liqid SVP for Solution Architecture Martin Cooper said the work shows shared CXL memory can lift GPU inference throughput 2.8× over DRAM, with no vLLM patches and no changes to the GPU servers. ARCHITECTURE Breaking the KV cache memory wall using Dynamo and Liqid CXL shared memory Micron has found that KV cache memory wall limitations on a GPU server can be substantially reduced by using a Liqid CXL-accessed memory pool to enlarge the DRAM cache. A Micron team is building a GPU cluster for the US Department of Energy’s Pacific Northwest National Laboratory PNNL to run next-generation, large-scale scientific, AI, HPC and data-intensive workloads. The cluster is called Great Abaco https://www.liqid.com/products/abaco , and Dr. Skyler Windh and his team at Micron developed a high-capacity shared memory system architecture for AI and HPC workloads. It uses CXL 2.0. The system is described in a 60-minute webinar and the background is this: as Large Language Model LLM inference scales across multi-GPU deployments, the Key-Value KV Cache becomes a major performance bottleneck. Each GPU typically keeps its own isolated cache, which can spill to host DRAM. When different GPUs process the same prompt prefix, each one repeats the same prefill computation. CXL fabric-attached memory FAM addresses this by providing multiple GPUs with a shared, byte-addressable memory pool, allowing other GPUs to reuse KV Cache entries computed by one GPU. Windh and his team built cross-host KV sharing into Nvida Dynamo and ran it on Micron’s Great Abaco cluster. Abaco https://www.liqid.com/products/abaco is a shared-memory system architecture for AI and HPC, designed by Micron with Liqid as the memory fabric and orchestration partner. A single Abaco rack presents up to 160TB of DDR5 as an addressable pool that one or multiple hosts in the rack can claim, share, or release without a reboot and without re-cabling. Applications reach the pool through a standard file system interface, so most code runs unmodified. The cluster uses H200 NVL GPUs in a Liqid PCIe Gen5 chassis, with four Liqid CXL memory chassis forming one shared pool. On Llama3-70B with 90 percent shared prefixes, a 32 GB cache per server delivered 2.8× the throughput and 2.9× faster time to first token than DRAM of the same size. Hosts reusing each other’s KV blocks got 3× the throughput with half the memory, and CXL memory ran within 99 percent of host DRAM. Ie. CXL FAM performs on par with a host DRAM baseline. The Micron team integrated CXL FAM-based KV Cache sharing into an open-source multi-GPU inference framework, with: - A block-level cache manager that allocates in CXL memory - Sequence-hash-based cache discovery across GPUs - An asynchronous transfer engine that moves blocks between CXL and GPU memory without stalling the inference scheduler - A CUDA host-registration workflow that enabled direct memory access DMA between GPUs and CXL memory Liqid SVP for Solution Architecture Martin Cooper says this Micron work shows that shared CXL memory can lift GPU inference throughput 2.8× over DRAM. Liqid puts GPUs and CXL memory on one PCIe fabric, so memory scales separately from compute. The serving stack stays as it is: no vLLM patches and no changes to the GPU servers. A shared pool lets you right-size each server and buy the memory once. You can access the webinar 's 30-slide presentation deck here https://computeexpresslink.org/wp-content/uploads/2026/10/CXL Consortium KV Cache Sharing FINAL.pdf , and watch the whole webinar here https://www.youtube.com/watch?v=cGTqLDHrcgs - it's has many YouTube adverts though. You can download a transcript we created here