Understanding Nvidia Grace-Blackwell Unified Memory Through Microbenchmarks

A placement-controlled microbenchmark study submitted to arXiv on 14 October 2026 finds that CUDA Unified Memory on NVIDIA's GB200 Grace-Blackwell system delivers wildly different bandwidth depending on where data sits, making a single "Unified Memory bandwidth" figure misleading. On GB200, a 1 GB STREAM-style triad reaches 3963 GB/s from local HBM but only 304 GB/s through host-mapped CPU/system memory, while a 256 MB streaming kernel with cold CPU-initialized managed memory reaches 14.1 GB/s versus 4606 GB/s with managed prefetch and 4504 GB/s on reuse. At 125% of CUDA-visible memory, prefetched and reused managed memory fall from above 4.6 TB/s to 968 GB/s and 1214 GB/s, which the authors summarize in a measurement-calibrated model of compute, access, migration, eviction, synchronization and handoff costs, using GH200 as a Grace-Hopper baseline and GB10/DGX Spark for validation.

Computer Science Distributed, Parallel, and Cluster Computing Submitted on 14 Oct 2026 Title:Understanding NVIDIA Grace-Blackwell Unified Memory Through Placement-Controlled Microbenchmarks View PDF https://arxiv.org/pdf/2610.20354 HTML experimental https://arxiv.org/html/2610.20354v1 Abstract:NVIDIA Grace--Blackwell systems make CPU--GPU memory increasingly unified at the programming level, but unified addressability does not make the memory system uniform. A CUDA managed allocation on GB200 may be served from GPU-local HBM, accessed through host/system memory, demand-migrated, prefetched, reused after GPU access, or evicted and reloaded under capacity pressure. The problem is that these states share a programming interface while exercising different hardware paths and runtime mechanisms, making a single Unified Memory bandwidth'' number misleading. This paper presents a placement-controlled microbenchmark study of CUDA Unified Memory on NVIDIA GB200, using GH200 as a Grace--Hopper baseline and GB10/DGX Spark as a validation platform. Our methodology separates local HBM access, host/system-memory access, explicit movement, managed first touch, prefetch, reuse, access-pattern sensitivity, oversubscription, and CPU--GPU handoff, then summarizes the observations with a measurement-calibrated model of compute, access, migration, eviction, synchronization, and handoff costs. On GB200, a 1~GB STREAM-style triad reaches 3963~GB/s from local HBM but 304~GB/s through host-mapped CPU/system memory; for a 256~MB streaming kernel, cold CPU-initialized managed memory reaches 14.1~GB/s, while managed prefetch and reuse reach 4606~GB/s and 4504~GB/s. Below the capacity boundary, prefetched and reused managed memory remain above 4.6~TB/s, but at 125\% of CUDA-visible memory they fall to 968~GB/s and 1214~GB/s, showing that Grace--Blackwell unified memory must be understood through placement, access history, locality, reuse, and capacity pressure. Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .