{"slug": "a-new-lens-on-multi-gpu-comms-api-level-memory-models-for-portable-optimization", "title": "A New Lens on Multi-GPU Comms: API-Level Memory Models for Portable Optimization", "summary": "A paper submitted to arXiv on 12 October 2026 proposes API-level memory models for device-side multi-GPU communication APIs such as rocSHMEM, MSCCL++ and NCCL/RCCL GIN, which currently lack a rigorous memory model. The authors define one such model for device-side communication, implemented in AMD's recently open-sourced rocSHMEM library, and show that formal relaxations enabling cache bypassing for network data improve a collective communication workload by up to 2.48x, a fused compute+communication workload by up to 1.77x, and a state-of-the-art expert-parallel communication workload by up to 1.18x.", "body_md": "# Computer Science > Distributed, Parallel, and Cluster Computing\n\n  [Submitted on 12 Oct 2026]\n\n# Title:A New Lens on Multi-GPU Communication: API-Level Memory Models for Portable Optimization\n\n[View PDF](https://arxiv.org/pdf/2610.16190)\n\nAbstract:As network communication increasingly becomes a performance bottleneck for ML and HPC workloads, it is more important than ever to optimize multi-GPU data transfers. Recently, low-level device APIs have emerged that allow programmers to perform fine-grain optimizations via one-sided put/get operations triggered directly from the device. However, the communication libraries providing these device APIs (e.g., rocSHMEM, MSCCL++, and NCCL/RCCL GIN) currently lack a rigorous memory model defining how their primitives interact with the underlying memory systems. Without this, emerging HPC and ML workloads aggressively optimizing network communication via device APIs can be brittle, error-prone, and face compatibility and inter-operability challenges.\n\nThis paper proposes the use of API-level memory models for these emerging device communication APIs. An API-level memory model defines the memory semantics of communication API calls in a way that abstracts lower-level details from the API user and simplifies development. This work defines one such model for device-side communication (implemented in AMD's recently open sourced rocSHMEM library) and demonstrates how formal relaxations to such a model can enable low-level LLM performance optimizations while retaining a well-defined standardized interface. Specifically, we describe how updating the SHMEM API implemented by rocSHMEM can properly support cache bypassing for network data, improving performance of a collective communication workload by up to 2.48x, a fused compute+communication workload by up to 1.77x, and a state-of-the-art expert-parallel communication workload by up to 1.18x.\n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/a-new-lens-on-multi-gpu-comms-api-level-memory-models-for-portable-optimization", "canonical_source": "https://arxiv.org/abs/2610.16190", "published_at": "2026-10-19 04:57:19+00:00", "updated_at": "2026-10-19 05:19:00.963300+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-research", "large-language-models", "mlops"], "entities": ["arXiv", "rocSHMEM", "AMD", "MSCCL++", "NCCL", "RCCL"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-new-lens-on-multi-gpu-comms-api-level-memory-models-for-portable-optimization", "markdown": "https://wpnews.pro/news/a-new-lens-on-multi-gpu-comms-api-level-memory-models-for-portable-optimization.md", "text": "https://wpnews.pro/news/a-new-lens-on-multi-gpu-comms-api-level-memory-models-for-portable-optimization.txt", "jsonld": "https://wpnews.pro/news/a-new-lens-on-multi-gpu-comms-api-level-memory-models-for-portable-optimization.jsonld"}}