{"slug": "building-efficient-multi-gpu-systems-for-llms", "title": "Building Efficient Multi-GPU Systems for LLMs", "summary": "UC Davis assistant professor and Meta AI research scientist Yang Zhou presented UCCL, a family of systems for improving multi-GPU large language model performance, including UCCL-Fuse for overlapping computation and communication within GPU kernels, UCCL-EP and UCCL-Lite for execution across diverse hardware, and UCCL-Tran for multipath communication over existing RDMA fabric without new hardware. Zhou said the open-source UCCL work has been deployed by AMD, NVIDIA, AWS, Swiss AI, Red Hat, IBM, Google, JD.com, Xiaomi, Tencent, SiliconFlow and Moonshot AI/KIMI, with ideas adopted at scale by xAI, OpenAI and Meta. The systems target reduced inference latency and higher communication throughput, and Zhou closed with directions for training coding agents to build and optimize distributed GPU systems.", "body_md": "# Building Efficient Multi-GPU Systems for LLMs\n\n## Abstract\n\nLarge language models increasingly rely on multiple GPUs, but communication overhead and synchronization limit their efficiency. This talk presents UCCL, a family of systems that improves LLM performance through efficient parallelism and extensible networking. UCCL-Fuse overlaps computation and communication within GPU kernels, while UCCL-EP and UCCL-Lite support efficient execution across diverse hardware. UCCL-Tran brings adaptable software transport to existing RDMA fabric, enabling multipath communication without new hardware. Together, these systems reduce inference latency and improve communication throughput, with adoption across industry frameworks. The talk concludes with directions for training coding agents to build and optimize distributed GPU systems.\n\n## Bio\n\nYang Zhou is an Assistant Professor at UC Davis and an AI Research Scientist at Meta TBD Lab. He was a Member of Technical Staff at xAI and a postdoc at UC Berkeley SkyLab working with Ion Stoica. He got his Ph.D. from Harvard, advised by Minlan Yu and James Mickens. He has received a MLsys best paper award and a Google PhD fellowship. He recently worked on UCCL for efficient GPU communication, with OSS deployed by AMD, NVIDIA, AWS, Swiss AI, Red Hat, IBM, Google, JD.com, Xiaomi, Tencent, SiliconFlow, and Moonshot AI/KIMI, and ideas adopted at scale by xAI, OpenAI, and Meta.", "url": "https://wpnews.pro/news/building-efficient-multi-gpu-systems-for-llms", "canonical_source": "https://systems.seas.harvard.edu/seminar/2026-10-20-yang-zhou/", "published_at": "2026-10-17 19:00:00+00:00", "updated_at": "2026-10-17 20:07:56.260018+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-chips", "ai-research"], "entities": ["Yang Zhou", "UC Davis", "Meta", "UCCL", "UCCL-Fuse", "UCCL-Tran", "xAI", "NVIDIA"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/building-efficient-multi-gpu-systems-for-llms", "markdown": "https://wpnews.pro/news/building-efficient-multi-gpu-systems-for-llms.md", "text": "https://wpnews.pro/news/building-efficient-multi-gpu-systems-for-llms.txt", "jsonld": "https://wpnews.pro/news/building-efficient-multi-gpu-systems-for-llms.jsonld"}}