Abstract #
Large language models increasingly rely on multiple GPUs, but communication overhead and synchronization limit their efficiency. This talk presents UCCL, a family of systems that improves LLM performance through efficient parallelism and extensible networking. UCCL-Fuse overlaps computation and communication within GPU kernels, while UCCL-EP and UCCL-Lite support efficient execution across diverse hardware. UCCL-Tran brings adaptable software transport to existing RDMA fabric, enabling multipath communication without new hardware. Together, these systems reduce inference latency and improve communication throughput, with adoption across industry frameworks. The talk concludes with directions for training coding agents to build and optimize distributed GPU systems.
Bio #
Yang Zhou is an Assistant Professor at UC Davis and an AI Research Scientist at Meta TBD Lab. He was a Member of Technical Staff at xAI and a postdoc at UC Berkeley SkyLab working with Ion Stoica. He got his Ph.D. from Harvard, advised by Minlan Yu and James Mickens. He has received a MLsys best paper award and a Google PhD fellowship. He recently worked on UCCL for efficient GPU communication, with OSS deployed by AMD, NVIDIA, AWS, Swiss AI, Red Hat, IBM, Google, JD.com, Xiaomi, Tencent, SiliconFlow, and Moonshot AI/KIMI, and ideas adopted at scale by xAI, OpenAI, and Meta.