NVIDIA announced multi-device inference support in TensorRT 11.0, enabling native high-performance multi-GPU inference for generative AI workloads. The feature integrates with NCCL for distributed col…
Shrijith Venkatramana, a developer building git-lrc, explains that NVIDIA Collective Communications Library (NCCL) is the critical infrastructure enabling multi-GPU training of large language models. …
VLLM, a model-serving engine for large language models, introduced a small op-level IR to resolve the tension between acting as a compiler target and a hand-tuned kernel dispatcher. The IR allows vLLM…
Apple released JACCL, an open-source collective communication library for distributed AI training over Thunderbolt 5, at WWDC 2026. The library enables clustering 2–4 Macs with 50–60 Gbps bandwidth an…
Researchers from UC Berkeley and other institutions released CommBench, a benchmark of over 100 GPU communication problems, to evaluate whether large language models can generate correct and efficient…
Researchers from UC Berkeley's UCCL project released mKernel, a library of persistent CUDA kernels that fuse intra-node NVLink communication, inter-node RDMA, and compute into a single kernel to addre…
A developer has built an open-source agent that correlates NCCL AllReduce stalls with TCP retransmits on the same host, revealing network bottlenecks that GPU dashboards miss. The tool attaches uprobe…
Hugging Face researchers released a new method called delta weight sync that reduces the per-step weight transfer in asynchronous reinforcement learning from 1.2 GB to as little as 20 MB for a 0.6B pa…