NVIDIA's DOCA GPUNetIO has evolved from a standalone GPU-centric packet-processing framework into a unified GDA-KI foundation now integrated across NVSHMEM, NCCL, Aerial 5G SDK, UCX/NIXL, NVQLink with…
AWS EFA's SRD transport moves CUDA buffers roughly 40x faster than tuned TCP, about 13x faster through a shuffle, and roughly 2x faster on the PDS-H Q9 benchmark, according to a blog post recording co…
Researchers Junnan Dong, Linhao Luo, Senlei Zhang and colleagues at Tencent Youtu Lab and Monash University proposed WFM, a Wiki Foundation Model for encoding and retrieving LLM Wiki knowledge bases t…
NVIDIA Dynamo-Triton release 26.07 now enables TensorRT multi-device inference, allowing a single TensorRT network to execute across multiple GPUs using NCCL-backed distributed collectives, with full …
An 8-GPU tensor-parallel (TP8) rig should prioritize PCIe topology over CPU choice, according to guidance recommending that GPUs communicate directly through a Gen5 switch rather than routing peer-to-…
PyTorch 2.14 shipped on September 15 with 2,995 commits from 487 contributors, adding fault-tolerant distributed training as a first-class framework concept via a rewritten NCCL backend (nccl2) with i…
Researchers proposed WFM, a Wiki Foundation Model for scalable, agent-native knowledge representation and retrieval, detailed in arXiv paper 2609.18182v1. WFM formalizes a Wiki Graph schema coupling d…
Reinforcement learning has become the standard final stage of training frontier language models, with GRPO on verifiable rewards now the standard recipe since DeepSeek-R1 popularized it, according to …
Nvidia's NVLink 6 interconnect adds multi-layer resiliency designed to keep large GPU clusters running through link failures instead of crashing entire training and inference jobs. The system reroutes…
NVIDIA released CUDA Toolkit 13.4, adding Windows on Arm support, early developer preview for the NVIDIA Rubin GPU architecture, and enhanced GPU management capabilities including Multi-Process Servic…
A technical explainer on distributed training and inference details how work is divided across CPUs, GPUs, and multi-node clusters, distinguishing data, pipeline, and tensor parallelism and sharding, …
A developer's final part of a CUDA/ROCm optimization series details implementing Mixture of Experts (MoE) inference, including a custom top-k gating kernel, expert parallelism with All-to-All communic…
A developer's third installment in a GPU optimization series details implementing distributed training for large language models using CUDA and ROCm, covering All-Reduce, Ring-AllReduce, and ZeRO shar…
PyTorch's DistributedDataParallel (DDP) enables multi-GPU training by synchronizing gradients across all GPUs via AllReduce, ensuring each GPU maintains identical model weights. The article explains t…
A new paper, FlowPrefill by Hsieh et al., proposes preempting long LLM inference prefills mid-forward-pass to rescue urgent requests that would otherwise miss their time-to-first-token (TTFT) service-…
The 5th edition of 'Programming Massively Parallel Processors' by Wen-mei W. Hwu, David B. Kirk, and Izzat El Hajj, published by Morgan Kaufmann, introduces new chapters on filtering, wavefront parall…
A new study from researchers at an undisclosed institution demonstrates GPU collective communication kernels that reduce latency to within 7% of the hardware Speed-of-Light lower bound, achieving subs…
A hardware-software co-design prototype called PIM-HBM explores zero-copy memory view fusion and fault telemetry for LLM training, using JAX/XLA integration and branchless CUDA kernels to mitigate com…
Databricks AI engineers detailed how they maintain GPU reliability at scale, describing failure modes including crashed jobs, silent slowdowns, and numerical corruption, and outlining a multi-stage he…
The UCCL team released rdmatop, a real-time terminal UI that monitors RDMA traffic across any Linux device including NVIDIA ConnectX, AWS EFA, and Broadcom NICs. The tool reads RDMA netlink to provide…