A developer's third installment in a GPU optimization series details implementing distributed training for large language models using CUDA and ROCm, covering All-Reduce, Ring-AllReduce, and ZeRO shar…
AMD's QuickReduce library now supports INT3 quantization for all-reduce communication in multi-GPU LLM inference, achieving a 22% reduction in on-wire data volume compared to INT4 on AMD Instinct MI35…
AMD has brought PyTorch Monarch to its Instinct GPUs with ROCm, enabling single-controller distributed training for large language models. The port addresses reliability challenges at scale by providi…
AMD Strix Halo cluster setup guide details how to configure a two-node system linked via Intel E810 RoCE v2 for distributed vLLM inference using Tensor Parallelism. The guide covers hardware prerequis…
Maincode, an Australian AI company, built a custom inference engine in Rust for the AMD MI355X GPU that bypasses the ROCm runtime entirely, achieving lower overhead on critical operations like tensor-…