A technical explainer on distributed training and inference details how work is divided across CPUs, GPUs, and multi-node clusters, distinguishing data, pipeline, and tensor parallelism and sharding, …
Higgsfield AI, a generative AI company with $500M in annual revenue run rate and 25M+ users, is hiring an ML Systems Performance Engineer (MFU) for its Almaty, Kazakhstan office. The role focuses on o…
AI applications run in data centers, and their performance depends on a five-layer stack of software, hardware, and infrastructure, not just the model. The article explains that training and inference…
Independent researcher Alpamys Makazhan released Soup, a Show HN project that fine-tunes a full Llama-3.1-8B model in NF4 quantization with a 3.32 GB VRAM peak at 119.6 tokens/sec on a 4 GB RTX 3050 L…
The PyTorch Foundation, which expanded into a multi-project foundation in April 2025, now hosts six projects including PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors. PyTorch released version …
Synthetic Sciences launched OpenScience, an open-source AI workbench that automates the full scientific research loop—literature review, hypothesis formation, code writing, experiment execution, and w…
A developer details practical QLoRA fine-tuning using Axolotl and Unsloth, explaining how parameter-efficient methods like LoRA and QLoRA enable training multi-billion parameter models on a single con…
Google's GKE Labs introduced OpenRL, an open-source self-hosted API for post-training and fine-tuning large language models on Kubernetes clusters. The project decouples reinforcement learning infrast…
Frontier LLMs including GPT-5.5, Gemini 3 Pro, and Opus 4.7 failed to correctly solve over two-thirds of problems in ParallelKernelBench, a new benchmark for multi-GPU kernel generation. The benchmark…
The PyTorch Foundation has opened nominations for its 2026 Contributor Awards, recognizing individuals who strengthen projects like PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors through techn…
Alibaba's 1.7B parameter Qwen3-TTS voice cloning model was fine-tuned using Fully Sharded Data Parallel (FSDP) with PyTorch and Ray, demonstrating memory-efficient distributed training across 4 GPUs. …
DeepSpeed has integrated the Muon Optimizer, a memory-efficient optimizer that uses a single momentum buffer and Newton-Schulz orthogonalization to improve training convergence, particularly for 2D we…
Researchers at Microsoft have introduced AutoSP, a compiler-based solution that automatically converts standard training code into multi-GPU sequence parallel code for long-context language model trai…