Synthetic Sciences launched OpenScience, an open-source AI workbench that automates the full scientific research loop—literature review, hypothesis formation, code writing, experiment execution, and w…
A developer details practical QLoRA fine-tuning using Axolotl and Unsloth, explaining how parameter-efficient methods like LoRA and QLoRA enable training multi-billion parameter models on a single con…
Google's GKE Labs introduced OpenRL, an open-source self-hosted API for post-training and fine-tuning large language models on Kubernetes clusters. The project decouples reinforcement learning infrast…
Frontier LLMs including GPT-5.5, Gemini 3 Pro, and Opus 4.7 failed to correctly solve over two-thirds of problems in ParallelKernelBench, a new benchmark for multi-GPU kernel generation. The benchmark…
The PyTorch Foundation has opened nominations for its 2026 Contributor Awards, recognizing individuals who strengthen projects like PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors through techn…
Alibaba's 1.7B parameter Qwen3-TTS voice cloning model was fine-tuned using Fully Sharded Data Parallel (FSDP) with PyTorch and Ray, demonstrating memory-efficient distributed training across 4 GPUs. …
DeepSpeed has integrated the Muon Optimizer, a memory-efficient optimizer that uses a single momentum buffer and Newton-Schulz orthogonalization to improve training convergence, particularly for 2D we…
Researchers at Microsoft have introduced AutoSP, a compiler-based solution that automatically converts standard training code into multi-GPU sequence parallel code for long-context language model trai…