AMD's open-source GEAK agent-driven framework automated the optimization of the DeepSeekV4 MLA kernel, achieving a 2.10x improvement in end-to-end throughput and a 3.71x reduction in time-to-first-tok…
AMD's QuickReduce library now supports INT3 quantization for all-reduce communication in multi-GPU LLM inference, achieving a 22% reduction in on-wire data volume compared to INT4 on AMD Instinct MI35…
NVIDIA released Nemotron-Labs-Diffusion, a single model that eliminates the need for a separate draft model in speculative decoding, achieving 6.82 accepted tokens per forward pass in self-speculation…
AMD GPUs running SGLang Diffusion on ROCm achieve 1.5x to 6.3x speedups over Hugging Face Diffusers for image generation and editing tasks, as demonstrated on models including FLUX.1-dev, Qwen-Image, …
Amazon Web Services announced new capabilities for Amazon SageMaker HyperPod, including inference data capture, direct model deployment from community hubs, NVMe storage for reduced latency, and Route…
OpenBMB and Tsinghua NLP lab released MiniCPM5-1B, a 1-billion-parameter open-source AI model designed to run on older smartphones. The model achieves top rankings in its class for reasoning and tool …
NVIDIA's closed-source cuda-checkpoint tool, which freezes and restores CUDA processes, suffers from slow PCIe transfers. Engineers reverse-engineered the tool to understand why checkpoint transfers f…
Tencent released Hy3, a 295B Mixture-of-Experts model with Apache 2.0 weights, achieving a 78.0 SWE-bench Verified score and leading open-weight models on tool and search benchmarks. A free API on Ope…
AMD introduced SGLang-ATOM, a bridge connecting the SGLang serving framework with ATOM's ROCm-native execution path to accelerate large language model inference on AMD Instinct GPUs. The integration u…
Microsoft Foundry now offers a curated catalog of Hugging Face open-weight models deployable on Foundry Managed Compute, with pre-staged weights in Azure and built-in enterprise security, governance, …
Tencent released Hy3, an open-source 295B-parameter Mixture-of-Experts model with 21B active parameters and 256K context length, under the Apache License 2.0. The model achieves state-of-the-art resul…
SGLang development is being augmented with agent-assisted workflows that encode procedural engineering knowledge into executable skills, covering LLM serving, GPU kernels, diffusion pipelines, and pro…
Tencent released Hy3, an Apache-2.0 open-weight Mixture-of-Experts model with 295B total parameters, 21B active parameters per token, and a 256K context window. The release includes official artifacts…
NVIDIA announced that its Confidential Computing technology for Blackwell GPUs achieves up to 98% of the inference performance of non-secure solutions, enabling hardware-rooted AI security without sig…
Speculative decoding accelerates large language model inference by 20-50% without quality loss, using a draft-verify mechanism that generates multiple tokens per forward pass. The technique amortizes …
MiniMax released M3, a 428-billion-parameter open-weight model, on June 7, achieving 59.0% on SWE-Bench Pro—slightly outperforming GPT-5.5's 58.6%—at $0.30 per million input tokens, making it 16 times…
Shanghai AI Laboratory's Agents-A1, a 35-billion-parameter Mixture-of-Experts model, matches or outperforms trillion-parameter rivals by training on longer task sequences instead of increasing paramet…
RadixArk released Miles, an open-source PyTorch-native framework for large-scale LLM reinforcement learning post-training. The framework integrates SGLang for rollout, NVIDIA Megatron-LM for training,…
PyTorch introduced the Cross-Repository CI Relay (CRCR), an automated pipeline that triggers and tracks CI in downstream repositories whenever changes are made to pytorch/pytorch. Results are displaye…
LLM inference optimization can reduce serving costs by 5-10x and latency by 3-5x, often determining whether an AI feature ships. The bottleneck is memory bandwidth during autoregressive decoding, and …