Researchers introduced OCGQuant, a post-training quantization method that pairs outlier channels with low-magnitude companions to reduce quantization error in NVFP4 low-bit inference, achieving the lo…
Baseten's technical breakdown of LLM inference efficiency identifies two categories of engineering choices: those that trade latency for throughput, such as batch sizing, tensor parallelism, expert pa…
Inco AI launched day-0 support for Z.ai's GLM 5.3 model, releasing DFlash 2 and NVFP4 checkpoints alongside an Inco Engine endpoint that delivers up to 4.4× throughput versus the native FP8 checkpoint…
A developer found a scale-selection rule for NVFP4 gradient quantization that reduces mean squared error by 14% on real gradient tensors compared to the state-of-the-art MS-EDEN estimator, but trainin…
A developer benchmarked the Qwen3.8-27B model with NVFP4 quantization and multi-token prediction (MTP) on an RTX 5090, comparing tier configurations and sweeping spec-draft-n values at 64K context. Th…
NVIDIA's open-weights Nemotron 3.5 Lightning mixture-of-experts model, with 30 billion total parameters but only 3 billion active, is designed for agentic grunt work and can be run locally on consumer…
Developer Nick Mitchko released DeepSeek-V4-Flash-0731-Latent-Reasoning, a self-contained model that performs reasoning in latent space via a trained CoLaR head, with all weights in one Hugging Face r…
NVIDIA released Ising Calibration 1.5, an open-source vision language model for fully automated quantum processor calibration, which is 11.4% smaller at BF16 precision and 86.68% better than its prede…
Hugging Face has integrated Nunchaku 4-bit diffusion inference natively into Diffusers, enabling users to load quantized checkpoints with a simple from_pretrained() call and no local CUDA compilation.…
NVIDIA's Nemotron 3 Embed 1B embedding model took first place on every recall task in Zep's benchmark of 5,954 production queries, outperforming a 4B competitor with a third of the parameters and surp…
AMD has integrated an NVFP4 emulation pipeline into vLLM that enables AMD Instinct MI355 accelerators to serve standard NVFP4 quantized checkpoints directly, dequantizing weights to BF16 on-the-fly at…
ARCQuant, a new framework for Large Language Model inference, uses the NVFP4 numerical format to achieve up to 3x speedup on GPUs while maintaining accuracy comparable to full-precision baselines. The…
NVIDIA released the Nemotron 3 Ultra NVFP4 checkpoint, a quantized model that achieves up to 5.9x higher inference throughput than GLM-5.1 754B FP4 on decode-heavy workloads while matching BF16 accura…
NVIDIA's June 2026 DGX Spark update introduces automated four-node clustering via Cluster Assistant, enabling local inference of models up to 700B parameters. The update also delivers a 2.6x throughpu…
NVIDIA announced that its full-stack inference and training optimizations, including the GB200 NVL72 rack-scale system and NVIDIA Dynamo software, can maximize AI factory energy efficiency, reducing t…
A critical bug in Google's Gemma 4 causes it to malform tool calls under real load, affecting vLLM, llama.cpp, Ollama, and oobabooga. A developer open-sourced a diagnosis, repair, and experimental LoR…
NVIDIA released a guide showing how to optimize transformer-based models for low-precision training using Hopper and Blackwell GPUs, focusing on FP8 and NVFP4 formats. The method translates model conf…
NVIDIA released Nemotron 3 Ultra, a 550B-parameter hybrid Mamba-Transformer model with 55B active parameters, achieving up to 6x higher inference throughput than state-of-the-art LLMs while maintainin…
NVIDIA has introduced the NVFP4 training recipe in TransformerEngine, enabling 4-bit mixed-precision pre-training on Blackwell GPUs with no measurable accuracy loss compared to FP8 baselines. The reci…
NVIDIA Nemotron 3 Ultra, a 550-billion parameter open large language model, is now available for one-click deployment on Amazon SageMaker JumpStart. The model delivers 5x faster inference and up to 30…