An optimized training stack can reach roughly 51% model FLOPs utilization (MFU) on consumer GPUs, according to the LLMQ paper, leaving headroom for a user currently getting about 65 TFLOPS of useful c…
DeepL trained and served its next-generation large language models in FP8 precision on a 544-NVIDIA H100 Tensor Core GPU DGX SuperPOD, producing translations that outperform its previous models by 1.4…
A September 22, 2026 arXiv paper reports that greedy decoding in large language models is not precision-invariant: the same model, prompt and decoding algorithm produced different outputs in BF16 vers…
A study measuring 54 configurations of Qwen2.5-7B-Instruct on vLLM 0.12 across L4, A100, and H100 GPUs found that 18 of 36 configurations reach the cost, quality, and latency Pareto frontier, with com…
Independent AI researcher Jinho Jang has published altered FP8 weights for Z.ai's open-weight GLM-5.3-Flash model through his DeAlignAI project, claiming the modified model complied with all 320 promp…
Inco AI launched day-0 support for Z.ai's GLM 5.3 model, releasing DFlash 2 and NVFP4 checkpoints alongside an Inco Engine endpoint that delivers up to 4.4× throughput versus the native FP8 checkpoint…
An engineer at Jahn AI ran a per-prompt regression check before recommending FP8 quantization for Qwen3-8B serving on an RTX PRO 6000 Blackwell, which delivered a 1.5x throughput gain (1,725 to 2,597 …
Snapshot compression using LZ4 or Zstandard can make elastic inference viable by reducing network I/O overhead during cold starts, according to a technical analysis. The approach involves weight quant…
The KV cache accelerates LLM inference on GPUs by storing prior token key/value projections instead of recomputing them, reducing per-step attention cost from quadratic to linear. Decode is memory-ban…
A developer detailed the architectural differences between NVIDIA's Ampere and Hopper GPU architectures, focusing on tensor core and memory bandwidth improvements. The Hopper architecture introduces T…
NVIDIA released a guide showing how to optimize transformer-based models for low-precision training using Hopper and Blackwell GPUs, focusing on FP8 and NVFP4 formats. The method translates model conf…
Enterprises are burning millions on cloud LLM tokens due to inefficient agentic systems, prompting a shift to open-weight models on dedicated infrastructure to eliminate marginal token costs and achie…
NVIDIA released CompileIQ, an AI-powered compiler auto-tuning framework that uses evolutionary and genetic algorithms to optimize GPU compilers for individual workloads. The tool, included in NVIDIA C…