MTP

Google released new Gemma 4 checkpoints optimized with Quantization-Aware Training (QAT) to reduce model memory footprint for local deployment on edge devices and consumer GPUs. The QAT process minimi…

Speculative decoding optimizes LLM inference by using a cheap draft model to predict multiple tokens, which are then verified in a single forward pass of the target model, reducing memory-bandwidth bo…

A developer discovered that standard quantization pipelines for large language models silently discard multi-token prediction (MTP) heads, causing speculative decoding speedups to vanish despite the b…