09:33
2026-09-17
leoniemonigatti.com
large-language-models
Speculative decoding makes LLMs go brrr
Speculative decoding, an inference optimization introduced independently by Chen et al. at DeepMind and Leviathan et al. at Google in 2023, cuts LLM decoding latency without changing output quality by…