{"slug": "notes-on-mtp-eagle-3-and-dflash", "title": "Notes on MTP, EAGLE-3 and DFlash", "summary": "A developer reports that speculative decoding techniques MTP, EAGLE-3, and DFlash can significantly speed up local inference of large language models in llama-cpp. Testing on an NVidia 3060 RTX 12 GB GPU with models like Qwen 3.6 35B and Gemma 4 26B, the developer found that using a lower spec-draft-n-max setting of 1 yields better tokens per second, with DFlash slightly outperforming EAGLE-3 on Gemma 4 26B. MTP speculative decoding improved a coding model's speed from 24 to 39 tokens per second, a 38% speedup.", "body_md": "**Speculative decoding** is a huge speedup for local inference of LLMs in `llama-cpp`\n\n(and similar programs). Recently I have tried to adjust the settings for MTP, EAGLE-3 and DFlash to get higher **tokens per second** on my NVidia 3060 RTX 12 GB GPU.\n\nFor larger models like Qwen 3.6 35B and Gemma 4 26B, I have found that using a **lower** `spec-draft-n-max`\n\nsetting (for `llama-cpp`\n\n) is beneficial. I have found 1 works better than 2 or 3 (or 0) for MPT, EAGLE-3 and DFlash on these larger models.\n\nI compared **DFlash** with **EAGLE-3** in Gemma 4 26B, and found that with a `spec-draft-n-max`\n\nof 1, DFlash was **slightly faster** than EAGLE-3. For consumer GPUs like the 3060, it seems beneficial to reduce the speculation being done, and use smaller speculative models (the DFlash model I have is smaller than the EAGLE-3 one).\n\nOverall, I found that **MTP speculative decoding** improved the tokens per second in my coding model (Qwen 3.6 35B AutoRound) from 24 to 39 tokens per second, which is a 38% speedup. It is important on some GPUs to use a lower value for `spec-draft-n-max`\n\n, even setting it to 1.", "url": "https://wpnews.pro/news/notes-on-mtp-eagle-3-and-dflash", "canonical_source": "https://www.dotnetperls.com/2026_7_7_notes-mtp-eagle-dflash", "published_at": "2026-07-07 07:00:00+00:00", "updated_at": "2026-07-20 15:27:51.947431+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["llama-cpp", "NVidia 3060 RTX", "Qwen 3.6 35B", "Gemma 4 26B", "MTP", "EAGLE-3", "DFlash"], "alternates": {"html": "https://wpnews.pro/news/notes-on-mtp-eagle-3-and-dflash", "markdown": "https://wpnews.pro/news/notes-on-mtp-eagle-3-and-dflash.md", "text": "https://wpnews.pro/news/notes-on-mtp-eagle-3-and-dflash.txt", "jsonld": "https://wpnews.pro/news/notes-on-mtp-eagle-3-and-dflash.jsonld"}}