{"slug": "trying-dflash-a-diffusion-model-approach-to-parallel-draft-token-generation-on", "title": "Trying \"DFlash,\" a Diffusion-Model Approach to Parallel Draft-Token Generation, on Gemma", "summary": "A developer benchmarked DFlash, a diffusion-model-based speculative decoding drafter from Z-Lab, against Gemma-4-12B-it's native Assistant MTP model on an RTX 3060 with 12GB VRAM using llama.cpp. DFlash failed to outperform the Assistant model on the JavaScript coding task, though the testing highlighted where the diffusion drafting approach is and isn't effective. DFlash, introduced in the paper \"DFlash: Block Diffusion for Flash Speculative Decoding,\" predicts multiple tokens in parallel and claims broad model support and gains over EAGLE-3.", "body_md": "In the concept edition and the implementation/benchmark edition, we covered a speed-up technique for LLM generation called MTP (Multi-Token Prediction). To recap briefly: a lightweight \"draft model\" predicts a handful of tokens ahead of time, and the main model checks them all at once. When the guesses are right, you leap ahead several tokens in a single step, which is what makes the whole thing feel faster.\n\nThere's more than one way to build that draft model, and the one we're looking at this time, \"DFlash,\" takes an unusual approach: it uses a diffusion model — the kind of technique you'd normally associate with image generation — to predict multiple tokens all at once instead of one at a time. It claims to support a wide range of models and to significantly outperform EAGLE-3, an existing approach. Those are the claims worth testing directly.\n\nThe catch is that benchmarks like this are usually measured in an environment the vendor sets up. It's harder to find a case where someone ran it on their own GPU, against an opponent that already has a dedicated, well-optimized MTP model of its own — Gemma-4's native Assistant model.\n\nSo this time, using the same setup as the implementation/benchmark edition (an RTX 3060 with 12GB VRAM, llama.cpp, the same JavaScript coding task), we directly compared DFlash against Gemma-4-12B-it's Assistant model. The short version: DFlash did not outperform the Assistant model. The reasons are technically clear, though, and they draw a fairly clear picture of where DFlash is strong and where it isn't — that's what we'll dig into below.\n\nTwo new token-prediction techniques appeared in quick succession:\n\nDSpark<sup>1</sup> was released to speed up inference specifically for DeepSeek's own DeepSeek-V4, and it only supports DeepSeek-V4 / DeepSeek-V4-Flash.\n\nDFlash, on the other hand, supports a much wider range of models, each with its own dedicated DFlash model. Like Google's Assistant model, it's designed to be bolted on as an add-on to achieve token prediction.\n\nZ-Lab is a research group led by Zhijian Liu<sup>2</sup>, an assistant professor at UC San Diego who is also a research scientist at NVIDIA. The lab works across the algorithm, systems, and application layers to make AI smaller, faster, and more efficient.\n\nThis time, we wanted to understand how much of a speed-up DFlash actually delivers, verified through hands-on testing.\n\nDFlash is introduced in the paper \"DFlash: Block Diffusion for Flash Speculative Decoding\"<sup>3</sup>.\n\nSpeculative token prediction itself isn't new — the earliest paper on the idea was published by Google DeepMind in 2023, \"Accelerating Large Language Model Decoding with Speculative Sampling\"<sup>4</sup>.\n\nImprovements continued quietly from there, culminating in 2025 in an MTP draft model called EAGLE-3. Even so, it apparently never escaped the autoregressive paradigm, and in the end didn't deliver a dramatic speed improvement.\n\nMeanwhile, diffusion models — the noise-removal mechanism commonly used in image generation — have been making their way into the LLM space. It started with Meta's LLaDa, and in Japan, KDDI's ELYZA Lab team released a model called \"ELYZA-Diffusion-Instruct-1.0-Dream-7B\"<sup>5</sup>.\n\nZ-Lab's DFlash brings that diffusion-model property into the MTP draft model. The basic mechanism is shown below.\n\n``` php\n[Input Token (N)] --> [Main Model] -- h_on ----------------------------------------> [Predicted: N+1]\n                                       |                                             ^\n                                       v                                             |\n                                  [KVCache]                                          |\n                                       |                                             |\n+--------------------------------------+--------------------------------+            |\n| DFlash Drafter                       v (KV data injection)            |            |\n|                                 [KVCache]                             |            |\n|                                      |                                |            |\n|                              [Diffusion Model]                        |            |\n|                                      |                                |            |\n|                               [Token Decoder]                         |            |\n|                                      |                                |            |\n|                            +---------v--------+                       |            |\n|                            | Predicted N+2    |                       |            |\n|                            |       +          |                       |            |\n|                            | Predicted N+3    |                       |            |\n|                            |       +          |                       |            |\n|                            | Predicted N+4    |                       |            |\n|                            +---------+--------+                       |            |\n+--------------------------------------+--------------------------------+            |\n                                       |                                             |\n+--------------------------------------+---------------------------------------------+-------+\n| Processing inside Main Model         v                                             |       |\n|                             * Uses causal attention to mask;                       |       |\n|                               computes probs in parallel                           |       |\n|                                      +------------------------------+              |       |\n|                                      v                              v              |       |\n|                         (Match found)                  (No match)                  |       |\n|                         Include matching portion       Nothing included in output  |       |\n|                         -> Predicted: N+2                                          |       |\n|                         -> Predicted: N+3                                          |       |\n+------------------------------------------------------------------------------------+-------+\n```\n\n*Figure 1: DFlash mechanism*\n\nLooking at the mechanism, it resembles the Assistant model implemented in Gemma, but the big difference is in how the draft model itself predicts tokens. The latter half — the token-evaluation stage — closely mirrors Gemma's approach.\n\nWith DFlash, the KV cache is built independently. As before, when the main model predicts token N+1, it uses that state to update its own KV cache. Data extracted from that update is then injected into DFlash's own KV cache, which is where DFlash's processing begins.\n\nGemma's Assistant model runs this prediction step using a very small neural network, sequentially and at high speed, producing as many predicted tokens as needed before handing them off to the evaluation logic.\n\nDFlash, by contrast, doesn't use an autoregressive model inside its small neural network — it uses a diffusion model. Here, much like generating an image, it produces all of the needed predicted tokens, in the correct order, in one shot. The longer the maximum token length, the longer this takes, but it's still dramatically faster than doing it sequentially. What follows is the same as before: causal-attention-based masking runs in parallel, and the result determines which tokens are allowed to be output together.\n\nSo the biggest contributor to any speed advantage comes down to the parts marked with blue and red boxes in the middle of the figure below.\n\n``` php\n[For Gemma Assistant]\n[Input Token (N)] --> [Main Model] --> [Predicted: N+1]\n                            |\n                            +-> [Predicted: N+2] -(seq)-> [Predicted: N+3] -(seq)-> [Predicted: N+4]\n                                                                    |\n                                                                    v\n                                                     [Verify] --> Up to n OK --> [Confirmed: N+2, N+3...]\n\n[For DFlash]\n[Input Token (N)] --> [Main Model] --> [Predicted: N+1]\n                            |\n                            |   +-- [Predicted: N+2]\n                            +---+-- [Predicted: N+3]   <-- (Outputs all at once, order included)\n                            |   +-- [Predicted: N+4]\n                            |               |\n                            |               v\n                            +--------->  [Verify] --> Up to n OK --> [Confirmed: N+2, N+3...]\n```\n\n*Figure 2: Difference between Gemma's Assistant model and DFlash*\n\nFirst, get the model files.\n\n```\nhf download google/gemma-4-12B-it\nhf download z-lab/gemma4-12B-it-DFlash\n```\n\nThis time we're using llama.cpp build 9850. It's a good idea to grab the latest build. Use the Python conversion tool included with it to convert each model to GGUF format. Start with the main model.\n\n``` bash\n$ python convert_hf_to_gguf.py \\\n~/.cache/huggingface/hub/models--google--gemma-4-12B-it/snapshots/cxxxxxx...a/ \\\n--outtype bf16 --outfile gemma-4-12B-it-b16.gguf\n```\n\nNext, convert the DFlash model to GGUF.\n\nThe important thing here is `--target-model-dir`. When converting a DFlash model, it needs to be given a reference to the main model — that's the parameter it uses to build the converted DFlash output.\n\n``` bash\n$ python convert_hf_to_gguf.py \\\n~/.cache/huggingface/hub/models--google--gemma-4-12B-it/snapshots/cxxxxxx...a/ \\\n--outtype bf16 --outfile models/gemma-4-12B-it-DFlash-b16.gguf \\\n--target-model-dir \\\n~/.cache/huggingface/hub/gemma-4-12B-it-qat-q4_0-unquantized/snapshots/c202...a/\n```\n\nIf this fails with the message below, download `tokenizer.model` directly from the `google/gemma-3-12b-it` repository on Hugging Face and place it in the cache directory.\n\n```\nINFO:hf-to-gguf:DFlash: Using tokenizer from target model: /home/aiuser/.cache/huggingface/hub/models--google--gemma-4-12B-it/snapshots/5854...7ced3\n\nTraceback (most recent call last):\n  File \"/data/user_data/aiuser/llama.cpp/conversion/qwen.py\", line 57, in set_vocab\n    self._set_vocab_sentencepiece()\n  File \"/data/user_data/aiuser/llama.cpp/conversion/base.py\", line 1589, in _set_vocab_sentencepiece\n    tokens, scores, toktypes = self._create_vocab_sentencepiece()\n  File \"/data/user_data/aiuser/llama.cpp/conversion/base.py\", line 1606, in _create_vocab_sentencepiece\n    raise FileNotFoundError(f\"File not found: {tokenizer_path}\")\nFileNotFoundError: File not found: /home/aiuser/.cache/huggingface/hub/models--google--gemma-4-12B-it/snapshots/585...ced3/tokenizer.model\n```\n\nQuantizing these produces 4-bit models.\n\n```\n$ /opt/llama/bin/llama-quantize \\\nmodels/gemma-4-12B-it-b16.gguf models/gemma-4-12B-it-q4_0.gguf q4_0\n\n$ /opt/llama/bin/llama-quantize \\\nmodels/gemma-4-12B-it-DFlash-b16.gguf models/gemma-4-12B-it-DFlash-q4_0.gguf q4_0\n```\n\nAt runtime, add the following arguments. Since the DFlash model acts as an add-on draft/MTP model, use `--model-draft` to point to it and enable MTP.\n\n```\n/opt/llama/bin/llama-server --model /opt/llama/models/gemma-4-12B-it-q4_0.gguf \\\n--model-draft /opt/llama/models/gemma-4-12B-it-DFlash-q4_0.gguf \\\n-t 4 --prio 2 --temp 1.0 --top-p 0.95 --top-k 64 --host 0.0.0.0 --port 8001 \\\n--device CUDA0 -mg 0 -sm layer --fit on -fa on -c 163840 -ctv q8_0 -ctk q8_0 \\\n--no-warmup --no-cache-prompt --cache-ram 0 \\\n--spec-type draft-dflash --spec-draft-n-max 4 --spec-draft-device CUDA0 \\\n--chat-template-kwargs '{\"enable_thinking\":true}'\n```\n\nA quick rundown of the arguments:\n\n`--model-draft <gguf file>`:`-sm layer`:`--spec-type draft-dflash`:`--spec-draft-n-max 4`:`--spec-draft-device CUDA0`:\nWe ran the following simplified verification setup.\n\nMemory usage came out as follows.\n\n| Data category | Assistant CUDA0 | Assistant CPU | DFlash CUDA0 | DFlash CPU | \n|---|---|---|---|---|\n| Weight data | 6,390.19 | 540.00 | 6,637.69 | 787.50 | \n| KV cache data | 1,360.00 | — | 1,360.00 | — | \n| KV cache for Sliding Window | 765.00 | — | 765.00 | — | \n| Gated DeltaNet compute buffer | 533.80 | 180.80 | 533.80 | 180.80 | \n| MTP model weight data | 226.90 | 144.00 | 390.42 | — | \n| MTP token-to-piece cache size | — | — | 1.94 | — | \n| MTP model KV cache size | — | — | 640.00 | — | \n| MTP model Sliding Window KV cache size | — | — | 136.00 | — | \n| MTP model Gated DeltaNet compute buffer | 532.78 | 180.79 | 519.50 | 176.02 | \n| **Total** | **9,808.67** | **1,045.59** | **10,984.35** | **1,226.38** | \n\n*Table 1: Memory usage breakdown by MTP method (units: MiB). This run used text-only mode, so multimodal-model requirements are excluded.*\n\nOf that, the memory used specifically by the MTP model itself was as follows — roughly double for DFlash.\n\n|  | Assistant CUDA0 | Assistant CPU | DFlash CUDA0 | DFlash CPU | \n|---|---|---|---|---|\n| MTP model memory usage | 759.68 | 324.79 | 1,687.86 | 176.02 | \n\n*Table 2: Total memory used by the MTP model, by MTP method (units: MiB)*\n\nThe main driver here is how the two approaches use the KV cache. The Assistant model shares its KV cache with the main Gemma model. DFlash, by contrast, keeps its own independent KV cache — meaning it has to hold the same cache structure as the main Gemma model a second time. That's the biggest factor behind the difference.\n\nAs in the previous verification, we compared token ingestion speed turn by turn. There was no major throughput difference between the Assistant model and DFlash for reading tokens in.\n\n| Turn | Assistant (tok/s) | DFlash (tok/s) | \n|---|---|---|\n| 1 | 138.34 | 53.72 | \n| 2 | 710.71 | 652.47 | \n| 3 | 733.53 | 739.52 | \n| 4 | 735.43 | 708.22 | \n\n*Figure 3: Token read-in speed by turn*\n\nFor output speed per turn, the Assistant model was clearly ahead — DFlash consistently trailed by about 10–20 tokens per second.\n\n| Turn | Assistant (tok/s) | DFlash (tok/s) | \n|---|---|---|\n| 1 | 62.80 | 39.37 | \n| 2 | 62.04 | 48.42 | \n| 3 | 66.68 | 52.40 | \n| 4 | 62.77 | 48.73 | \n\n*Figure 4: Token output speed by turn*\n\nHere is the raw measurement data behind the numbers above. Note that time is in milliseconds.\n\n**Reading**\n\n| Turn | Assistant time (ms) | Assistant tokens | Assistant tps | DFlash time (ms) | DFlash tokens | DFlash tps | \n|---|---|---|---|---|---|---|\n| 1 | 181 | 25 | 138.34 | 577 | 31 | 53.72 | \n| 2 | 3,004 | 2,135 | 710.71 | 3,374 | 2,202 | 652.47 | \n| 3 | 6,361 | 4,666 | 733.53 | 6,391 | 4,727 | 739.52 | \n| 4 | 9,559 | 7,030 | 735.43 | 10,173 | 7,205 | 708.22 | \n\n**Generation**\n\n| Turn | Assistant time (ms) | Assistant tokens | Assistant tps | DFlash time (ms) | DFlash tokens | DFlash tps | \n|---|---|---|---|---|---|---|\n| 1 | 33,186 | 2,084 | 62.80 | 54,413 | 2,142 | 39.37 | \n| 2 | 40,281 | 2,499 | 62.04 | 51,096 | 2,474 | 48.42 | \n| 3 | 35,046 | 2,337 | 66.68 | 46,701 | 2,447 | 52.40 | \n| 4 | 45,532 | 2,858 | 62.77 | 55,774 | 2,718 | 48.73 | \n\n*Table 3: Raw measurements from the logs*\n\nOverall, inference with DFlash trailed the Assistant model by under 20 seconds. The Assistant model was already a fairly well-optimized setup going in, which may be part of why the diffusion model's theoretical advantage didn't stand out here.\n\nAs before, output speed increased somewhat over the course of each turn for both the Assistant model and DFlash, but the Assistant model was faster throughout. (We're omitting the detailed per-token scatter plots for each model here, since the turn-by-turn averages above and the raw log data in Table 3 already capture the trend.) In a few cases we also observed DFlash's output speed peaking around 1,700 tokens and then dropping off slightly after that.\n\nAs in the previous verification, let's also look at the token acceptance rate.\n\n| Turn | Acceptance | Accepted | Generated | \n|---|---|---|---|\n| 1 | 41.03% | 1,331 | 3,244 | \n| 2 | 53.36% | 1,684 | 3,156 | \n| 3 | 62.02% | 1,744 | 2,812 | \n| 4 | 57.26% | 1,892 | 3,304 | \n\n*Table 4: DFlash token acceptance rate*\n\nComparing this against the Assistant model:\n\n|  | Assistant Acceptance | Assistant Accepted | Assistant Generated | DFlash Acceptance | DFlash Accepted | DFlash Generated | \n|---|---|---|---|---|---|---|\n| Max | 88.27% | 2,196 | 2,648 | 62.02% | 1,892 | 3,304 | \n| Min | 75.34% | 1,564 | 2,064 | 41.03% | 1,331 | 2,812 | \n| Avg | 81.06% | 1,868 | 2,305 | 53.42% | 1,663 | 3,129 | \n\n*Table 5: Acceptance rate comparison — Assistant (left) vs. DFlash (right)*\n\nThis suggests that DFlash's underwhelming result comes down to its lower acceptance rate. If model tuning progresses further and a faster-processing diffusion model becomes possible, DFlash might eventually surpass the Assistant model.\n\nThrough this verification, we confirmed that DFlash falls a bit short of the Assistant model. But maybe tweaking the parameters would speed things up? With that in mind, we tried the following.\n\nThe predicted-token count is currently set to 4. That's the same value DFlash used in the previous Assistant-model verification, kept for comparability. Raising this significantly might help — so we bumped `--spec-draft-n-max` from 4 to 15.\n\n```\n/opt/llama/bin/llama-server --model /opt/llama/models/gemma-4-12B-it-q4_0.gguf \\\n--model-draft /opt/llama/models/gemma-4-12B-it-DFlash-q4_0.gguf \\\n-t 4 --prio 2 --temp 1.0 --top-p 0.95 --top-k 64 --host 0.0.0.0 --port 8001 \\\n--device CUDA0 -mg 0 -sm layer --fit on -fa on -c 163840 -ctv q8_0 -ctk q8_0 \\\n--no-warmup --no-cache-prompt --cache-ram 0 \\\n--spec-type draft-dflash --spec-draft-n-max 15 --spec-draft-device CUDA0 \\\n--chat-template-kwargs '{\"enable_thinking\":true}'\n```\n\nHere's what came out of running it:\n\n```\nprompt eval time = 122.83 ms / 31 tokens ( 3.96 ms per token, 252.39 tokens per second)\neval time = 48833.63 ms / 1893 tokens ( 25.80 ms per token, 38.76 tokens per second)\ntotal time = 48956.46 ms / 1924 tokens\ngraphs reused = 1444\ndraft acceptance = 0.10901 ( 1174 accepted / 10770 generated), mean len = 2.64\n```\n\nThe complete opposite of what we hoped for: acceptance dropped, and speed dropped along with it.\n\nIt generated a lot more candidate tokens, but almost all of them were rejected — simply lengthening the prediction window clearly wasn't the answer.\n\nHere's the breakdown by turn:\n\n| Turn | Acceptance | Accepted | Generated | \n|---|---|---|---|\n| 1 | 10.90% | 1,174 | 10,770 | \n| 2 | 13.59% | 2,059 | 15,150 | \n| 3 | 19.54% | 2,090 | 10,695 | \n| 4 | 19.30% | 2,310 | 11,970 | \n\n|  | Acceptance | Accepted Tokens | Generated Tokens | \n|---|---|---|---|\n| Max | 19.54% | 2,310 | 15,150 | \n| Min | 10.90% | 1,174 | 10,695 | \n| Avg | 15.83% | 1,908 | 12,146 | \n\nThe `mean_len=2.64` value in the log above appears to represent roughly \"how many tokens tend to get accepted.\" Looking at the per-turn breakdown above, the average was 15.83%<sup>6</sup>, suggesting that `--spec-draft-n-max=4` was in fact the right setting for this model.\n\nWhat if we raised the model's own output precision instead? We tried rebuilding DFlash on that basis.\n\nFor this run, since we'd need to manually apply the same quantization level, we couldn't use an Unsloth Dynamic 2.0–quantized model the way we did for the Assistant model. Instead, we rebuilt using Gemma-4-12B-it-QAT (including the DFlash model), hoping it might improve performance.\n\nThe results, unfortunately, were not what we expected. Output was so slow that we stopped measuring after the second turn.\n\n```\nprompt eval time = 475.69 ms / 31 tokens ( 15.34 ms per token, 65.17 tokens per second)\neval time = 102682.33 ms / 1963 tokens (52.31 ms per token, 19.12 tokens per second)\ntotal time = 103158.02 ms / 1994 tokens\ngraphs reused = 1953\ndraft acceptance = 0.00025 ( 2 accepted / 7844 generated), mean len = 1.00\n```\n\nThis configuration is not viable: throughput dropped further, and the acceptance rate fell to 0.00025, or 0.25% — the lowest we observed in this test. With `mean_len=1.00`, essentially every generated token was rejected, a clear indication that DFlash should not be used in this configuration.\n\nIt looks like the DFlash model Z-Lab provided is really only usable with the vanilla Gemma-4-12B-it model. That's unfortunate — if that's the case, the Assistant model, which comes with a properly matched MTP model for each release, seems like the more practical choice.\n\nThe likely cause here is that Gemma-4-Assistant's draft model is simply fast enough that it processed tokens more quickly than DFlash's draft model.\n\nAs covered earlier, the speed advantage a diffusion-based drafter like DFlash offers over a conventional MTP draft model comes from being able to output candidate tokens \"all at once.\" Against that, Gemma-Assistant has the following working strongly in its favor, which appears to have canceled out DFlash's advantage:\n\nFirst, Gemma-Assistant comes with KV-cache sharing built in from the start, while DFlash keeps an independent KV cache that has to be injected fresh every time. That's an advantage for Gemma-Assistant.\n\nOn top of that, Gemma-Assistant's hidden dimension is only 1,024, with a 4-layer structure. DFlash's hidden dimension, as noted at the end of this article, matches the main model at 3,840, with 5 layers — meaning its compute cost is far heavier than Gemma-Assistant's.\n\nHowever good DFlash's diffusion model is at generating tokens in a single batch, more compute per step still means more time spent per step.\n\nIn this case, we should conclude that Gemma-Assistant simply already had a more optimized setup, and DFlash wasn't able to get ahead of it.\n\nIt's also worth remembering that DFlash's original benchmark comparison was against EAGLE-3, not Gemma-Assistant — so it's possible Gemma-Assistant is simply a stronger comparison baseline than the one DFlash was originally built to outperform.\n\nThat said, while the established option produced better results this time, it's possible a pairing with, say, Qwen3.5 would have produced a more favorable result.\n\nThis time we introduced DFlash, a technique from the UC San Diego (UCSD) research team Z-Lab that takes the \"MTP\" technology covered previously and pushes it in a new direction. Applying a diffusion model to a draft model was a genuinely ambitious idea, but in this test it wasn't able to beat Gemma's native draft model, Assistant.\n\nTracing the cause, the token acceptance rate was lower than the native draft model's, and that penalty appears to have been the dominant factor. Without a way to understand how to raise that acceptance rate, it's hard to say at this point whether further speed gains are realistic.\n\nWe also think the matchup itself didn't help. Gemma-4's Assistant model is built with practicality in mind and makes full use of Gemma-4's shared KV-cache mechanism, whereas DFlash, as before, has to write to an independent KV cache — and that overhead seems to have tipped the balance toward lower throughput.\n\nOn the broader question of putting diffusion models to work in LLMs, LLaDa and Dream are the well-known names, but more recently Google itself has released a model called Diffusion Gemma. We're currently in the middle of testing it ourselves, and the throughput numbers so far are notably high. We plan to cover those results in a dedicated article.\n\nAs Coding Agent usage keeps climbing, LLM throughput has become a common pain point, and rising VRAM costs have only sharpened the demand for models that can deliver solid throughput even on unified memory.\n\nThis kind of research is very much a \"fail your way forward\" field, and new techniques are emerging as we speak. We'll keep watching this space and try to keep up.\n\n**DFlash: Block Diffusion for Flash Speculative Decoding**\n\nJian Chen, Yesheng Liang, Zhijian Liu\n\n[https://arxiv.org/pdf/2602.06036](https://arxiv.org/pdf/2602.06036)\n\n**Accelerating Large Language Model Decoding with Speculative Sampling**\n\nDeepMind: Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre and John Jumper\n\n[https://arxiv.org/pdf/2302.01318](https://arxiv.org/pdf/2302.01318)\n\n**DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence**\n\nDeepSeek-AI\n\n[https://arxiv.org/pdf/2606.19348](https://arxiv.org/pdf/2606.19348)\n\n**[Speculative decoding] feat: add DFlash support**\n\n[https://github.com/ggml-org/llama.cpp/pull/22105](https://github.com/ggml-org/llama.cpp/pull/22105)\n\n*(We've omitted the architecture diagram here to keep the English edition concise — the description below covers the key points.)*\n\nThe DFlash model, which adds MTP capability to Gemma-4-12B-it, is architecturally distinct from Gemma-4-Assistant: rather than sharing a KV cache the way Assistant does, it's a fully separate model. In terms of its activation function and related choices, it's actually closer to a Qwen-style model. Its LA/GA notation follows Gemma-4's own convention — LA meaning Sliding Window Attention and GA meaning Full Attention.\n\nDuring prediction, state vectors are pulled from layers 1, 10, 19, 27, 36, and 45, concatenated, and normalized. That combined information is then injected into the draft side's KV cache.\n\nThe input to the draft model is a query built by combining the embedding vectors of the N confirmed tokens with as many MASK tokens as the number of predictions needed.\n\nUnlike the Assistant model, this approach uses non-causal attention, and internally relies on Masked Diffusion — that's the key structural difference.\n\nThis mechanism lets DFlash derive all of its predicted tokens at once; from there, the rest of the pipeline follows the same logic as the Assistant model, outputting whichever tokens get accepted.\n\nEAGLE3 is version 3 of the EAGLE (Extrapolation Algorithm for Greater Language Model Efficiency) series, documented in the following papers:\n\nDevelopment has been led by Yuhui Li of Peking University, first author on these papers, together with a joint research team spanning Peking University, Microsoft Research, the University of Waterloo, and the Vector Institute. In the open-source community, they operate under the name SafeAILab.\n\nSuccessor models are also in development: SafeAILab released EAGLE3.1 in May 2026, and a team at AWS AI Labs has developed P-EAGLE (Parallel-Drafting EAGLE), released in February 2026.\n\nBoth treat the sequential bottleneck of causal attention as the core problem, and both are focused on how to parallelize that part of the pipeline further.\n\n*This article is an English adaptation of the original Japanese post published on Zenn: [\"ドラフトトークンを並列生成する拡散モデル方式「DFlash」をGemmaで試した\"](https://zenn.dev/highreso/articles/ab3dbc20ce58ff), by Yuichi Tominaga.*\n\n[https://arxiv.org/pdf/2606.19348](https://arxiv.org/pdf/2606.19348) ↩\n\n[https://zhijianliu.com/](https://zhijianliu.com/), [https://z-lab.ai/](https://z-lab.ai/) ↩\n\n[https://arxiv.org/pdf/2602.06036](https://arxiv.org/pdf/2602.06036) ↩\n\n[https://arxiv.org/pdf/2302.01318](https://arxiv.org/pdf/2302.01318) ↩\n\n[https://huggingface.co/elyza/ELYZA-Diffusion-Instruct-1.0-Dream-7B](https://huggingface.co/elyza/ELYZA-Diffusion-Instruct-1.0-Dream-7B) ↩\n\nAverage acceptance rate across the four turns shown in the table below. ↩", "url": "https://wpnews.pro/news/trying-dflash-a-diffusion-model-approach-to-parallel-draft-token-generation-on", "canonical_source": "https://dev.to/oooocean66/trying-dflash-a-diffusion-model-approach-to-parallel-draft-token-generation-on-gemma-41o8", "published_at": "2026-09-14 07:02:56+00:00", "updated_at": "2026-09-14 07:31:40.198629+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "machine-learning", "ai-infrastructure", "developer-tools"], "entities": ["DFlash", "Z-Lab", "Zhijian Liu", "UC San Diego", "NVIDIA", "Gemma-4-12B-it", "llama.cpp", "EAGLE-3"], "alternates": {"html": "https://wpnews.pro/news/trying-dflash-a-diffusion-model-approach-to-parallel-draft-token-generation-on", "markdown": "https://wpnews.pro/news/trying-dflash-a-diffusion-model-approach-to-parallel-draft-token-generation-on.md", "text": "https://wpnews.pro/news/trying-dflash-a-diffusion-model-approach-to-parallel-draft-token-generation-on.txt", "jsonld": "https://wpnews.pro/news/trying-dflash-a-diffusion-model-approach-to-parallel-draft-token-generation-on.jsonld"}}