Currently LLMs generate text in an autoregressive way—one token after another. Each successive token depends on the tokens before it. However with DFlash, an innovation created by Z-Lab, we can use diffusion and speed up LLM token generation.
With DFlash, a diffusion model is used to speculate what tokens the underlying LLM will generate next. Then if those tokens are correct, they are accepted—this is the same as other forms of speculative decoding, like MTP, but with a diffusion model as well.
I downloaded the raw weights of Qwen 3 8B and quantized them along with a DFlash model. I found:
• Using DFlash in llama-cpp
gives a clear speedup generating code—for example, having the model generate Java code went from 58 to 123 tokens per second.
• Other text, like thinking text or informational text, is sped up slightly but not as much—it seems no better than MTP in this case.
So with DFlash, we have a new state of the art for code generation. Unfortunately, DFlash uses VRAM, so it works best on models that are not filling up your GPU. Basically it seems to make models that are already fast (because they fit on the GPU) even faster.