cd /news/artificial-intelligence/dneo-your-parallel-draft-model-needs… · home › topics › artificial-intelligence › article
[ARTICLE · art-171523] src=arxiv.org ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

DNeo: Your Parallel Draft Model Needs Better Target Feature Encoding

A new speculative-decoding architecture called DNeo generates text 1.3x faster than the prior state-of-the-art DFlash-2 without increasing drafter footprint or degrading generation quality, according to an arXiv paper (2611.21925v1). DNeo introduces Mixture-of-Features Attention (MoFA), a lightweight feature-encoding method with a working memory that records previously rejected draft proposals, addressing three limits the authors identify in standard feature encoding: lossy projections that cause "residual mixing," no memory of prior proposals, and uniform sampling of target-layer features. The architecture adds 2-tap causal convolutions, prefix-only cross-attention and non-uniform target layer selection to speed parallel draft-token verification.

recommendation

When building a parallel draft model for speculative decoding, replace lossy uniform feature projections with MoFA plus rejection memory and non-uniform target layer selection to gain 1.3x speed at equal quality.

more in playbook: Inference and self-hosting
by read1 min views6 publishedNov 18, 2026

arXiv:2611.21925v1 Announce Type: new Abstract: Speculative-diffusion decoding uses a small parallel drafter model to generate b draft tokens in one forward pass, which then undergo parallel verification via a larger target model. Faster inference can be achieved when multiple draft tokens pass verification simultaneously. Recent parallel draft architectures (DFlash, DSpark) improve draft quality by conditioning on rich features computed by the target, through a mechanism known as feature encoding. However, standard feature encoding mechanisms have three limitations: (1) they are lossy, applying encoding projections that are destructive to the underlying target feature-space via a discovered effect known as residual mixing, (2) they are memory-less, with no record of previously processed proposals, and (3) they utilize features sampled uniformly from across the target layers, omitting important hidden states. To address these issues, we propose Mixture-of-Features Attention (MoFA), a lightweight feature-encoding approach equipped with a working memory that encodes previously rejected proposals. Leveraging MoFA, rejection memory, and a set of further enhancements, including 2-tap causal convolutions, prefix-only cross-attention, and non-uniform target layer selection, the final architecture, DNeo, achieves 1.3x faster generation speed than SoTA (DFlash-2) without impacting drafter footprint or generation quality. DNeo marks a new state of the art in high quality, fast LLM inference.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @dneo 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/dneo-your-parallel-d…] indexed:0 read:1min 2026-11-18 · —