Tencent's DSpine is a 2B draft model that speeds up Qwen3.8-27B decoding by nearly 5x. Here's how its layer-wise design beats DFlash2.
What is DSpine? #
DSpine is a draft model built by Tencent that speeds up text generation from the Qwen3.8-27B language model. It’s a small helper network, around 2 billion parameters, that sits alongside the much larger 27 billion parameter model and predicts blocks of upcoming tokens before the big model has to compute them one at a time. The big model then verifies those guesses in a single pass, keeping whatever turns out correct. In local testing with SGLang on an H100 GPU, this setup pushed Qwen3.8-27B from about 33 tokens per second to around 156 tokens per second, a speedup of roughly 4.8x. Tencent has released DSpine under the Apache 2.0 license.
TL;DR #
- DSpine is a 2B parameter draft model that works alongside Qwen3.8-27B to speed up inference without changing the big model’s outputs.
- It uses layer-wise causal injection , meaning each position in a draft block passes its guess to its neighbor at every layer of the network, not just at the end.
- Local testing showed roughly 4.8x throughput gains , taking Qwen3.8-27B from about 33 tokens/second to 156 tokens/second on an H100.
- DSpine outperforms DFlash2 on acceptance length , the average number of draft tokens the big model accepts per round, across both general benchmarks like GSM8K and MATH-500 and agent benchmarks like SWE-bench Pro.
- Tencent reports around 11% higher throughput than DFlash2 on average for Qwen3.8-27B, with end-to-end agent benchmarks showing speedups of 2.7x to 3.8x over standard autoregressive decoding.
- It runs through SGLang with a documented serving command, requiring specific flags for speculative decoding, linear attention, and Mamba cache settings.
- The model needs a capable GPU to run well , with the creator noting it worked on an H100 but not on an A100.
Without shadow IT.
Every app, approved
✓
✓
✓
✓
✓
How does speculative decoding work? #
Large language models normally generate one token at a time. Each token requires a full forward pass through every layer of the network, which is slow, especially for a 27 billion parameter model. Speculative decoding speeds this up by introducing a smaller, faster “draft” model that guesses several tokens ahead. The big model then checks all of those guessed tokens in a single pass instead of generating them one by one.
If the guesses are right, the big model accepts them and moves on. The moment it hits a wrong guess, it keeps everything before that point, corrects the first error, and discards the rest. The final output is identical to what the big model would have produced on its own, it’s just arrived at faster. The key metric here is “acceptance length,” the average number of draft tokens accepted per verification round. The higher that number, the fewer expensive passes through the big model are needed, and the faster the whole system runs.
How does DSpine’s architecture differ from DFlash, DFlash2, and Eagle3? #
Different speculative decoding methods make different trade-offs between speed and guess quality, and that trade-off comes down to how the draft tokens are generated.
Eagle3-style helpers generate one token at a time. This is careful, since each guess can take into account what came before it, but it’s slow because there’s no parallelism in the drafting step itself.
DFlash generates an entire block of guesses in one parallel pass, which is fast, but each position in that block is predicted independently, without knowledge of its neighbors. That means tokens within the same block can contradict each other, and the big model stops accepting as soon as it hits the first mismatch, which caps how many tokens get accepted per round.
Newer approaches like Domino, DSpark, and DFlash2 try to fix this by adding a correction step after the main parallel pass, a kind of cleanup pass that reconciles the guesses.
DSpine takes a different approach. According to Tencent’s model card, it uses “adjacent causal injection,” where each layer of the draft network passes predicted predecessor features to neighboring positions as the guesses move through the network’s depth. In plainer terms, instead of guessing a block of tokens independently and fixing it up afterward, DSpine lets each token’s guess inform its neighbor’s guess at every layer, so the whole block is refined together as it’s being built, not corrected after the fact. At the end, DSpine picks tokens left to right using a lookup table, which keeps the final selection fast. The released checkpoint for Qwen3.8-27B uses five layers, a block size of 16, and an 8,192 token sliding window.
How much faster is DSpine than DFlash2? #
Tencent’s own benchmarks, run with SGLang, BF16 precision, and FlashAttention 3, compare DSpine against the official DFlash2 draft model for Qwen3.8-27B. The comparison used a draft block size of 16 for DSpine versus 8 for DFlash2.
mission-criticalenterprise applications.
Dr. A. Okafor Cardiology · St. Mercy Hospital
- Patient records*✓ Granted*
- Surgery schedule*✓ Granted*
- Billing export*✕ Denied*
On general benchmarks measuring acceptance length (how many draft tokens get accepted per verification round), DSpine beat DFlash2 across every task tested at both greedy decoding (temperature 0) and sampling (temperature 1):
- GSM8K: 7.20 vs 5.69 (greedy)
- MATH-500: 6.49 vs 5.49 (greedy)
- HumanEval: 5.71 vs 4.87 (greedy)
- MBPP: 6.14 vs 5.04 (greedy)
- MT-Bench: 4.48 vs 4.11 (greedy)
On agent-style benchmarks, which simulate tool-use conversations like coding and web tasks, DSpine’s advantage held up too. On WorkBuddy Office under low reasoning effort, DSpine hit an acceptance length of 5.63 versus DFlash2’s 4.24. On SWE-bench Pro, DSpine reached 4.99 versus 4.19.
End-to-end throughput numbers (measured in tokens per second against standard autoregressive decoding) show the practical payoff. On WorkBuddy Office with low reasoning, DSpine delivered a 3.84x speedup over autoregressive decoding compared to DFlash2’s 3.17x. On τ²-bench, DSpine reached 3.20x versus DFlash2’s 2.96x. Tencent reports this averages out to roughly 11% higher throughput than DFlash2 across the board for Qwen3.8-27B.
How was DSpine tested locally, and what did it show? #
Independent local testing using SGLang on an H100 GPU ran a set of eight prompts through Qwen3.8-27B, first with DSpine enabled as the draft model, then again without any speculative decoding helper at all.
With DSpine active, the model generated at an overall rate of about 156 tokens per second, with individual prompts ranging from roughly 95 to 235 tokens per second. Without DSpine, running Qwen3.8-27B on its own, the same test dropped to about 33 tokens per second. That works out to roughly a 4.8x speedup from adding the draft model, closely matching the kind of gains Tencent reports in its own agent benchmarks.
VRAM usage also shifted: with DSpine loaded alongside the big model, consumption sat just under 79 GB on an 80 GB H100. Without the draft model, VRAM usage dropped to just under 72 GB. The creator also noted the setup didn’t work on an A100 GPU, suggesting hardware compatibility matters here, likely tied to the attention backends and Mamba-related flags required by the serving command.
Is DSpine worth using? #
For anyone already serving Qwen3.8-27B and looking for faster inference without retraining or quantizing the base model, DSpine looks like a solid option based on the numbers available. It’s open-sourced under Apache 2.0, integrates directly with SGLang, and the benchmarks, both Tencent’s official ones and independent local testing, point to real, substantial throughput gains over running the 27B model alone, and modest but consistent gains over DFlash2 specifically. The catch is the setup overhead. Serving DSpine requires a fairly involved SGLang command with specific flags for speculative decoding, linear attention backends, and Mamba cache configuration, plus a GPU capable enough to handle it (the H100 worked, the A100 reportedly did not). For teams running inference at scale on compatible hardware, that setup cost is likely worth it. For casual or resource-constrained use, the bar to entry is higher.
Frequently Asked Questions #
What is a draft model in speculative decoding?
Without shadow IT.
A draft model is a smaller, faster neural network that predicts several upcoming tokens before the main, larger model verifies them. Because checking a block of guesses in parallel is cheaper than generating each token sequentially, this can significantly speed up generation without changing the final output.
How big is the DSpine draft model compared to Qwen3.8-27B?
DSpine is about 2 billion parameters, compared to Qwen3.8-27B’s 27 billion parameters. Despite being roughly 13 times smaller, it’s able to accurately predict multiple tokens ahead for the larger model.
Does DSpine change the output quality of Qwen3.8-27B?
No. Speculative decoding, including DSpine, is designed to produce outputs identical to what the base model would generate on its own. The big model always verifies draft tokens and corrects any mismatches, so speed improves without any change in output quality.
What GPU do you need to run DSpine?
Local testing confirmed it running on an H100 GPU with around 79 GB of VRAM consumed when serving Qwen3.8-27B with DSpine. The same setup reportedly failed on an A100 GPU, so hardware compatibility should be checked before deployment.
How is DSpine different from DFlash2?
DFlash2 generates a block of draft tokens in parallel and then applies a correction step afterward to fix inconsistencies between neighboring guesses. DSpine instead lets each position’s guess inform its neighbors at every layer of the network as the block is built, an approach Tencent calls adjacent causal injection, which the company’s benchmarks show results in higher acceptance rates and better throughput.