{"slug": "tencent-dspine-faster-speculative-decoding-for-qwen3-8-27b", "title": "Tencent DSpine: Faster Speculative Decoding for Qwen3.8-27B", "summary": "Tencent released DSpine, a five-layer open-weight draft model for speculative decoding that speeds up inference on Qwen3.8-27B by roughly 2.7x to 3.8x over autoregressive decoding on Tencent's WorkBuddy agent benchmarks, the company reported. DSpine proposes blocks of up to 16 tokens at once using an \"adjacent causal injection\" technique that passes predicted features between neighboring positions across layers, and it beat the prior DFlash2 draft model on acceptance length, including 7.20 versus 5.69 on GSM8K at greedy decoding. The checkpoint (tencent/Qwen3.8-27B-DSpine-b16) ships as a drop-in SGLang speculative-decoding backend under Apache 2.0 with no changes to the base Qwen3.8-27B weights, alongside an arXiv technical report and GitHub repo.", "body_md": "# Tencent DSpine: Faster Speculative Decoding for Qwen3.8-27B\n\nTencent's DSpine draft model speeds up Qwen3.8-27B inference up to 3.8x over autoregressive decoding. Here's how it works and how to deploy it.\n\n## What is DSpine?\n\nDSpine is a draft-model architecture from Tencent for speculative decoding, released as an open-weight checkpoint paired with Qwen3.8-27B. It generates multiple candidate tokens per step in parallel, then lets the full model verify them in a single pass, which cuts the number of expensive forward passes needed to produce long outputs. In Tencent’s published benchmarks, it beats the prior DFlash2 draft model on both acceptance length and end-to-end throughput, reaching up to roughly 3.8x speedup over plain autoregressive decoding on agent workloads.\n\n## TL;DR\n\n- **DSpine** is a five-layer draft model built for Qwen3.8-27B that proposes blocks of up to 16 tokens at once instead of decoding one token at a time.\n- The architecture uses **adjacent causal injection** , passing predicted features from each position to its neighbor across layers so parallel draft tokens still respect causal order.\n- On general benchmarks like GSM8K and HumanEval, DSpine’s **acceptance length** (tokens accepted per verification step) consistently beats DFlash2, for example 7.20 versus 5.69 on GSM8K at greedy decoding.\n- On Tencent’s **WorkBuddy agent benchmarks** (Code, Web, Office) plus SWE-bench Pro and tau-squared-bench, DSpine delivers end-to-end throughput speedups between roughly 2.7x and 3.8x over autoregressive decoding, outpacing DFlash2 in every tested configuration.\n- DSpine ships as a drop-in **SGLang** speculative-decoding backend, configurable via command-line flags with no changes to the base Qwen3.8-27B weights.\n- The checkpoint and code are released under **Apache 2.0** , with the technical report posted to arXiv alongside a GitHub repo and project page.\n\n## How does DSpine’s draft architecture work?\n\n*regulatory compliance and security.*\n\nSpeculative decoding works by having a small, fast “draft” model guess several tokens ahead, which a larger target model then verifies in one batched step. If the draft guesses correctly often enough, you get several tokens for roughly the cost of one full forward pass. The catch is that draft models historically struggle to parallelize their own token generation without breaking the causal dependency between tokens. Position 5 normally needs to know what happened at position 4.\n\nDSpine’s answer, described in its documentation as “adjacent causal injection,” has each of its five layers pass a predicted feature for one position directly to the next position, layer by layer. That lets the model generate a whole block of draft tokens at once while still threading causal information through network depth rather than through strict sequential decoding. The released checkpoint uses a block size of 16 (DSpine proposes up to 16 tokens per round) and supports an 8,192-token sliding window for long-context generation.\n\nOn the serving side, DSpine’s inference uses dynamic verification referencing a technique called DSpark, and it can enable something called ReplaySSM, which appears to optimize caching behavior in the speculative decoding loop. The draft model itself is distributed as a single safetensors checkpoint (`tencent/Qwen3.8-27B-DSpine-b16`) that sits alongside the base Qwen3.8-27B weights rather than replacing them.\n\n## How much faster is DSpine than DFlash2?\n\nTencent benchmarked DSpine directly against DFlash2, the prior open draft model for the same base model, using identical SGLang 0.5.22 infrastructure, BF16 precision, FlashAttention 3, and concurrency of 8. The comparison used a block size of 16 for DSpine versus 8 for DFlash2, DFlash2’s officially recommended setting.\n\nOn general-purpose benchmarks, DSpine posted higher acceptance length across every task tested, at both greedy decoding (temperature 0) and sampled decoding (temperature 1):\n\n- GSM8K: 7.20 vs 5.69 (greedy)\n- MATH-500: 6.49 vs 5.49 (greedy)\n- HumanEval: 5.71 vs 4.87 (greedy)\n- MBPP: 6.14 vs 5.04 (greedy)\n- MT-Bench: 4.48 vs 4.11 (greedy)\n\nAcceptance length measures how many draft tokens, on average, the target model accepts before rejecting one and falling back to its own generation. Higher is better because it means fewer wasted draft tokens and fewer verification rounds per output.\n\nThe gap holds up under more realistic, tool-using workloads too. On Tencent’s WorkBuddy Bench (covering coding, web, and office-document agent tasks) plus the external SWE-bench Pro and tau-squared-bench suites, DSpine’s acceptance length beat DFlash2 across the board, for instance 5.63 vs 4.24 on WorkBuddy Office at low reasoning effort, and 5.53 vs 4.70 on tau-squared-bench.\n\n## What does the throughput improvement look like in practice?\n\nAcceptance length is a useful proxy, but the number builders actually care about is wall-clock throughput. Tencent measured end-to-end tokens-per-second across concurrent request replay, comparing plain autoregressive (AR) decoding, DFlash2, and DSpine on the same agent benchmarks.\n\nThe results show consistent gains:\n\n- WorkBuddy Code: AR at 193-253 tok/s, DFlash2 at 2.33x-2.85x, DSpine at 2.71x-3.18x\n- WorkBuddy Web: DFlash2 at 2.59x-2.91x, DSpine at 3.13x-3.33x\n- WorkBuddy Office: DFlash2 at 3.08x-3.17x, DSpine reaching **3.84x** , the highest multiplier reported\n- SWE-bench Pro: DFlash2 at 2.72x-2.87x, DSpine at 2.95x-3.19x\n- tau-squared-bench: DFlash2 at 2.56x-2.96x, DSpine at 2.87x-3.20x\n\nThe pattern is the same throughout: DSpine’s larger draft blocks and causal-injection architecture translate higher acceptance rates into real throughput gains, typically adding another 0.3x-0.7x of speedup on top of what DFlash2 already delivers over plain decoding. The biggest gains show up on WorkBuddy Office, where DSpine nearly quadruples raw decoding speed.\n\n## How do you deploy DSpine?\n\n*Without shadow IT.*\n\nDSpine is built to run as an SGLang speculative-decoding backend rather than a standalone serving stack. Setup involves cloning Tencent’s DSpine GitHub repo, installing it as a Python package, and pointing SGLang at both the base Qwen3.8-27B weights and the DSpine draft checkpoint:\n\n```\nsglang serve \\\n  --model-path Qwen/Qwen3.8-27B \\\n  --speculative-algorithm DSPINE \\\n  --speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \\\n  --speculative-num-draft-tokens 16 \\\n  --dtype bfloat16 \\\n  --attention-backend fa3 \\\n  --speculative-draft-attention-backend fa3 \\\n  --linear-attn-backend triton \\\n  --enable-linear-replayssm-spec \\\n  --mamba-ssm-dtype float32 \\\n  --mamba-radix-cache-strategy extra_buffer \\\n  --max-mamba-cache-size 64 \\\n  --mem-fraction-static 0.80 \\\n  --context-length 131072 \\\n  --max-running-requests 8 \\\n  --cuda-graph-max-bs-decode 8 \\\n  --min-free-slots-delay 1 \\\n  --sampling-backend flashinfer \\\n  --reasoning-parser qwen3 \\\n  --tool-call-parser qwen3_coder \\\n  --trust-remote-code\n```\n\nKey flags to note: `--speculative-num-draft-tokens 16` matches the checkpoint’s block size, `--enable-linear-replayssm-spec` turns on the ReplaySSM optimization, and the `--mamba-*` flags configure cache behavior for that component. The setup requires FlashAttention 3 and the FlashInfer sampling backend, and it was validated against SGLang version 0.5.22, so matching that version (or newer compatible releases) is worth checking before deployment.\n\n## Is DSpine worth adopting over DFlash2?\n\nFor teams already running Qwen3.8-27B on SGLang with speculative decoding, the benchmarks suggest DSpine is a straightforward upgrade: better acceptance length, better throughput, same base model, same serving framework, open license. The tradeoffs to weigh are practical rather than architectural. DSpine’s larger block size (16 vs 8) and ReplaySSM caching add configuration surface area, and the deployment command has more moving parts than a vanilla SGLang setup. Teams running other base models won’t benefit directly, since this checkpoint is trained specifically against Qwen3.8-27B’s weights.\n\nThe benchmarks are also Tencent’s own, run on their chosen hardware and concurrency settings (concurrency of 8 throughout), so actual gains at different batch sizes, context lengths, or GPU configurations may vary. Still, the fact that the comparison was run head-to-head against a named, specific competitor (DFlash2) on identical infrastructure makes the numbers easier to trust than vaguer claims.\n\n## Frequently Asked Questions\n\n### What is speculative decoding and why does it matter?\n\nSpeculative decoding speeds up large language model inference by using a small draft model to propose several tokens ahead, which the full target model then verifies in one batched pass instead of generating each token sequentially. When the draft’s guesses are accurate, this cuts the number of expensive forward passes needed per output, directly lowering latency and increasing throughput without changing the target model’s outputs.\n\n### How is DSpine different from DFlash2?\n\nBoth are draft models for Qwen3.8-27B used in speculative decoding, but DSpine uses a five-layer “adjacent causal injection” architecture that passes predicted features between neighboring positions across layers, letting it propose larger blocks of tokens (16 vs DFlash2’s 8) while maintaining higher acceptance length and throughput in Tencent’s published benchmarks.\n\n### What hardware or software do I need to run DSpine?\n\nDSpine is designed to run through SGLang (version 0.5.22 was used in Tencent’s benchmarks) with BF16 precision and FlashAttention 3. It requires the base Qwen3.8-27B model weights plus the separate DSpine draft checkpoint, and the FlashInfer sampling backend for full compatibility with the published configuration.\n\n### Does DSpine change the output quality of Qwen3.8-27B?\n\n*Nothing gets lost.*\n\n**312 apps** across 41 teams · all in one place\n\nSpeculative decoding, including DSpine, is designed to be a lossless acceleration technique: the target model still verifies every proposed token, so the final output distribution matches what the base model would have produced on its own. The benefit is purely in reducing the number of sequential forward passes needed to generate that output.\n\n### Where can I get the DSpine model and code?\n\nThe checkpoint `tencent/Qwen3.8-27B-DSpine-b16` is hosted on Hugging Face under an Apache 2.0 license, with accompanying code on Tencent’s GitHub repository and a technical report posted to arXiv describing the architecture in detail.", "url": "https://wpnews.pro/news/tencent-dspine-faster-speculative-decoding-for-qwen3-8-27b", "canonical_source": "https://www.mindstudio.ai/blog/tencent-dspine-speculative-decoding/", "published_at": "2026-11-12 00:00:00+00:00", "updated_at": "2026-11-12 10:23:40.984315+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-research", "mlops"], "entities": ["Tencent", "DSpine", "Qwen3.8-27B", "DFlash2", "SGLang", "WorkBuddy", "GSM8K", "HumanEval"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tencent-dspine-faster-speculative-decoding-for-qwen3-8-27b", "markdown": "https://wpnews.pro/news/tencent-dspine-faster-speculative-decoding-for-qwen3-8-27b.md", "text": "https://wpnews.pro/news/tencent-dspine-faster-speculative-decoding-for-qwen3-8-27b.txt", "jsonld": "https://wpnews.pro/news/tencent-dspine-faster-speculative-decoding-for-qwen3-8-27b.jsonld"}}