Qwen3.8-27B DSpine-b16: Tencent's Open 2B Drafter for Faster Local Inference

Tencent published Qwen3.8-27B-DSpine-b16 on Hugging Face, a roughly 2B-parameter speculative-decoding drafter that pairs with the Qwen3.8-27B target model under an Apache 2.0 license. The five-layer drafter uses block size 16 and an 8,192-token sliding window, and Tencent's model card reports acceptance lengths above DFlash2 (block 8) across GSM8K, MATH-500, HumanEval, MBPP and MT-Bench, plus up to roughly 3.8x autoregressive throughput on WorkBuddy Office (low) agent workloads. Tencent evaluated the drafter in SGLang 0.5.22 with BF16 weights, FlashAttention 3 and concurrency 8, and did not publish training code on the card.

Tencent published Qwen3.8-27B-DSpine-b16 https://huggingface.co/tencent/Qwen3.8-27B-DSpine-b16?ref=explainx on Hugging Face — a ~2B-parameter drafter that pairs with Qwen3.8-27B https://huggingface.co/Qwen/Qwen3.8-27B?ref=explainx for speculative decoding in SGLang. The October DSpine paper https://arxiv.org/abs/2609.36173?ref=explainx and our earlier Qwen3-8B DSpine guide https://explainx.ai/blog/tencent-dspine-speculative-decoding-drafter-qwen3-8b-27-percent-longer-acceptance-vs-dflash-2026 listed a 27B checkpoint as roadmap-only; that line item is now a downloadable Apache 2.0 artifact with benchmark tables on coding, chat and long agent runs. If you self-host Qwen3.8-27B for coding agents https://explainx.ai/blog/plannotator-plan-code-review-workspace-coding-agents-claude-code-codex-github-trending-2026 or batch jobs, the practical question is whether a small drafter pays for its VRAM. Tencent's card answers with acceptance-length tables and end-to-end throughput on WorkBuddy-style sandboxes — numbers you should reproduce on your GPU before you trust them for production. TL;DR | Question | Answer | |---|---| | What shipped? | Five-layer DSpine drafter, block size 16, 8,192-token sliding window | | Parent model | Qwen/Qwen3.8-27B separate download | | License | Apache 2.0 | | Runtime cited | SGLang 0.5.22, BF16, FlashAttention 3 | | Baseline on card | DFlash2 block 8 vs DSpine block 16 | | Agent story | WorkBuddy Code/Web/Office, SWE-bench Pro, τ²-bench throughput up to ~3.8× AR on Office low | | Training code | Not on the card; see Tencent/DSpine repo | What DSpine does in one paragraph Speculative decoding trades a cheap draft model for fewer full passes through a large target. Tencent's DSpine method https://explainx.ai/blog/tencent-dspine-speculative-decoding-drafter-qwen3-8b-27-percent-longer-acceptance-vs-dflash-2026 drafts a block of tokens in parallel but injects each position's predicted features into its neighbor at every layer — "adjacent causal injection" — so parallel guesses stay more consistent than classic parallel drafters like DFlash. The 27B checkpoint is the same architecture family as the 8B release, scaled to a larger target backbone. For token economics background, see what tokens per second actually measures https://explainx.ai/blog/what-is-tps-tokens-per-second-llm-speed-explained-2026 and how Baseten-style serving stacks https://explainx.ai/blog/baseten-deepseek-ultrafast-2060-tps-rust-inference-engine-2026 chase wall-clock wins. Architecture at a glance The Tencent model card https://huggingface.co/tencent/Qwen3.8-27B-DSpine-b16?ref=explainx describes a five-layer draft network, block size 16 , and an 8,192-token sliding window. Evaluation used SGLang 0.5.22 with BF16 weights, FlashAttention 3, concurrency 8, dynamic verification referencing DSpark with ReplaySSM enabled — details that matter if your install drifts from their stack. Benchmarks: acceptance length Tencent compares DFlash2 official 8-token blocks against DSpine 16-token blocks on general tasks at greedy and sampled decoding: | Task | DFlash2 T=0 | DSpine T=0 | DFlash2 T=1 | DSpine T=1 | |---|---|---|---|---| | GSM8K | 5.69 | 7.20 | 5.45 | 6.31 | | MATH-500 | 5.49 | 6.49 | 5.27 | 5.66 | | HumanEval | 4.87 | 5.71 | 4.34 | 4.71 | | MBPP | 5.04 | 6.14 | 4.79 | 5.41 | | MT-Bench | 4.11 | 4.48 | 3.89 | 4.03 | Longer accepted prefixes generally mean higher throughput when verification is cheap. These are Tencent-measured figures on their harness; your acceptance length moves with batch size, quantization and context length. Agent workloads: where the card pushes hardest The card's newer section targets agent trajectories , not just static benchmarks. Tencent built WorkBuddy Bench subsets Code, Web, Office using CodeBuddy sandbox replays: up to 32,768 new tokens per request, tool-call prefixes replayed in order, prefix caching on. Acceptance length low reasoning excerpt : | Benchmark | DFlash2 | DSpine | |---|---|---| | WorkBuddy Code | 4.19 | 5.08 | | WorkBuddy Web | 3.95 | 4.91 | | WorkBuddy Office | 4.24 | 5.63 | | SWE-bench Pro | 4.19 | 4.99 | | τ²-bench | 4.70 | 5.53 | End-to-end throughput tokens/s, speedup vs autoregressive in parentheses — low reasoning: | Benchmark | AR | DFlash2 | DSpine | |---|---|---|---| | WorkBuddy Code | 253 | 590 2.33× | 686 2.71× | | WorkBuddy Web | 247 | 640 2.59× | 773 3.13× | | WorkBuddy Office | 210 | 665 3.17× | 806 3.84× | | SWE-bench Pro | 242 | 658 2.72× | 713 2.95× | | τ²-bench | 301 | 891 2.96× | 964 3.20× | At xhigh reasoning the multipliers shrink slightly but stay above 2.5× on several rows. That pattern fits the intuition that long CoT traces stress verification more than short chat turns — similar to how SparseRead-style context trimming https://explainx.ai/blog/sparseread-token-efficient-agent-reading-layer-2026 attacks cost on the input side while drafters attack cost on the output side. A branching checkpoint stem illustrating a new open-weight drafter release for Qwen3.8-27B speculative decoding How to try it today 1. Clone and install Tencent/DSpine https://github.com/Tencent/DSpine?ref=explainx pip install -e . . 2. Download Qwen/Qwen3.8-27B and tencent/Qwen3.8-27B-DSpine-b16 about 4,873 downloads in the last month on the card when we read it . 3. Launch SGLang with the card's full flag set — notably DSPINE , 16 draft tokens, fa3 attention backends, qwen3 reasoning and tool parsers, and --enable-linear-replayssm-spec . sglang serve \ --model-path Qwen/Qwen3.8-27B \ --speculative-algorithm DSPINE \ --speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \ --speculative-num-draft-tokens 16 \ --dtype bfloat16 \ --attention-backend fa3 \ --speculative-draft-attention-backend fa3 \ --enable-linear-replayssm-spec VRAM planning: you hold both models. On a single GPU that often means quantization on the 27B parent, tensor parallel, or a second GPU for drafts — the same trade space as GLM-5.3 two-station vLLM setups https://explainx.ai/blog/glm-5-3-two-dgx-stations-vllm-tensor-parallel-753b-nvfp4-607-tokens-per-second-guide-2026 at a smaller scale. Pair this with llama.cpp v1.0 stability work https://explainx.ai/blog/llama-cpp-v1-0-0-stable-libllama-libggml-semantic-versioning-what-changes-2026 if you split local vs server paths: DSpine's documented path here is SGLang-first , not llama.cpp. DSpine vs DFlash2 vs waiting for vLLM | Path | When it fits | |---|---| | DSpine + SGLang | You already serve Qwen with SGLang; you want the card's exact algorithm flags | | DFlash2 | You want Tencent's prior parallel drafter baseline on the same parent | | Nvidia DFlash 2 on Muse Spark | Different ecosystem — see Nvidia's vLLM speculative recipe https://explainx.ai/blog/nvidia-muse-spark-open-dflash-2-draft-model-speculative-decoding-vllm-2026 | | No drafter | Simplicity wins when latency targets are already met | The Hugging Face card credits DFlash, SpecForge, SGLang and FlashInfer — useful breadcrumbs if you debug kernel mismatches. What developers should verify locally The model card's Hacker News thread https://news.ycombinator.com/item?id=50413991&ref=explainx when we checked had no substantive hands-on comments yet — early release noise only. Until community runs land: 1. Match SGLang version 0.5.22 cited . 2. Compare AR vs DSpine on your agent prompt mix, not only Tencent's benches. 3. Watch tool-call integrity ; speculative stacks can interact badly with parsers if drafts desynchronize mid-trajectory. 4. Track memory : ReplaySSM and mamba cache flags change footprint. Command reference: full model-card serve line Tencent's card is unusually explicit about flags because DSpine touches linear attention , mamba radix cache , and FlashInfer sampling in one stack. Missing one flag often silently disables the path you thought you enabled. | Flag | Role | |---|---| | --speculative-algorithm DSPINE | Selects adjacent-injection drafting | | --speculative-num-draft-tokens 16 | Matches b16 block training | | --enable-linear-replayssm-spec | Turns on ReplaySSM path cited in eval | | --linear-attn-backend triton | Linear attention kernel choice | | --mamba-ssm-dtype float32 | Stability for SSM state | | --mamba-radix-cache-strategy extra buffer | Cache strategy for speculative tree | | --max-mamba-cache-size 64 | Caps SSM cache footprint | | --reasoning-parser qwen3 | Parses Qwen3 reasoning channels | | --tool-call-parser qwen3 coder | Aligns tool traces with parent | If you already run Qwen3.8-27B on SGLang without speculation, diff your script against this table before blaming the drafter for slowdowns. Paper vs card: keep the baselines straight The arXiv paper https://arxiv.org/abs/2609.36173?ref=explainx established DSpine against DFlash on smaller Qwen3 checkpoints with different acceptance-length math than this 27B card. Our October Qwen3-8B post https://explainx.ai/blog/tencent-dspine-speculative-decoding-drafter-qwen3-8b-27-percent-longer-acceptance-vs-dflash-2026 explains why 27.8% vs DFlash and ~4% vs retrained DFlash2 can both appear in Tencent materials without contradiction. On the 27B card, the meaningful comparison is DFlash2 vs DSpine under the same Open-PerfectBlend retraining story for general benchmarks, plus the agent replay suite. Do not import 8B paper headlines into a 27B procurement slide without re-benchmarking. Memory and hardware planning Rough planning questions for a single host: | Component | Planning question | |---|---| | 27B parent BF16 | Does it fit beside a 2B drafter on one 48GB card, or do you quantize the parent? | | Concurrent requests | Card uses concurrency 8 — scaling users may need tensor parallel on the parent | | Context 131072 | Long agent runs need the card's context-length flag, not defaults | | ReplaySSM / mamba cache | Extra state beyond vanilla KV — watch OOM when enabling speculation | Teams already splitting parents across GPUs for tensor-parallel GLM jobs https://explainx.ai/blog/glm-5-3-two-dgx-stations-vllm-tensor-parallel-753b-nvfp4-607-tokens-per-second-guide-2026 can treat the drafter as a small satellite model pinned to the least-loaded device. Cost math sketch illustrative Suppose autoregressive decoding costs you $X per million output tokens on rented H100 time. A 2.7× throughput gain on WorkBuddy Code low — one row from the card — suggests the same wall-clock budget produces 2.7× more tokens if verification overhead stays stable. That is not automatic savings: - Draft model VRAM may force a second GPU or a smaller parent batch . - Failed speculative paths add wasted draft FLOPs . - Agent traces with tools may spend more time in CPU sandbox than in matmul. Still, for high-volume Rainshare-style quota sharing https://explainx.ai/blog/rainshare-mutual-aid-ai-coding-plan-quota-2026 or internal codegen fleets, a verified 2×+ speedup on Qwen is worth a weekend bake-off. When not to adopt yet Skip the upgrade path if: - You are on vLLM-only serving with no SGLang appetite. - Your workload is mostly embedding or rerank , not long generation. - You need bit-identical replay against a non-speculative baseline for compliance. - You cannot spare engineering time to pin SGLang 0.5.22 and the DSpine commit. In those cases, watch the checkpoint for community reports rather than rushing production. Research line: adjacent causal injection in plain language Parallel drafters guess many tokens at once, but guesses at position k often ignore what position k-1 already committed at early layers. DSpine routes predecessor features forward through depth so that by the time the block exits, later positions have seen a causal chain without serial token steps. That is why Tencent pairs the method with dynamic verification referencing DSpark on the card — verification and drafting co-evolved in their stack. Readers digging into kernels should start at github.com/Tencent/DSpine https://github.com/Tencent/DSpine?ref=explainx rather than treating the Hugging Face repo as the whole system. Limitations and honesty - Training code for custom drafters is still a moving target on the GitHub README. - Benchmarks are author-run ; we have not reproduced the 3.8× Office throughput claim. - Qwen3.8-27B parent weights are a separate download and license check from the drafter. - Block size 16 may not be optimal on every GPU class; treat 16 as the card default, not a universal law. What this means for builders Tencent is shipping open drafters on the same cadence as Qwen3.8 parents , which keeps pressure on closed APIs for cost per agent step. If DSpine's agent tables hold up in the wild, teams running self-hosted coding harnesses https://explainx.ai/blog/tencent-loopforge-self-hosted-sop-coding-agents-claude-codex-2026 on Qwen gain another knob besides quantization and batching. The near-term homework is boring and valuable: one benchmark script, one production-like trace, one cost line before you flip a fleet. Related reading Model card metrics and SGLang flags are accurate as of November 14, 2026. Re-check Hugging Face and github.com/Tencent/DSpine before you deploy.