{"slug": "qwen3-8-27b-dspine-b16-tencent-s-open-2b-drafter-for-faster-local-inference", "title": "Qwen3.8-27B DSpine-b16: Tencent's Open 2B Drafter for Faster Local Inference", "summary": "Tencent published Qwen3.8-27B-DSpine-b16 on Hugging Face, a roughly 2B-parameter speculative-decoding drafter that pairs with the Qwen3.8-27B target model under an Apache 2.0 license. The five-layer drafter uses block size 16 and an 8,192-token sliding window, and Tencent's model card reports acceptance lengths above DFlash2 (block 8) across GSM8K, MATH-500, HumanEval, MBPP and MT-Bench, plus up to roughly 3.8x autoregressive throughput on WorkBuddy Office (low) agent workloads. Tencent evaluated the drafter in SGLang 0.5.22 with BF16 weights, FlashAttention 3 and concurrency 8, and did not publish training code on the card.", "body_md": "**Tencent published [Qwen3.8-27B-DSpine-b16](https://huggingface.co/tencent/Qwen3.8-27B-DSpine-b16?ref=explainx) on Hugging Face — a ~2B-parameter drafter that pairs with [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B?ref=explainx) for speculative decoding in SGLang.** The October [DSpine paper](https://arxiv.org/abs/2609.36173?ref=explainx) and our earlier [Qwen3-8B DSpine guide](https://explainx.ai/blog/tencent-dspine-speculative-decoding-drafter-qwen3-8b-27-percent-longer-acceptance-vs-dflash-2026) listed a 27B checkpoint as roadmap-only; that line item is now a downloadable Apache 2.0 artifact with benchmark tables on coding, chat and long agent runs.\n\nIf you self-host Qwen3.8-27B for [coding agents](https://explainx.ai/blog/plannotator-plan-code-review-workspace-coding-agents-claude-code-codex-github-trending-2026) or batch jobs, the practical question is whether a small drafter pays for its VRAM. Tencent's card answers with acceptance-length tables and end-to-end throughput on WorkBuddy-style sandboxes — numbers you should reproduce on your GPU before you trust them for production.\n\n## TL;DR\n\n| Question | Answer | \n|---|---|\n| What shipped? | Five-layer DSpine drafter, block size 16, 8,192-token sliding window | \n| Parent model | Qwen/Qwen3.8-27B (separate download) | \n| License | Apache 2.0 | \n| Runtime cited | SGLang 0.5.22, BF16, FlashAttention 3 | \n| Baseline on card | DFlash2 (block 8) vs DSpine (block 16) | \n| Agent story | WorkBuddy Code/Web/Office, SWE-bench Pro, τ²-bench throughput up to ~3.8× AR on Office (low) | \n| Training code | Not on the card; see Tencent/DSpine repo | \n\n## What DSpine does (in one paragraph)\n\nSpeculative decoding trades a cheap draft model for fewer full passes through a large target. [Tencent's DSpine method](https://explainx.ai/blog/tencent-dspine-speculative-decoding-drafter-qwen3-8b-27-percent-longer-acceptance-vs-dflash-2026) drafts a block of tokens in parallel but injects each position's predicted features into its neighbor at every layer — \"adjacent causal injection\" — so parallel guesses stay more consistent than classic parallel drafters like DFlash.\n\nThe 27B checkpoint is the same architecture family as the 8B release, scaled to a larger target backbone. For token economics background, see [what tokens per second actually measures](https://explainx.ai/blog/what-is-tps-tokens-per-second-llm-speed-explained-2026) and how [Baseten-style serving stacks](https://explainx.ai/blog/baseten-deepseek-ultrafast-2060-tps-rust-inference-engine-2026) chase wall-clock wins.\n\n## Architecture at a glance\n\nThe [Tencent model card](https://huggingface.co/tencent/Qwen3.8-27B-DSpine-b16?ref=explainx) describes a **five-layer** draft network, **block size 16**, and an **8,192-token** sliding window. Evaluation used **SGLang 0.5.22** with BF16 weights, FlashAttention 3, concurrency 8, dynamic verification referencing DSpark with **ReplaySSM** enabled — details that matter if your install drifts from their stack.\n\n## Benchmarks: acceptance length\n\nTencent compares **DFlash2** (official 8-token blocks) against **DSpine** (16-token blocks) on general tasks at greedy and sampled decoding:\n\n| Task | DFlash2 (T=0) | DSpine (T=0) | DFlash2 (T=1) | DSpine (T=1) | \n|---|---|---|---|---|\n| GSM8K | 5.69 | 7.20 | 5.45 | 6.31 | \n| MATH-500 | 5.49 | 6.49 | 5.27 | 5.66 | \n| HumanEval | 4.87 | 5.71 | 4.34 | 4.71 | \n| MBPP | 5.04 | 6.14 | 4.79 | 5.41 | \n| MT-Bench | 4.11 | 4.48 | 3.89 | 4.03 | \n\nLonger accepted prefixes generally mean higher throughput when verification is cheap. These are **Tencent-measured** figures on their harness; your acceptance length moves with batch size, quantization and context length.\n\n## Agent workloads: where the card pushes hardest\n\nThe card's newer section targets **agent trajectories**, not just static benchmarks. Tencent built **WorkBuddy Bench** subsets (Code, Web, Office) using CodeBuddy sandbox replays: up to **32,768** new tokens per request, tool-call prefixes replayed in order, prefix caching on.\n\n**Acceptance length (low reasoning excerpt):**\n\n| Benchmark | DFlash2 | DSpine | \n|---|---|---|\n| WorkBuddy Code | 4.19 | 5.08 | \n| WorkBuddy Web | 3.95 | 4.91 | \n| WorkBuddy Office | 4.24 | 5.63 | \n| SWE-bench Pro | 4.19 | 4.99 | \n| τ²-bench | 4.70 | 5.53 | \n\n**End-to-end throughput (tokens/s, speedup vs autoregressive in parentheses) — low reasoning:**\n\n| Benchmark | AR | DFlash2 | DSpine | \n|---|---|---|---|\n| WorkBuddy Code | 253 | 590 (2.33×) | **686 (2.71×)** | \n| WorkBuddy Web | 247 | 640 (2.59×) | **773 (3.13×)** | \n| WorkBuddy Office | 210 | 665 (3.17×) | **806 (3.84×)** | \n| SWE-bench Pro | 242 | 658 (2.72×) | **713 (2.95×)** | \n| τ²-bench | 301 | 891 (2.96×) | **964 (3.20×)** | \n\nAt **xhigh** reasoning the multipliers shrink slightly but stay above 2.5× on several rows. That pattern fits the intuition that long CoT traces stress verification more than short chat turns — similar to how [SparseRead-style context trimming](https://explainx.ai/blog/sparseread-token-efficient-agent-reading-layer-2026) attacks cost on the input side while drafters attack cost on the output side.\n\nA branching checkpoint stem illustrating a new open-weight drafter release for Qwen3.8-27B speculative decoding\n\n## How to try it today\n\n1. Clone and install [Tencent/DSpine](https://github.com/Tencent/DSpine?ref=explainx) (`pip install -e .` ).\n2. Download **Qwen/Qwen3.8-27B** and**tencent/Qwen3.8-27B-DSpine-b16** (about 4,873 downloads in the last month on the card when we read it).\n3. Launch SGLang with the card's full flag set — notably `DSPINE` , 16 draft tokens,`fa3` attention backends,`qwen3` reasoning and tool parsers, and`--enable-linear-replayssm-spec` .\n\n```\nsglang serve \\\n  --model-path Qwen/Qwen3.8-27B \\\n  --speculative-algorithm DSPINE \\\n  --speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \\\n  --speculative-num-draft-tokens 16 \\\n  --dtype bfloat16 \\\n  --attention-backend fa3 \\\n  --speculative-draft-attention-backend fa3 \\\n  --enable-linear-replayssm-spec\n```\n\nVRAM planning: you hold **both** models. On a single GPU that often means quantization on the 27B parent, tensor parallel, or a second GPU for drafts — the same trade space as [GLM-5.3 two-station vLLM setups](https://explainx.ai/blog/glm-5-3-two-dgx-stations-vllm-tensor-parallel-753b-nvfp4-607-tokens-per-second-guide-2026) at a smaller scale.\n\nPair this with [llama.cpp v1.0 stability work](https://explainx.ai/blog/llama-cpp-v1-0-0-stable-libllama-libggml-semantic-versioning-what-changes-2026) if you split local vs server paths: DSpine's documented path here is **SGLang-first**, not llama.cpp.\n\n## DSpine vs DFlash2 vs waiting for vLLM\n\n| Path | When it fits | \n|---|---|\n| **DSpine + SGLang** | You already serve Qwen with SGLang; you want the card's exact algorithm flags | \n| **DFlash2** | You want Tencent's prior parallel drafter baseline on the same parent | \n| **Nvidia DFlash 2 on Muse Spark** | Different ecosystem — see [Nvidia's vLLM speculative recipe](https://explainx.ai/blog/nvidia-muse-spark-open-dflash-2-draft-model-speculative-decoding-vllm-2026) | \n| **No drafter** | Simplicity wins when latency targets are already met | \n\nThe Hugging Face card credits **DFlash, SpecForge, SGLang and FlashInfer** — useful breadcrumbs if you debug kernel mismatches.\n\n## What developers should verify locally\n\nThe model card's [Hacker News thread](https://news.ycombinator.com/item?id=50413991&ref=explainx) (when we checked) had **no substantive hands-on comments yet** — early release noise only. Until community runs land:\n\n1. Match **SGLang version** (0.5.22 cited).\n2. Compare **AR vs DSpine** on*your* agent prompt mix, not only Tencent's benches.\n3. Watch **tool-call integrity** ; speculative stacks can interact badly with parsers if drafts desynchronize mid-trajectory.\n4. Track **memory** : ReplaySSM and mamba cache flags change footprint.\n\n## Command reference: full model-card serve line\n\nTencent's card is unusually explicit about flags because DSpine touches **linear attention**, **mamba radix cache**, and **FlashInfer sampling** in one stack. Missing one flag often silently disables the path you thought you enabled.\n\n| Flag | Role | \n|---|---|\n| `--speculative-algorithm DSPINE` | Selects adjacent-injection drafting | \n| `--speculative-num-draft-tokens 16` | Matches b16 block training | \n| `--enable-linear-replayssm-spec` | Turns on ReplaySSM path cited in eval | \n| `--linear-attn-backend triton` | Linear attention kernel choice | \n| `--mamba-ssm-dtype float32` | Stability for SSM state | \n| `--mamba-radix-cache-strategy extra_buffer` | Cache strategy for speculative tree | \n| `--max-mamba-cache-size 64` | Caps SSM cache footprint | \n| `--reasoning-parser qwen3` | Parses Qwen3 reasoning channels | \n| `--tool-call-parser qwen3_coder` | Aligns tool traces with parent | \n\nIf you already run Qwen3.8-27B on SGLang without speculation, diff your script against this table before blaming the drafter for slowdowns.\n\n## Paper vs card: keep the baselines straight\n\nThe [arXiv paper](https://arxiv.org/abs/2609.36173?ref=explainx) established DSpine against **DFlash** on smaller Qwen3 checkpoints with different acceptance-length math than this 27B card. Our [October Qwen3-8B post](https://explainx.ai/blog/tencent-dspine-speculative-decoding-drafter-qwen3-8b-27-percent-longer-acceptance-vs-dflash-2026) explains why **27.8% vs DFlash** and **~4% vs retrained DFlash2** can both appear in Tencent materials without contradiction.\n\nOn the 27B card, the meaningful comparison is **DFlash2 vs DSpine** under the same Open-PerfectBlend retraining story for general benchmarks, plus the **agent replay** suite. Do not import 8B paper headlines into a 27B procurement slide without re-benchmarking.\n\n## Memory and hardware planning\n\nRough planning questions for a single host:\n\n| Component | Planning question | \n|---|---|\n| **27B parent BF16** | Does it fit beside a 2B drafter on one 48GB card, or do you quantize the parent? | \n| **Concurrent requests** | Card uses concurrency 8 — scaling users may need tensor parallel on the parent | \n| **Context 131072** | Long agent runs need the card's context-length flag, not defaults | \n| **ReplaySSM / mamba cache** | Extra state beyond vanilla KV — watch OOM when enabling speculation | \n\nTeams already splitting parents across GPUs for [tensor-parallel GLM jobs](https://explainx.ai/blog/glm-5-3-two-dgx-stations-vllm-tensor-parallel-753b-nvfp4-607-tokens-per-second-guide-2026) can treat the drafter as a **small satellite model** pinned to the least-loaded device.\n\n## Cost math sketch (illustrative)\n\nSuppose autoregressive decoding costs you **$X per million output tokens** on rented H100 time. A **2.7× throughput** gain on WorkBuddy Code (low) — one row from the card — suggests the same wall-clock budget produces **2.7× more tokens** if verification overhead stays stable.\n\nThat is not automatic savings:\n\n- Draft model VRAM may force a **second GPU** or a**smaller parent batch** .\n- Failed speculative paths add **wasted draft FLOPs** .\n- Agent traces with tools may spend more time in **CPU sandbox** than in matmul.\n\nStill, for high-volume [Rainshare-style quota sharing](https://explainx.ai/blog/rainshare-mutual-aid-ai-coding-plan-quota-2026) or internal codegen fleets, a verified 2×+ speedup on Qwen is worth a weekend bake-off.\n\n## When not to adopt yet\n\nSkip the upgrade path if:\n\n- You are on **vLLM-only** serving with no SGLang appetite.\n- Your workload is **mostly embedding or rerank** , not long generation.\n- You need **bit-identical** replay against a non-speculative baseline for compliance.\n- You cannot spare engineering time to pin **SGLang 0.5.22** and the DSpine commit.\n\nIn those cases, watch the checkpoint for community reports rather than rushing production.\n\n## Research line: adjacent causal injection in plain language\n\nParallel drafters guess many tokens at once, but guesses at position *k* often ignore what position *k-1* already committed at early layers. DSpine routes predecessor features forward **through depth** so that by the time the block exits, later positions have seen a causal chain **without** serial token steps.\n\nThat is why Tencent pairs the method with **dynamic verification referencing DSpark** on the card — verification and drafting co-evolved in their stack. Readers digging into kernels should start at [github.com/Tencent/DSpine](https://github.com/Tencent/DSpine?ref=explainx) rather than treating the Hugging Face repo as the whole system.\n\n## Limitations and honesty\n\n- **Training code** for custom drafters is still a moving target on the GitHub README.\n- Benchmarks are **author-run** ; we have not reproduced the 3.8× Office throughput claim.\n- **Qwen3.8-27B** parent weights are a separate download and license check from the drafter.\n- Block size **16** may not be optimal on every GPU class; treat 16 as the card default, not a universal law.\n\n## What this means for builders\n\nTencent is shipping **open drafters on the same cadence as Qwen3.8 parents**, which keeps pressure on closed APIs for cost per agent step. If DSpine's agent tables hold up in the wild, teams running [self-hosted coding harnesses](https://explainx.ai/blog/tencent-loopforge-self-hosted-sop-coding-agents-claude-codex-2026) on Qwen gain another knob besides quantization and batching.\n\nThe near-term homework is boring and valuable: one benchmark script, one production-like trace, one cost line before you flip a fleet.\n\n## Related reading\n\n*Model card metrics and SGLang flags are accurate as of November 14, 2026. Re-check Hugging Face and github.com/Tencent/DSpine before you deploy.*", "url": "https://wpnews.pro/news/qwen3-8-27b-dspine-b16-tencent-s-open-2b-drafter-for-faster-local-inference", "canonical_source": "https://explainx.ai/blog/qwen3-8-27b-dspine-b16-open-drafter-sglang-speculative-decoding-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-14 00:00:00+00:00", "updated_at": "2026-11-14 17:24:19.355915+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-tools", "ai-agents"], "entities": ["Tencent", "Qwen3.8-27B-DSpine-b16", "Qwen3.8-27B", "Hugging Face", "SGLang", "DFlash2", "WorkBuddy Bench", "CodeBuddy"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/qwen3-8-27b-dspine-b16-tencent-s-open-2b-drafter-for-faster-local-inference", "markdown": "https://wpnews.pro/news/qwen3-8-27b-dspine-b16-tencent-s-open-2b-drafter-for-faster-local-inference.md", "text": "https://wpnews.pro/news/qwen3-8-27b-dspine-b16-tencent-s-open-2b-drafter-for-faster-local-inference.txt", "jsonld": "https://wpnews.pro/news/qwen3-8-27b-dspine-b16-tencent-s-open-2b-drafter-for-faster-local-inference.jsonld"}}