Tencent published Qwen3.8-27B-DSpine-b16 on Hugging Face β a ~2B-parameter drafter that pairs with Qwen3.8-27B for speculative decoding in SGLang. The October DSpine paper and our earlier Qwen3-8B DSpine guide listed a 27B checkpoint as roadmap-only; that line item is now a downloadable Apache 2.0 artifact with benchmark tables on coding, chat and long agent runs.
If you self-host Qwen3.8-27B for coding agents or batch jobs, the practical question is whether a small drafter pays for its VRAM. Tencent's card answers with acceptance-length tables and end-to-end throughput on WorkBuddy-style sandboxes β numbers you should reproduce on your GPU before you trust them for production.
TL;DR #
| Question | Answer |
|---|---|
| What shipped? | Five-layer DSpine drafter, block size 16, 8,192-token sliding window |
| Parent model | Qwen/Qwen3.8-27B (separate download) |
| License | Apache 2.0 |
| Runtime cited | SGLang 0.5.22, BF16, FlashAttention 3 |
| Baseline on card | DFlash2 (block 8) vs DSpine (block 16) |
| Agent story | WorkBuddy Code/Web/Office, SWE-bench Pro, ΟΒ²-bench throughput up to ~3.8Γ AR on Office (low) |
| Training code | Not on the card; see Tencent/DSpine repo |
What DSpine does (in one paragraph) #
Speculative decoding trades a cheap draft model for fewer full passes through a large target. Tencent's DSpine method drafts a block of tokens in parallel but injects each position's predicted features into its neighbor at every layer β "adjacent causal injection" β so parallel guesses stay more consistent than classic parallel drafters like DFlash.
The 27B checkpoint is the same architecture family as the 8B release, scaled to a larger target backbone. For token economics background, see what tokens per second actually measures and how Baseten-style serving stacks chase wall-clock wins.
Architecture at a glance #
The Tencent model card describes a five-layer draft network, block size 16, and an 8,192-token sliding window. Evaluation used SGLang 0.5.22 with BF16 weights, FlashAttention 3, concurrency 8, dynamic verification referencing DSpark with ReplaySSM enabled β details that matter if your install drifts from their stack.
Benchmarks: acceptance length #
Tencent compares DFlash2 (official 8-token blocks) against DSpine (16-token blocks) on general tasks at greedy and sampled decoding:
| Task | DFlash2 (T=0) | DSpine (T=0) | DFlash2 (T=1) | DSpine (T=1) |
|---|---|---|---|---|
| GSM8K | 5.69 | 7.20 | 5.45 | 6.31 |
| MATH-500 | 5.49 | 6.49 | 5.27 | 5.66 |
| HumanEval | 4.87 | 5.71 | 4.34 | 4.71 |
| MBPP | 5.04 | 6.14 | 4.79 | 5.41 |
| MT-Bench | 4.11 | 4.48 | 3.89 | 4.03 |
Longer accepted prefixes generally mean higher throughput when verification is cheap. These are Tencent-measured figures on their harness; your acceptance length moves with batch size, quantization and context length.
Agent workloads: where the card pushes hardest #
The card's newer section targets agent trajectories, not just static benchmarks. Tencent built WorkBuddy Bench subsets (Code, Web, Office) using CodeBuddy sandbox replays: up to 32,768 new tokens per request, tool-call prefixes replayed in order, prefix caching on.
Acceptance length (low reasoning excerpt):
| Benchmark | DFlash2 | DSpine |
|---|---|---|
| WorkBuddy Code | 4.19 | 5.08 |
| WorkBuddy Web | 3.95 | 4.91 |
| WorkBuddy Office | 4.24 | 5.63 |
| SWE-bench Pro | 4.19 | 4.99 |
| ΟΒ²-bench | 4.70 | 5.53 |
End-to-end throughput (tokens/s, speedup vs autoregressive in parentheses) β low reasoning:
| Benchmark | AR | DFlash2 | DSpine |
|---|---|---|---|
| WorkBuddy Code | 253 | 590 (2.33Γ) | 686 (2.71Γ) |
| WorkBuddy Web | 247 | 640 (2.59Γ) | 773 (3.13Γ) |
| WorkBuddy Office | 210 | 665 (3.17Γ) | 806 (3.84Γ) |
| SWE-bench Pro | 242 | 658 (2.72Γ) | 713 (2.95Γ) |
| ΟΒ²-bench | 301 | 891 (2.96Γ) | 964 (3.20Γ) |
At xhigh reasoning the multipliers shrink slightly but stay above 2.5Γ on several rows. That pattern fits the intuition that long CoT traces stress verification more than short chat turns β similar to how SparseRead-style context trimming attacks cost on the input side while drafters attack cost on the output side.
A branching checkpoint stem illustrating a new open-weight drafter release for Qwen3.8-27B speculative decoding
How to try it today #
- Clone and install Tencent/DSpine (
pip install -e .). - Download Qwen/Qwen3.8-27B andtencent/Qwen3.8-27B-DSpine-b16 (about 4,873 downloads in the last month on the card when we read it).
- Launch SGLang with the card's full flag set β notably
DSPINE, 16 draft tokens,fa3attention backends,qwen3reasoning and tool parsers, and--enable-linear-replayssm-spec.
sglang serve \
--model-path Qwen/Qwen3.8-27B \
--speculative-algorithm DSPINE \
--speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \
--speculative-num-draft-tokens 16 \
--dtype bfloat16 \
--attention-backend fa3 \
--speculative-draft-attention-backend fa3 \
--enable-linear-replayssm-spec
VRAM planning: you hold both models. On a single GPU that often means quantization on the 27B parent, tensor parallel, or a second GPU for drafts β the same trade space as GLM-5.3 two-station vLLM setups at a smaller scale.
Pair this with llama.cpp v1.0 stability work if you split local vs server paths: DSpine's documented path here is SGLang-first, not llama.cpp.
DSpine vs DFlash2 vs waiting for vLLM #
| Path | When it fits |
|---|---|
| DSpine + SGLang | You already serve Qwen with SGLang; you want the card's exact algorithm flags |
| DFlash2 | You want Tencent's prior parallel drafter baseline on the same parent |
| Nvidia DFlash 2 on Muse Spark | Different ecosystem β see Nvidia's vLLM speculative recipe |
| No drafter | Simplicity wins when latency targets are already met |
The Hugging Face card credits DFlash, SpecForge, SGLang and FlashInfer β useful breadcrumbs if you debug kernel mismatches.
What developers should verify locally #
The model card's Hacker News thread (when we checked) had no substantive hands-on comments yet β early release noise only. Until community runs land:
- Match SGLang version (0.5.22 cited).
- Compare AR vs DSpine onyour agent prompt mix, not only Tencent's benches.
- Watch tool-call integrity ; speculative stacks can interact badly with parsers if drafts desynchronize mid-trajectory.
- Track memory : ReplaySSM and mamba cache flags change footprint.
Command reference: full model-card serve line #
Tencent's card is unusually explicit about flags because DSpine touches linear attention, mamba radix cache, and FlashInfer sampling in one stack. Missing one flag often silently disables the path you thought you enabled.
| Flag | Role |
|---|---|
--speculative-algorithm DSPINE |
Selects adjacent-injection drafting |
--speculative-num-draft-tokens 16 |
Matches b16 block training |
--enable-linear-replayssm-spec |
Turns on ReplaySSM path cited in eval |
--linear-attn-backend triton |
Linear attention kernel choice |
--mamba-ssm-dtype float32 |
Stability for SSM state |
--mamba-radix-cache-strategy extra_buffer |
Cache strategy for speculative tree |
--max-mamba-cache-size 64 |
Caps SSM cache footprint |
--reasoning-parser qwen3 |
Parses Qwen3 reasoning channels |
--tool-call-parser qwen3_coder |
Aligns tool traces with parent |
If you already run Qwen3.8-27B on SGLang without speculation, diff your script against this table before blaming the drafter for slowdowns.
Paper vs card: keep the baselines straight #
The arXiv paper established DSpine against DFlash on smaller Qwen3 checkpoints with different acceptance-length math than this 27B card. Our October Qwen3-8B post explains why 27.8% vs DFlash and ~4% vs retrained DFlash2 can both appear in Tencent materials without contradiction.
On the 27B card, the meaningful comparison is DFlash2 vs DSpine under the same Open-PerfectBlend retraining story for general benchmarks, plus the agent replay suite. Do not import 8B paper headlines into a 27B procurement slide without re-benchmarking.
Memory and hardware planning #
Rough planning questions for a single host:
| Component | Planning question |
|---|---|
| 27B parent BF16 | Does it fit beside a 2B drafter on one 48GB card, or do you quantize the parent? |
| Concurrent requests | Card uses concurrency 8 β scaling users may need tensor parallel on the parent |
| Context 131072 | Long agent runs need the card's context-length flag, not defaults |
| ReplaySSM / mamba cache | Extra state beyond vanilla KV β watch OOM when enabling speculation |
Teams already splitting parents across GPUs for tensor-parallel GLM jobs can treat the drafter as a small satellite model pinned to the least-loaded device.
Cost math sketch (illustrative) #
Suppose autoregressive decoding costs you $X per million output tokens on rented H100 time. A 2.7Γ throughput gain on WorkBuddy Code (low) β one row from the card β suggests the same wall-clock budget produces 2.7Γ more tokens if verification overhead stays stable.
That is not automatic savings:
- Draft model VRAM may force a second GPU or asmaller parent batch .
- Failed speculative paths add wasted draft FLOPs .
- Agent traces with tools may spend more time in CPU sandbox than in matmul.
Still, for high-volume Rainshare-style quota sharing or internal codegen fleets, a verified 2Γ+ speedup on Qwen is worth a weekend bake-off.
When not to adopt yet #
Skip the upgrade path if:
- You are on vLLM-only serving with no SGLang appetite.
- Your workload is mostly embedding or rerank , not long generation.
- You need bit-identical replay against a non-speculative baseline for compliance.
- You cannot spare engineering time to pin SGLang 0.5.22 and the DSpine commit.
In those cases, watch the checkpoint for community reports rather than rushing production.
Research line: adjacent causal injection in plain language #
Parallel drafters guess many tokens at once, but guesses at position k often ignore what position k-1 already committed at early layers. DSpine routes predecessor features forward through depth so that by the time the block exits, later positions have seen a causal chain without serial token steps.
That is why Tencent pairs the method with dynamic verification referencing DSpark on the card β verification and drafting co-evolved in their stack. Readers digging into kernels should start at github.com/Tencent/DSpine rather than treating the Hugging Face repo as the whole system.
Limitations and honesty #
- Training code for custom drafters is still a moving target on the GitHub README.
- Benchmarks are author-run ; we have not reproduced the 3.8Γ Office throughput claim.
- Qwen3.8-27B parent weights are a separate download and license check from the drafter.
- Block size 16 may not be optimal on every GPU class; treat 16 as the card default, not a universal law.
What this means for builders #
Tencent is shipping open drafters on the same cadence as Qwen3.8 parents, which keeps pressure on closed APIs for cost per agent step. If DSpine's agent tables hold up in the wild, teams running self-hosted coding harnesses on Qwen gain another knob besides quantization and batching.
The near-term homework is boring and valuable: one benchmark script, one production-like trace, one cost line before you flip a fleet.
Related reading #
Model card metrics and SGLang flags are accurate as of November 14, 2026. Re-check Hugging Face and github.com/Tencent/DSpine before you deploy.