SGLang publishes one-GPU Qwen3.8-27B recipes, claims 206.1 tokens per second

SGLang, the open-source inference project associated with Ying Sheng and Banghua Zhu, published deployment recipes for Alibaba's Qwen3.8-27B model, enabling single-GPU serving with NVFP4 quantization and DFlash2 speculative decoding, and reported a throughput of 206.1 tokens per second on one NVIDIA RTX 5090. The recipes, announced around August 20, 2026, follow Alibaba's release of the model's open weights by about five days, but the speed figures require independent replication and output-quality checks.

SGLang publishes one-GPU Qwen3.8-27B recipes, claims 206.1 tokens per second Ying Sheng and Banghua Zhu's inference project added NVFP4 and DFlash2 recipes, with project-reported throughput of 206.1 tokens per second on one RTX 5090. By RuntimeWire Staff /author/runtimewire-staff · Published Primary source: Qwen / Alibaba https://x.com/Alibaba Qwen/status/2090709994761339190 Why it matters Open weights gain practical value when developers can serve them on attainable hardware. SGLang's single-GPU recipes make Qwen3.8-27B more accessible, while its project-published speed figures still require independent replication and output-quality checks. The SGLang open-source inference project https://github.com/sgl-project/sglang?ref=runtimewire , associated with Ying Sheng https://x.com/ying11231?ref=runtimewire , has added deployment recipes for Alibaba https://www.alibabagroup.com/en-US/about-alibaba?ref=runtimewire 's Qwen3.8-27B https://github.com/AlibabaCloud-Official/Qwen3.8-27B?ref=runtimewire , giving developers a documented route to run the 27-billion-parameter model with NVFP4 quantization and DFlash2 speculative decoding on a single GPU. Support circulated around August 20, 2026, when the cookbook update and Qwen announcement appeared. The update followed Alibaba's release of the model's open weights by about five days, extending the launch from downloadable parameters into configurations that developers can attempt to reproduce on Blackwell hardware. Qwen's post announcing the SGLang recipes https://x.com/Alibaba Qwen/status/2090709994761339190?ref=runtimewire SGLang's original post about the Qwen3.8-27B cookbook update https://twitter.com/sgl project/status/2090587564545806719?ref=runtimewire That handoff matters because model weights alone rarely amount to a usable deployment. Sheng and SGLang's contributors are handling the less visible serving layer of the release cycle. SGLang's January 2024 overview https://www.lmsys.org/blog/2024-01-17-sglang/?ref=runtimewire paired a structured language for model programs with RadixAttention, a runtime technique for reusing cached prompt prefixes. Sheng later co-founded RadixArk https://www.radixark.com/?ref=runtimewire , a commercial entity built around SGLang, with Banghua Zhu. Zhu is publicly associated with SGLang and RadixArk. Compressing a 27B model for one GPU The new cookbook page https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B?ref=runtimewire provides Qwen3.8-27B /models/qwen/qwen3.8-27b deployment recipes using NVFP4 and DFlash2. Alibaba's model card https://huggingface.co/Qwen/Qwen3.8-27B?ref=runtimewire describes a dense native multimodal model with 27 billion parameters, image and video support, a 262,144-token native context window and an extension path to 1 million tokens using YaRN. The weights carry an Apache 2.0 license. The Qwen model card reports a 48.0152 score on WildClawBench; that figure does not by itself establish a ranking against other hosted systems. Hugging Face's evaluation metadata https://huggingface.co/Qwen/Qwen3.8-27B?ref=runtimewire lists the model's WildClawBench overall rank as 8. The same model card reports an Agents' Last Exam score of 42.9 and a Pass@1 rate of 20.4. Those model-published and platform-listed evaluations are separate from SGLang's serving measurements. NVFP4 stores key model operations in a four-bit floating-point format designed for NVIDIA's Blackwell generation. Cutting numerical precision can reduce the memory and compute required to serve a model, though operators still need to test the resulting checkpoint and runtime combination for output quality and stability. The speed claims still need independent validation DFlash2 uses a smaller draft model to propose future tokens before Qwen3.8-27B verifies them. Accepted proposals allow the target model to produce several tokens during one verification pass. Actual gains depend on the prompt, draft quality, hardware, batch size and memory configuration. The cookbook's DFlash2 recipe https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B?ref=runtimewire uses the separate incoai/Qwen3.8-27B-DFlash2 draft checkpoint and sets --speculative-num-draft-tokens 8 . That adds another checkpoint and another compatibility surface to an already hardware-specific deployment. The cookbook's benchmark table https://github.com/sgl-project/sglang/blob/main/docs/cookbook/autoregressive/Qwen/Qwen3.8-27B.mdx?ref=runtimewire says SGLang validated the RTX 5090 and RTX PRO 6000 configurations with an 8,192-token input, a 1,024-token output and concurrency of one. It says the DGX Spark configurations booted and served under those settings, while clarifying that the project did not take DGX Spark throughput or acceptance-length measurements in that validation. SGLang reports 206.1 tokens per second on one RTX 5090 and 38.28 tokens per second on a DGX Spark https://x.com/sgl project/status/2088281320422322413?ref=runtimewire . Those are project-published measurements, not independently replicated benchmarks. SGLang has not published acceptance-rate data or output-quality results for those measurements. A separate SGLang issue https://github.com/sgl-project/sglang/issues/34895?ref=runtimewire reported severely repetitive output from an unofficial NVFP4 conversion of Qwen3.8-27B because an FP8 scaling value for the model's output head was not loaded. The report concerns a different checkpoint from the RadixArk NVFP4 model in the cookbook and does not establish a defect in the new recipe. It does show why exact checkpoint and runtime combinations matter when deploying mixed-precision models. RadixArk's distribution strategy RadixArk reportedly launched with a $100M seed round at a $400M post-money valuation. TechCrunch reported the financing https://techcrunch.com/2026/01/21/sources-project-sglang-spins-out-as-radixark-with-400m-valuation-as-inference-market-explodes/?ref=runtimewire , while RadixArk's financing release https://www.vcaonline.com/news/2026050502/radixark-launches-with-100-million-in-seed-funding-led-by-accel-to-grow-sglang-and-democratize-frontier-ai-infrastructure/?ref=runtimewire identified Accel as the lead, with Spark Capital and angel investors including Intel CEO Lip-Bu Tan and xAI co-founder Igor Babuschkin /article/xai-cofounder-igor-babuschkin-personalized-ai-startup participating. The financing belongs to RadixArk, not Alibaba's Qwen project. The funding gives Sheng and Zhu room to turn rapid model support into a distribution channel. Each cookbook entry puts SGLang between an open-model lab and developers deciding how to run its weights. Alibaba benefits when outside inference projects make Qwen easier to deploy. Qwen is part of Alibaba's broader AI and cloud strategy, where wider model use could support demand for paid compute and related services. Sheng's larger bet is that open models need an open serving layer to become credible building blocks. The Qwen3.8-27B recipes are a modest release measured in configuration files and project-published performance claims. RadixArk needs to repeat that work with each model cycle, translating a lab's weights into software another engineering team can run.