{"slug": "qwen3-8-2-4t-a95b-specs-architecture-and-benchmarks-explained", "title": "Qwen3.8-2.4T-A95B: Specs, Architecture, and Benchmarks Explained", "summary": "Alibaba's Qwen team released Qwen3.8-2.4T-A95B, an open-weight mixture-of-experts language model with 2.4 trillion total parameters and 95 billion active per token, supporting 262,144 tokens of native context extensible to about 1,010,000 tokens. The model, which serves as the base for Qwen3.8-Max, scores competitively with Opus 4.8 and GPT 5.6 Sol (max) on coding-agent benchmarks like Terminal Bench 2.1 and PaperBench, according to Qwen.", "body_md": "# Qwen3.8-2.4T-A95B: Specs, Architecture, and Benchmarks Explained\n\nQwen3.8-2.4T-A95B specs: 2.4T total/95B active MoE parameters, 262K context, and benchmark scores versus Opus 4.8 and GPT 5.6.\n\nDrafted with Claude from source material, checked by automated verification, and reviewed before release. [How we make these](/editorial-standards).\n\n## What is Qwen3.8-2.4T-A95B?\n\nQwen3.8-2.4T-A95B is Qwen’s newest open-weight flagship language model, released as the base for Qwen3.8-Max. It’s a mixture-of-experts (MoE) model with 2.4 trillion total parameters, of which 95 billion are active per forward pass. Qwen describes it as the first Qwen-Max-class model made available as open weights, built on the architecture introduced with Qwen3.5 and aimed at coding, professional research, and long-horizon agentic tasks.\n\n## TL;DR\n\n**Qwen3.8-2.4T-A95B** is a 2.4 trillion parameter MoE model with only 95 billion parameters active per token, keeping inference cost closer to a much smaller dense model.- The model natively supports\n**262,144 tokens of context**, extensible up to roughly 1,010,000 tokens, making it viable for very large codebases or document sets. - Qwen3.8-Max, the hosted API version built on this checkpoint, adds\n**vision input, non-thinking mode, a default 1M context window, and built-in tools**. - On coding-agent benchmarks like Terminal Bench 2.1 and PaperBench, Qwen3.8-Max scores competitively with\n**Opus 4.8 and GPT 5.6 Sol (max)**, sometimes edging ahead. - The architecture mixes\n**Gated DeltaNet linear attention with Gated Attention layers** in a repeating pattern, paired with a 512-expert MoE routing 10 routed experts plus 1 shared expert per token. - The release includes a\n**reasoning_effort parameter** for tunable reasoning depth and a**preserve_thinking** option to retain reasoning context across turns, features aimed at agentic workflows. - Model weights are distributed in\n**safetensors format across 213 shards**, compatible with vLLM, SGLang, and similar inference stacks.\n\n## What are the core specs of Qwen3.8-2.4T-A95B?\n\n##\nPlans first.\n*Then code.*\n\nRemy writes the spec, manages the build, and ships the app.\n\nThe model card lists 2.4 trillion parameters total with 95 billion activated per token, a hidden dimension of 8192, and 92 layers. The token embedding and output layers are padded to 248,320 entries. Context length is 262,144 tokens natively, extensible to about 1,010,000 tokens for applications that need it.\n\nThe layer structure follows a repeating pattern: 23 blocks of “3x (Gated DeltaNet then MoE), 1x (Gated Attention then MoE).” This mixes linear attention (Gated DeltaNet) with standard gated attention layers rather than using one attention mechanism throughout. Gated DeltaNet layers use 128 linear attention heads for values and 16 for queries/keys, each with a head dimension of 128. The Gated Attention layers use 64 query heads and 4 key/value heads, a head dimension of 256, and rotary position embeddings of dimension 64.\n\nThe MoE component has 512 experts total, with 10 routed experts plus 1 shared expert activated per token, and an expert intermediate dimension of 2048. The model also uses multi-token prediction (MTP) trained across multiple steps, a technique intended to improve training efficiency and downstream generation quality.\n\n## How does the MoE architecture affect efficiency?\n\nMixture-of-experts design is the reason a model with 2.4 trillion total parameters can run with a compute footprint closer to a 95-billion-parameter dense model. Instead of activating every parameter for every token, the router selects a small subset of experts (10 routed plus 1 shared, out of 512 available) for each token. That keeps FLOPs per token much lower than a dense model of equivalent total size, while still letting the model store far more specialized knowledge across its expert pool.\n\nThe tradeoff is memory. Even though only 95B parameters compute per token, all 2.4T parameters need to be loaded for inference, which means serving this model requires substantial VRAM or multi-node setups regardless of the lighter compute cost per token. The weights ship as 213 safetensors shards, reflecting the scale of the full checkpoint.\n\nThe hybrid attention scheme (Gated DeltaNet plus Gated Attention) is a separate efficiency lever aimed at long-context handling. Linear attention variants like Gated DeltaNet generally scale better with sequence length than standard softmax attention, which is one reason the model can natively handle 262K tokens and stretch to around 1M with extension.\n\n## How does Qwen3.8-Max compare to Opus 4.8 and GPT 5.6?\n\nQwen published benchmark comparisons between Qwen3.8-Max, its own predecessor Qwen3.7-Max, and external frontier models referred to as Opus 4.8, Fable 5, and GPT 5.6 Sol (max). Results vary by task category.\n\nOn coding-agent benchmarks, Qwen3.8-Max scored 86.6 on Terminal Bench 2.1, ahead of Opus 4.8 (84.6) and Fable 5 (84.6), though behind GPT 5.6 Sol (max) at 88.8. On PaperBench, Qwen3.8-Max led the group at 93.0, ahead of GPT 5.6 Sol (max) at 90.5 and Opus 4.8 at 80.3. On SWE-bench Pro, Qwen3.8-Max posted 67.7, behind Fable 5’s 80.0 but ahead of GPT 5.6 Sol (max) at 64.6.\n\nOther benchmarks showed more mixed results. On DeepSWE 1.1, Qwen3.8-Max scored 56.6, trailing GPT 5.6 Sol (max) at 73.0 and Fable 5 at 70.0. On FrontierSWE, it scored 73.5, behind Fable 5’s 88.8 but well ahead of Opus 4.8’s 70.0 and its own predecessor Qwen3.7-Max at 40.7.\n\n## Other agents ship a demo. Remy ships an app.\n\nReal backend. Real database. Real auth. Real plumbing. Remy has it all.\n\nCompared to Qwen3.7-Max specifically, the jump is large and consistent: Qwen3.8-Max improved on every benchmark listed, including a jump from 21.6 to 56.6 on DeepSWE 1.1 and from 40.7 to 73.5 on FrontierSWE. That gap illustrates the scale of the generational improvement Qwen is claiming for this release, independent of how it stacks up against closed competitors.\n\nOn general-agent benchmarks like CoWorkBench and WorkSpaceBench, Qwen3.8-Max scored in a similar range to Opus 4.8 and GPT 5.6 Sol (max), generally a few points behind Fable 5, which topped most categories in the released tables.\n\n## What’s different between Qwen3.8-2.4T-A95B and Qwen3.8-Max?\n\nQwen3.8-2.4T-A95B is the open-weight checkpoint: the raw model in Hugging Face Transformers format, compatible with inference engines like vLLM, SGLang, and TokenSpeed. Anyone can download it, host it, and run it on their own infrastructure, provided they have the hardware to hold a 2.4 trillion parameter model in memory.\n\nQwen3.8-Max is the managed API version hosted on Qwen Cloud, built from the same underlying model but with additional product features layered on top: vision input support, a non-thinking mode for faster responses without extended reasoning, a default context window of 1 million tokens, and built-in tools. The benchmark table in Qwen’s release compares Qwen3.8-Max (the API product) against other frontier models, not the raw open checkpoint directly, since the hosted version includes the extended context and tooling that shape real-world benchmark performance.\n\n## What is reasoning_effort and why does it matter for agents?\n\nQwen3.8 introduces a `reasoning_effort`\n\nparameter that lets developers tune how much internal reasoning the model performs before responding. Lower settings favor speed, higher settings favor more thorough step-by-step reasoning, useful for harder coding or research tasks where accuracy matters more than latency.\n\nPaired with this is `preserve_thinking`\n\n, which retains reasoning context from earlier turns in a conversation rather than discarding it after each response. For long-horizon agentic tasks, where a model needs to plan, act, observe environment feedback, and revise its plan over many steps, retaining prior reasoning can reduce redundant re-derivation and improve consistency across a multi-step task. Qwen frames both features as part of a broader push toward more reliable end-to-end task completion, rather than just improving single-turn question answering.\n\n## Frequently Asked Questions\n\n### How many parameters does Qwen3.8-2.4T-A95B have?\n\nIt has 2.4 trillion total parameters, with 95 billion active per token due to its mixture-of-experts architecture. Only a subset of the model’s 512 experts (10 routed plus 1 shared) compute for any given token.\n\n### What is the context length of Qwen3.8-2.4T-A95B?\n\nThe model natively supports 262,144 tokens and can be extended up to approximately 1,010,000 tokens. The hosted Qwen3.8-Max API defaults to a 1 million token context window.\n\n### Is Qwen3.8-2.4T-A95B open source?\n\nThe weights are released openly on Hugging Face under a license labeled “other” in the model’s metadata, and they’re compatible with common inference frameworks including vLLM and SGLang. Check the specific license terms on the model card before commercial use.\n\n### How does Qwen3.8-Max compare to GPT 5.6 and Opus 4.8?\n\nResults are mixed across benchmarks. Qwen3.8-Max leads on some tasks like PaperBench and Terminal Bench 2.1 relative to Opus 4.8, but trails GPT 5.6 Sol (max) and Fable 5 on others like DeepSWE 1.1 and FrontierSWE. No single model dominates across all reported categories.\n\n### What hardware is needed to run Qwen3.8-2.4T-A95B?\n\nQwen doesn’t publish specific VRAM requirements in the model card, but a 2.4 trillion parameter model, even with sparse MoE activation, requires enough memory across GPUs or nodes to hold all weights simultaneously. The 213-shard safetensors release reflects that scale. Most users without large multi-GPU clusters will likely access the model through the Qwen Cloud API rather than self-hosting.", "url": "https://wpnews.pro/news/qwen3-8-2-4t-a95b-specs-architecture-and-benchmarks-explained", "canonical_source": "https://www.mindstudio.ai/blog/qwen3-8-2-4t-a95b-model-overview/", "published_at": "2026-08-12 00:00:00+00:00", "updated_at": "2026-08-12 21:09:31.491828+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "generative-ai", "ai-research"], "entities": ["Qwen", "Qwen3.8-2.4T-A95B", "Qwen3.8-Max", "Opus 4.8", "GPT 5.6 Sol (max)", "Terminal Bench 2.1", "PaperBench"], "alternates": {"html": "https://wpnews.pro/news/qwen3-8-2-4t-a95b-specs-architecture-and-benchmarks-explained", "markdown": "https://wpnews.pro/news/qwen3-8-2-4t-a95b-specs-architecture-and-benchmarks-explained.md", "text": "https://wpnews.pro/news/qwen3-8-2-4t-a95b-specs-architecture-and-benchmarks-explained.txt", "jsonld": "https://wpnews.pro/news/qwen3-8-2-4t-a95b-specs-architecture-and-benchmarks-explained.jsonld"}}