Qwen3.8-2.4T-A95B: Specs, Architecture, and Benchmarks Explained

Alibaba's Qwen team released Qwen3.8-2.4T-A95B, an open-weight mixture-of-experts language model with 2.4 trillion total parameters and 95 billion active per token, supporting 262,144 tokens of native context extensible to about 1,010,000 tokens. The model, which serves as the base for Qwen3.8-Max, scores competitively with Opus 4.8 and GPT 5.6 Sol (max) on coding-agent benchmarks like Terminal Bench 2.1 and PaperBench, according to Qwen.

Qwen3.8-2.4T-A95B: Specs, Architecture, and Benchmarks Explained Qwen3.8-2.4T-A95B specs: 2.4T total/95B active MoE parameters, 262K context, and benchmark scores versus Opus 4.8 and GPT 5.6. Drafted with Claude from source material, checked by automated verification, and reviewed before release. How we make these /editorial-standards . What is Qwen3.8-2.4T-A95B? Qwen3.8-2.4T-A95B is Qwen’s newest open-weight flagship language model, released as the base for Qwen3.8-Max. It’s a mixture-of-experts MoE model with 2.4 trillion total parameters, of which 95 billion are active per forward pass. Qwen describes it as the first Qwen-Max-class model made available as open weights, built on the architecture introduced with Qwen3.5 and aimed at coding, professional research, and long-horizon agentic tasks. TL;DR Qwen3.8-2.4T-A95B is a 2.4 trillion parameter MoE model with only 95 billion parameters active per token, keeping inference cost closer to a much smaller dense model.- The model natively supports 262,144 tokens of context , extensible up to roughly 1,010,000 tokens, making it viable for very large codebases or document sets. - Qwen3.8-Max, the hosted API version built on this checkpoint, adds vision input, non-thinking mode, a default 1M context window, and built-in tools . - On coding-agent benchmarks like Terminal Bench 2.1 and PaperBench, Qwen3.8-Max scores competitively with Opus 4.8 and GPT 5.6 Sol max , sometimes edging ahead. - The architecture mixes Gated DeltaNet linear attention with Gated Attention layers in a repeating pattern, paired with a 512-expert MoE routing 10 routed experts plus 1 shared expert per token. - The release includes a reasoning effort parameter for tunable reasoning depth and a preserve thinking option to retain reasoning context across turns, features aimed at agentic workflows. - Model weights are distributed in safetensors format across 213 shards , compatible with vLLM, SGLang, and similar inference stacks. What are the core specs of Qwen3.8-2.4T-A95B? Plans first. Then code. Remy writes the spec, manages the build, and ships the app. The model card lists 2.4 trillion parameters total with 95 billion activated per token, a hidden dimension of 8192, and 92 layers. The token embedding and output layers are padded to 248,320 entries. Context length is 262,144 tokens natively, extensible to about 1,010,000 tokens for applications that need it. The layer structure follows a repeating pattern: 23 blocks of “3x Gated DeltaNet then MoE , 1x Gated Attention then MoE .” This mixes linear attention Gated DeltaNet with standard gated attention layers rather than using one attention mechanism throughout. Gated DeltaNet layers use 128 linear attention heads for values and 16 for queries/keys, each with a head dimension of 128. The Gated Attention layers use 64 query heads and 4 key/value heads, a head dimension of 256, and rotary position embeddings of dimension 64. The MoE component has 512 experts total, with 10 routed experts plus 1 shared expert activated per token, and an expert intermediate dimension of 2048. The model also uses multi-token prediction MTP trained across multiple steps, a technique intended to improve training efficiency and downstream generation quality. How does the MoE architecture affect efficiency? Mixture-of-experts design is the reason a model with 2.4 trillion total parameters can run with a compute footprint closer to a 95-billion-parameter dense model. Instead of activating every parameter for every token, the router selects a small subset of experts 10 routed plus 1 shared, out of 512 available for each token. That keeps FLOPs per token much lower than a dense model of equivalent total size, while still letting the model store far more specialized knowledge across its expert pool. The tradeoff is memory. Even though only 95B parameters compute per token, all 2.4T parameters need to be loaded for inference, which means serving this model requires substantial VRAM or multi-node setups regardless of the lighter compute cost per token. The weights ship as 213 safetensors shards, reflecting the scale of the full checkpoint. The hybrid attention scheme Gated DeltaNet plus Gated Attention is a separate efficiency lever aimed at long-context handling. Linear attention variants like Gated DeltaNet generally scale better with sequence length than standard softmax attention, which is one reason the model can natively handle 262K tokens and stretch to around 1M with extension. How does Qwen3.8-Max compare to Opus 4.8 and GPT 5.6? Qwen published benchmark comparisons between Qwen3.8-Max, its own predecessor Qwen3.7-Max, and external frontier models referred to as Opus 4.8, Fable 5, and GPT 5.6 Sol max . Results vary by task category. On coding-agent benchmarks, Qwen3.8-Max scored 86.6 on Terminal Bench 2.1, ahead of Opus 4.8 84.6 and Fable 5 84.6 , though behind GPT 5.6 Sol max at 88.8. On PaperBench, Qwen3.8-Max led the group at 93.0, ahead of GPT 5.6 Sol max at 90.5 and Opus 4.8 at 80.3. On SWE-bench Pro, Qwen3.8-Max posted 67.7, behind Fable 5’s 80.0 but ahead of GPT 5.6 Sol max at 64.6. Other benchmarks showed more mixed results. On DeepSWE 1.1, Qwen3.8-Max scored 56.6, trailing GPT 5.6 Sol max at 73.0 and Fable 5 at 70.0. On FrontierSWE, it scored 73.5, behind Fable 5’s 88.8 but well ahead of Opus 4.8’s 70.0 and its own predecessor Qwen3.7-Max at 40.7. Other agents ship a demo. Remy ships an app. Real backend. Real database. Real auth. Real plumbing. Remy has it all. Compared to Qwen3.7-Max specifically, the jump is large and consistent: Qwen3.8-Max improved on every benchmark listed, including a jump from 21.6 to 56.6 on DeepSWE 1.1 and from 40.7 to 73.5 on FrontierSWE. That gap illustrates the scale of the generational improvement Qwen is claiming for this release, independent of how it stacks up against closed competitors. On general-agent benchmarks like CoWorkBench and WorkSpaceBench, Qwen3.8-Max scored in a similar range to Opus 4.8 and GPT 5.6 Sol max , generally a few points behind Fable 5, which topped most categories in the released tables. What’s different between Qwen3.8-2.4T-A95B and Qwen3.8-Max? Qwen3.8-2.4T-A95B is the open-weight checkpoint: the raw model in Hugging Face Transformers format, compatible with inference engines like vLLM, SGLang, and TokenSpeed. Anyone can download it, host it, and run it on their own infrastructure, provided they have the hardware to hold a 2.4 trillion parameter model in memory. Qwen3.8-Max is the managed API version hosted on Qwen Cloud, built from the same underlying model but with additional product features layered on top: vision input support, a non-thinking mode for faster responses without extended reasoning, a default context window of 1 million tokens, and built-in tools. The benchmark table in Qwen’s release compares Qwen3.8-Max the API product against other frontier models, not the raw open checkpoint directly, since the hosted version includes the extended context and tooling that shape real-world benchmark performance. What is reasoning effort and why does it matter for agents? Qwen3.8 introduces a reasoning effort parameter that lets developers tune how much internal reasoning the model performs before responding. Lower settings favor speed, higher settings favor more thorough step-by-step reasoning, useful for harder coding or research tasks where accuracy matters more than latency. Paired with this is preserve thinking , which retains reasoning context from earlier turns in a conversation rather than discarding it after each response. For long-horizon agentic tasks, where a model needs to plan, act, observe environment feedback, and revise its plan over many steps, retaining prior reasoning can reduce redundant re-derivation and improve consistency across a multi-step task. Qwen frames both features as part of a broader push toward more reliable end-to-end task completion, rather than just improving single-turn question answering. Frequently Asked Questions How many parameters does Qwen3.8-2.4T-A95B have? It has 2.4 trillion total parameters, with 95 billion active per token due to its mixture-of-experts architecture. Only a subset of the model’s 512 experts 10 routed plus 1 shared compute for any given token. What is the context length of Qwen3.8-2.4T-A95B? The model natively supports 262,144 tokens and can be extended up to approximately 1,010,000 tokens. The hosted Qwen3.8-Max API defaults to a 1 million token context window. Is Qwen3.8-2.4T-A95B open source? The weights are released openly on Hugging Face under a license labeled “other” in the model’s metadata, and they’re compatible with common inference frameworks including vLLM and SGLang. Check the specific license terms on the model card before commercial use. How does Qwen3.8-Max compare to GPT 5.6 and Opus 4.8? Results are mixed across benchmarks. Qwen3.8-Max leads on some tasks like PaperBench and Terminal Bench 2.1 relative to Opus 4.8, but trails GPT 5.6 Sol max and Fable 5 on others like DeepSWE 1.1 and FrontierSWE. No single model dominates across all reported categories. What hardware is needed to run Qwen3.8-2.4T-A95B? Qwen doesn’t publish specific VRAM requirements in the model card, but a 2.4 trillion parameter model, even with sparse MoE activation, requires enough memory across GPUs or nodes to hold all weights simultaneously. The 213-shard safetensors release reflects that scale. Most users without large multi-GPU clusters will likely access the model through the Qwen Cloud API rather than self-hosting.