Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72

Alibaba released the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), a 2.4T-parameter mixture-of-experts model with 95B activated parameters per token, designed for long-context reasoning and agentic workloads. NVIDIA announced that the model achieves over 4,000 tokens per second per GPU and over 350 tokens per second per user on the NVIDIA GB300 NVL72 platform in FP8 precision on day one, with further optimizations expected. The model features a hybrid full and linear attention architecture, a context window up to one million tokens, and configurable reasoning levels (low/high/xhigh).

Alibaba released the open weights for Qwen3.8-2.4T-A95B Qwen3.8-Max , its largest open-weight model, bringing near-frontier capabilities to the open ecosystem. It has 2.4T total parameters with 95B activated per token. It has 2.4T total parameters with 95B activated per token. It’s a fine-grained mixture of experts MoE architecture with a hybrid of full and linear attention, a context window of up to one million tokens, and an output length of up to 128K, designed for demanding reasoning and agentic workloads. Deploying a 2.4T parameter open-weight model requires data-center-scale accelerated compute. Inference at this scale depends on extreme co-design across chips, system architecture, and software. NVIDIA is working with the open-source ecosystem to bring the model to multinode deployments through optimized kernels, inference runtimes, and distributed serving recipes. Without additional model tuning, the model achieves a throughput of over 4K tokens per second per GPU and over 350 tokens per second per user on NVIDIA GB300 NVL72 in FP8 precision on Day 0. Further optimizations, including NVFP4 precision, are expected to deliver enhanced performance gains over time. Architectural innovations for long-context inference Qwen3.8-2.4T-A95B is built for the hardest agentic workloads like coding, large-scale document analysis, and long-running multi-step workflows. Unlike chat-first models that send a single prompt and receive a single reply, agentic applications accumulate system instructions, tool outputs, retrieved documents, code, logs, and multi-step reasoning traces across a workflow. As context grows, attention, compute, and KV cache memory become the binding constraints. The full-attention and linear-attention hybrid architecture addresses this, and the model alternates between the two. In the full-attention layers, every token attends to every other token, and in the linear-attention layers, the growing KV cache is replaced with a bounded recurrent state. Qwen3.8-2.4T-A95B keeps both compute and memory bounded as context scales to up to one million tokens. Fine-grained MoE makes the 2.4T parameter count practical to serve. Instead of a small number of large experts, capacity is distributed across a larger population of smaller experts, improving specialization and routing efficiency per unit of activated compute. A learned router activates only the experts needed per token, so serving costs track active parameters, not the full 2.4T parameters, delivering frontier-scale capacity at a fraction of the cost of a comparable dense model. Built-in reasoning controls low/high/xhigh enable developers to configure inference depth per request, trading compute for reasoning quality depending on the task: dial up for complex multi-step reasoning or dial down for high-throughput document processing. Qwen3.8-2.4T-A95B optimized performance on GB300 NVL72 The GB300 NVL72 features a rack-scale architecture that integrates 72 NVIDIA Blackwell Ultra GPUs into a single platform. Its large, 72-GPU NVIDIA NVLink domain enables efficient all-to-all communication at 130 TB/s, eliminating bottlenecks that appear when expert traffic must cross traditional off-the-shelf networks. Out of the box, Qwen3.8~~ ~~2.4T-A95B running on NVIDIA Blackwell GB300 NVL72 delivers over 4K tokens per second per GPU and over 350 tokens per second per user , enabling AI factories to run large-parameter models in production at high throughput and low latency. Post-train Qwen3.8-2.4T-A95B and choose a serving path NVIDIA supports multiple inference stacks to meet a variety of developer needs. SGLang https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8 , vLLM https://recipes.vllm.ai/Qwen/Qwen3.8-2.4T-A95B , and NVIDIA Dynamo https://docs.nvidia.com/dynamo/dev/recipes/qwen-3-8-2-4t-a95b provide open-source inference recipes for developers who require greater control over performance on the NVIDIA-accelerated platform. It’s also available to deploy via a model-free NVIDIA NIM https://catalog.ngc.nvidia.com/orgs/nim/nvidia/containers/sglang-model-free-nim/- , a single inference container that serves any supported model. Download the model weights and deploy on Day-0 to serve fine-tuned checkpoints, and scale to production. Developers can post-train the model for domain-specific use cases using NVIDIA NeMo AutoModel https://github.com/NVIDIA-NeMo/Automodel/tree/main/docs/model-coverage/llm/qwen/qwen3-8-max.mdx , a PyTorch-native fine-tuning library with Day-0 Hugging Face checkpoint support. Train directly on existing checkpoints without model conversion, with support for full SFT or memory-efficient LoRA fine-tuning. Get started with Qwen3.8-2.4T-A95B Download Qwen3.8-2.4T-A95B model weights from Hugging Face https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B or ModelScope https://modelscope.cn/models/Qwen/Qwen3.8-2.4T-A95B and deploy with a model-free NVIDIA NIM https://catalog.ngc.nvidia.com/orgs/nim/nvidia/containers/sglang-model-free-nim/- from NVIDIA NGC.