{"slug": "qwen3-8-max-beat-claude-on-three-benchmarks-the-full-story-is-less-convenient", "title": "Qwen3.8-Max Beat Claude on Three Benchmarks. The Full Story Is Less Convenient", "summary": "Alibaba has released Qwen3.8-Max, a 2.4-trillion-parameter mixture-of-experts model with 95 billion active parameters per token and a one-million-token context window. The company claims the model operated autonomously for 16 days to build a software project, but an analysis of benchmark results shows it leads on some tests while trailing on others, such as SWE-bench Pro, where Claude models score higher.", "body_md": "Alibaba has released Qwen3.8-Max, its largest and most capable AI model to date.\n\nThe headline specifications are absurd: 2.4 trillion total parameters, 95 billion activated per token, multimodal input, and a one-million-token context window.\n\nBut the specification sheet is not the most interesting part.\n\nAlibaba says Qwen3.8-Max operated autonomously for roughly 16 days, starting with an empty repository and building a working software project through issues, code changes, testing, pull requests, and self-correction.\n\nThat sounds impressive. It also sounds suspiciously like the sort of claim that deserves more inspection than a celebratory repost.\n\nSo I examined Alibaba's announcement, the public repository, and its benchmark results to answer a more useful question:\n\n**Does Qwen3.8-Max actually beat Claude, or did the benchmark department simply have an excellent week?**\n\nPrefer the two-minute version?\n\nQwen3.8-Max is Alibaba's new flagship mixture-of-experts model.\n\nIt contains **2.4 trillion parameters in total**, with approximately **95 billion active during each forward pass**. This architecture allows Alibaba to scale the model's capacity without paying the full inference cost of a dense 2.4-trillion-parameter model on every token.\n\nThe model also supports a **one-million-token context window**, making it suitable for large repositories, long documents, persistent agent sessions, and other tasks where context compression usually arrives carrying a shovel.\n\nAlibaba calls it the first Qwen model at Max scale that will receive an open-weight release. However, there is an important distinction:\n\n**The API is available now. The model weights are scheduled for release next week.**\n\nSo calling it an \"open-weight model\" is reasonable when discussing Alibaba's release plan, but saying the weights are already available would be inaccurate.\n\nYou can find the specifications and release details in the [official Qwen3.8-Max announcement](https://qwen.ai/blog?id=qwen3.8).\n\nAlibaba asked Qwen3.8-Max to create a project called `oh-my-cli`\n\nfrom an empty repository.\n\nInstead of responding to a single prompt and stopping, the model worked through a continuous engineering loop:\n\nAccording to Alibaba, the repository had accumulated **265 commits, 127 pull requests, and 151 issues** after approximately 16 days of autonomous operation.\n\nThe complete project history is available in the public [ oh-my-cli GitHub repository](https://github.com/qwen-code-dev-bot/oh-my-cli).\n\nThat transparency matters. Most autonomous-agent demonstrations give us a polished video and ask us to believe that nothing caught fire outside the frame. Here, developers can inspect the issues, commits, pull requests, tests, and failures.\n\nStill, this does not prove that the model can autonomously build any production system for 16 days. It proves that Qwen3.8-Max performed this particular task inside a structured environment with automated testing and feedback loops.\n\nThat is still meaningful, just narrower than the marketing headline.\n\nAlibaba published a large benchmark table comparing Qwen3.8-Max with Claude Opus 4.8, Claude Fable 5, GPT-5.6 Sol, and Qwen3.7-Max.\n\nHere are the most relevant results:\n\n| Benchmark | Qwen3.8-Max | Claude Opus 4.8 | Claude Fable 5 | Result |\n|---|---|---|---|---|\n| Terminal Bench 2.1 | 86.6 | 84.6 | 84.6 | Qwen leads |\n| PaperBench | 93.0 | 80.3 | 88.8 | Qwen leads |\n| OSWorld-Verified | 86.1 | 83.4 | 85.0 | Qwen leads |\n| SWE-bench Pro | 67.7 | 69.2 | 80.0 | Claude leads |\n| FrontierSWE | 73.5 | 70.0 | 88.8 | Mixed; Fable 5 leads |\n\nThese results suggest three areas where Qwen3.8-Max looks particularly strong.\n\nIts **86.6 score on Terminal Bench 2.1** puts it ahead of both Claude models in Alibaba's comparison.\n\nThat makes Qwen especially interesting for command-line agents, environment setup, testing, deployment workflows, and tasks that require repeated tool use rather than a single code-generation response.\n\nQwen3.8-Max scored **93.0 on PaperBench**, ahead of Claude Fable 5's 88.8 and Opus 4.8's 80.3.\n\nAlibaba also demonstrated a five-day research task in which the model reproduced a paper's experimental pipeline, ran 33 rounds of GPU training, and then searched for improvements to the original method.\n\nThis is potentially more useful than another model becoming marginally better at generating React components nobody requested.\n\nOn **OSWorld-Verified**, which evaluates an agent's ability to operate computer environments, Qwen3.8-Max scored 86.1.\n\nThe model uses visual output as part of its feedback loop. It can inspect an interface, identify errors, revise its plan, and try again. That matters for browser agents, desktop automation, document workflows, UI testing, and multimodal development.\n\nThe \"Qwen kills Claude\" headline falls apart once we examine harder repository-level engineering tasks.\n\nOn **SWE-bench Pro**, Qwen3.8-Max scored 67.7. Claude Fable 5 scored 80.0.\n\nOn **FrontierSWE**, Qwen scored 73.5 while Fable 5 reached 88.8.\n\nThat is not a rounding error. It suggests Claude remains stronger when a task requires deep repository understanding, architectural judgment, and reliable changes across a complicated codebase.\n\nThe more honest conclusion is:\n\nThere is another caveat: these scores come from Alibaba's evaluation table. Different benchmarks used different harnesses, time limits, context settings, and judging methods. The numbers are useful, but independent testing will matter more than launch-day charts.\n\nQwenCloud provides an Anthropic-compatible API, allowing Claude Code to use Qwen3.8-Max without replacing the Claude Code interface.\n\nFirst, install Claude Code:\n\n```\nnpm install -g @anthropic-ai/claude-code\n```\n\nThen configure it to use Qwen:\n\n```\nexport ANTHROPIC_MODEL=\"qwen3.8-max\"\nexport ANTHROPIC_SMALL_FAST_MODEL=\"qwen3.8-max\"\nexport ANTHROPIC_BASE_URL=\"https://dashscope-intl.aliyuncs.com/apps/anthropic\"\nexport ANTHROPIC_AUTH_TOKEN=\"YOUR_QWEN_API_KEY\"\n\nclaude\n```\n\nYou will need a QwenCloud API key. The international endpoint may differ depending on your account or deployment region, so check the current [QwenCloud documentation](https://www.qwencloud.com/) before configuring it.\n\nAnd please do not paste your real API key into a public DEV article. Becoming an involuntary cloud-compute philanthropist is rarely part of the content strategy.\n\nNot casually.\n\nAlthough only 95 billion parameters are active during each pass, the full model contains 2.4 trillion parameters. Open weights do not magically convert that into something your laptop can run between Chrome tabs.\n\nOnce the weights are released, practical deployment will likely require substantial multi-GPU infrastructure, aggressive quantization, or a hosted inference provider.\n\nFor most individual developers, QwenCloud will be the realistic way to use the full model.\n\nQwen3.8-Max does not kill Claude.\n\nIt does something more consequential: it brings frontier-scale agent capabilities closer to the open-weight ecosystem.\n\nIts strongest argument is not a single benchmark score. It is the combination of long-horizon execution, terminal performance, multimodal feedback, research reproduction, and a public 16-day development trace.\n\nBased on the evidence available today:\n\nQwen3.8-Max is not the model that makes Claude irrelevant.\n\nIt is the model that makes the frontier race significantly less comfortable, and that is far more interesting.\n\nHave you tested Qwen3.8-Max in QwenCloud or Claude Code? Share the task, harness, and result in the comments. \"It felt smarter\" is emotionally valid, but logs are sexier.", "url": "https://wpnews.pro/news/qwen3-8-max-beat-claude-on-three-benchmarks-the-full-story-is-less-convenient", "canonical_source": "https://dev.to/aditi_gupta_8d81622a592aa/qwen38-max-vs-claude-what-the-16-day-coding-run-and-benchmarks-really-show-3bje", "published_at": "2026-08-04 11:34:00+00:00", "updated_at": "2026-08-04 11:49:41.017957+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research"], "entities": ["Alibaba", "Qwen3.8-Max", "Claude Opus 4.8", "Claude Fable 5", "GPT-5.6 Sol", "Qwen3.7-Max", "oh-my-cli"], "alternates": {"html": "https://wpnews.pro/news/qwen3-8-max-beat-claude-on-three-benchmarks-the-full-story-is-less-convenient", "markdown": "https://wpnews.pro/news/qwen3-8-max-beat-claude-on-three-benchmarks-the-full-story-is-less-convenient.md", "text": "https://wpnews.pro/news/qwen3-8-max-beat-claude-on-three-benchmarks-the-full-story-is-less-convenient.txt", "jsonld": "https://wpnews.pro/news/qwen3-8-max-beat-claude-on-three-benchmarks-the-full-story-is-less-convenient.jsonld"}}