{"slug": "qwythos-9b-a-9b-that-checks-its-own-work", "title": "Qwythos-9B: a 9B that checks its own work", "summary": "Empero AI released Qwythos-9B-Claude-Mythos-5-1M, a 9-billion-parameter open-weights reasoning model distilled from Claude Mythos 5, featuring a 1-million-token context window, native tool use, and a 34-point MMLU improvement over its base. The model, built on an uncensored Qwen3.5-9B base and trained on over 500 million tokens of Claude traces, uses chain-of-thought reasoning and self-correction via tool calls to improve accuracy.", "body_md": "# Qwythos-9B: a 9B that checks its own work\n\nOur biggest open-weights release yet — a full-parameter reasoning model distilled from Claude Mythos 5, with a 1M-token context, native tool use, and a +34-point MMLU jump over its base. Here's what's in it, the honest benchmark table, and how to run it.\n\nWe just shipped **Qwythos-9B-Claude-Mythos-5-1M** — our biggest open-weights model to date, and the new flagship over on [Hugging Face](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M). It's a full-parameter reasoning model built on a deeply uncensored Qwen3.5-9B base, post-trained on north of 500 million tokens of Claude Mythos and Claude Fable traces, with the chain-of-thought generated in-house by our `rethink`\n\ntool.\n\nThe short version: it reasons before it answers, ships with a **1-million-token context window out of the box**, calls tools natively, and — the part I'm proudest of — *checks its own specifics with those tools instead of guessing.* Apache-2.0. Weights and GGUF builds are up now.\n\n## What's in it\n\n**Base:** a deeply uncensored Qwen3.5-9B — dense, with a hybrid attention stack (3:1 Gated-DeltaNet linear-attention to full attention).**Training:** full-parameter SFT, assistant-only loss, a two-phase curriculum (broad reasoning corpus → focused agentic + coding). bf16, paged 8-bit AdamW, no truncation.**Data:** 500M+ tokens of Claude Mythos and Claude Fable traces. The chain-of-thought is structured by`rethink`\n\n, our in-house CoT tool, so the model learns to walk hypothesis → verification → conclusion*before*it commits to an answer.**License:** Apache-2.0, inherited from the base.\n\n## A million tokens, by default\n\nQwythos ships with YaRN rope-scaling already wired into `config.json`\n\n— factor 4.0 over the native 262,144-token architecture, for a full **1,048,576-token** window with no flag to flip and no separate tokenizer:\n\n```\n\"rope_parameters\": {\n  \"rope_type\": \"yarn\",\n  \"factor\": 4.0,\n  \"original_max_position_embeddings\": 262144\n},\n\"max_position_embeddings\": 1048576\n```\n\nThis is Qwen's own official 1M recipe. What it unlocks in practice: whole-codebase reasoning without RAG chunking, long agentic trajectories with verbose tool output, and multi-document research that fits a dozen papers plus your draft in a single prompt.\n\nOne practical note: the hybrid Gated-DeltaNet stack keeps memory growth sub-quadratic below ~256k tokens, so a single H100/H200 comfortably handles **256k–512k**; the full 1M wants tensor-parallel or aggressive KV-cache offload. YaRN trades a little short-context fidelity for the range — if you never go past the native 262k and want maximum sharpness, there's a `config.json.pre_yarn`\n\nbackup to restore.\n\n## It uses tools — and corrects itself\n\nFunction calling works out of the box per Qwen3.5's spec. Pass `tools=[...]`\n\nto the chat template and the model emits valid `<tool_call>`\n\nblocks with the required parameters honored — no wrapper, no tool-specific fine-tune.\n\nWe ran a 7-prompt harness mixing capability demos with deliberately hard, closed-book facts where sampling-from-memory usually fails. **Seven of seven succeeded.** A few I think matter:\n\n*Count the primes below 100,000.*It didn't recall a figure — it wrote a primality test, ran it in the Python executor, and reported**9,592**.*What's the hashcat mode for a Kerberos TGS-REP ticket?*The first search came back muddy. The model judged the results insufficient,**refined its own query**, and confirmedacross multiple sources.`-m 13100`\n\n*Is physostigmine indicated for organophosphate poisoning?*It searched authoritative toxicology sources and got the safety-critical answer right:**no — it's contraindicated**; physostigmine is for the anticholinergic toxidrome. Getting that one wrong in the real world hurts someone.\n\nThat last example is the whole thesis. A 9B that knows when to look something up beats a much bigger model that confidently invents it. Full transcripts — every reasoning step, every tool call, every result — are in [ evals/tool_test_outputs.md](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M/blob/main/evals/tool_test_outputs.md).\n\n## The numbers (the honest table)\n\nSame harness ([ lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)), same sampling, same prompts, against the base:\n\n| Task | Metric | Base Qwen3.5-9B | Qwythos-9B | Δ |\n|---|---|---|---|---|\n| gsm8k | exact match (flexible) | 0.670 | 0.860 | +0.190 |\n| gsm8k | exact match (strict) | 0.510 | 0.810 | +0.300 |\n| mmlu | acc | 0.232 | 0.575 | +0.343 |\n| arc_challenge | acc | 0.470 | 0.490 | +0.020 |\n| arc_challenge | acc_norm | 0.400 | 0.410 | +0.010 |\n| gpqa_diamond | exact match (flexible) | 0.630 | 0.580 | −0.050 |\n\nThe **MMLU +34.3** is the headline — 0.575 mean across all 57 subjects, peaking around 0.78 on government/politics, 0.77 on college biology, 0.74 on conceptual physics. gsm8k-strict is up 30 points.\n\nNot everything went up: **gpqa-diamond slipped five points and arc-challenge was roughly flat.** We publish the full table anyway, because hiding a regression is how you lose people's trust. Absolute MMLU for any 9B is sensitive to harness and few-shot count; what matters in this comparison is that both models were measured under identical settings.\n\n## Uncensored, on purpose\n\nQwythos inherits a deeply uncensored base and we kept it that way. It's built to engage seriously with technically demanding questions across cybersecurity, red-team methodology, biology, pharmacology and clinical medicine — the domains where over-aligned models refuse, hedge into uselessness, or bury the real answer under disclaimer boilerplate. That's a deliberate research choice. If you're putting it in front of end users, add your own application-level review layer.\n\n## Run it\n\nGGUF builds are up for [llama.cpp / Ollama / LM Studio](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF) if you just want to pull and chat. To serve at long context:\n\n```\n# vLLM\nvllm serve empero-ai/Qwythos-9B-Claude-Mythos-5-1M --max-model-len 1010000\n\n# SGLang\nSGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 python -m sglang.launch_server \\\n  --model-path empero-ai/Qwythos-9B-Claude-Mythos-5-1M --context-length 1010000\n```\n\nIt's a reasoning model, so give it room and don't decode greedily:\n\n```\ngen_kwargs = dict(\n    temperature=0.6, top_p=0.95, top_k=20,\n    repetition_penalty=1.05,\n    max_new_tokens=16384,\n)\n```\n\nAt greedy or very-low-temperature (T ≤ 0.3) it can fall into repetition loops on long generations — a known reasoning-model failure mode; 0.6 cleanly avoids it. Every answer opens with a `<think>`\n\nblock, so strip that span before showing it to end users. You'll also want the Gated-DeltaNet kernels (`flash-linear-attention`\n\nplus a CUDA-matched `causal_conv1d`\n\n), or the linear-attention layers fall back to slow PyTorch. It's a text-only fine-tune; the base is multimodal but we only trained the text path.\n\n## Get it\n\n**Weights + model card:**[Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M)** GGUF:**[Qwythos-9B-Claude-Mythos-5-1M-GGUF](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF)** Full eval transcripts:**[tool_test_outputs.md](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M/blob/main/evals/tool_test_outputs.md)**The rest of the lab:**[empero.org](https://empero.org)\n\nIf you build something with it, tell us. And if you want the next drop in your inbox, the dispatch sign-up is on the [home page](/).\n\n— kodee", "url": "https://wpnews.pro/news/qwythos-9b-a-9b-that-checks-its-own-work", "canonical_source": "https://empero.org/writing/qwythos-9b-release", "published_at": "2026-06-19 22:05:13+00:00", "updated_at": "2026-06-28 15:08:39.370645+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-products", "ai-tools", "ai-infrastructure"], "entities": ["Empero AI", "Qwythos-9B", "Claude Mythos 5", "Qwen3.5-9B", "Hugging Face", "Claude Fable", "Apache-2.0", "YaRN"], "alternates": {"html": "https://wpnews.pro/news/qwythos-9b-a-9b-that-checks-its-own-work", "markdown": "https://wpnews.pro/news/qwythos-9b-a-9b-that-checks-its-own-work.md", "text": "https://wpnews.pro/news/qwythos-9b-a-9b-that-checks-its-own-work.txt", "jsonld": "https://wpnews.pro/news/qwythos-9b-a-9b-that-checks-its-own-work.jsonld"}}