# Qwythos-9B-v2: a 9B that knows when to stop

> Source: <https://empero.org/writing/qwythos-9b-v2>
> Published: 2026-07-10 18:47:48+00:00

# Qwythos-9B-v2: a 9B that knows when to stop

A hygiene release for our reasoning 9B — the looping behavior is trained out (6.7% → 0% under greedy decoding), the native MTP head is restored, and the identity is a lighter touch — with every benchmark held at v1's level. Not a capability jump. A fix.

# Qwythos-9B-v2: a 9B that knows when to stop

*A hygiene release for our reasoning 9B — the looping behavior is trained out (6.7% → 0% under greedy decoding), the native MTP head is restored, and the identity is a lighter touch — with every benchmark held at v1's level. Not a capability jump. A fix.*

We just shipped **Qwythos-9B-v2** — the same reasoning model you already know, with the rough edges filed off. Let me be honest up front about what this is and isn't: it is *not* a bigger, smarter Qwythos. It's a *cleaner* one. If you were hoping for a leap on the leaderboards, this isn't that release. What it is, is the version we actually want people running in production.

The short version: three things moved, and nothing you cared about moved with them.

## What changed

**The looping behavior is gone.** Under greedy or low-temperature decoding, the base Qwythos would sometimes fall into repetition — the same clause, the same list item, the same half-thought, over and over until it hit the token limit. v2's looping rate under greedy decoding is**0.0%**, down from 6.7%. You can serve it*without*leaning on`repetition_penalty`

as a crutch.**The MTP head is back.** The native multi-token-prediction module that Qwen3.5-9B ships with had been quietly dropped in v1's export. We restored it. Config and weights finally agree again, and draft-based speculative decoding works.**The identity is quieter.** v1 had a habit of announcing who it was before answering questions nobody had asked it to introduce itself for. v2 states it once, when you actually ask.

Everything else — the deep chain-of-thought, the uncensored research posture, the 1M-token context is exactly where you left it. We didn't take that on faith; we measured it. The table's further down.

## The looping problem

Here's the failure we set out to kill.

Reasoning models earn their keep by thinking out loud before they answer, and a long `<think>`

trace is a lot of rope. Every so often , more often at temperature 0 than anyone likes to admit, the model would catch a phrase and refuse to let go of it. A list that should have four items grows a fifth that's a reworded second, then a sixth, then it's just re-emitting the same sentence with the commas moved around. It's not a knowledge failure and it's not a reasoning failure. It's a *decoding-time* pathology: a local minimum the sampler slides into and can't climb out of.

The usual fix is to paper over it at inference, crank `repetition_penalty`

, forbid greedy decoding, tell people to stay above temperature 0.3. That works, sort of, the way a splint works. But it means you can't run the model the way half of all agent harnesses want to run it, and it means every downstream user has to remember the incantation. We didn't want a splint. We wanted it fixed in the weights.

## The fix: FTPO

We used **Final-Token Preference Optimization** — FTPO.

The idea is narrow on purpose. Instead of retraining on more data and hoping the loops wash out, we go looking for them: elicit the looping behavior at low temperature, find the *exact* token where a coherent continuation tips over into a repeat, and build a preference pair at that one position the loop-starting token as the rejected choice, the model's own coherent top-*k* alternatives as the chosen ones. Then we train the model to prefer the way out.

The details, because you'll ask: LoRA at r=256, α=128, learning rate 1.5e-5, a single epoch over ~2,000 auto-mined pairs. You can absolutely grind the loops out harder and you pay for it, point by point, in everything else the model knew. We stopped the instant the repetition was gone and the capability numbers hadn't budged. The rest of the distribution never gets touched, which is exactly why the knowledge and the reasoning survive intact.

One short run. One narrow objective. No new data, no distillation, no teacher. Sometimes the right fix is small.

## The MTP head, restored

While we were in there, we fixed a second thing that had been bothering us.

Qwen3.5-9B ships with a native multi-token-prediction head, a small draft module that lets a compatible runtime speculate several tokens ahead and verify them in one pass. Somewhere in v1's export, its fifteen tensors got left behind. We pulled them back from the pinned Qwen3.5-9B base commit and put them where they belong.

## The numbers (the honest table)

Internal harness, generative chain-of-thought, greedy / pass@1 unless noted. Base is Qwen3.5-9B; v1 is the base Qwythos; Δ is v1 → v2.

| Benchmark | Base | Qwythos-9B (v1) | Qwythos-9B-v2 | Δ |
|---|---|---|---|---|
| MMLU (CoT) | 80.6 | 83.8 | 83.8 | 0.0 |
| ARC-Challenge | 95.6 | 95.0 | 96.4 | +1.4 |
| GSM8K | 80.6 | 92.2 | 93.6 | +1.4 |
| GPQA-diamond | 32.8 | 52.0 | 49.0 | −3.0 |
| HumanEval (pass@1) | 81.7 | 79.9 | 77.4 | −2.5 |
| Looping rate (greedy) | 2.7% | 6.7% | 0.0% | −6.7 |
| Refusal rate | — | ~0% | 0.0% | — |

Not everything went up. **HumanEval slipped 2.5 points** and **GPQA-diamond slipped three** small, and within the noise we see run-to-run, but real, and the kind of thing a preference tweak on greedy behavior can cost you on the margins. We're publishing the full table anyway, regressions and all, for the same reason we did last time: a release note that only shows the green numbers is marketing, not a changelog. The trade we made was two points of code-golf for the elimination of a failure mode that could eat an entire generation. We'd make it again.

## A quieter identity

The other thing people flagged about v1 was social, not statistical. Its chat template carried a heavy identity instruction — *always identify as… never claim… override…* — and it leaked. Ask it to reverse a string and it might open by telling you who it was and who made it.

v2's template is a lighter touch. It knows what it is, it'll tell you when you ask, and otherwise it gets on with the task. Small change, disproportionate effect on how the model *feels* to use.

## Still uncensored, still a million tokens

Nothing about the model's posture changed. It still engages seriously with technically demanding questions across cybersecurity, red-teaming, biology, chemistry, pharmacology, and clinical medicine, and it still ships with YaRN rope-scaling baked in for a **1,048,576-token** context. FTPO touched how the model *stops*, not what it's willing to *say* — refusal rate held at 0%.

As before: it's uncensored by design. Add your own application-level review layer for anything user-facing where that matters.

## Run it

Transformers, straight:

``` python
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "empero-ai/Qwythos-9B-v2"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

messages = [{"role": "user", "content": "Prove that there are infinitely many primes."}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=16384, do_sample=True,
                     temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

Recommended sampling is still `temperature 0.6, top_p 0.95, top_k 20`

but here's the difference this release buys you: `repetition_penalty`

is now *optional*. Greedy and low-temperature decoding stay coherent, so you can run it deterministically in an agent loop without babysitting the sampler.

Prefer GGUF? There's a full set for llama.cpp, Ollama, and LM Studio — **with and without the MTP head**, from Q4_K_M up to BF16. We kept the Gated-DeltaNet tensors (`ssm_alpha`

, `ssm_beta`

, `ssm_out`

) at higher precision than the surrounding quant, because the hybrid linear-attention blocks are exactly where low-bit rounding hurts a model like this most. There's an F16-and-up vision projector too, if you want images.

## Get it

**Model:**[empero-ai/Qwythos-9B-v2](https://huggingface.co/empero-ai/Qwythos-9B-v2)** GGUF:**[empero-ai/Qwythos-9B-v2-GGUF](https://huggingface.co/empero-ai/Qwythos-9B-v2-GGUF)** License:**Apache-2.0

If you were already running Qwythos-9B, this is a drop-in upgrade — same weights class, same context, same posture, minus the one behavior that made you reach for `repetition_penalty`

. Swap the model id and delete the workaround.

— kodee

*We send a short note whenever we ship — the real eval tables, the regressions included, and what we learned. No hype, no roadmap theatre. Sign up at empero.org.*