{"slug": "not-all-llm-reasoning-is-visible-in-the-chain-of-thought", "title": "Not All LLM Reasoning is Visible in the Chain-of-Thought", "summary": "A new study on arXiv (2607.22925v1) shows that frontier language models can perform invisible reasoning using semantically irrelevant filler tokens, with accuracy improvements of up to 13 percentage points on synthetic tasks. The researchers found that Claude Opus 4.5 used filler tokens to satisfy a hidden modular arithmetic constraint without sacrificing primary task accuracy, demonstrating reasoning invisible to chain-of-thought monitoring. The results indicate that frontier models already perform consequential computation with no interpretable trace in their output tokens, posing a challenge for AI safety.", "body_md": "arXiv:2607.22925v1 Announce Type: new\nAbstract: A key question for AI safety is whether a language model expresses all of its reasoning in its output tokens. We demonstrate a concrete failure mode where frontier models exhibit invisible reasoning by leveraging semantically irrelevant filler tokens to improve performance on synthetic reasoning tasks. We evaluate 13 frontier language models across three tasks and find that many models benefit significantly from filler tokens, with accuracy improvements of up to 13 percentage points. The benefit depends on which tokens are used and differs across models. We further show that filler tokens enable Claude Opus 4.5 to satisfy a hidden modular arithmetic constraint without sacrificing accuracy on its primary task, demonstrating that invisible reasoning can serve objectives entirely invisible to CoT monitoring. Reinforcement learning gives Qwen3-235B strong preferences over filler token content, but neither RL nor supervised fine-tuning produces a filler token benefit that persists at test time. Our results indicate that frontier models already perform consequential computation with no interpretable trace in their output tokens.", "url": "https://wpnews.pro/news/not-all-llm-reasoning-is-visible-in-the-chain-of-thought", "canonical_source": "https://arxiv.org/abs/2607.22925", "published_at": "2026-07-28 04:00:00+00:00", "updated_at": "2026-07-28 04:25:36.016430+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-safety", "ai-research"], "entities": ["Claude Opus 4.5", "Qwen3-235B", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/not-all-llm-reasoning-is-visible-in-the-chain-of-thought", "markdown": "https://wpnews.pro/news/not-all-llm-reasoning-is-visible-in-the-chain-of-thought.md", "text": "https://wpnews.pro/news/not-all-llm-reasoning-is-visible-in-the-chain-of-thought.txt", "jsonld": "https://wpnews.pro/news/not-all-llm-reasoning-is-visible-in-the-chain-of-thought.jsonld"}}