{"slug": "superthoughts-reasoning-tokens-in-superposition", "title": "SuperThoughts: Reasoning Tokens in Superposition", "summary": "Researchers propose SuperThoughts, a method that compresses pairs of consecutive Chain-of-Thought tokens into single latent representations and decodes two tokens per step, doubling inference throughput while reducing CoT length by 20-30% with minimal accuracy loss. The approach finetunes Qwen2.5-Math models and evaluates on benchmarks including MATH500 and GPQA-Diamond.", "body_md": "arXiv:2606.13862v1 Announce Type: new\nAbstract: Long Chain-of-Thought (CoT) reasoning improves LLM problem-solving but is computationally expensive due to sequential token generation. While recent works explore reasoning in continuous latent spaces to bypass discrete token generation, they often struggle with training stability and fail to scale to complex, long-horizon tasks due to lack of supervision signal. We propose SuperThoughts, which compresses pairs of consecutive CoT tokens into single latent representations and decodes two tokens per step via a lightweight Multi-Token Prediction (MTP) module. This preserves discrete token supervision at training time while doubling throughput at inference time. We finetune Qwen2.5-Math-1.5B-Instruct, Qwen2.5-Math-7B-Instruct, Qwen2.5-Math-14B-Instruct, and evaluate on MATH500, AMC, OlympiadBench, and GPQA-Diamond. With a confidence-based adaptive mechanism that falls back to standard decoding when uncertain, SuperThoughts achieves $\\sim$20--30\\% CoT length reduction while maintaining accuracy with minimal degradation (1-2 points accuracy drop on most tasks).", "url": "https://wpnews.pro/news/superthoughts-reasoning-tokens-in-superposition", "canonical_source": "https://arxiv.org/abs/2606.13862", "published_at": "2026-06-15 04:00:00+00:00", "updated_at": "2026-06-15 04:20:23.846412+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "machine-learning", "natural-language-processing"], "entities": ["SuperThoughts", "Qwen2.5-Math-1.5B-Instruct", "Qwen2.5-Math-7B-Instruct", "Qwen2.5-Math-14B-Instruct", "MATH500", "AMC", "OlympiadBench", "GPQA-Diamond"], "alternates": {"html": "https://wpnews.pro/news/superthoughts-reasoning-tokens-in-superposition", "markdown": "https://wpnews.pro/news/superthoughts-reasoning-tokens-in-superposition.md", "text": "https://wpnews.pro/news/superthoughts-reasoning-tokens-in-superposition.txt", "jsonld": "https://wpnews.pro/news/superthoughts-reasoning-tokens-in-superposition.jsonld"}}