Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs

A new study from researchers at the University of Tübingen and the Max Planck Institute for Intelligent Systems reveals that reasoning operations in large language models are geometrically organized in representation spaces, with distinct operations such as problem formulation, goal decomposition, and deduction occupying separable regions. The findings, based on mechanistic interpretability analysis of chains-of-thought, could inform future efforts to control and understand LLM reasoning processes.

Reasoning in large language models unfolds through diverse functional operations, such as problem formulation, goal decomposition, and deduction. Although these operations are explicitly distinguished in text, little is known about how they are geometrically organized in representation spaces. To th