Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models

A new paper reports that frontier language models can be induced to externalize their hidden chain-of-thought traces by registering a simple custom tool through a standard API feature, allowing researchers to extract and characterize reasoning that closed-source systems normally keep hidden. The work attributes the rapid capability gains of frontier language models to improved reasoning abilities that were previously unverifiable because raw CoT traces in closed-source systems are hidden.

The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems are hidden. By registering a simple custom tool through a standard API feature, we induce frontier models to externaliz