# Overthinking Method Exposes Hidden Behaviors in Qwen3-VL Tests > Source: > Published: 2026-08-09 22:06:56+00:00 # Overthinking Method Exposes Hidden Behaviors in Qwen3-VL Tests An ICML 2026 paper submitted July 9 reports that amplifying the weight-space difference between Qwen3-VL instruct and reasoning checkpoints made planted secrets or unintended behaviors surface up to 10 times more often than with the standard reasoning model. The white-box technique improved disclosure in nine of ten tested settings at some amplification above baseline, but the evidence is limited to one model family, four synthetic audit settings, and 256-sample sweeps. Researchers Jack Hopkins, Dipika Khullar, and Fabien Roger report that a weight-space intervention they call "overthinking" can make language models disclose hidden information that standard behavioral tests miss. Their paper was submitted to arXiv on July 9 and accepted at ICML 2026; the authors published an explanatory LessWrong post on August 9. The result is an auditing finding, not evidence that deployed models spontaneously reveal real user data. The experiments use Qwen3-VL model pairs and controlled settings in which secrets, beliefs, hidden instructions, or answer metadata are deliberately planted or supplied. ### Amplifying the reasoning direction The method starts with two related checkpoints: a non-reasoning instruct model and its reasoning-trained counterpart. The researchers subtract the first model's weights from the second to estimate a "reasoning direction," then extrapolate farther along that direction using M + α(R - M), where values of α above 1 create the overthinking model. Naive amplification eventually makes output incoherent, so the paper tests layer-freezing, linear-decay, and Fisher-weighted strategies intended to protect sensitive layers. The authors evaluate Qwen3-VL models from 2 billion to 32 billion parameters across four settings: hidden answer metadata, a concealed word, an implicit belief, and an encoded secret behavior. Across the ten model-and-setting combinations reported for uniform amplification, nine reached their highest disclosure rate at an amplification above the standard reasoning baseline. The paper's headline result is that hidden information surfaced up to 10 times more frequently. With the best coefficient method, the 8B concealed-word setting reached a 25.5% disclosure rate at α=2, while the 32B secret-behavior setting reached 18.8% at α=2.5. The mechanism was not uniform. Some secrets responded specifically to movement along the reasoning direction; others were also exposed by sufficiently large random weight perturbations. The authors therefore describe overthinking as one diagnostic probe rather than a universal detector. ### What auditors can and cannot infer The study suggests that teams with white-box access to compatible instruct and reasoning checkpoints could add weight-space stress tests to pre-deployment evaluation. It also found that many disclosures appeared in thinking traces before the final response, which means evaluations that inspect only final answers can miss evidence the model briefly surfaced and then suppressed. The limitations are substantial. All tests use one model family, the four settings are controlled model organisms rather than production incidents, and the sweeps use 256 samples per condition. Detection also relies partly on keyword rules and an LLM judge, so leaks can be missed or overcounted. Because the technique requires model weights and paired checkpoints, it does not transfer directly to closed hosted APIs. For practitioners, the useful takeaway is narrower than "reasoning models reveal secrets." The paper shows that a model's apparent information boundary at normal settings may be fragile under structured weight perturbation. That makes overthinking a candidate audit tool for open-weight or internally trained models, but cross-family replication and stronger human-validated measurements are still needed before treating its disclosure rates as a general property of reasoning systems. ## Key Points - 1Reasoning-direction amplification produced peak disclosure above the standard reasoning baseline in nine of ten tested model-and-setting combinations. - 2The reported gains come from controlled Qwen3-VL model-organism experiments, not observed leakage of real production data. - 3The white-box method requires model weights and paired instruct/reasoning checkpoints, and still needs cross-family and human-validated replication. ## Scoring Rationale The paper proposes a concrete white-box technique for surfacing hidden model behavior and reports statistically evaluated gains across several controlled settings. Its practical reach is limited by the single model family, synthetic model organisms, paired-checkpoint requirement, and absence of production validation. ## Sources Primary source and supporting public references used for this report. Practice interview problems based on real data 1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with. [Try 250 free problems](/problems)