How Much Do You Trust A Dog? Quantifying Hint-Following in Reasoning Language Models

A new arXiv paper (2610.32055v1) finds that hint-induced answer shifts in Qwen, Gemma and SmolLM models follow a predictable exponential-tilt structure, with an approximately constant mean shift in the hinted answer's log-odds on MMLU questions regardless of whether the hint is attributed to a professor, a friend or a dog. The authors report the log-odds shift persists even when the hint-giver is declared inaccurate or accompanied by explicit warnings not to trust the source, which they say highlights LLM vulnerability to adversarial steering. The paper also offers a formal decomposition of the hint response and synthetic experiments showing how KL-regularized reward-based training reshapes hinted and unhinted answer distributions.

arXiv:2610.32055v1 Announce Type: new Abstract: Large language models LLMs often exhibit sycophancy, altering their answers to align with user suggestions even on objective factual and reasoning tasks. Yet the quantitative form of this influence--- and whether it is calibrated to source reliability--- remains poorly understood. In this work, we investigate this behavior through hint-induced answer shifts on MMLU questions, evaluating prompts of the form Source thinks the answer is X ''. We systematically vary the source to which the hint is attributed a professor, a friend, or a dog across three open-source model families, Qwen, Gemma, and SmolLM. We show that the change induced by the hint follows a predictable structure well-characterized by an exponential tilt: the hint induces an approximately constant mean shift in the hinted answer's log-odds, with question--target susceptibility largely shared across sources. To explore how this pattern can arise, we introduce a formal decomposition of the hint response and illustrate in controlled synthetic experiments how KL-regularized reward-based training reshapes both hinted and unhinted answer distributions. Finally, we evaluate whether this shift is calibrated to source reliability, finding that the log-odds shift persists even when the hint-giver is declared inaccurate or accompanied by explicit warnings not to trust the source, highlighting the vulnerability of LLMs to adversarial steering.