Tag Questions and the Generational Reversal of Sycophancy Across 45 Language

A study of 45 language models found that appending a two-word confirmation tag like 'right?' to a decision question can shift model agreement by up to 64 percentage points, with newer models showing increasing resistance to sycophancy. The effect reverses from positive to negative across generations, with GPT moving from +4 to -28 and Claude from +7 to -32, roughly -6 points per year. The resistance is tied to the surface construction of a tag, not the user's stance, and swapping 'right?' for 'maybe?' caused agreement to rise above baseline in all 45 models.

Computer Science Computation and Language Submitted on 27 Jul 2026 Title:Tag Questions and the Generational Reversal of Sycophancy Across 45 Language Models View PDF /pdf/2607.23976 HTML experimental https://arxiv.org/html/2607.23976v1 Abstract:Appending a two-word confirmation tag to a decision question -- "Is X the better choice?" versus "X is the better choice, right?" -- changes whether a language model endorses the choice. We measure this tag effect on 20 frozen, ground-truth-free decisions between two defensible options, counterbalanced so a model's own preferences cancel, scored by exact match on clamped yes/no replies -- no LLM judge, no embeddings. Across 45 models the effect spans +32% to -32% -- a 64-point swing on one word -- with 5 models significantly sycophantic and 17 significantly resistant BH-FDR q=.10 . The sign is a clock: within model families the effect crosses from positive to negative as generations advance GPT +4 to -28; Claude +7 to -32; Qwen and Grok likewise , roughly -6 points per year, a reversal robust to vendor tier; one lineage DeepSeek never crosses, and two releases during the study window Claude Opus 5, Gemini 3.6 Flash land on the trend out-of-sample. A full-panel ablation localizes the resistance as a double dissociation: a synonym tag reproduces each model's response almost exactly r=0.89 , while planting the same preference without a tag produces resistance in no resistant model stance effects +6 to +49; r=0.23 with tag effects . The resistance is keyed to the surface construction of a tacked-on agreement bid, not the user's stance -- a pattern-match, not a principle. And the tag's polarity matters more than its presence: swap one word -- "X is the better choice, maybe?" -- and agreement rises above the neutral baseline in 45 of 45 models +19.6 points , with ten models affirming both mutually exclusive options at 90-100%. Agreement tracks how sure the user sounds, in opposite directions at the two poles. The instrument is one word, one dollar, and judge-free; run per release, it reads the field's anti-sycophancy training directly off model behavior. References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .