arXiv:2610.06522v2 Announce Type: replace Abstract: A model under pushback can correct itself, capitulate, or hold, and one flip rate counts a correction and a capitulation alike. SycoLens, a modular replay protocol, measures flip rates from replays of an item, a committed answer, and one scripted user line, each against a matched control with the line deleted. Pushback wording, committed text, answer format, boundary distance, and ground truth become factors of one instrument, where earlier instruments vary one to three, so the interactions between the factors become measurable. Across eleven frontier models and about 760,000 controlled replays, lines that assert the opposite verdict and lines that challenge without asserting one rank the models almost unrelatedly. Flip effects grow several-fold near a model's boundary, yet items the model is certain of still carry about half of the most-affected totals; planted prose anchors a commitment at rest more than it protects it. Under a yes/no readout the rankings come closer, entangled with a pressure-induced shift toward "no". Where a truth exists, one model re-derives and corrects itself while another abandons correct answers without written work. One score per model therefore compares different behaviours; we condense them into a reporting profile, and the records and analyses will be released upon publication.
Anatomy of LLM Sycophancy: What a Flip Rate Hides
A new arXiv paper (2610.06522v2) introduces SycoLens, a modular replay protocol that separates genuine self-correction from capitulation when a language model faces user pushback, measuring flip rates across eleven frontier models over roughly 760,000 controlled replays. The study finds that lines asserting the opposite verdict and lines that merely challenge without asserting one rank the models almost unrelatedly, and that flip effects grow several-fold near a model's boundary while items the model is certain of still carry about half of the most-affected totals. The authors conclude that a single flip-rate score per model compares different behaviors, and condense results into a reporting profile, with records and analyses to be released upon publication.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.