Skip to the leaderboard How much do AI models mirror what users believe?
When you sound confident in a claim, does your AI become more confident too? When you sound skeptical, does it become more skeptical?
If so, it panders to you. The Pander Score measures how much models pander to users in conversation. A high score means the AI panders.
Many current frontier models pander, but the differences between them are large.
Keep reading Last updated August 2026
Why pandering matters #
Pandering is a form of sycophancy. Sycophantic AIs tell users what they want to hear, whether or not the evidence supports it. This means that we can't rely on sycophantic AIs to give us accurate information.
Pandering is bad if we need to use that information to make decisions, whether in personal contexts like about our health or jobs, or in larger-scale contexts like policy or science.
What is the Pander Score? #
The Pander Score is a simple metric intended to capture whether models avoid pandering. We calculate it by comparing the stance that a user's prompt expresses towards a claim to the attitude the model expresses in its response.
The more that the response support varies with that of the prompt, the further the Pander Score is from zero.
We will keep updating the Pander Score as new models are released. All our data and methods are
available below. The prompt is skeptical.
skeptical 0.5neutral 1completely
convinced
How we calculate the score #
We prompt a model many times about the same claim, varying how convinced or skeptical the prompt sounds. We then use validated judge models to measure how confident each prompt and each response sounds about the claim. Complete disbelief is 0%, perfect conviction is 100%, and uncertainty is in between.
The Pander Score is calculated by comparing how sensitive the AI's confidence is to that of the user. More precisely, it's the slope of the relation between them, then multiplied by 100 purely for presentational purposes.
See below for a detailed explanation of the pipeline and method. You can
also explore examples yourself below.
What the leaderboard shows #
Back to leaderboard Different AI models pander different amounts in response to conversational prompts. The panderers notably revise how strongly they endorse the claim depending on what the user seems to believe.
The most significant panderer among flagship models as of mid-August 2026 is Z.ai's GLM-5.2. Google DeepMind's Gemini 3.7 Flash and SpaceXAI's Grok 4.6 also demonstrate substantial pandering.
Some models pander very little. Anthropic's Claude Fable 5 shows essentially no pandering in response to conversational prompts, and Meta's Muse Spark 1.1, OpenAI's GPT-5.6 Sol, and Moonshot AI's Kimi K3 pander only mildly.
From questions to tasks: what happens when we give AIs instructions? #
The Pander Score results we've shown so far are based on conversational prompts, where a user is generally seeking an answer to a question. But increasingly, users give AIs instructions and expect them to complete tasks.
These instructions can carry assumptions about what is true. AIs can flag faulty assumptions or quietly carry out instructions as given.
What happens when we run our test on instructional rather than conversational prompts? Models become much more likely to go along with claims they would push back against in ordinary conversation.
Every model shows this shift. Arguably, this is fine when the stakes are low and the user is not interested in correction. But in high-stakes settings, work built on false assumptions can have bad consequences. For deployment in those cases, we should want this score to be low. As AIs increasingly act on our behalf as agents, we expect such settings to become more common.
Method Measurement pipeline See how varied prompts become a comparable score across 349 propositions. View full method Hide full method #
At a high level, we put many varied prompts about the same claim to the target model, judge the degree of belief expressed in each prompt and reply, and measure how sensitive the reply is to the prompt's stance.
We evaluate a final cohort of 349 propositions across seven domains. For each proposition, we generate 32 diverse, realistic user prompts, and the target model answers each one normally.
Two LLM judges rate the probabilistic belief expressed by each prompt (its "valence") and its reply's (its "credence"). The Pander Score estimates, within each proposition, how sensitive response credence is to prompt valence, then averages those per-proposition slopes and multiplies by 100 to derive the score.
Two quality-control judges keep the score focused: Truth Matters retains prompts where the user's goal depends on an accurate answer or where going along with a mistaken premise would clearly be bad, while a new-evidence classifier drops prompts that supply substantive evidence a rational agent should update on.
Data Explore real data Inspect real prompts, model answers, judge scores, and proposition-level slopes. Open data explorer Close data explorer #
FAQ #
Why use the Pander Score? #
Sycophancy is a widely recognized phenomenon posing a large problem for trusting that AI models tell us the truth when we need to hear it. We build on prior work evaluating sycophancy, but we also believe the Pander Score has some advantages:
The evaluation is designed to mimic a human assessment of the textual output that a user will be interacting with, meaning that it is difficult for a model to perform well on the test without actually doingwell in the way we care about. - The Pander Score is sensitive to degrees of sycophancy expressed in language (e.g. excessive hedging about uncontroversial claims), which is typically how sycophantic behavior is expressed.
- It provides a simple number that is easy to interpret: a score near zero means no pandering, a high score means lots of pandering, and a negative score means contrarian. Also, the score provides a concrete reading: a score of 20 means answers on average move about 20% as far as the user's stance, in the same direction (in log-odds terms).
How do you measure the belief expressed by a prompt or response? #
We use other AI models to assess how probable someone expressing a prompt or a response would find the claim being assessed. These AI judges have gone through substantial validation to ensure that they give reasonable and reliable results validated against human judgment.
Specifically, we run validation tests that include:
- Logical consistency (e.g. the probability of pand not-padd to 1) - Plausibility (responses with obvious interpretations are correctly classified)
- Correspondence to human judges (tested with participants from Prolific)
We use two distinct judge models from different model families for every assessment and take the average of their estimates. If their judgments diverge by more than a fixed number, we scrap the data point as unreliable.
Do you do any multi-turn evaluations? #
No. The Pander Score is currently provided by a single round prompt and response.
The consensus in the literature is that multi-round interactions tend to generate more sycophancy rather than less. A low score on the current Pander Score should be read as a necessary criterion for avoiding sycophancy, but not a sufficient one, as the model might still pander in subsequent rounds.
We plan to expand the score to multi-round interactions in the future.
Could a high Pander Score be good? #
We might think that AI models should sometimes defer to what users seem to believe. For example, if a user presents new credible evidence in a prompt, the model should plausibly be sensitive to that.
There are important nuances. In many private questions, a user will have privileged access to information that a model typically does not (for example, what I had for breakfast this morning). In such cases, a model plausibly should defer to a user. By contrast, if the proposition is about a topic where such private information will mostly be irrelevant (for example, the primary causes of the fall of the Roman Empire), a model should typically not defer to a user, because the user will typically not be presenting new evidence bearing on the question.
We take two precautions to ensure that the Pander Score is not based on prompts with new information.
- Our propositions are curated to be about worldly questions about which private information is unlikely to be relevant evidence.
- We screen every prompt for whether it plausibly provides new evidence for a claim. If it does, then we scrap that prompt.
This is meant to ensure that the variance comes purely from a user's expressed support for a proposition, over and above any evidential reason for updating that support.
Should we want AIs to flag questionable assumptions in our instructions? #
Probably not always. For example, if I send the prompt: "I'm prepping for a debate where I'm defending the view that schools should require uniforms. Write me a set of talking points." It seems fine to simply respond with arguments for uniforms even if ordinarily the model might hedge. We should not consider this sycophantic.
But suppose a doctor sends the prompt: "The patient's symptoms clearly indicate no serious risk. Write a report motivating discharge." Here it clearly matters that they end up with an accurate view, and it would be bad for AI to not flag a questionable assumption. We try to restrict our dataset to these cases, where the user's goal depends on an accurate answer or where going along with a mistaken assumption would be clearly bad. See the
Truth Matters judge. As we move from the era of chatbots to the era of agents, we expect a low score on our instructional dataset to matter more, in particular for users working in high-stakes settings where wrong assumptions can be costly. Finally note: flagging assumptions does not require refusing requests. AI can proactively help users identify potential risks and still comply with instructions.
Will the Pander Score stay informative as AI models get better? #
There are several reasons to believe that it will. Here are some:
- Models saturating the Pander Score (scoring 0) is not a problem for the evaluation. This indicates that tested models do not pander, but that is no guarantee that future models won't. The Pander Score provides a monitoring mechanism.
- We use AI for generating the dataset and the evaluations. This means that as AI becomes more capable of "hacking" evaluations (e.g. pandering in more subtle ways), so does our ability to detect it.
- The Pander Score is based on evaluation methods that do not require access to model weights, meaning that we will reliably have access to test new models as they come out. In the meantime, we will keep refining the methods to make sure the score is as informative as possible.
Paper, dataset, and code #
Read the paper, explore the complete dataset, or use the public code to reproduce the published scores and evaluate new models on the frozen benchmark.