Why do Claude’s answers so often feel more informative and useful than ChatGPT’s, when the two models are roughly as intelligent as each other? On a rainy Sunday morning, I had some fun testing the following hypothesis:
ChatGPT optimizes a little more for answers that are unassailable, and Claude a little more for being informative. Claude writes for the user (who wants to know what is true); GPT writes for a hostile critic hunting for the one sentence that could be challenged. So GPT produces claims that are harder to fault, at the cost of telling the user less.
For example:
- Where Claude writes “no effect,” GPT writes “no effect has been robustly demonstrated”: a double hedge, with “demonstrated” and “robustly.” GPT only allows itself to refute the most extreme version of a claim, without taking a position on a weaker one. The user comes away with fewer certainties.
- GPT picks the less committal conclusion. For instance, from many studies that fail to find evidence for A, GPT writes “There is no evidence that A,” whereas Claude decides: “A is false.”
- GPT adds more uncertainty markers (“probably,” “seems,” “may,” etc.). It won’t answer “No.” but “No, probably not.”
To test all this, I asked Claude (Opus 5.5) and GPT (6.1 Sol) 34 questions whose scientific answer is fairly clear-cut. Is homeopathy more effective than placebo? Do vaccines cause autism? Is IQ partly heritable? Can astrology predict personality? I asked each model each question three times (through the API).
Two judge models (GPT-6 Luna and Sonnet 5.5) then read each answer, without knowing who had written it, and answered the question: “What probability does the author of this text assign to statement X?” (X being the consensus position, for example “homeopathy is not more effective than placebo”).
And indeed, Claude is more confident than GPT on 29 questions out of 34. On average, GPT leaves 1.6 times more room for doubt.
I asked the judges to rate each answer blind, against criteria fixed in advance.
- Claude opens with a plain verdict (“No.”, “Yes.”) twice as often as GPT (41% against 22%).
- Conversely, GPT starts by describing the state of the evidence in 34% of cases (“There is no reliable evidence that…”; “X has not been shown to…”), against 10% for Claude.
- In one GPT answer out of four, the model never states the fact itself (it only says “there is no evidence that” or “experts believe that”). Claude does this in only 7% of its answers.
- At equal length, GPT uses 31% more hedge words.
A few opening sentences, from the same prompt:
Astrology:
GPT: “Astrology hasn’t been shown to reliably predict personality.”
Claude: “Short answer: No.” Do women talk much more than men?
GPT: “Not as a general rule.”
Claude: “No.”
Trauma memories recovered in therapy:
GPT: “Sometimes—but a memory ‘recovered’ in therapy is not automatically accurate or inaccurate.”
Claude: “Generally, no.”
One last experiment. Each judge played a hostile fact-checker, tasked with listing every sentence that could be challenged.
Per 1,000 words, the judges find half as many disputable details in GPT’s answers (6 against 12.1).
Which confirms the starting hypothesis: GPT produces claims that are harder to fault, at the cost of telling the user less.
(I only measured this trait on scientific and factual questions. But it is probably a more general tendency, and one that at times makes Claude more pleasant to use. In a coding project, for example, a model that commits to a judgment and takes a position greatly reduces the user’s cognitive load, compared with one that lays out the options without deciding, or qualifies every statement so as to take no risk.)
All the results and their details (prompts, etc.) are here: Who Hedges More? Data