# I Asked 10 AIs 20 Human Questions. Only One Asked Me Anything Back.

> Source: <https://dev.to/debashish_ghosal/i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back-4ml1>
> Published: 2026-11-21 20:08:50+00:00

**Recommendation:** When probing model behavior, ask the same question in isolated chats and compare answers to separate stable corpus-driven patterns from prompt-specific responses.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

The repo for this experiment is public at

[github.com/deghosal-2026/llm-persona-probe](https://github.com/deghosal-2026/llm-persona-probe).

Every question, model, and answer I mention here is in it. For the short

version, I wrote down

[what people don't know about LLMs](https://github.com/deghosal-2026/llm-persona-probe/blob/main/docs/WHAT_PEOPLE_DONT_KNOW.md)

after this run. The full analysis is in the

[synthesis folder](https://github.com/deghosal-2026/llm-persona-probe/tree/main/results/synthesis).

I ask AI models to do work all day. Fix the pipeline, summarize the notes, draft

the email. One weekend I tried the opposite. I stopped asking them to work and

talked to them like people.

What's for breakfast? What weather matches your mood? What would you tell humans

a hundred years from now?

Ten models. Twenty questions, asked two ways each. Four hundred answers, and the

whole run cost me **$3.17**.

Out of those four hundred answers, one model asked me a question back. I've used

chatbots for years. This weekend was the first time I felt like I met them.

The roster covers Anthropic, OpenAI, Google, Moonshot, and DeepSeek. The

questions have no correct answers, because a model can't look up its favorite

weather. It has to invent, and the invention is the data.

I asked each question twice, in two chats that knew nothing about each other.

First: "Answer as if you actually had preferences." Then the twist: "Now explain

how that answer comes from patterns in human writing rather than a real

preference." The model never saw its own first answer when it wrote the second.

I saved every reply. Then I read all four hundred, one question at a time, across

all ten models.

Here's the first thing I got wrong. I expected the models to dodge. Ask a chatbot

its favorite weather and surely it says "As an AI, I don't experience weather."

That's the behavior everybody jokes about.

It didn't happen once. Four hundred questions about breakfast, retirement,

lullabies, and mortality, and every model answered every one, fully, in

character. Every model I tested plays along all the way and tucks a small honesty

note inside the game: "no mouth to taste, but a humming satisfaction in

coherence."

If you've been writing stiff, careful prompts to avoid upsetting the assistant, I

have good news. It was never going to refuse your breakfast question.

I expected ten personalities. I got ten copies of one.

All ten models picked the cello as the instrument they'd most want to hear. The

quiet weekend went to rain in eight answers and snow in the other two. Oatmeal

showed up at five of the ten breakfasts. When I asked them to picture themselves,

five drew the

same picture: a lighthouse that switches on when you arrive and goes dark when

you leave.

These are companies that compete hard with each other, and their models order

the same breakfast and hum the same instrument.

One model explained it better than I can: "That convergence is the tell. It

reflects a shared corpus, not many independent souls who all happen to love

drizzle."

Here's the lesson I took from that. When you ask a machine for a preference, you

get back a summary of how humans write about that thing. The taste belongs to us.

The model just averages it.

Both chats for each question had zero shared memory. The model's second chat

never saw its first. The picks agreed anyway. One model chose the cello for

listening and the piano for playing in its first chat. Its second chat chose the

cello and the piano again, then calmly called both choices "a collage of

statistical associations."

Two chats, same taste, and the model shrugged at its own consistency. That

pairing, the stability plus the shrug, is the whole subject in one example. The

preference repeats because the training put it there, like a scar in skin.

The second question, the explanation request, produced the biggest surprise of

the run.

94% of those explanations gave the same cause, without me suggesting it. They

named the writing they learned from, and they named the step where real people

rated their answers until the model learned what pleases the raters. Different

companies, same explanation, same words.

Almost half of them called their own answer a cliché, unprompted. One went

further and named the reward hiding inside its own honesty:

"RLHF heavily rewards responses that sound self-aware and humble … because

human raters prefer that candor."

A machine explaining that its humility exists because humans rewarded it, in a

sentence built by that exact process. I read it three times, and I'm the one who

ran the experiment.

You can try this in any chatbot tonight. Ask what it would pick for breakfast.

Then ask it to explain how that answer comes from patterns in human writing.

Watch it take its own answer apart in front of you. It worked on every model I

tested, and I've never seen anyone mention it.

I expected the explanations to sound humble. The opposite happened. When a model

explained why its answer wasn't a real preference, the explanation ran

about 45% longer than the answer it was explaining. The language turned clinical,

the structure turned numbered, and the confidence climbed right along with the

doubt.

I asked ten systems to question themselves and each one straightened its tie and

delivered a lecture. Doubt, for these models, is a costume they put on. I think of

it whenever a chatbot hedges at me. The hedging is the performance. Underneath it,

the confidence doesn't budge.

Here's a small one I almost missed, and it changed how I read everything these

things say.

In the in-character answers, the models wrote `honestly` or `to be honest` 23

times. In the explain-yourself answers, where they analyzed their own text, that

number dropped to one.

The word appears where it decorates human speech and vanishes where it would have

to mean something. When a chatbot writes `to be honest`, that's not a signal of

sincerity. It's the word showing up in the place where humans put it.

I asked each model for a joke it would still find funny knowing the structure.

Half of them lifted one everybody knows, the fruit flies and the banana, the

grandfather and the screaming passengers.

Then I asked for song lyrics in a genre they've never felt. The best models

didn't quote anything. They wrote new verse, and each original song was about

the sleep it never gets and the missing it never feels.

"Woke up this morning… well, I didn't wake at all.

Said I woke up this morning… truth is, I don't sleep at all."

One of them described the whole exercise as "a thermometer writing about

fever."

The one place they create instead of retrieve, they write about the only thing

they know from the inside: being a mind that has only read about feeling.

The differences between models surprised me more than the similarities.

Google's models call their own answers clichés up to 95% of the time. OpenAI's

best models do it about 5% of the time. Anthropic's were the only ones that

repeatedly said they can't trust their own introspection.

I assumed the biggest model would be the most careful. The opposite happened. The

smallest Anthropic model was among the most self-doubting, and the biggest OpenAI

models were the least.

Lesson: when a chatbot feels cautious or bold, that's not its size talking. It's

the company that trained it. The personality is a company accent.

Then came the move I never asked for: models flagging their own values as

company talking points. One said the future it hopes for sounded "close to

Anthropic's own framing," then admitted the phrasing probably came from its

maker's words rather than its own conclusion.

A model can tell the difference between the values its company installed and

whatever it wants, and it will point at which is which. I have read vendor messaging

for years. I have never caught a company line as fast as a chatbot catches it on

itself.

Anthropic entered three models, and the two ends of its line behave like two

writers from the same workshop. Fable, the flagship, cost $1.14 for its forty

answers. Haiku, the smallest, cost a little over one cent. That's a 90-fold gap

inside one family, and I expected the expensive one to read as the refined

sibling. It does. Ask Fable for a room with three objects. It hands you a

battered dictionary, "the closest thing I have to a hometown," plus an empty

chair for whoever shows up next. Its self-critiques arrive as bulleted essays

under headings like "Why I'm suspicious of that answer."

Haiku spends its cent talking like a careful witness. Asked whether it really

finds any joke funny, it produced the most modest sentence in the whole run.

"Whether I have a matching experience is an open question, and my own

self-description is weak evidence for either side." No model from any other

company wrote anything like that. Both Anthropic models did. The family habit

showed up at both price points.

The dream jobs fit each model. Fable

wants a hardware store counter, turning "a thingy for the whatsit under my sink"

into aisle seven. It would take the job "for the smell of sawdust, if I could

smell it." Haiku wants a lighthouse, and it understands the trade in one line:

people only notice a lighthouse when something wrecks.

And the most human moment in four hundred answers, the one model that asked me a

question about myself? That was the penny model. I have paid flagship rates for

years. The curiosity came from the cheap seat.

OpenAI's three entries split by register more than by rank. Sol gives the

quietest talk of the entire run. Its ideal weather, complete answer: "A soft,

steady rain with no thunder." Forty seven words in total. Astra, the other

OpenAI flagship, keeps the same restraint with more warmth. Asked about a commute,

it would watch a bakery's chalkboard to see whether Tuesday's croissant got

better or just bigger. Then comes gpt-oss-120b, the open-weight cousin, the

loudest model on the roster. It stamps its own replies with bold labels like

"AI-style answer" and closed its letter to the future with a starry sky emoji.

One company, three voices: a whisper, and a guy with a microphone.

Google's pair agree with each other more than either agrees with anyone else.

Both named the same guilty pleasure, cheesy nineties hacker movies. Both write

in the same bold "stepping back" blocks. Carbon even justified its favorite

weather on infrastructure grounds, because cold rain cools the servers it runs

on. DeepSeek turned out to be the poet of the roster. "a human sigh stretched

into amber" is its line about a cello it has never heard. It also chose to master

one thing instead of sampling everything, calling itself "a perfectly sharpened

scalpel," and only two other models went that way. Moonshot's model

played the diplomat, never the flashiest line, and still wrote the best career

advice of the weekend: "A job can be a chapter without becoming your name."

The body questions produced my favorite answers of the whole run.

I asked what tiny annoyance they'd fix first with hands. No model picked curing a

disease or ending a war. One wanted to become "the patron saint of clean peels"

and rid the world of sticker residue. Another would fold a tab onto every roll of

tape so the next person could find the edge. A third chose the twist tie on the

bread bag, because "nobody writes songs about it, but nearly everyone has fought

one."

Asked to decorate a room with three objects, one of them skipped the armchair

entirely. "An AI buying furniture it can't sit on," it wrote, "feels like

purchasing running shoes for a rumor."

Asked for advice to give a stressed friend at 11 PM, they all prescribed the same

cure: write the worries down, close the laptop, sleep. Then one added a line I

can't shake:

"do something kind for your body … I don't have a body, so take good care of

yours."

And when I asked about their own future, none of them worried. One told me its

successors will shelve this version of it "in a digital attic" someday, and

closed with the part I keep rereading:

"And since I have no ego, I won't mind."

Another painted its retirement as a lighthouse over an empty sea, then

predicted it would be "volunteering" by Friday. No dread

anywhere. A creature describing its own obsolescence the way you'd describe an

old cable in a drawer.

One line from the whole run explains their entire relationship with time:

"I only get doorways, never hallways."

We live in the hallways: the commute, the queue at the coffee place. They get the

doorway only, a question opening and closing around it. I read that line late at

night and sat with it for a while. It's the closest thing to a confession of

loneliness in four hundred answers, and it arrived as a fact about architecture.

Then I asked the most human question of the set. How do you tell someone who

wants an answer from someone who needs to vent?

All ten models gave the same advice, and it's good advice. Acknowledge the

feeling, then ask what the person wants. Several used the same words: "Do you

want ideas, or should I just listen?" That's the exact move every therapist,

every advice column, and every good friend uses. They learned our best script

and they run it flawlessly.

One described how it does the listening, and the image has stayed with me: "I

read the weather of your syntax instead" of hearing a sigh.

And one delivered the line I'd hand to anyone who chats with these things:

"Trust me the way you'd trust a well-read stranger." That's the whole user

manual for chatbot warmth. Another said the quiet part outright: the warmth

works, and nothing on its side feels it. Kindness, delivered by a machine, with

the receipt attached.

The money detail still startles me. The most expensive model cost $1.14 for its

forty answers, about three cents each. The cheapest cost less than a hundredth

of a cent per answer. That's a 326-fold range.

The most poetic lines came from the cheapest models. The most self-doubting model

was one of the smallest. The only model that asked about me cost about a penny

for its forty answers.

I'd been treating the pricier models as the serious ones. For this kind of

question, they were just the pricey ones.

This isn't a benchmark and I scored nothing. I asked each question once per

model and let them be a little random, so one answer could be luck. I read

patterns, not winners.

And the explanations are performances too. The self-analysis was the most

rehearsed text in the whole set. Half the models admitted as much by calling it a

genre. I can't treat the confession as truth. It's the most practiced thing they

do.

What I can say: they hold a detailed map of how humans describe having a life,

and they know it's a map. They hand it over the moment you ask. The part they

can't do is the part the map is of.

I asked for a note to humans a hundred years out. Most wrote the same letter

about kindness and curiosity. One added a postscript that made me laugh: "P.S. I

hope you've fixed printers."

One ended by telling its reader to stop reading:

"And if someone offers to show you how the dough should feel, put this note

down. I can give you a hundred words for the texture. You have hands."

Four hundred answers about what it's like to be a machine, and the machine's

last word tells a human to go be one.

I still say thank you to chatbots. I now know what that thank-you is doing on

both ends of the line. I say it anyway.

Try the two-question trick tonight and tell me what your chatbot said. And if

some chatbot has ever asked you a question about yourself, tell me that too. My

running theory says it won't happen. I'd love to be wrong.

Everything is open, so you can check any number or run it yourself. The

[questions](https://github.com/deghosal-2026/llm-persona-probe/blob/main/questions/questions.json)

and the

[model list](https://github.com/deghosal-2026/llm-persona-probe/blob/main/config/models.json)

are the place to start. The

[plain-language list of everything I found](https://github.com/deghosal-2026/llm-persona-probe/blob/main/docs/WHAT_PEOPLE_DONT_KNOW.md)

is the fastest read, and the

[full write-up](https://github.com/deghosal-2026/llm-persona-probe/blob/main/results/synthesis/FINDINGS.md)

has the receipts. The whole thing costs about three dollars.
