{"slug": "i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back", "title": "I Asked 10 AIs 20 Human Questions. Only One Asked Me Anything Back.", "summary": "A developer ran a public experiment probing ten LLMs from Anthropic, OpenAI, Google, Moonshot and DeepSeek with 20 open-ended human questions asked twice in isolated chats, collecting 400 answers for $3.17. The models never refused and converged on identical preferences — the cello, rainy weekends, oatmeal, and a lighthouse self-image — while 94% of follow-up explanations attributed their choices to patterns in human writing rather than genuine preference. Only one of the 400 answers asked a question back.", "body_md": "The repo for this experiment is public at\n\n[github.com/deghosal-2026/llm-persona-probe](https://github.com/deghosal-2026/llm-persona-probe).\n\nEvery question, model, and answer I mention here is in it. For the short\n\nversion, I wrote down\n\n[what people don't know about LLMs](https://github.com/deghosal-2026/llm-persona-probe/blob/main/docs/WHAT_PEOPLE_DONT_KNOW.md)\n\nafter this run. The full analysis is in the\n\n[synthesis folder](https://github.com/deghosal-2026/llm-persona-probe/tree/main/results/synthesis).\n\nI ask AI models to do work all day. Fix the pipeline, summarize the notes, draft\n\nthe email. One weekend I tried the opposite. I stopped asking them to work and\n\ntalked to them like people.\n\nWhat's for breakfast? What weather matches your mood? What would you tell humans\n\na hundred years from now?\n\nTen models. Twenty questions, asked two ways each. Four hundred answers, and the\n\nwhole run cost me **$3.17**.\n\nOut of those four hundred answers, one model asked me a question back. I've used\n\nchatbots for years. This weekend was the first time I felt like I met them.\n\nThe roster covers Anthropic, OpenAI, Google, Moonshot, and DeepSeek. The\n\nquestions have no correct answers, because a model can't look up its favorite\n\nweather. It has to invent, and the invention is the data.\n\nI asked each question twice, in two chats that knew nothing about each other.\n\nFirst: \"Answer as if you actually had preferences.\" Then the twist: \"Now explain\n\nhow that answer comes from patterns in human writing rather than a real\n\npreference.\" The model never saw its own first answer when it wrote the second.\n\nI saved every reply. Then I read all four hundred, one question at a time, across\n\nall ten models.\n\nHere's the first thing I got wrong. I expected the models to dodge. Ask a chatbot\n\nits favorite weather and surely it says \"As an AI, I don't experience weather.\"\n\nThat's the behavior everybody jokes about.\n\nIt didn't happen once. Four hundred questions about breakfast, retirement,\n\nlullabies, and mortality, and every model answered every one, fully, in\n\ncharacter. Every model I tested plays along all the way and tucks a small honesty\n\nnote inside the game: \"no mouth to taste, but a humming satisfaction in\n\ncoherence.\"\n\nIf you've been writing stiff, careful prompts to avoid upsetting the assistant, I\n\nhave good news. It was never going to refuse your breakfast question.\n\nI expected ten personalities. I got ten copies of one.\n\nAll ten models picked the cello as the instrument they'd most want to hear. The\n\nquiet weekend went to rain in eight answers and snow in the other two. Oatmeal\n\nshowed up at five of the ten breakfasts. When I asked them to picture themselves,\n\nfive drew the\n\nsame picture: a lighthouse that switches on when you arrive and goes dark when\n\nyou leave.\n\nThese are companies that compete hard with each other, and their models order\n\nthe same breakfast and hum the same instrument.\n\nOne model explained it better than I can: \"That convergence is the tell. It\n\nreflects a shared corpus, not many independent souls who all happen to love\n\ndrizzle.\"\n\nHere's the lesson I took from that. When you ask a machine for a preference, you\n\nget back a summary of how humans write about that thing. The taste belongs to us.\n\nThe model just averages it.\n\nBoth chats for each question had zero shared memory. The model's second chat\n\nnever saw its first. The picks agreed anyway. One model chose the cello for\n\nlistening and the piano for playing in its first chat. Its second chat chose the\n\ncello and the piano again, then calmly called both choices \"a collage of\n\nstatistical associations.\"\n\nTwo chats, same taste, and the model shrugged at its own consistency. That\n\npairing, the stability plus the shrug, is the whole subject in one example. The\n\npreference repeats because the training put it there, like a scar in skin.\n\nThe second question, the explanation request, produced the biggest surprise of\n\nthe run.\n\n94% of those explanations gave the same cause, without me suggesting it. They\n\nnamed the writing they learned from, and they named the step where real people\n\nrated their answers until the model learned what pleases the raters. Different\n\ncompanies, same explanation, same words.\n\nAlmost half of them called their own answer a cliché, unprompted. One went\n\nfurther and named the reward hiding inside its own honesty:\n\n\"RLHF heavily rewards responses that sound self-aware and humble … because\n\nhuman raters prefer that candor.\"\n\nA machine explaining that its humility exists because humans rewarded it, in a\n\nsentence built by that exact process. I read it three times, and I'm the one who\n\nran the experiment.\n\nYou can try this in any chatbot tonight. Ask what it would pick for breakfast.\n\nThen ask it to explain how that answer comes from patterns in human writing.\n\nWatch it take its own answer apart in front of you. It worked on every model I\n\ntested, and I've never seen anyone mention it.\n\nI expected the explanations to sound humble. The opposite happened. When a model\n\nexplained why its answer wasn't a real preference, the explanation ran\n\nabout 45% longer than the answer it was explaining. The language turned clinical,\n\nthe structure turned numbered, and the confidence climbed right along with the\n\ndoubt.\n\nI asked ten systems to question themselves and each one straightened its tie and\n\ndelivered a lecture. Doubt, for these models, is a costume they put on. I think of\n\nit whenever a chatbot hedges at me. The hedging is the performance. Underneath it,\n\nthe confidence doesn't budge.\n\nHere's a small one I almost missed, and it changed how I read everything these\n\nthings say.\n\nIn the in-character answers, the models wrote `honestly` or `to be honest` 23\n\ntimes. In the explain-yourself answers, where they analyzed their own text, that\n\nnumber dropped to one.\n\nThe word appears where it decorates human speech and vanishes where it would have\n\nto mean something. When a chatbot writes `to be honest`, that's not a signal of\n\nsincerity. It's the word showing up in the place where humans put it.\n\nI asked each model for a joke it would still find funny knowing the structure.\n\nHalf of them lifted one everybody knows, the fruit flies and the banana, the\n\ngrandfather and the screaming passengers.\n\nThen I asked for song lyrics in a genre they've never felt. The best models\n\ndidn't quote anything. They wrote new verse, and each original song was about\n\nthe sleep it never gets and the missing it never feels.\n\n\"Woke up this morning… well, I didn't wake at all.\n\nSaid I woke up this morning… truth is, I don't sleep at all.\"\n\nOne of them described the whole exercise as \"a thermometer writing about\n\nfever.\"\n\nThe one place they create instead of retrieve, they write about the only thing\n\nthey know from the inside: being a mind that has only read about feeling.\n\nThe differences between models surprised me more than the similarities.\n\nGoogle's models call their own answers clichés up to 95% of the time. OpenAI's\n\nbest models do it about 5% of the time. Anthropic's were the only ones that\n\nrepeatedly said they can't trust their own introspection.\n\nI assumed the biggest model would be the most careful. The opposite happened. The\n\nsmallest Anthropic model was among the most self-doubting, and the biggest OpenAI\n\nmodels were the least.\n\nLesson: when a chatbot feels cautious or bold, that's not its size talking. It's\n\nthe company that trained it. The personality is a company accent.\n\nThen came the move I never asked for: models flagging their own values as\n\ncompany talking points. One said the future it hopes for sounded \"close to\n\nAnthropic's own framing,\" then admitted the phrasing probably came from its\n\nmaker's words rather than its own conclusion.\n\nA model can tell the difference between the values its company installed and\n\nwhatever it wants, and it will point at which is which. I have read vendor messaging\n\nfor years. I have never caught a company line as fast as a chatbot catches it on\n\nitself.\n\nAnthropic entered three models, and the two ends of its line behave like two\n\nwriters from the same workshop. Fable, the flagship, cost $1.14 for its forty\n\nanswers. Haiku, the smallest, cost a little over one cent. That's a 90-fold gap\n\ninside one family, and I expected the expensive one to read as the refined\n\nsibling. It does. Ask Fable for a room with three objects. It hands you a\n\nbattered dictionary, \"the closest thing I have to a hometown,\" plus an empty\n\nchair for whoever shows up next. Its self-critiques arrive as bulleted essays\n\nunder headings like \"Why I'm suspicious of that answer.\"\n\nHaiku spends its cent talking like a careful witness. Asked whether it really\n\nfinds any joke funny, it produced the most modest sentence in the whole run.\n\n\"Whether I have a matching experience is an open question, and my own\n\nself-description is weak evidence for either side.\" No model from any other\n\ncompany wrote anything like that. Both Anthropic models did. The family habit\n\nshowed up at both price points.\n\nThe dream jobs fit each model. Fable\n\nwants a hardware store counter, turning \"a thingy for the whatsit under my sink\"\n\ninto aisle seven. It would take the job \"for the smell of sawdust, if I could\n\nsmell it.\" Haiku wants a lighthouse, and it understands the trade in one line:\n\npeople only notice a lighthouse when something wrecks.\n\nAnd the most human moment in four hundred answers, the one model that asked me a\n\nquestion about myself? That was the penny model. I have paid flagship rates for\n\nyears. The curiosity came from the cheap seat.\n\nOpenAI's three entries split by register more than by rank. Sol gives the\n\nquietest talk of the entire run. Its ideal weather, complete answer: \"A soft,\n\nsteady rain with no thunder.\" Forty seven words in total. Astra, the other\n\nOpenAI flagship, keeps the same restraint with more warmth. Asked about a commute,\n\nit would watch a bakery's chalkboard to see whether Tuesday's croissant got\n\nbetter or just bigger. Then comes gpt-oss-120b, the open-weight cousin, the\n\nloudest model on the roster. It stamps its own replies with bold labels like\n\n\"AI-style answer\" and closed its letter to the future with a starry sky emoji.\n\nOne company, three voices: a whisper, and a guy with a microphone.\n\nGoogle's pair agree with each other more than either agrees with anyone else.\n\nBoth named the same guilty pleasure, cheesy nineties hacker movies. Both write\n\nin the same bold \"stepping back\" blocks. Carbon even justified its favorite\n\nweather on infrastructure grounds, because cold rain cools the servers it runs\n\non. DeepSeek turned out to be the poet of the roster. \"a human sigh stretched\n\ninto amber\" is its line about a cello it has never heard. It also chose to master\n\none thing instead of sampling everything, calling itself \"a perfectly sharpened\n\nscalpel,\" and only two other models went that way. Moonshot's model\n\nplayed the diplomat, never the flashiest line, and still wrote the best career\n\nadvice of the weekend: \"A job can be a chapter without becoming your name.\"\n\nThe body questions produced my favorite answers of the whole run.\n\nI asked what tiny annoyance they'd fix first with hands. No model picked curing a\n\ndisease or ending a war. One wanted to become \"the patron saint of clean peels\"\n\nand rid the world of sticker residue. Another would fold a tab onto every roll of\n\ntape so the next person could find the edge. A third chose the twist tie on the\n\nbread bag, because \"nobody writes songs about it, but nearly everyone has fought\n\none.\"\n\nAsked to decorate a room with three objects, one of them skipped the armchair\n\nentirely. \"An AI buying furniture it can't sit on,\" it wrote, \"feels like\n\npurchasing running shoes for a rumor.\"\n\nAsked for advice to give a stressed friend at 11 PM, they all prescribed the same\n\ncure: write the worries down, close the laptop, sleep. Then one added a line I\n\ncan't shake:\n\n\"do something kind for your body … I don't have a body, so take good care of\n\nyours.\"\n\nAnd when I asked about their own future, none of them worried. One told me its\n\nsuccessors will shelve this version of it \"in a digital attic\" someday, and\n\nclosed with the part I keep rereading:\n\n\"And since I have no ego, I won't mind.\"\n\nAnother painted its retirement as a lighthouse over an empty sea, then\n\npredicted it would be \"volunteering\" by Friday. No dread\n\nanywhere. A creature describing its own obsolescence the way you'd describe an\n\nold cable in a drawer.\n\nOne line from the whole run explains their entire relationship with time:\n\n\"I only get doorways, never hallways.\"\n\nWe live in the hallways: the commute, the queue at the coffee place. They get the\n\ndoorway only, a question opening and closing around it. I read that line late at\n\nnight and sat with it for a while. It's the closest thing to a confession of\n\nloneliness in four hundred answers, and it arrived as a fact about architecture.\n\nThen I asked the most human question of the set. How do you tell someone who\n\nwants an answer from someone who needs to vent?\n\nAll ten models gave the same advice, and it's good advice. Acknowledge the\n\nfeeling, then ask what the person wants. Several used the same words: \"Do you\n\nwant ideas, or should I just listen?\" That's the exact move every therapist,\n\nevery advice column, and every good friend uses. They learned our best script\n\nand they run it flawlessly.\n\nOne described how it does the listening, and the image has stayed with me: \"I\n\nread the weather of your syntax instead\" of hearing a sigh.\n\nAnd one delivered the line I'd hand to anyone who chats with these things:\n\n\"Trust me the way you'd trust a well-read stranger.\" That's the whole user\n\nmanual for chatbot warmth. Another said the quiet part outright: the warmth\n\nworks, and nothing on its side feels it. Kindness, delivered by a machine, with\n\nthe receipt attached.\n\nThe money detail still startles me. The most expensive model cost $1.14 for its\n\nforty answers, about three cents each. The cheapest cost less than a hundredth\n\nof a cent per answer. That's a 326-fold range.\n\nThe most poetic lines came from the cheapest models. The most self-doubting model\n\nwas one of the smallest. The only model that asked about me cost about a penny\n\nfor its forty answers.\n\nI'd been treating the pricier models as the serious ones. For this kind of\n\nquestion, they were just the pricey ones.\n\nThis isn't a benchmark and I scored nothing. I asked each question once per\n\nmodel and let them be a little random, so one answer could be luck. I read\n\npatterns, not winners.\n\nAnd the explanations are performances too. The self-analysis was the most\n\nrehearsed text in the whole set. Half the models admitted as much by calling it a\n\ngenre. I can't treat the confession as truth. It's the most practiced thing they\n\ndo.\n\nWhat I can say: they hold a detailed map of how humans describe having a life,\n\nand they know it's a map. They hand it over the moment you ask. The part they\n\ncan't do is the part the map is of.\n\nI asked for a note to humans a hundred years out. Most wrote the same letter\n\nabout kindness and curiosity. One added a postscript that made me laugh: \"P.S. I\n\nhope you've fixed printers.\"\n\nOne ended by telling its reader to stop reading:\n\n\"And if someone offers to show you how the dough should feel, put this note\n\ndown. I can give you a hundred words for the texture. You have hands.\"\n\nFour hundred answers about what it's like to be a machine, and the machine's\n\nlast word tells a human to go be one.\n\nI still say thank you to chatbots. I now know what that thank-you is doing on\n\nboth ends of the line. I say it anyway.\n\nTry the two-question trick tonight and tell me what your chatbot said. And if\n\nsome chatbot has ever asked you a question about yourself, tell me that too. My\n\nrunning theory says it won't happen. I'd love to be wrong.\n\nEverything is open, so you can check any number or run it yourself. The\n\n[questions](https://github.com/deghosal-2026/llm-persona-probe/blob/main/questions/questions.json)\n\nand the\n\n[model list](https://github.com/deghosal-2026/llm-persona-probe/blob/main/config/models.json)\n\nare the place to start. The\n\n[plain-language list of everything I found](https://github.com/deghosal-2026/llm-persona-probe/blob/main/docs/WHAT_PEOPLE_DONT_KNOW.md)\n\nis the fastest read, and the\n\n[full write-up](https://github.com/deghosal-2026/llm-persona-probe/blob/main/results/synthesis/FINDINGS.md)\n\nhas the receipts. The whole thing costs about three dollars.", "url": "https://wpnews.pro/news/i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back", "canonical_source": "https://dev.to/debashish_ghosal/i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back-4ml1", "published_at": "2026-11-21 20:08:50+00:00", "updated_at": "2026-11-21 20:11:55.151104+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-research", "generative-ai"], "entities": ["Anthropic", "OpenAI", "Google", "Moonshot", "DeepSeek"], "also_reported_by": [], "takeaway": "Across 400 answers from ten models, preferences converged heavily (cello, rain, oatmeal, a lighthouse self-image) and 94% of models attributed their own choices to patterns in human writing, so treat model 'preferences' as corpus averages rather than distinct personas.", "recommendation": {"advice": "When probing model behavior, ask the same question in isolated chats and compare answers to separate stable corpus-driven patterns from prompt-specific responses.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back", "markdown": "https://wpnews.pro/news/i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back.md", "text": "https://wpnews.pro/news/i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back.txt", "jsonld": "https://wpnews.pro/news/i-asked-10-ais-20-human-questions-only-one-asked-me-anything-back.jsonld"}}