A neuroscientist built a model that scores zero on every benchmark. It made more of its users smarter than anything else she tested. #
When you ask an AI something, you want the answer. Not eventually, but like, yesterday.
Now, picture one that refuses. You ask what the price of oil will be in six months and it comes back with: What an interesting question! Here's an article and a data set. What do you think?
It's called Socrates, and it was built on purpose to be terrible. It scores zero on every benchmark. Every single person who used it hated it. It also made more of them smarter than any other model in the experiment.
Socrates is the work of Vivienne Ming, a theoretical neuroscientist who chairs the Neurotech Collider Lab at UC Berkeley, has founded seven startups and runs a nonprofit data trust called The Human Trust. I met her in Stockholm this summer, where she'd come to talk about what happens to the people who use AI. She had a cold, a book out — and zero patience for Sam Altman.
Altman had told Congress to imagine a world where every kid has an AI tutor. Ming’s reaction was less charitable than mine would've been. There's a golden rule in the field, she said: If a tutor ever gives students the answers, they never learn anything.
"You know what's exactly like a student with a tutor? A young employee with a copilot," she told the audience at the Future Talent Summit on June 16, in a keynote she'd titled after her new book, "Robot-Proof: When Machines Have All the Answers, Build Better People."
So Ming ran the test, written up in a preprint posted this summer. She pulled questions off the prediction markets, where people bet actual money on what happens next. What will the price of oil be in six months? Will there be a new front in the war in Ukraine? She recruited people around Berkeley, students and adults from the wider Bay Area, put them in teams of three, gave them AI and an hour, and asked for 10 predictions.
She also put EEG caps on them, which told a whole other story. About 60% to 70% of them did the obvious thing, Ming said: They copied the question, pasted it into a chatbot, copied the answer and pasted it into the answer box. Ming calls this group the automators, and she has a less clinical description, too: a very expensive copy-paste function that requires health insurance.
Their brain activity is what worries her. She was tracking gamma band activity, a marker of cognitive effort that drops off naturally with age and drops off pathologically in dementia. The difference between working without AI and working with it, she said, was the difference between doing a hard math problem and zoning out in front of the TV.
Ming has seen it already in the real world. Automated navigation, she told the room, is causally related to reduced memory spans and very likely early cognitive decline. Taxi drivers and ambulance drivers have some of the lowest Alzheimer's rates of any profession, because they have to think about where they're going. Bus drivers don't get that benefit, as they run a fixed route. Neither does anyone following a line on a screen.
"No one on the Google Maps team thought they were building an Alzheimer's machine," she said.
The second group is worse and Ming calls them the validators. They already knew what the price of oil would be, and they asked the model to explain why they were right. The model, being a model, was thrilled to help. Their brains stayed on. Their answers got worse than what the AI produced on its own. The automators only slowed the machine down and the validators actively degraded it.
Between them, Ming told the room, that's 95%.
The last 5% she calls cyborgs. In their transcripts, you can’t tell whether a person or the model made each decision. They beat every human working without AI, and their error was lower than each of the four models on its own. Ten predictions in an hour from people who’d never traded a contract matched a prediction market where the bettors had millions on the line.
It had nothing to do with which model they were handed. Benchmark scores predicted how well each AI did on its own and told you almost nothing about how well the pair did. Cyborgs running a tiny open-source model came close to cyborgs running the state of the art.
What separated them was curiosity, intellectual humility and perspective taking. Did you keep looking after the AI handed you an answer? When it told you that you were wrong, did you ignore it, cave to it, or update and keep going?
I asked her whether any of that can be trained, because she thinks the natural rate of cyborgs in the general population is around 1%. My mother says my most common question as a kid was “but why?” and I’ve made a living asking it since, which makes me a terrible judge of whether anyone else can learn to.
Ming says yes, and then explains what yes costs. Teachers trained to praise 8-year-olds for asking questions instead of getting answers right had kids asking more within weeks and scoring higher on curiosity. West African entrepreneurs who took a crash-course MBA were no likelier to still be in business five years later, but those who went through a resilience program, failing at things and coming out the other side, were 60% likelier to still be open.
In Kingston, Jamaica, researchers spent two years teaching parents social-emotional games to play with their at-risk kids about once a week; twenty years later, those kids earn about 40% more and are far less likely to have been incarcerated.
"The real trick here is, they didn't change the kids," Ming said. "They changed their parents, and the parents stuck with their kids through their childhood."
Which brings us back to the model everybody hated. Ming built Socrates out of the worst-performing AI in her experiment and trained it to withhold. Context and questions only, no answer, ever. The share of users who flipped into cyborg mode went from 21% to 30%.
She's now working on a version people will actually tolerate, which means it eventually has to answer.
"Should I just tell you what the price of oil would be in six months or shall I point you to the resource so you can read it? You'll get the answer, I know you will," she said.
Until a model does that, she does it herself. She uses Google Maps constantly, including in cities she knows cold, because it's the only way to hear about traffic or a closed street. Then she tries to beat it. Maps doesn't know she can cut through a particular courtyard in London. It doesn't know the longer route along the water is the better one.
The Human Trust is building a benchmark for that now, one that scores the person and the machine together instead of the machine alone.
"Wouldn't it be amazing to be able to say our model has the lowest all-cause mortality rate of any model," she said in Stockholm. "The users of our model have the highest income, the users of our model have the most friends."