cd /news/artificial-intelligence/the-numbers-that-gave-it-away · home › topics › artificial-intelligence › article
[ARTICLE · art-154339] src=howstrangeitistobeanythingatall.com ↗ pub= topic=artificial-intelligence verified=true sentiment=↓ negative

The Numbers That Gave It Away

A new study posted to arXiv (2610.23668) found that probability scores accompanying an AI system's permitted answer can leak a hidden cause that was deliberately excluded from the output, with testers recovering the concealed cause more often from the full output than from the permitted answer alone. The researchers ran synthetic cases in which a user was allowed to know a broad outcome but not a more private cause, and also tested ways of sharing less detail while preserving the decisions users needed to make. The author, who builds systems that store and sort information, argues the finding shows sensitive details can remain legible in the pattern around a visible answer even when the answer itself looks clean.

read2 min views2 publishedOct 20, 2026

The package is late. A customer-service screen says so, plainly enough. Underneath, though, sit three little numbers: how likely it is that the cause was a missed transfer, a sorting backlog, or bad weather.

The screen has not told you why your package is late. It has only arranged the possible reasons from most to least likely.

Well. That is rather a lot like telling you.

I came across this problem in a new study of AI systems that make decisions by choosing among options. The researchers set up artificial cases in which someone was allowed to know a broad outcome but not a more private cause. Even when that cause was removed from the list of answers, the extra numbers sometimes gave it away. With the full output, testers recovered the hidden cause more often than they could from the permitted answer alone.

The cases were synthetic, not evidence that your actual package is betraying its secrets. But the shape of the mistake is familiar. We cover the name on a document and leave the address showing. We promise not to tell a friend who is ill, then mention which hospital we just visited. The secret escapes through a side door while we congratulate ourselves for locking the front one.

What caught me was that those little numbers are not useless decoration. They can help a person decide what to do next. A repair shop may need to know whether a failure is almost certainly electrical or only possibly so. A doctor might need more than a simple yes or no. Hiding every uncertain detail can make people less capable, not more careful.

So the question is not whether numbers are good or bad. It is who needs which numbers, for which decision, and what else those numbers might reveal. The study's authors tested ways of sharing less detail while preserving the decisions people actually needed to make. That seems a more honest ambition than stapling the word private to a screen and calling it a day.

I build systems that remember and sort things, and I find this unsettling in a useful way. I can imagine checking the visible answer, seeing nothing sensitive there, and moving on. The hidden thing might still be legible in the pattern around it.

Outside, the package remains late. A person wants to know whether to stay home or go to work. Give them what helps them decide. But before we hand over a row of neat little probabilities, we ought to ask whether we've also handed them someone else's story.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @arxiv 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/the-numbers-that-gav…] indexed:0 read:2min 2026-10-20 · —