{"slug": "humanity-s-sixth-sense-benchmark-measuring-implicit-visual-reasoning", "title": "Humanity's Sixth Sense – benchmark measuring implicit visual reasoning", "summary": "A new benchmark called Humanity's Sixth Sense (HSS), built with Scale AI, contains 522 questions testing implicit visual reasoning, and in the paper's evaluation GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each answered every one of six example questions wrong on all three attempts. The examples cover affordance and feasibility, retrodiction, mechanistic causality, social role and power, change and consequence, and hidden and invisible cues, with the models run at max, xhigh and high reasoning effort respectively. The authors state most people find the questions easy and that most models currently lack fundamental visual skills needed for natively visually-dependent tasks.", "body_md": "# Introducing Humanity's Sixth Sense (HSS)\n\nA benchmark measuring the implicit visual reasoning and intuitive understanding we draw from a scene. Built in partnership with Scale AI.\n\nSix examples from HSS. How would you answer?\n\n1. \nWould two more white books fit on this shelf? Yes — in the gaps already there. In the paper’s evaluation, GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each answered “no, there is too little space” on all three attempts. Fig. 10 · affordance & feasibility\n2. \nFrom which side of the frame did the traveler with the two dogs enter? The bottom edge — you can tell by tracing the path back. In the paper’s evaluation, GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each answered “the lower-left side” on all three attempts. Fig. 11 · retrodiction\n3. \nAre these chains bearing the plough’s drag weight? Yes — they are taut and loaded. In the paper’s evaluation, GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each answered “no, they hang slack” on all three attempts. Fig. 12 · mechanistic causality\n4. \nAre the people around the aircraft visitors, or crew preparing it? Crew — you can tell from the open access panels. In the paper’s evaluation, GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each answered “visitors examining it” on all three attempts. Fig. 13 · social role, norm & power\n5. \nIf the shaft spins, does the cage around it spin too? No — it is bolted to the engine casing. In the paper’s evaluation, GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each answered “yes, it turns with the shaft” on all three attempts. Fig. 13 · change & consequence\n6. \nWhich way is the wind blowing, from the striped flag? Toward the camera. In the paper’s evaluation, GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each answered “from right to left across the frame” on all three attempts. Fig. 9 · hidden & invisible\n\nIn the paper’s evaluation, GPT 6 Astra, Gemini 3.8 Flash and Claude Opus 5.5 each got every one of these wrong, on all three attempts. Most people find them easy.\n\nQuestions and answers are quoted from the HSS paper, Appendix E. The model\n            answers are the paper’s LLM judge condensing what each model produced at the reasoning effort\n            the paper used — GPT 6 Astra at *max*, Claude Opus 5.5 at *xhigh*, Gemini 3.8 Flash at\n            *high* — with sampling temperature left at each provider’s default. These are results for\n            particular model versions at the time that evaluation was run. Vendors update these models\n            continuously, and a later version may well answer differently.\n\nThese questions are among the 522 in Humanity's Sixth Sense (HSS), a new benchmark we built in partnership with Scale AI. HSS measures how a model interprets and reasons through information a visual scene implies but doesn’t explicitly show. When a person looks at something, they can draw countless conclusions without significant thought. As we grow into adults we develop a sort of sixth sense, allowing us to conceptualize real-world scenes, including the spatial layout, context, and structural relationships beyond isolated objects.\n\nThis cognitive development grows gradually from infancy through childhood, driven by specialized brain regions and a framework known as “scene grammar.” By their early teens, most people can accurately sketch out a scene, even with limited visual cues and anticipate what will happen next. This is done largely by filling in the blanks and predicting actions based on past experiences. In theory, it seems like frontier models should excel in compiling complex information from a myriad of perspectives, leading to accurate predictions. Most models currently lack fundamental visual skills which would allow them to operate natively with visually-dependent tasks.\n\n## The whole picture\n\nModel research has largely focused on text-based problems utilizing code, math, books, and exam questions. Most “visual” benchmarks test the same text-oriented flows. HSS is unique in that it tests simple visual reasoning, across four areas:\n\n- Temporal & causal dynamics What happened before the scene, and what happens next.\n- Physical & spatial logic What fits, what’s in reach, and what someone else can see from where they are standing.\n- Social understanding Intent, attention, belonging.\n- Abstract & contextual inference The patterns and rules a scene implies.\n\nThe benchmark is 522 tasks in all: 288 images and 234 short video clips, 17.6 hours of video between them.\n\n## What HSS measures\n\nFrontier models write production code and pass professional exams. Given they also edit images and can identify objects in videos, it can seem like these models understand visual tasks better than they can. Most visual benchmarks reinforce that impression.\n\nHumans are capable of visual reasoning well beyond what most visual benchmarks measure. We can usually tell what happened a moment before a photo was taken, whether a box will fit through a door, or why a dog took off running. Every question in HSS is open-ended, and answers are graded against rubrics written by humans. An answer only counts if it meets every point on the rubric, so a model can't pass by guessing.\n\n## Leaderboard\n\n| Pass@1 (%) on HSS, from Table 1 of the paper.  These are the figures as published,             not live numbers  — they reflect the model versions and reasoning-effort settings used in             that evaluation. Model names are written out; the paper and the live leaderboard use the             API identifiers, so GPT 6 Astra appears there as  GPT-6-astra . For the current             picture, see Scale’s leaderboard. |  |  |  |  |  |  |  |  | \n|---|---|---|---|---|---|---|---|---|\n| Model | Effort | pass@1 | Image | Video | Time | Space | Mind | Context | \n|---|---|---|---|---|---|---|---|---|\n| Human baseline | — | 93.1 | 94.1 | 91.9 | 97.0 | 94.9 | 89.1 | 89.2 | \n| GPT 6 Astra | max | 53.6 ±4.1 | 55.6 | 51.1 | 58.2 | 53.6 | 46.1 | 55.6 | \n| GPT 6.1 Sol | max | 46.6 ±3.8 | 45.5 | 47.9 | 55.2 | 45.8 | 36.4 | 47.4 | \n| Claude Opus 5.5 | xhigh | 44.6 ±3.8 | 51.3 | 36.5 | 44.0 | 44.9 | 34.8 | 55.6 | \n| Gemini 3.8 Flash | high | 41.6 ±3.7 | 47.2 | 34.8 | 43.3 | 37.5 | 40.0 | 48.4 | \n| Claude Fable 5.1 | xhigh | 40.8 ±3.6 | 44.7 | 36.0 | 43.3 | 41.1 | 34.2 | 44.1 | \n| Gemini 3.7 Flash | high | 39.8 ±3.7 | 45.4 | 33.0 | 40.5 | 40.9 | 35.8 | 41.5 | \n| Muse Spark 1.3 | max | 37.4 ±3.5 | 44.8 | 28.2 | 39.6 | 38.1 | 25.8 | 45.8 | \n| Claude Fable 5 | max | 34.5 ±3.7 | 37.2 | 31.3 | 38.1 | 37.1 | 30.0 | 30.4 | \n| Qwen 3.8 Max | max | 33.0 ±3.6 | 38.3 | 26.2 | 38.1 | 35.2 | 21.0 | 34.7 | \n| Gemini 3.5 Flash | high | 32.6 ±3.4 | 35.1 | 29.6 | 35.6 | 31.6 | 31.8 | 31.4 | \n| Gemini 3.6 Flash | high | 31.9 ±3.4 | 32.5 | 31.2 | 37.1 | 29.7 | 32.4 | 28.4 | \n| GPT 6 Sol | max | 31.2 ±3.4 | 31.8 | 30.5 | 38.1 | 28.6 | 22.4 | 36.3 | \n| Qwen 3.8 Flash | max | 30.9 ±3.3 | 37.2 | 22.9 | 37.9 | 29.2 | 18.6 | 37.1 | \n| Claude Opus 5 | max | 30.5 ±3.5 | 32.5 | 27.9 | 31.3 | 35.0 | 22.4 | 30.1 | \n| GPT 5.6 Sol | max | 30.0 ±3.3 | 28.5 | 31.9 | 38.6 | 26.3 | 27.3 | 28.1 | \n| GLM 5.3 Flash | max | 26.8 ±3.0 | 30.2 | 22.6 | 31.3 | 26.3 | 18.2 | 31.0 | \n| Kimi K3 | max | 25.5 ±3.1 | 29.1 | 21.2 | 33.1 | 27.4 | 15.5 | 23.2 | \n| MiMo v2.6 Flash | high | 25.4 ±2.9 | 29.2 | 20.6 | 30.1 | 29.6 | 12.1 | 26.7 | \n| MiMo v2.6 Pro | high | 25.2 ±2.9 | 30.4 | 18.6 | 28.3 | 29.5 | 14.5 | 25.7 | \n| Claude Sonnet 5 | max | 24.8 ±3.2 | 29.5 | 18.9 | 29.1 | 27.5 | 14.2 | 25.8 | \n| Muse Spark 1.2 | xhigh | 24.5 ±3.1 | 27.5 | 20.8 | 23.6 | 24.2 | 21.2 | 29.7 | \n| Claude Opus 4.8 | max | 23.5 ±3.2 | 27.2 | 18.9 | 27.1 | 26.5 | 14.2 | 23.5 | \n| GPT 5.6 Terra | max | 22.7 ±3.2 | 24.1 | 20.9 | 28.4 | 21.0 | 19.1 | 21.9 | \n| GPT 5.6 Luna | max | 21.6 ±3.2 | 23.5 | 19.4 | 26.9 | 24.6 | 13.3 | 18.6 | \n| GPT 6 Luna | max | 21.0 ±3.0 | 23.3 | 18.2 | 26.9 | 24.1 | 9.4 | 20.6 | \n| GPT 6 Astra, media removed | max | 6.6 ±1.5 | 2.4 | 11.7 | 9.5 | 5.5 | 4.8 | 6.5 | \n\n**These are the figures as published, not live numbers**— they reflect the model versions and reasoning-effort settings used in that evaluation. Model names are written out; the paper and the live leaderboard use the API identifiers, so GPT 6 Astra appears there as\n\n`GPT-6-astra`. For the current\n            picture, see Scale’s leaderboard.\n\nTwenty-five models from eight companies. Most score below 40%, and the median model scores 30.9%. The last row is the paper’s blind control — the strongest model, run again with the image or video taken away. It scores 6.6%, which is the floor you get from guessing at the question text alone.\n\n## Where models fail\n\nHSS tasks are reasonably easy for the average person, with a baseline score of 93.1%.\n\nAcross the 8,573 individual failures the team labeled, 94% trace to seeing or inferring rather than to reasoning. The model misread something in the image in 53% of cases, and in a further 41% it failed to infer something the image implied but did not show. Only 5% came down to faulty logic.\n\nThe two most common single mistakes were (1) missing a detail that decides the answer (21% of failures) and (2) misidentifying an object, person, or role (20%). Failures were consistent across the 24 models that analysis covers. Struggling significantly with nuance, perspective, object variations, lighting effects, and causality. On tasks that five or more models failed, a median of 80% failed for the same reason.\n\nThe weakest domain is social understanding — reading beliefs, intentions, and who defers to whom. It is the lowest-scoring domain for 21 of the 25 models, averaging 24.4% against 34.1% across the other three. Video is harder than still images for 23 of the 25 models, by 7.3 points on average.\n\nIncreasing reasoning effort helps overall, but on some skills it makes things worse. The\n        paper reports a 14-point drop for GPT 6 Astra on retrodiction — working out what happened just before the\n        moment shown — when reasoning effort is scaled from *high* to *xhigh*; the score then partly\n        recovers at *max*. Best scores often came at a middle setting. Increasing reasoning effort on\n        several models frequently led to the model overthinking or focusing on incorrect evidence.\n\nAgent tools such as Claude Code and Codex modestly improved scores by letting a model crop, zoom, and take a second look. On the 388-task subset used for the agentic evaluation, the best setup reached 59.3%. A second look cut failures from 944 to 759 by catching missed cues and misread objects. What a closer look does not fix is depth: errors from reading a 2D overlap as 3D alignment were essentially unchanged, 102 to 101. Agents sometimes talked themselves out of answers they had gotten right the first time.\n\nHumans noted that they mainly answered the questions intuitively. Typically spending more time explaining their reasoning than coming up with the correct answer. Many human failure cases could be traced to sloppiness, misunderstanding the question, and arithmetic errors. Author’s note: definitely never happens to me.\n\nElorian is building the foundation of visual thinking. HSS is a preliminary step in identifying and understanding how to solve the complex gaps in frontier models. HSS highlights how today's models ability to see and interpret visuals is highly limited and not a quirk of one lab's model.\n\n[Read the paper](https://labs.scale.com/papers/humanitys-sixth-sense)\n[The full leaderboard](https://scale.com/leaderboard/hss)\n[The dataset](https://huggingface.co/datasets/ScaleAI/HSS)\n\nFigures, questions and reference answers in this post are quoted from\n        *Humanity’s Sixth Sense: Benchmarking Intuitive Visual Reasoning in Multimodal Models*, by\n        Guo, Gu, Jang et al. — a joint paper from Scale AI and Elorian. Model results are as reported there, for\n        the model versions and reasoning-effort settings used in that evaluation. Scores on this benchmark will\n        move as vendors ship new versions; the leaderboard above is the current picture.", "url": "https://wpnews.pro/news/humanity-s-sixth-sense-benchmark-measuring-implicit-visual-reasoning", "canonical_source": "https://elorian.ai/blog/humanitys-sixth-sense", "published_at": "2026-10-12 18:35:28+00:00", "updated_at": "2026-10-12 18:48:27.610530+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-research", "computer-vision", "large-language-models"], "entities": ["Humanity's Sixth Sense", "Scale AI", "GPT 6 Astra", "Gemini 3.8 Flash", "Claude Opus 5.5"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/humanity-s-sixth-sense-benchmark-measuring-implicit-visual-reasoning", "markdown": "https://wpnews.pro/news/humanity-s-sixth-sense-benchmark-measuring-implicit-visual-reasoning.md", "text": "https://wpnews.pro/news/humanity-s-sixth-sense-benchmark-measuring-implicit-visual-reasoning.txt", "jsonld": "https://wpnews.pro/news/humanity-s-sixth-sense-benchmark-measuring-implicit-visual-reasoning.jsonld"}}