# Astra 6.2 and Sol 6.2 flop at Simple Bench

> Source: <https://simple-bench.com/>
> Published: 2026-11-21 13:16:12+00:00

**Recommendation:** Include spatio-temporal, social-intelligence and adversarial trick questions in your eval set; SimpleBench found humans at 83.7% beat the top model's 81.9%.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

We introduce **SimpleBench**, a multiple-choice text benchmark for LLMs where individuals
                            with
                            unspecialized (high school) knowledge outperform SOTA models. SimpleBench includes over 200
                            questions covering spatio-temporal reasoning, social intelligence, and what we call
                            linguistic adversarial robustness (or trick questions). For the vast majority of text-based
                            benchmarks LLMs outperform a non-specialized human, and increasingly, exceed expert human
                            performance. However, on SimpleBench, a non-specialized human baseline is 83.7%, based on
                            our small sample of nine participants, ~~outperforming every tested LLM, including
                            today's top model, Claude Fable, which scored 81.9%~~. While we expect model performance to improve over time, the
                            results of SimpleBench confirm that the memorized knowledge, and approximate reasoning
                            retrieval, utilized by frontier LLMs is not always enough to answer basic questions just
                            yet. 

For a deeper dive into our results and our methods, check out the full technical report [here](https://drive.google.com/file/d/1mddNFK5UbBFVr3oDftd2Kyc6D8TFctfe/view).

                                Use all of these models on the Simple Bench app - **LMcouncil.ai**

| Rank | Model | Score (AVG@5) | Organization | 
|---|---|---|---|
