{"slug": "without-search-even-the-best-ai-models-get-1-in-4-answers-wrong", "title": "Without search, even the best AI models get 1 in 4 answers wrong", "summary": "Top AI models answer roughly 1 in 4 specific-fact questions incorrectly when they cannot search, according to Artificial Analysis data cited by Randy Olson. The AA-Omniscience dataset covers 6,000 questions on specific facts across 42 work topics, answered without search, with wrong answers counted as a share of answers given and declined questions excluded. Olson advised that models look facts up when accuracy matters, and noted the chart was made with evident-charts, his open source agent skill for explanatory charts.", "body_md": "Randy Olson on X: \"Question 2 from @ArtificialAnlys data: how often do top AI models get specific facts wrong without search?\nEven the best get about 1 in 4 answers wrong. If a fact matters, have the model look it up.\nMade with evident-charts, my open source agent skill for explanatory charts.\"\n\nQuestion 2 from @ArtificialAnlys data: how often do top AI models get specific facts wrong without search?\nEven the best get about 1 in 4 answers wrong. If a fact matters, have the model look it up.\nMade with evident-charts, my open source agent skill for explanatory charts.\n\nQuestion 2 from @ArtificialAnlys data: how often do top AI models get specific facts wrong without search?\nEven the best get about 1 in 4 answers wrong. If a fact matters, have the model look it up.\nMade with evident-charts, my open source agent skill for explanatory charts.\n\nData: AA-Omniscience, 6,000 questions about specific facts across 42 work topics, answered without search. Wrong answers as a share of answers given; declined questions don't count.\nGet the evident-charts agent skill here:", "url": "https://wpnews.pro/news/without-search-even-the-best-ai-models-get-1-in-4-answers-wrong", "canonical_source": "https://twitter.com/randal_olson/status/2105298222562582591", "published_at": "2026-10-03 06:49:06+00:00", "updated_at": "2026-10-03 07:06:14.052979+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-search", "ai-tools"], "entities": ["Randy Olson", "Artificial Analysis", "AA-Omniscience", "evident-charts"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/without-search-even-the-best-ai-models-get-1-in-4-answers-wrong", "markdown": "https://wpnews.pro/news/without-search-even-the-best-ai-models-get-1-in-4-answers-wrong.md", "text": "https://wpnews.pro/news/without-search-even-the-best-ai-models-get-1-in-4-answers-wrong.txt", "jsonld": "https://wpnews.pro/news/without-search-even-the-best-ai-models-get-1-in-4-answers-wrong.jsonld"}}