{"slug": "new-artificial-analysis-index-widens-the-gap-between-open-and-closed-models", "title": "New Artificial Analysis index widens the gap between open and closed models", "summary": "Artificial Analysis released version 5 of its Intelligence Index, which now draws 60 percent of its score from private test sets to prevent \"benchmaxxing,\" and the revision widened the measured gap between open and closed models from 3 points to 12 points. Kimi K3.1 Pro, which launched in fourth place, fell to ninth behind eight closed models, while Grok 4.7 lost 12 points. AI researcher Nathan Lambert attributed part of the shift to reinforcement learning infrastructure, noting that keeping pace across multiple domains at once requires sophisticated RL setups that not every lab has.", "body_md": "# New Artificial Analysis index widens the gap between open and closed models\n\n**Artificial Analysis overhauled its Intelligence Index, causing Kimi K3.1 Pro to drop five spots in the rankings.** The model originally launched in [fourth place](https://the-decoder.com/chinese-ai-model-kimi-k3-1-pro-is-closing-in-on-anthropics-claude-opus-5-5/), but it falls to ninth behind eight closed models in [version 5](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5). Artificial Analysis now puts the gap between open models and the top of the leaderboard at 12 points, up from 3 points previously.\n\nFor the first time, 60 percent of the index relies on private test sets to prevent \"benchmaxxing,\" the practice of training models specifically on known test questions. Kimi K3.1 Pro's drop suggests the old index made some models look better than they were. Grok 4.7 also lost 12 points.\n\nAI researcher [Nathan Lambert](https://www.interconnects.ai/p/artificial-analysis-v5-and-the-benchmarkers) thinks benchmaxxing is only part of the story. Many training gains today happen in reinforcement learning environments with tasks that resemble the benchmarks. Keeping up across multiple domains at once takes sophisticated RL infrastructure, and not every lab has it.\n\nClosed test sets do have a downside, since outsiders can no longer spot errors in the prompts. The AA-Omniscience knowledge test proves those errors exist. Artificial Analysis had to revise ambiguous and flawed questions in that benchmark for the new index.\n\n```\nAI News Without the Hype – Curated by Humans\n\n\t\t\t\t\tSubscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive \"AI Radar\" frontier report six times a year, full archive access, and access to our comment section.\t\t\t\t\n\n\t\t\t\t\tSubscribe now\n```\n\n[Artificial Analysis](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5)", "url": "https://wpnews.pro/news/new-artificial-analysis-index-widens-the-gap-between-open-and-closed-models", "canonical_source": "https://the-decoder.com/new-artificial-analysis-index-widens-the-gap-between-open-and-closed-models/", "published_at": "2026-10-28 15:04:43+00:00", "updated_at": "2026-10-28 15:15:46.185401+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "machine-learning"], "entities": ["Artificial Analysis", "Kimi K3.1 Pro", "Grok 4.7", "Nathan Lambert", "AA-Omniscience"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/new-artificial-analysis-index-widens-the-gap-between-open-and-closed-models", "markdown": "https://wpnews.pro/news/new-artificial-analysis-index-widens-the-gap-between-open-and-closed-models.md", "text": "https://wpnews.pro/news/new-artificial-analysis-index-widens-the-gap-between-open-and-closed-models.txt", "jsonld": "https://wpnews.pro/news/new-artificial-analysis-index-widens-the-gap-between-open-and-closed-models.jsonld"}}