Last week, Kimi K3.1 Pro, an LLM from the Beijing, China-based company Moonshot AI, debuted to a surprising amount of fanfare, particularly if you care about benchmarks, which are meant to be standardized measurements for models’ capabilities. It charted at a very respectable—in fact somewhat shocking—fourth place on the marquee Artificial Analysis benchmark called Intelligence Index v4. It also claimed the top spot on Artificial Analysis’ Terminal Bench 4.0, which is focused on computer use.
Kimi K3.1 Pro’s success was a minor earthquake in the AI world. The Decoder pointed out that if it had released its model weights publicly at its debut, it would have been the highest-scoring open weight model by a wide margin—not to mention the highest scoring Chinese model. “That would shrink the gap between open and proprietary models from 12 to just 3 points,” the Decoder wrote.
But that never technically happened, and then on Tuesday, Artificial Analysis announced a new version of its Intelligence Index called Intelligence Index v5. Artificial Analysis called it “our biggest upgrade since v4 launched in January.” 60% of the new test apparently involves private test questions, or “held-out answers,” according to Artificial Analysis. Both of these concepts point to Artificial Analysis seeking to curb “benchmaxxing,” the practice of preparing models for evaluations by feeding them old questions available online in repositories like Github and (famously) Hugging Face.
The effect has been devastating for Kimi K3.1 Pro’s ranking. It fell from its 4th place slot to 9th. Artificial Analysis itself pointed out in the announcement of the new Index that “The open weights gap widens from 3 points to 12.” Again, Moonshot still hasn’t released the model’s weights, and won’t until November 4. But in spirit, it was a return to normal for models in the open weights world.
Google’s Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5, and second only to Claude Opus 5.5 on the Artificial Analysis Coding Agent Index
Gemini 4 Carbon scores 59 on Intelligence Index v5, driven by leading performance in complex reasoning,… pic.twitter.com/gSH65FSKzL
— Artificial Analysis (@ArtificialAnlys) October 28, 2026 Kimi K3.1 Pro’s brief moment in the sun created none of the apocalyptic panic of DeepSeek’s release in January of last year. Still it was a surprising bump in the road for critics of the Chinese open-weight approach. Anthropic in particular seems to fret about open weights being a danger to the public, writing in a report about one such model that since it was “released as an open-weight model, users can reconfigure it to remove its refusals with little change in its capabilities.”
Anthropic also often denounces the extractive practice of model distillation, which open-weight models often rely on to increase their capabilities. Nvidia CEO Jensen Huang has pushed back at this, saying distillation is just competition.
A day after the release of Intelligence Index v5, there was a new champ to go with the new index. Google’s Gemini 4 Carbon, which was evaluated, but not released, was already ranked number 1 when it was announced on Wednesday, and Google heavily featured that fact in its blog post about the model. The second sentence of the announcement reads, “Carbon ranks #1 on the recently updated v5.0 of Artificial Analysis Intelligence Index at 59.”
There may be other explanations for the drop in performance for models by Moonshot, as well as other Chinese models. Physicist Stephen Hsu wrote on X that he believed the U.S. models were outmatching them in math and science performance. “I’ve been talking to a lot of Chinese labs recently and they have not prioritized this area at all,” he wrote.
Artificial Analysis, for its part, is not a standardizing body or an academic institution. It’s a private firm founded, it appears, in 2023, making it impressively influential for such a young company.