Mistral's Le Chonk claims European AI crown but third-party tests disagree

Mistral AI released Mistral Large 4, nicknamed Le Chonk, on October 6 as a mixture-of-experts model with 1.05 trillion total parameters (49 billion active per token), a 1.6 billion-parameter vision encoder and a one-million-token context window, with open weights landing October 27 after three weeks of API-only preview. Independent benchmarking firm Artificial Analysis placed Le Chonk between DeepSeek's V4.1 Flash and OpenAI's GPT-6 Luna, and on the Vals AI benchmark Large 4 scored 48.05% of available points, ranking 32nd of 44 models, well short of Mistral's claim that it is the strongest open-weight model developed outside China. Mistral evaluated its coding benchmarks privately before the testing harness was public, so those numbers are not yet independently reproducible, though the company says Large 4 is state-of-the-art among open-weight models on cybersecurity, finance and legal workloads, the one area where testers have not pushed back.

Mistral calls its new trillion-parameter model the strongest open-weight system built outside China. Independent testers who actually ran it rank it 32nd out of 44. Mistral AI unveiled Mistral Large 4 on October 6, a model it immediately nicknamed Le Chonk, and the name turned out to be the easy part. The French lab built a mixture-of-experts system with 1.05 trillion total parameters, though only 49 billion are active on any given token, paired with a 1.6 billion-parameter vision encoder and a context window that stretches to one million tokens. It's natively multimodal, fluent in more than 160 languages, and according to Mistral, it beats every open-weight model built in the US or Europe by a wide margin. The open weights themselves are landing this week, on October 27, after three weeks of API-only preview access. That timing matters. Alibaba's Qwen team has spent the back half of 2026 making Western labs nervous, previewing four Qwen4 tiers at its Apsara Conference on September 22 and shipping Qwen3.8-Flash-Next as open weights back in August. DeepSeek's R2 and V4 models have kept pace on raw benchmark scores. Mistral's own comparisons for Le Chonk put it up against exactly that field: DeepSeek V4, Alibaba's Qwen3.8 Max, Moonshot, and Z.AI, on coding, agentic workflows, finance, and legal tasks. But Mistral's language is more careful than the breathless coverage suggests. The company calls Le Chonk the strongest open-weight model developed outside China, not the strongest open-weight model, period, and it concedes outright that the system still lags the frontier in coding. Here's the problem. Artificial Analysis, the independent benchmarking firm that labs increasingly treat as a neutral referee, ran its own tests and placed Le Chonk somewhere between DeepSeek's V4.1 Flash and OpenAI's entry-level GPT-6 Luna, well behind the flagship tier Mistral was implicitly inviting comparison to. On the Vals AI benchmark, Large 4 scored 48.05% of available points and landed 32nd out of 44 models ranked. That's not a close call, and it's not the picture Mistral's own launch materials painted. Part of the gap comes down to who graded the homework. Mistral evaluated its coding benchmarks privately, before the testing harness was made public, which means those specific numbers aren't independently reproducible yet. A company publishing its best results on a test nobody else can run is not fraud, but it's not proof either, and the open-weight community has gotten sharper about calling that distinction out since Qwen and DeepSeek releases started getting the same treatment. Mistral does have one benchmark where the gap between its claim and the independent read actually narrows: cybersecurity. The company says Large 4 is state-of-the-art among open-weight models on cybersecurity, finance, and legal workloads, and that's the one area where testers haven't pushed back hard. Unsloth found and fixed a bug in Mistral Medium 3.5 and the story reveals how much of the open-weight model race is actually a tooling race https://startupfortune.com/unsloth-found-and-fixed-a-bug-in-mistral-medium-35-and-the-story-reveals-how-much-of-the-open-weight-model-race-is-actually-a-tooling-race/ Unsloth has identified and fixed an implementation bug in Mistral Medium 3.5, highlighting a reality about open-weight AI models that headline benchmark comparisons rarely capture: practical performance depends as much on the tooling ecosystem as on the model weights themselves. The episode raises questions about how much of the open-weight model... - unsloth bug fix in mistral medium 3.5 https://startupfortune.com/unsloth-found-and-fixed-a-bug-in-mistral-medium-35-and-the-story-reveals-how-much-of-the-open-weight-model-race-is-actually-a-tooling-race/ - open weight models tooling optimization race https://startupfortune.com/unsloth-found-and-fixed-a-bug-in-mistral-medium-35-and-the-story-reveals-how-much-of-the-open-weight-model-race-is-actually-a-tooling-race/ So why launch a 1.05-trillion-parameter model and immediately admit it trails on coding, the single benchmark category everyone actually watches? Money, mostly, and timing. Mistral closed a €3 billion Series D in September, led by Samsung Electronics with Nvidia, ASML, and the EU's Scaleup Fund joining in, pushing its post-money valuation past €21 billion, according to Sifted and confirmed by Orrick, the law firm that advised on the deal. That's nearly double the €11.7 billion valuation Mistral carried after its Series C just twelve months earlier. Le Chonk is the first model built with that war chest, and Mistral has said more are coming. You don't raise €3 billion and then quietly skip the "beats China" headline, even when the honest claim is narrower than the one that gets picked up. Frankly, the real story here isn't whether Le Chonk beats Qwen or DeepSeek on a leaderboard. It's that Europe now has a lab with the capital and the ambition to compete on the same axis as the US and Chinese labs, and that's new. Mistral spent its first few years positioned as the scrappy open-source alternative to OpenAI; a trillion-parameter flagship with a vision encoder and a million-token context window is a different kind of company making a different kind of bet. Whether Le Chonk actually earns the comparisons Mistral is inviting will get settled over the next few weeks, once the weights are out and anyone can run the coding benchmarks themselves rather than taking the company's word for it. Until then, the gap between Mistral's framing and what Artificial Analysis and Vals AI are reporting is the most honest number in this story. Also read: Nvidia releases Nemotron 3.5 Transcribe, a free 30 billion parameter speech model https://startupfortune.com/nvidia-releases-nemotron-35-transcribe-a-free-30-billion-parameter-speech-model/ • FuriosaAI's valuation jumps fourfold to $2.9 billion as it chases Nvidia https://startupfortune.com/furiosaais-valuation-jumps-fourfold-to-29-billion-as-it-chases-nvidia/ • Samsung joins SK Hynix and Micron declaring AI memory sold out for years https://startupfortune.com/samsung-joins-sk-hynix-and-micron-declaring-ai-memory-sold-out-for-years/ This article is posted in AI News https://startupfortune.com/category/ai/ , check it out for more related stories. Join the discussion Open in the community → https://startupfortune.com/community/ Almost there. Sign in and your reply posts straight away.