MiniCPM5-2B claims 2B-class open-source SOTA, beating 4B models too. We check its benchmark comparisons against hands-on coding, reasoning, and language tests.
What is MiniCPM5-2B and why is it getting attention? #
MiniCPM5-2B is a dense 2 billion parameter language model from ModelBest, a Beijing-based AI company spun out of Tsinghua University’s NLP lab, released as the second entry in the MiniCPM5 series after MiniCPM5-1B. It’s built as a LlamaForCausalLM architecture with 42 layers, grouped-query attention (16 heads for Q, 2 for KV), and a native context window of 131,072 tokens. The model card claims “2B-class open-source SOTA,” meaning it beats other small open models on ModelBest’s benchmark suite and, more strikingly, holds up against several 4B-class models too. That second claim is the one worth scrutinizing.
TL;DR #
- MiniCPM5-2B posts an average score of 53.9 on ModelBest’s own comparison suite, which it says beats every larger model in the chart including a 4B-class model scoring 51.1.
- The comparison set draws criticism because some baseline models cited in early commentary (like Qwen2.5-2B and Gemma-2B) are noticeably older generations, though the official model card also lists more current models like Qwen3.5-2B, Gemma-4-E2B-it, and granite-4.2-3B.
- Hands-on testing with a hard Oracle SQL partition bug, a historical fact-check on the Treaty of Nerchinsk, and a scientific reasoning problem all came back correct, though the model reasoned at length before settling on answers.
- A self-contained HTML coding test simulating a rotating kebab on a spit, complete with lighting, texture, and physics, produced a genuinely impressive visual result for a model this size.
- Multilingual translation of a term of endearment across a dozen languages was the weak spot, with several entries fabricated or simply wrong.
- The model needs roughly 4GB of VRAM for weights alone , with total usage climbing toward 44GB if you max out the KV cache for long-context workloads, though that’s adjustable.
- The training pipeline uses a three-stage process (base, mid, post-training) plus a reinforcement learning and on-policy distillation (OPD) step that ModelBest says accounts for roughly 11 points of improvement on reasoning benchmarks.
One coffee. One working app. #
You bring the idea. Remy manages the project.
How does MiniCPM5-2B’s benchmark comparison actually work? #
ModelBest published a radar chart plotting MiniCPM5-2B against a mix of same-size and larger models across nine axes: code reasoning, math reasoning, instruction following, general knowledge, long context, tool use, coding agent, search agent, and general agent. Each axis is normalized independently to 100%, and the blue MiniCPM5-2B line sits outside or matches the other three lines on nearly every axis, giving an average of 53.9.
The official comparison set listed on the model card includes Qwen3.5-2B, Gemma-4-E2B-it, and LFM2.5-2.6B as same-size peers, with Qwen3.5-4B, granite-4.2-3B, Nemotron-3-Nano-4B, Gemma-4-E4B-it, and LFM2.5-8B-A1B listed as larger reference points. In the radar chart itself, the four plotted lines are MiniCPM5-2B (53.9), Qwen3.5-4B (51.1), granite-4.2-3B (42.7), and LFM2.5-2.6B (33.2). That means the headline “beats 4B models” claim rests on outperforming a single 4B model, Qwen3.5-4B, by less than three points on average, not a sweep of the entire 4B class.
The video’s creator flagged a separate concern before the more recent model card update: some early comparisons referenced older models like Qwen2.5-2B and Gemma-2B-it, which are prior generations rather than current releases. Whether you’re looking at the older or newer comparison set, the pattern holds: MiniCPM5-2B’s advantage is most pronounced in coding, reasoning, math, and long-context axes, and the chart format itself (each axis independently normalized to its own max) tends to visually flatter whichever model has the most balanced spread rather than the largest absolute margin.
What does hands-on testing show about coding and reasoning? #
Independent testing on locally-hosted MiniCPM5-2B (served via SGLang with an Hermes-style agent setup) put the model through several targeted tasks rather than standard benchmarks.
On a deliberately tricky Oracle SQL bug, one designed so the query runs without errors but silently pulls data from the wrong partition, the model correctly diagnosed the issue after a long chain-of-thought process involving file reads and tool calls. The reasoning trace was long and looped back on itself multiple times before arriving at the right fix.
On a factual/historical question about the Treaty of Nerchinsk (1689), asking what language it was written in and whether Russian and Chinese negotiators communicated directly, the model correctly identified Latin as the authoritative text and noted the negotiation went through Jesuit interpreters rather than direct speaker-to-speaker exchange. Both parts of the answer were accurate.
On a scientific reasoning prompt combining math and science knowledge, the model reached the correct final answer, though it recomputed the same intermediate steps three or four times before committing to a response, a pattern that would burn through tokens quickly on an API-billed deployment.
The standout result was a coding test: generating a single self-contained HTML file simulating a rotating doner kebab on a rotisserie, complete with CSS, JavaScript, lighting effects, and physics-plausible motion. For a 2B model, the visual output (glow effects, texture, rotation) was notably more polished than expected, described as impressive relative to model size.
Where does the model fall short? #
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
The clearest weak point in testing was multilingual nuance. Asked to translate a term of endearment (“sweetheart” / Urdu “janeman”) into its natural, culturally appropriate equivalent across roughly a dozen languages, the model got some right (Hindi, Spanish, Indonesian) but fabricated or simply invented plausible-sounding words for several others, including some claimed correct answers in Mandarin, Korean, German, and Icelandic that don’t hold up. This is a common failure mode for small models asked to produce idiomatic, low-frequency vocabulary rather than pattern-matched translation, and it stands in contrast to the model’s stronger performance on structured reasoning tasks.
Response style was also flagged as a minor issue: outputs read as somewhat bloated and less smooth than comparably sized models from other labs, with visible reasoning bleeding into the final response text in places.
Is MiniCPM5-2B worth running locally? #
For a model in the 2B parameter class, MiniCPM5-2B is worth trying if your use case is coding assistance, tool-use agents, or long-context document work on constrained hardware. The weights themselves need only about 4GB of VRAM, and the model supports quantized formats (GGUF, GPTQ, MLX for Apple Silicon) alongside the full BF16 release, so it fits laptops and modest GPUs, not just data-center cards. The 131,072 token context window is a genuine differentiator for a model this small. The tradeoff is inference cost per query. The model reasons extensively, sometimes redundantly, before producing final answers, which is good for accuracy but means minimal or low reasoning-effort settings are worth configuring if you’re paying per token or want fast turnaround. For multilingual or idiomatic language tasks, treat outputs with skepticism and verify against a native speaker or reference source.
The “SOTA” framing is defensible within the narrow scope ModelBest defines: among the specific models it chose to compare against, MiniCPM5-2B does come out ahead on average, including against one 4B model. But it’s not a sweep of the entire 4B class, and it’s not compared against every current 2B-3B competitor on the market. Treat the radar chart as a starting point for interest, not as proof the model beats every alternative in its class.
Frequently Asked Questions #
What is MiniCPM5-2B’s context length?
131,072 tokens, which is long-context support built directly into the base model rather than added via a separate extension technique.
How much VRAM does MiniCPM5-2B need to run?
Model weights require roughly 4GB of VRAM. Total usage can rise toward 44GB if the KV cache is set to maximum for long-context inference, but that figure scales down if you reduce the KV cache setting.
Does MiniCPM5-2B actually beat 4B parameter models?
According to ModelBest’s own comparison chart, it edges out one specific 4B model, Qwen3.5-4B, by about three points on average (53.9 vs 51.1). It’s not shown beating every 4B-class model, and the comparison set is chosen by ModelBest rather than being an independent leaderboard.
What training approach does MiniCPM5-2B use?
A three-stage pipeline: base pre-training, mid-training, and post-training. The post-training stage includes supervised fine-tuning followed by reinforcement learning with specialized teacher models for math, code, agentic tasks, and writing, merged back into the final model via on-policy distillation (OPD). ModelBest attributes a meaningful share of the model’s benchmark gains to this final RL and distillation step.
What formats is MiniCPM5-2B available in?
BF16 checkpoints at every training stage (base, mid-train, SFT, final), plus GGUF for llama.cpp/Ollama/LM Studio, MLX for Apple Silicon, GPTQ 4-bit quantization, and a LiteRT-LM version for mobile and edge deployment.