Baidu has put the raw pre-training weights of ERNIE-5.5-Flash-Base on Hugging Face. It is a text-only Mixture-of-Experts model with 280B total parameters and about 14B active per token, trained on 33.5T tokens. The weights come in FP8 and BF16, and Baidu says the model supports up to 1M tokens for fine-tuning and inference, according to the model card and Baidu's technical blog.
This is a base model, not a chatbot. It is the checkpoint at the end of pre-training, with no mid-training or post-training. Baidu says the post-trained ERNIE 5.5 Flash will follow, but gives no date.
TL;DR: ERNIE-5.5-Flash-Base at a glance #
| Question | Answer |
|---|---|
| What is it? | Open-weight sparse MoE base model from Baidu |
| Size | 280B total, about 14B active per token |
| Training data | 33.5T tokens, 128K training context |
| Context | Up to 1M tokens for fine-tuning and inference |
| Weights | Hugging Face (FP8 and BF16), also ModelScope |
| Serving | SGLang and vLLM, completions endpoint |
| Instruction-tuned? | No. Post-trained version "will follow" |
| License | ERNIE 5.5 Flash Base Model License Agreement v1.0 (read it before commercial use) |
| Benchmarks | Self-reported by Baidu, not independently reproduced |
What did Baidu actually release? #
The release is the pre-training checkpoint and nothing else. There is no instruct model, no API listing in the sources we read, and no vision input. The card describes a 43-layer network: one dense layer and 42 MoE layers, hidden size 4096, 64 attention heads, and a vocabulary of 201,216 tokens.
Each MoE layer holds 512 routed experts, of which 10 fire per token, plus one shared expert. That is how a 280B model activates only about 14B parameters for each token. The ratio matters for cost: inference compute tracks the active parameters, while memory tracks the total.
Training used the Muon optimizer and FP8 mixed precision, according to the technical blog. The data mix stresses web text, repository-level code (commits, pull requests, CI-verified fixes) and long-context data.
If you are new to how MoE models route tokens, our explainers on Kimi K3's open weights and DeepSeek V4.1 Flash cache savings give useful background.
Three architecture ideas worth knowing #
Router hub linked to many small expert blocks, showing how a Multi-Router Latent MoE picks experts for ERNIE-5.5-Flash-Base
Baidu's technical blog names three changes. Each links to a paper listed on the model card. We have not read those papers for this post, so treat the descriptions below as Baidu's own summary.
Multi-Router Latent MoE. A normal MoE layer has one router that scores experts and keeps the top few. Baidu uses several independently initialized sigmoid routers that score experts jointly. The authors say the routing scores track how useful an expert is more closely, especially in middle and deeper layers. They also say active experts, expert FLOPs and communication cost stay the same. The experts work in a 2048-dimension latent space.
Virtual Head Attention. After attention, a light low-rank mixing step lets heads share information, with an identity skip connection. At inference it is fused into the attention kernel, so Baidu says the overhead is negligible.
ECHO. This replaces the usual multi-token-prediction module. It reuses the backbone's final layer to predict future tokens. The same layer then acts as the draft model for speculative decoding. Baidu claims a higher acceptance rate than a separate MTP module.
The blog also lists hybrid MLA and HCA attention with sparse attention in MLA layers, four residual streams with manifold-constrained hyper-connections, and the FlashMask-4 attention kernel. Baidu claims FlashMask-4 is up to 175% faster in the forward pass and 74% faster in the backward pass than FlashAttention-4 MaskMod on SM100 GPUs. That is a vendor claim about a kernel, not a result we can check.
How does it score against DeepSeek-V4-Flash-Base? #
Baidu compares the model with three other base models. All numbers below come from the model card, and all are self-reported.
| Benchmark | ERNIE-5.5-Flash-Base | DeepSeek-V4-Flash-Base | Hy3-preview-Base | MiMo-V2.5-Base |
|---|---|---|---|---|
| Total / active params | 280B / 14B | 284B / 13B | 295B / 21B | 310B / 15B |
| MMLU (5-shot) | 93.3 | 89.4 | 87.5 | 87.6 |
| MMLU-Pro (5-shot) | 78.4 | 69.2 | 66.9 | 65.7 |
| GPQA (5-shot) | 60.2 | 55.9 | 50.2 | 51.4 |
| SimpleQA (10-shot) | 34.1 | 29.1 | 25.9 | 20.8 |
| HumanEval+ (1-shot) | 82.3 | 69.5 | 78.0 | 74.4 |
| LiveCodeBench (1-shot) | 40.1 | 31.9 | 33.5 | 32.4 |
| RepoQA (0-shot) | 91.7 | 81.0 | 91.3 | 56.5 |
| LongBench-v2 (0-shot) | 48.7 | 44.5 | 39.8 | 44.6 |
The model does not win everywhere. Per the card, it trails on ChineseSimpleQA (73.0 vs 75.3 for DeepSeek-V4-Flash-Base), INCLUDE-lite (79.5 vs 79.8), FullStackBench (54.6 vs 56.7 for Hy3-preview-Base), HELMET-Rerank (56.2 vs 60.0), and InfiniteBench (46.5 vs 48.0 for DeepSeek, 46.7 for MiMo).
The pattern is clear: strong on English knowledge, reasoning and code, weaker on a few Chinese-language and very-long-context tests. That is a notable result for a Chinese lab's flagship base model.
How to read these numbers
Base-model benchmarks use few-shot prompts, and small prompt changes can move scores. Baidu ran all four models under one evaluation setup, but a setup chosen by the model's author is not a neutral test. Wait for third-party runs, for example from the Open LLM community or independent evaluators, before you treat a 13-point MMLU-Pro gap as fact.
Also note that the rivals are other base models. The numbers say nothing about how ERNIE-5.5-Flash-Base compares with a post-trained model such as a chat or agent release. For that comparison you will need the instruct version.
How to run it with SGLang or vLLM #
Arrow splitting a request toward small expert boxes with one lit, illustrating how an open-weight MoE like ERNIE-5.5-Flash-Base serves tokens
Baidu says both frameworks serve the model through the completions API. This is the SGLang launch command from the model card:
sglang serve \
--model-path baidu/ERNIE-5.5-Flash-Base \
--tp 4 \
--attn-dp-size 4 \
--context-length 131072 \
--max-running-requests 1024 \
--host 0.0.0.0 \
--port 30000
And the vLLM command, which needs vllm==0.32.0 plus the vLLM-ERNIE plugin:
vllm serve baidu/ERNIE-5.5-Flash-Base \
--kv-cache-dtype fp8 \
--enable-expert-parallel \
--data-parallel-size 4 \
--max-model-len 131072 \
--port 30000 \
--moe-backend humming
Because this is a base model, query the completions endpoint, not chat:
curl http://localhost:30000/v1/completions \
-H "Content-Type: application/json" \
-d '{"model": "baidu/ERNIE-5.5-Flash-Base", "prompt": "Large language models are", "max_tokens": 256, "temperature": 0}'
Both examples serve a 131,072-token window, which matches the 128K training context. The 1M-token figure is the context Baidu says fine-tuning and inference support. The sources do not give a launch command for it, so expect to raise the length setting and budget for much more KV-cache memory.
What hardware do you need?
The card does not give a minimum. The example commands use four-way parallelism. At FP8, 280B parameters take roughly 280 GB for weights alone, which is more than one 80 GB GPU holds. Plan for a multi-GPU node. This is our own arithmetic, not a Baidu figure.
What can you do with a base model? #
A base model is raw material. Typical uses, in the words of the card, are fine-tuning and reinforcement-learning research.
- Domain adaptation. Continue pre-training on your own corpus, such as legal, medical or code data.
- Instruction tuning. Train your own chat or agent behavior on top.
- Research. Study the new routing and speculative-decoding design with the weights in hand.
- Evaluation baselines. Measure how much a post-training recipe adds.
Most teams will wait for the instruct model. If you need something small that you can run today, Baidu's earlier Unlimited-OCR-2, a 3B MIT-licensed document parser, is a better fit.
The license question #
The model page names the "ERNIE 5.5 Flash Base Model License Agreement v1.0" and the notice "Copyright © 2026 Baidu. All Rights Reserved." It does not say in plain words whether commercial use is permitted. That is a real gap, because other recent open releases state Apache 2.0 or MIT clearly. See, for example, SenseNova Loop 9B, which ships under Apache 2.0.
Our advice: read the LICENSE file in the repository, and have counsel check any use that earns money. Do not assume the weights are as permissive as an MIT or Apache release.
Where does it fit among Chinese open models? #
China's labs now ship large open MoE models at a steady pace. Tencent's Hy4 is a 770B model whose official launch added vision and computer use, though its weights were still pending at launch; the earlier preview had promised open weights. DeepSeek's V4 Flash Vision is a 305B open model.
ERNIE-5.5-Flash-Base sits in the same size class as DeepSeek's Flash line, at 280B total and 14B active. Its difference is that Baidu publishes the pre-training checkpoint with a detailed architecture write-up, and it documents serving with two major inference engines.
What to watch next #
- The post-trained ERNIE 5.5 Flash. Baidu says it is coming. That model will decide whether this line competes in chat and agents.
- Independent benchmark runs. Third-party results on MMLU-Pro, HumanEval+ and long-context tasks will show if the self-reported lead holds.
- License clarity. A plain statement on commercial use would remove the main adoption risk.
- Community fine-tunes. Quantized and domain-tuned versions usually appear within days of a release like this.
Limits of this post #
We read the Hugging Face model card and Baidu's technical blog. We did not run the model, and we did not read the linked papers. Every benchmark and speed number here is Baidu's own. We will update this post when independent results or the instruct model appear.
Related reading #
Specs and benchmark figures are accurate as of November 5, 2026, and come from Baidu's published model card and technical blog.