Baidu open-sources ERNIE-5.5-Flash-Base, a 280B-parameter MoE — as a base model, not a chatbot
On November 5, 2026, Baidu posted the raw pre-training checkpoint of ERNIE-5.5-Flash-Base on Hugging Face: a text-only MoE with 280 billion total parameters, of which only 14 billion fire per token. It is a fine-tuning base, not an assistant — and the license still needs reading before any commercial use.
November 5, 2026. Baidu posts the raw pre-training checkpoint of ERNIE-5.5-Flash-Base on Hugging Face, in FP8 and BF16, and on ModelScope. November 5, 2026. The model is a text-only MoE with 280 billion total parameters, of which only 14 billion activate per token. November 5, 2026. Baidu says the post-trained version — the chatbot — “will follow,” with no date. Why it matters: a Chinese lab is shipping the base first and the assistant second, the exact reverse of the usual release cycle.
For a team tracking open models, this is not one more launch. It is a signal: for the big labs, open source is becoming a research and fine-tuning artifact first, and a ready-to-use product second.
What Baidu actually shipped: a base, not an assistant
The most important thing is in the name. ERNIE-5.5-Flash-Base is a base model — the checkpoint at the end of pre-training, with no mid-training and no post-training. There is no instruct version, no API listed in the public sources, and no vision input: it is text-only.
The spec sheet fits in a few numbers. The network has 43 layers: one dense layer and 42 MoE layers, a hidden size of 4096, 64 attention heads, and a vocabulary of 201,216 tokens. Each MoE layer holds 512 routed experts, of which 10 fire per token, plus one shared expert. That is how a 280 billion parameter model activates only 14 billion at inference.
Training used 33.5 trillion tokens, a 128,000-token training context, extendable to 1 million tokens for fine-tuning and inference. The optimizer is Muon, in FP8 mixed precision, and the data mix leans on web text, repository-level code (commits, pull requests, CI-verified fixes), and long-context data.
The practical consequence is direct: this model is for fine-tuning, not for chatting. To get an assistant, you teach it the behavior yourself — or wait for the post-trained release.
Three architecture ideas that explain the 280B/14B ratio
Baidu’s technical post highlights three novelties, each backed by a paper cited in the model card.
Multi-Router Latent MoE. A conventional MoE layer uses one router that scores experts and keeps the top few. Baidu uses several independently initialized sigmoid routers that score experts jointly. The authors say routing scores track an expert’s real usefulness more closely, especially in middle and deep layers, without increasing active experts, FLOPs, or communication cost. The experts work in a 2048-dimensional latent space.
Virtual Head Attention. After attention, a light low-rank mixing step lets heads share information, with an identity skip connection. At inference it is fused into the attention kernel, so Baidu claims the overhead is negligible.
ECHO. This module replaces the usual multi-token prediction: it reuses the backbone’s final layer to predict future tokens, and that same layer then acts as the draft model for speculative decoding. Baidu claims a higher acceptance rate than a separate MTP module.
The post also mentions hybrid MLA and HCA attention, four residual streams with manifold-constrained hyper-connections, and a FlashMask-4 attention kernel claimed to be up to 175% faster in the forward pass and 74% faster backward than FlashAttention-4 MaskMod on SM100 GPUs. That last figure is a vendor claim, not a result anyone can verify.
Self-reported benchmarks that beat DeepSeek-V4-Flash-Base on code
Every published score is self-reported by Baidu, measured in a single evaluation setup against three other base models: DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base. None has been independently reproduced yet.
On code, the gap is clear. HumanEval+ lands at 82.3 versus 69.5 for DeepSeek-V4-Flash-Base, and LiveCodeBench at 40.1 versus 31.9. RepoQA reaches 91.7 versus 81.0. On general knowledge and reasoning, MMLU is 93.3 (versus 89.4) and MMLU-Pro is 78.4 (versus 69.2). GPQA follows at 60.2 versus 55.9.
The model does not win everywhere. It trails on ChineseSimpleQA (73.0 versus 75.3), FullStackBench (54.6 versus 56.7 for Hy3-preview-Base), HELMET-Rerank (56.2 versus 60.0), and InfiniteBench (46.5 versus 48.0). The pattern reads clearly: strong on English, reasoning, and code, weaker on a few Chinese-language and very-long-context tests.
The reading should stay cautious. An evaluation setup chosen by the model’s author is not a neutral test, and these rivals are other base models, not post-trained assistants. A 13-point MMLU-Pro gap remains a claim until an independent evaluator confirms it.
The license is the real friction point
The last detail will decide adoption. The model page names an “ERNIE 5.5 Flash Base Model License Agreement v1.0” and the notice “Copyright © 2026 Baidu. All Rights Reserved.” — without saying in plain words whether commercial use is allowed.
That is a real gap, because other recent releases state Apache 2.0 or MIT unambiguously. A vague proprietary license on an “open” model forces you to read the LICENSE file and, for anything that earns money, to have counsel check the terms. The classic trap is treating weights as permissive because they are downloadable.
For teams that want to serve the model, SGLang and vLLM are supported, through the completions endpoint — not chat — since this is a base. The SGLang command from the model card says a lot about the hardware required:
sglang serve \
--model-path baidu/ERNIE-5.5-Flash-Base \
--tp 4 \
--attn-dp-size 4 \
--context-length 131072 \
--max-running-requests 1024 \
--host 0.0.0.0 \
--port 30000 The four-way parallelism is not incidental: at FP8, 280 billion parameters mean roughly 280 GB of weights, more than one 80 GB GPU holds. You need a multi-GPU node, and the cost math follows the total, even if inference only follows the 14 billion active parameters.
Verdict
ERNIE-5.5-Flash-Base is a serious release, but its appeal is narrow. Baidu shipped a research base — raw material for fine-tuning, not an assistant to deploy.
If you want a ready-to-use assistant, wait for the post-trained version: deploying a base as a chatbot means alignment work the vendor will do itself, better and faster. If you do domain adaptation or MoE architecture research, the model is worth a look: the weights are out, SGLang/vLLM support is announced, and the three architecture ideas — multi-routers, virtual heads, ECHO — are worth studying with the weights in hand. And in all cases, read the license before you sell anything: a downloadable model is not a permissive model until the text has been checked. The underlying signal is clear — for the big labs, open source is becoming a fine-tuning and research tool, not a distribution channel for finished assistants.