Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text

A Qwen3-based model named Qwenfant ranks 1st in the Multilingual track of the 2026 BabyLM Challenge, according to an arXiv submission by Lukas Edman and Alexander Fraser. The team improved on its baseline using data summarized and translated by an LLM, then added two changes: a larger model and a character-level vocabulary, both of which required LoRA-style fine-tuning to reach full potential. The work, arXiv:2610.19294v1, argues higher-quality training data can come from synthetic sources.

Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text By Lukas Edman, Alexander FraserSource: arXiv cs.CL https://arxiv.org/list/cs.CL/recent arXiv:2610.19294v1 Announce Type: new Abstract: This work concerns our submission for the Multilingual track of the 2026 BabyLM Challenge. We improve on our baseline using data that has been summarized and translated by an LLM https://www.machinebrief.com/glossary/llm , showing that higher-quality data can come from synthetic sources. We further improve with 2 simple changes: using a larger model, and using a character-level vocabulary. Both of these changes require LoRA https://www.machinebrief.com/glossary/lora -style fine-tuning https://www.machinebrief.com/glossary/fine-tuning for realizing their full potential. We dub our final Qwen3-based model Qwenfant, which, at the time of submission, ranks 1st in the Multilingual track. Get AI news in your inbox Daily digest of what matters in AI.