Lara Translate

Translated's Lara Mini, a 7.2B-parameter mixture-of-experts model with 0.6B active parameters per token, ranked #1 of 36 models in its class worldwide after training on 6T tokens, the company reported. Lara Mini was built as the multilingual model underlying Lara 4 Translation AI, covering 44 languages plus English, including all 24 EU official languages, with a 128K-token context window and a 151,643-token vocabulary. Translated said 39% of raw tokens were filtered out of its deduplicated pool and that no source was repeated more than 2 epochs, arguing that multilingual and multicultural data is "a source of it" rather than a cost to intelligence.

Every AI lab assumes more languages make a model worse.We bet the opposite. Born as the general-purpose LLM underneath Lara 4 Translation AI, Lara Mini was built as the most multilingual and multicultural model from day one. At the end of its 6T‑token training it ranks 1 of 36 models in its class worldwide. Diversity isn't a cost to intelligence. It's a source of it. The belief The big labs train mostly on English and treat other languages and cultures as noise that costs quality. The bet Translated, whose job is translation, built Lara on real text written in many languages and cultures. Seeing one idea through many of them, the model understands the idea better. The proof 1 of 36 models with up to 2B active parameters worldwide, trained on 6T tokens, 128K context. The run at a glance Architecture 0.6Bactive parameters per token 4 experts 7.2Btotal parameters, MoE with 128 experts, 1 shared 34layers: memory GDN , window and global attention 128Ktokens of context window Data 6Ttraining tokens 39%of raw tokens filtered 44 + ENlanguages, all 24 EU official ones 151,643tokens in vocabulary, low resource rebalanced How we built it A hybrid, sparse design for the most multilingual model A hybrid, sparse model: 34 layers, 128 experts Three kinds of layer, one repeating unit windowmemorywindowmemoryglobal 14 memory · Gated DeltaNet: a linear-attention state per head, updated token by token 13 window · attention over the last 512 tokens, RoPE positions 7 global · full attention, no positional encoding; the last layer is always global Mixture of experts in every layer 4 of 128 routed experts chosen per token by a router 1 shared expert always on 0.6B of 7.2B parameters active per token; hidden size 1,024, tied input and output embeddings Optimizer Muon + AdamW Each momentum update is orthogonalised Newton–Schulz before it is applied, so every direction of a weight matrix moves at a similar rate instead of a few dominating. 4,096-token context window in training; 151,643-token vocabulary. The most multilingual and multi-cultural model 6T training tokens from a pool deduplicated across all sources: 39% of raw tokens were filtered out. No source is repeated more than 2 epochs wiki and high-quality math excepted ; plain web is never repeated. English2.49TWeb HQ 1.34T · Synthetic HQ 0.66T · PDFs 0.32T · Web medium 0.15T · Wiki 0.02T Multilingual2.18TWeb 0.79T · Web HQ 0.66T · PDFs 0.4T · Synthetic 0.28T · Wiki 0.05T Code0.77TReal code 0.45T · Synthetic 0.31T Math0.35TMath HQ 0.21T · Math web 0.15T SFT-like0.21TQA 0.12T · Math 0.07T · Code 0.02T Three stages, one learning-rate decay Main · 4.8T0–80% · to step 572.2k Transition · 0.6T80–90% · to step 643.7k Anneal · 0.6T90–100% · to step 715.3k Learning-rate scheduleEvaluated model: end of training, 6T tokens. Each stage is a separate data blend loaded at its step boundary. Transition: a bridge between the two at the stable learning rate, where the mix already moves halfway toward the anneal's more high-quality, QA, code and math data so the model adapts before the decay. Anneal: the highest-quality mix, while the learning rate decays to zero over the last 10%. Each stage leans further into code, math and SFT-like data English Multilingual Code Math SFT-like English-only Main 44%37%12% Transition 35.5%35%14.5%8% Anneal 27%33%17%11%12% Whole run 41%36%13% Share of each stage's tokens. The data are shuffled, not fed domain by domain: every batch of 2,048 sequences mixes all domains in about these proportions. SFT-like QA, math and code instruction data : 2% in main, 3.5% over the whole run. Methodology 26 benchmarks, one harness, every model Overall = ¼ English + ¼ Multilingual + ¼ Math + ¼ Code. Every model is scored by us with lm-evaluation-harness and the same answer-format corrections. Each multilingual benchmark is averaged over our 44 non-English training languages. English10 NLI PAWS Belebele en COPA PIQA HellaSwag WinoGrande ARC-C MMLU MMLU-Pro Multilingual9 XNLI PAWS-X Belebele XCOPA Global PIQA NP Global PIQA P T-HellaSwag INCLUDE Global MMLU Math3 GSM8K MATH MGSM Code4 MBPP MBPP+ CRUX-I CRUX-O No peeking: the final training stage skips every document that holds a benchmark item. Results 1 of 36 in tiny models worldwide First on Overall, Text and Multilingual, 3.1 points ahead of granite-4.0-h-tiny on Overall. Overall Tiny models ranking 1 out of 36 models Category ranking Math & Code 1 Text 1 Multilingual 1 Benchmark definition. Our model is the final checkpoint of the 6T‑token run, with its context extended to 128K; its anneal skips every document holding a benchmark item. Its class: models with up to 2B active parameters gemma-4-E2B at its 2B effective size . Multilingual scores are averaged over the 44 non-English languages we train on, for every model. Rankings Top 12 tiny models 36 models with up to 2B active parameters gemma-4-E2B at its 2B effective size, 5.1B total . Multilingual on our training languages. Bars start from 0. OverallEuropeTextMath & Code 1Lara Mini Base A0.6B/7.2B 57.0 2IBM - granite-4.0-h-tiny 1B active 53.9 3Alibaba - Qwen3-1.7B53.8 4Falcon-H1-1.5B-Deep53.2 5Hunyuan-1.8B52.2 6LFM2.5-8B-A1B 1.7B active 50.8 7Trinity-Nano 1B active 50.7 8LFM3-1.7B49.2 9Qwen3.5-2B48.9 10Falcon-H1-1.5B48.8 11Qwen2.5-1.5B47.6 12granite-4.0-1b47.2 ⋯ 17Google - gemma-4-E2B41.9 Overall = ¼ English + ¼ Multilingual + ¼ Math + ¼ Code. 1Lara Mini Base A0.6B/7.2B 57.0 2Bielik-1.5B-v3 Qwen-based 40.9 3Hugging Face - SmolLM2-1.7B36.8 4Luciole-1B31.3 5Apertus-v1.1-1.5B31.2 6Kyutai - helium-1-2b30.7 7EuroMoE-2.6B-A0.6B-2512 0.5B active 29.4 8EuroLLM-1.7B27.6 9Apertus-v1.1-0.5B24.2 +16 points +39% over the next European model. European sovereign models of our class 9 , from European labs: up to 2B active parameters. Overall score, Multilingual on our training languages. 1Lara Mini Base A0.6B/7.2B 61.5 2IBM - granite-4.0-h-tiny 1B active 59.0 3Google - gemma-4-E2B58.3 4Trinity-Nano 1B active 57.9 5LFM2.5-8B-A1B 1.7B active 57.1 6Qwen3-1.7B57.0 7Qwen3.5-2B56.7 8Falcon-H1-1.5B-Deep55.2 9Qwen2.5-1.5B54.7 10Hunyuan-1.8B53.5 11LFM3-1.7B53.3 12Falcon-H1-1.5B53.3 Text = mean of English 10 benchmarks and Multilingual 9 benchmarks, our 44 non-English training languages . Separately: English 2 66.4 , Multilingual 1 56.7 . 1Lara Mini Base A0.6B/7.2B 52.5 2Falcon-H1-1.5B-Deep51.2 3Hunyuan-1.8B50.9 4Alibaba - Qwen3-1.7B50.6 5IBM - granite-4.0-h-tiny 1B active 48.9 6LFM3-1.7B45.1 7LFM2.5-8B-A1B 1.7B active 44.4 8Falcon-H1-1.5B44.4 9Trinity-Nano 1B active 43.5 10granite-4.0-1b43.1 11Qwen3.5-2B41.1 12Qwen2.5-1.5B40.5 ⋯ 26Google - gemma-4-E2B 5.1B total 25.4 Math & Code = mean of Math GSM8K, MATH, MGSM and Code MBPP, MBPP+, CRUX-I, CRUX-O . Math 1 59.9 , Code 5 45.1 . Efficiency 1 under 2B, with just 0.6B active Lara Mini Base outscores all 35 other models of its class, including models that use up to 3× more active parameters, such as Qwen3-1.7B and gemma-4-E2B. Overall score against active parameters, log scale. The step line is the frontier: no model with fewer active parameters scores higher. gemma-4-E2B is drawn at its 2B effective size 5.1B total . Limited release · who gets it What Lara Mini is for Lara Translate app Offline Translation Lara Mini is the base model behind Lara 4 offline translation feature for the next Lara Translate app. Your text never leaves the phone, and each translation still takes under a second. Translated customers Tiny Base LLM Selected Translated enterprise customers will have access. Lara Mini is designed to be fine-tuned for specialized sovereign applications, runs on mobile devices and on our cloud delivers an end-to-end latency under 20 ms delivering millions of tokens per sec. Research AI researchers Access for AI researchers to reproduce our results and build on them, pushing multilingual research further. What's next Three competitions, one rule We train the next stage only if we win a medal in the current one. Stage 1 Local Tiny · <2B active Nov 2026Lara Mini Base Win the tiny category with Lara Base A0.6B/7.2B . Dec 2026Lara Mini Lara 4 Mini and Lara Mini LLM, for ultra-low latency and mobile devices. Medal won → we train the National stage. No medal, we stop. Stage 2 National Small · <10B active Mar 2027Lara Piccola Base A7B/138B Enter the small category and win a medal there too. Apr 2027Lara 4 & Lara Piccola LLM Lara 4 translation model and Lara Piccola LLM, an EU-sovereign A7B/138B model. Medal won → we train the Olympics stage. No medal, we stop. Stage 3 Olympics Frontier · 40B+ active Dec 2027Lara Grande Base A60B/1.2T Enter the frontier category, against the world's best models. Mar 2028Lara 5 & Lara Grande LLM Lara 5 surpasses P95 pro translators. Lara Grande: an EU-sovereign frontier LLM, ahead of Chinese models and approaching Anthropic and OpenAI. Reference models:tiny Gemma 4 E2B · small Gemma 4B/26B · medium Qwen 17B/397B · frontier GPT-6 Astra, Claude 5.6, Kimi K3. Training budget Training single-run budgets, next releases Model Tokens FLOPs Hardware Days Training cost Lara 4 PiccolaMoE 6.7B / 138B 20T 8.0 × 10²³ 512 × H100 31 $1.0M Lara 4 GrandeMoE 60B / 1.2T 24T 8.6 × 10²⁴ 512 × Vera Rubin 25 $2.4M Lara 4 SaggiaMoE 600B / 12T 36T 1.3 × 10²⁶ 2,304 × Vera Rubin 83 $36.6M For reference Typical frontierMoE 150B / 3T 36T 3.2 × 10²⁵ 4,096 × B200 51 $18.8M Source: Lara 4 LLM Training Budget Planning, 3 Nov 2026 · FP8 · compute cost only · Saggia days estimated with the plan's method Next training Lara Piccola Candidate 60 layers and 138B parameters: the model to beat Gemma 4 31B. 138Btotal parameters 6.7Bactive per token Tokenizernew, rebalanced 60layers 2816hidden size 512 × 512experts, top-8 + 2 shared 400kH100 GPU-hours for 20T tokens 30 dayson 512 H100 GPUs $1MTraining run cost Goal: win the small model class <10B active Current leaders: Qwen3.5-122B-A10B, GLM-4.5-Air Good chance Goal: beat Gemma 4 31B on multilingual Beat highly multilingual dense models in the medium model class. Good chance 30.7 s per step at 32 nodes · same hybrid attention as Lara Mini Base GDN, window, global Technical report Get the technical report Architecture, training data, benchmarks and methodology behind Lara Mini Base, in one document. Leave your details and we'll email it to you. First name Last name Business email Company name Company website I consent to be contacted by Lara for further information and demo. I have read and agree to the Privacy Policy Submit Work with Lara Two ways to put Lara to work Interested in the generative model? Selected Translated enterprise customers will have access. Lara Mini is designed to be fine-tuned for specialized sovereign applications, runs on mobile devices and on our cloud delivers an end-to-end latency under 20 ms delivering millions of tokens per sec.