RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment

RightNowAI released RightNow-Arabic-0.5B-Turbo, a 518M-parameter Arabic-specialized language model that outperforms all existing sub-1B open models on Arabic benchmarks while running on edge devices. The model, built by injecting 27,032 Arabic tokens into Qwen2.5-0.5B and continuing pretraining on 504M Arabic tokens, achieves 35.9% mean accuracy on Arabic benchmarks and ties Falcon-H1-1.5B on COPA-ar at one-third the size. Quantized to 398 MB, the model delivers 635 tokens per second on a single H100, with all code, weights, and benchmarks released openly.

arXiv:2605.28827v1 Announce Type: new Abstract: Open Arabic large language models split into two classes: sub-1B multilingual models that treat Arabic as an afterthought Qwen2.5-0.5B, Falcon-H1-0.5B , and 7B-70B Arabic-specialized models that require a server to run Jais, AceGPT, ALLaM, SILMA . The one published attempt at a sub-2B Arabic-specialized model, Kuwain-1.5B, never released its weights. We present RightNow-Arabic-0.5B-Turbo, a 518M-parameter Arabic-specialized decoder LLM built on Qwen2.5-0.5B. The pipeline adds 27,032 Arabic tokens via mean-subtoken initialization, continues pretraining on 504M Arabic tokens on 8xH100 with FSDP, FlashAttention varlen packing, and Liger fused kernels, then applies supervised fine-tuning on 129,116 Arabic instruction pairs with response-only loss masking, direct preference optimization on 6,750 Arabic preference pairs, and weight soup merging across three checkpoints. On three lm-evaluation-harness Arabic benchmarks COPA-ar, Arabic HellaSwag, ArabicMMLU the merged model reaches 35.9% mean accuracy, beats every same-class open model, ties Falcon-H1-1.5B on COPA-ar 58.4% at one-third the size, and recovers 67% of SILMA-9B's mean at 1/18 the parameters. The edge build quantizes to 398 MB q4 k m and delivers 635 tokens/s at batch size 1 on a single H100 via llama.cpp. All code 5,555 lines across 25 scripts , weights bf16, int8, and four GGUF quantizations , and benchmark scripts are released at https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo.