{"slug": "index-slm-technical-report", "title": "Index SLM Technical Report", "summary": "Bilibili released Index-1.9B, a series of open small language models including base, pure, chat, and character variants. The 1.9B-parameter models achieve competitive benchmark scores against larger models, with the base model averaging 64.92 on standard tests. All models and evaluation code are open-sourced on GitHub.", "body_md": "arXiv:2607.09885v1 Announce Type: new\nAbstract: We present Index-1.9B, a series of open small language models developed at Bilibili. The series comprises four models: Index-1.9B-Base, a foundation model with 1.9 billion non-embedding parameters pre-trained on 2.8 trillion predominantly Chinese and English tokens; Index-1.9B-Pure, a control variant trained with an identical recipe but with all instruction-like data strictly filtered from the corpus; Index-1.9B-Chat, aligned from the base model with supervised fine-tuning and direct preference optimization; and Index-1.9B-Character, which augments the chat model with retrieval-augmented generation for few-shot role-playing customization. Pre-training employs a Warmup-Stable-Decay learning-rate schedule in which the concentration of curated data is raised substantially during the decay phase, together with a Norm-Head output layer that stabilizes training under large learning rates. On a suite of standard benchmarks covering examination, reasoning, mathematics, and code, Index-1.9B-Base attains an average score of 64.92, competitive with or exceeding open models of several times its size. We further report controlled studies on model depth, learning-rate magnitude and scheduling, the interaction between learning-rate decay and data quality, and the effect of including instruction data during pre-training, and we document an unexplained surge in benchmark performance midway through the constant-learning-rate phase. All models, together with evaluation code, are released at https://github.com/bilibili/Index-1.9B.", "url": "https://wpnews.pro/news/index-slm-technical-report", "canonical_source": "https://arxiv.org/abs/2607.09885", "published_at": "2026-07-14 04:00:00+00:00", "updated_at": "2026-07-14 04:33:19.424074+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "natural-language-processing", "ai-research"], "entities": ["Bilibili", "Index-1.9B", "Index-1.9B-Base", "Index-1.9B-Pure", "Index-1.9B-Chat", "Index-1.9B-Character", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/index-slm-technical-report", "markdown": "https://wpnews.pro/news/index-slm-technical-report.md", "text": "https://wpnews.pro/news/index-slm-technical-report.txt", "jsonld": "https://wpnews.pro/news/index-slm-technical-report.jsonld"}}