{"slug": "blimp-ro-a-benchmark-of-linguistic-minimal-pairs-for-romanian", "title": "BLiMP-RO: A Benchmark of Linguistic Minimal Pairs for Romanian", "summary": "Researchers introduced BLiMP-RO, a Romanian linguistic minimal-pair benchmark covering 50 grammatical phenomena across ten domains, comprising 493 linguist-constructed sentence pairs validated by native speakers in five experiments plus 5,942 semi-automatically generated, human-validated pairs. Training GPT-2 and GPT-BERT models from scratch on Romanian BabyBabelLM data and a newly collected 10-million-word corpus of child-targeted video captions, the authors found roughly 9 percentage points of accuracy improvement per tenfold increase in corpus size; at matched data sizes BabyBabelLM generally outperformed video captions, and GPT-BERT consistently outperformed GPT-2.", "body_md": "arXiv:2611.17836v1 Announce Type: new \nAbstract: Evaluating grammatical knowledge in language models requires benchmarks that cover diverse linguistic phenomena and are validated against human judgments. We introduce BLiMP-RO, a Romanian minimal-pair benchmark covering 50 grammatical phenomena across ten domains. It comprises 493 linguist-constructed sentence pairs, evaluated by native speakers in five experiments, and 5,942 additional semi-automatically generated, human-validated pairs. We further investigate grammatical generalization under limited linguistic input by training GPT-2 and GPT-BERT models from scratch on Romanian BabyBabelLM data and a newly collected corpus of 10 million words from child-targeted video captions. Increasing caption training data yields approximately 9 percentage points of accuracy improvement per tenfold increase in corpus size. At matched data sizes, BabyBabelLM generally outperforms video captions, while GPT-BERT consistently outperforms GPT-2. Our findings highlight how training data quantity, composition, and model architecture shape grammatical generalization in Romanian.", "url": "https://wpnews.pro/news/blimp-ro-a-benchmark-of-linguistic-minimal-pairs-for-romanian", "canonical_source": "https://www.machinebrief.com/news/blimp-ro-a-benchmark-of-linguistic-minimal-pairs-for-romania-pure", "published_at": "2026-11-16 05:00:00+00:00", "updated_at": "2026-11-16 05:25:24.543543+00:00", "lang": "en", "topics": ["natural-language-processing", "large-language-models", "ai-research", "machine-learning"], "entities": ["BLiMP-RO", "BabyBabelLM", "GPT-2", "GPT-BERT", "Romania"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/blimp-ro-a-benchmark-of-linguistic-minimal-pairs-for-romanian", "markdown": "https://wpnews.pro/news/blimp-ro-a-benchmark-of-linguistic-minimal-pairs-for-romanian.md", "text": "https://wpnews.pro/news/blimp-ro-a-benchmark-of-linguistic-minimal-pairs-for-romanian.txt", "jsonld": "https://wpnews.pro/news/blimp-ro-a-benchmark-of-linguistic-minimal-pairs-for-romanian.jsonld"}}