cd /news/natural-language-processing/blimp-ro-a-benchmark-of-linguistic-m… · home › topics › natural-language-processing › article
[ARTICLE · art-170346] src=machinebrief.com ↗ pub= topic=natural-language-processing verified=true sentiment=· neutral

BLiMP-RO: A Benchmark of Linguistic Minimal Pairs for Romanian

Researchers introduced BLiMP-RO, a Romanian linguistic minimal-pair benchmark covering 50 grammatical phenomena across ten domains, comprising 493 linguist-constructed sentence pairs validated by native speakers in five experiments plus 5,942 semi-automatically generated, human-validated pairs. Training GPT-2 and GPT-BERT models from scratch on Romanian BabyBabelLM data and a newly collected 10-million-word corpus of child-targeted video captions, the authors found roughly 9 percentage points of accuracy improvement per tenfold increase in corpus size; at matched data sizes BabyBabelLM generally outperformed video captions, and GPT-BERT consistently outperformed GPT-2.

by read1 min views1 publishedNov 16, 2026

arXiv:2611.17836v1 Announce Type: new Abstract: Evaluating grammatical knowledge in language models requires benchmarks that cover diverse linguistic phenomena and are validated against human judgments. We introduce BLiMP-RO, a Romanian minimal-pair benchmark covering 50 grammatical phenomena across ten domains. It comprises 493 linguist-constructed sentence pairs, evaluated by native speakers in five experiments, and 5,942 additional semi-automatically generated, human-validated pairs. We further investigate grammatical generalization under limited linguistic input by training GPT-2 and GPT-BERT models from scratch on Romanian BabyBabelLM data and a newly collected corpus of 10 million words from child-targeted video captions. Increasing caption training data yields approximately 9 percentage points of accuracy improvement per tenfold increase in corpus size. At matched data sizes, BabyBabelLM generally outperforms video captions, while GPT-BERT consistently outperforms GPT-2. Our findings highlight how training data quantity, composition, and model architecture shape grammatical generalization in Romanian.

── more in #natural-language-processing 4 stories · sorted by recency
── more on @blimp-ro 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/blimp-ro-a-benchmark…] indexed:0 read:1min 2026-11-16 · —