{"slug": "adage-a-language-agnostic-pipeline-for-analogical-reasoning-evaluation", "title": "ADAGE: A Language-Agnostic Pipeline for Analogical Reasoning Evaluation", "summary": "Researchers introduce ADAGE, a language-agnostic pipeline for analogical reasoning evaluation that uses native-speaker curation and LLM-assisted generation to create translation-free benchmarks. Testing on Arabic, Amharic, and Japanese, they find that 14 open-weight models show a consistent cultural reasoning gap, with accuracy dropping 12–52 percentage points relative to English.", "body_md": "arXiv:2607.23058v1 Announce Type: new\nAbstract: Multilingual reasoning evaluation overwhelmingly relies on translating English benchmarks, a practice that introduces linguistic artifacts and fails to test culturally-grounded reasoning. We introduce ADAGE (Analogical Difficulty-by-design Assessment for Grounded Evaluation), a language-agnostic pipeline that combines native-speaker curation with LLM-assisted generation to construct challenging, translation-free benchmarks for abstract analogical reasoning. We validate ADAGE by constructing benchmarks for Arabic, Amharic, and Japanese. Evaluating 14 open-weight models, we find a consistent cultural reasoning gap: models that perform well on English proverb reasoning struggle substantially on all three native benchmarks, with accuracy dropping by 12--52 percentage points relative to English. We release the pipeline, all three benchmarks, and the full evaluation suite.", "url": "https://wpnews.pro/news/adage-a-language-agnostic-pipeline-for-analogical-reasoning-evaluation", "canonical_source": "https://arxiv.org/abs/2607.23058", "published_at": "2026-07-28 04:00:00+00:00", "updated_at": "2026-07-28 04:26:00.466482+00:00", "lang": "en", "topics": ["artificial-intelligence", "natural-language-processing", "ai-research", "large-language-models"], "entities": ["ADAGE", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/adage-a-language-agnostic-pipeline-for-analogical-reasoning-evaluation", "markdown": "https://wpnews.pro/news/adage-a-language-agnostic-pipeline-for-analogical-reasoning-evaluation.md", "text": "https://wpnews.pro/news/adage-a-language-agnostic-pipeline-for-analogical-reasoning-evaluation.txt", "jsonld": "https://wpnews.pro/news/adage-a-language-agnostic-pipeline-for-analogical-reasoning-evaluation.jsonld"}}