LMCA

Anthropic and alignment researchers including Caspar Oesterheld and Emery Cooper released the Conceptual Reasoning Index (CRI), a 0–100 benchmark scoring AI reasoning in domains without ground truth, …

Anthropic collaborated on the release of the Conceptual Reasoning Index (CRI), a suite of three benchmarks—LMCA, ACCoRD, and DTBench—designed to measure AI models' ability to reason about conceptual q…

A researcher argues that judgment prediction tasks are poorly suited for benchmarking AI conceptual capabilities due to subjective disagreements and noise. The approach would involve measuring how wel…