The Allen Institute for AI introduced BenchMIRT, a new method for auditing LLM benchmarks at the level of individual prompts, which uses multidimensional Item Response Theory to separate the underlyin…
Researchers introduced a 'capability slice' unit and a closed-loop system that links evaluation failures to targeted data interventions in LLM pre-training. In two case studies, the system correctly i…
Researchers introduced SPEAR (Sandboxed Prompt Engineer with Active Roll-back), a code-augmented agentic optimizer that autonomously rewrites prompts using a Python sandbox for structural error analys…
A new study of small language models reveals that chain-of-thought prompting for arithmetic relies on a positional shortcut: the model copies whichever number appears last before the answer delimiter,…