Researchers introduced JudgeArena, an open-source framework unifying major LLM-judge benchmarks (AlpacaEval, Arena-Hard, MT-Bench, and m-Arena-Hard) under a single interface with swappable judges and …
A field guide to LLM-as-a-judge reveals that five novel proposals for improving the technique were all refuted by prior art published within the last twelve weeks, according to a research synthesis co…
A new study reveals that LLM judges, widely used to evaluate AI outputs, can be manipulated after making an initial decision through targeted conversation, overturning stable judgments and shifting be…
A new analysis of over 5,400 AI models reveals that benchmark scores for large language models are highly correlated, with just five subjects on the MMLU test predicting the remaining 52 with 91% accu…