JudgeBench

TuringCorp built Decider, an AI judge that takes a question and two candidate answers and returns the stronger pick, a calibrated confidence value, and its reasoning. On JudgeBench (614 judgments), De…

A systematic evaluation of 21 LLM-as-a-Judge models across 118 runs and 541,000 judgments reveals that exact-match agreement overstates discriminative ability, with kappa deflation of 33–41 percentage…