19:27
2026-07-22
dev.to
artificial-intelligence
An LLM judge is a biased instrument, not a measurement
A developer found that an LLM judge gave opposite results for the same eval run on consecutive days due to position bias, one of three systematic biases documented in the 2023 paper "Judging LLM-as-a-…