Evaluating the Evaluators: Investigating LLM Judges for Personalized Writing Style Assessment

A NeurIPS 2026 workshop paper from researchers at the University of Southern California and the University of Copenhagen finds that LLM judges' quality correlates only weakly with general model ability measured by MMLU (Pearson's ρ ≤ 0.2) when assessing personalized writing style. The study generated text with four LLMs across three long-form writing tasks, collected human pairwise preference labels and judge-LLM pairwise rankings, and found judge performance varies by writing task and evaluation criteria, with paired comparisons yielding more consistent results than independent scoring. The authors also report that the strongest LLMs do not most reliably reproduce human evaluations, hypothesizing exaggerated sensitivity to subtle details that human annotators and weaker LLMs ignore.

This paper was accepted at the Can We Trust AI Evaluation? Workshop at NeurIPS 2026. We evaluate how good large language model LLM judges are for personalized writing tasks. This is important when a user’s preferred writing style is known e.g., “upbeat” and an LLM judge is used to evaluate whether generated text adheres to this preference. We examine how the writing task, the combination of generator and judge LLM, the evaluation objective, and general commonsense and reasoning ability impact LLM judge performance. To this end, we generate text with four LLMs for three long-form writing tasks, collect human labels evaluating which of two responses is better, collect judge-LLM, pairwise-ranking annotations, and then compare the human and judge LLM annotations. We find that judge quality correlates weakly with general LLM ability measured using MMLU Pearson’s ρ ≤ 0.2 and varies by both writing task and evaluation criteria i.e., general helpfulness versus style personalization . We also find that LLM evaluators are more consistent and performant using paired comparisons rather than scoring responses independently. Finally, we find that the “strongest” LLMs do not most reliably reproduce the human evaluations, which we hypothesize relates to exaggerated sensitivity to subtle details that human annotators and weaker LLMs ignore. - ‡ Equal contribution - † University of Southern California - § University of Copenhagen - Work done while at Apple