Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond English
A new multilingual summary meta-evaluation dataset (BASSE) containing human judgments on 2,040 abstractive summaries reveals that proprietary judge LLMs correlate most strongly with human ratings, followed by criteria-sp…