A new arXiv paper (2609.10996v1) reports that verbalized confidence has become the more robust soft-scoring mechanism for LLM-as-a-Judge on post-2025 proprietary models, reversing the standard advice …
LLM-as-judge architectures are shifting from offline evaluation into agent runtime control flow, letting applications make judgment calls on AI-generated work as it happens, according to a technical a…
Andrew Ng outlined a roadmap for becoming an AI engineer, emphasizing a shift from model-centric to data and system-centric workflows. He identified three core pillars: data engineering for AI, the RA…
A new method, RubricForge, reduces the false-pass rate of LLM-as-a-judge agent evaluations by roughly half—down to 11.5% from 17.3% compared to standard G-Eval—by automatically evolving a frozen, huma…
A field guide to LLM-as-a-judge reveals that five novel proposals for improving the technique were all refuted by prior art published within the last twelve weeks, according to a research synthesis co…
G-Eval, a prompting framework from Microsoft researchers that uses a large language model as a reference-free evaluator of generated text, scores open-ended outputs in a way that aligns with human jud…
Researchers propose BINEVAL, a framework that decomposes LLM evaluation into atomic binary questions for interpretable, multi-dimensional scoring. The method matches or outperforms strong baselines on…
A developer evaluated six LLM-as-judge tools—DeepEval, Confident AI, Evidently, Braintrust, Promptfoo, and Future AGI—and found that none of them prioritize validating judge outputs against human labe…