A new methodology for tuning LLM judges focuses on reducing variance rather than just accuracy, as evaluators can flip decisions up to 15 points across runs. The approach, outlined by an unnamed autho…
ExploitHunter.app, an open-source, local-first AI security platform for authorized security research, has been launched by developer justsml. The platform features a deep evaluation suite, supports mu…
Mastra AI argues that model routers require rigorous testing beyond simple model selection, introducing scorers, datasets, and experiments to validate routing decisions across quality, cost, speed, an…
LLM benchmarks like MMLU and HumanEval are irrelevant for most businesses building AI products, as they measure generic performance rather than specific system tasks. Teams should instead build custom…