LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets

Researchers introduced LiveMACEBench, a process-aware benchmark for evaluating LLM agent capabilities in evolving markets, arguing that judging agents by outcomes alone obscures the capabilities that produce them. The benchmark targets environments where outcomes reflect a closed-loop interaction between agent behavior and changing external conditions.

Evaluating agents by outcomes alone can obscure the capabilities that produce them. This problem is especially pronounced in evolving environments, where outcomes reflect a closed-loop interaction between agent behavior and changing external conditions. We introduce LiveMACEBench, a process-aware be