When Aggregate Alignment Misleads: Auditing Policy Repair Without Per-State Expert Actions

A new study on agentic AI policy repair in a hotel-pricing simulator shows that LLM-based editors can nearly match benchmark performance using only region-level diagnostic feedback, achieving RevPAR 108.47 vs. benchmark 108.75. However, the research warns that aggregate alignment metrics can mislead: a tree editor with better behavioral distances produced lower revenue (98.91), indicating that policy repair should be evaluated by closed-loop outcomes rather than single distance measures.

arXiv:2607.03386v1 Announce Type: new Abstract: Agentic AI systems are increasingly used to edit, refine, and repair decision policies, but evaluating these edits is difficult when per-state expert action labels are unavailable. We study this problem in a hotel-pricing simulator where an agentic policy editor receives only region-level diagnostic feedback: summaries of how its price distribution differs from a benchmark policy across time, inventory, and market regions. The editor cannot observe benchmark actions, benchmark source code, reward numbers, or held-out outcomes, and may only propose constrained edits to a target-action table. On 5,000 held-out episodes, a multi-restart LLM editor reaches RevPAR 108.47 95% CI 107.61 - 109.34 , close to the benchmark policy's 108.75 107.81 - 109.68 , with paired gap LLM minus benchmark -0.276 and 95% CI -0.692, 0.146 . A cheap diagnostic projection already recovers much of the revenue 107.90 , so the LLM editor's distinctive gain is not raw revenue lift alone: it also reduces episode composition distance from 1.153 to 0.609. This is the strongest non-benchmark repair result. This profile is not explained by restart search alone: non-semantic proposers with up to 2,500 evaluations fall 8.77 - 14.57 RevPAR points short. Nor is it explained by plausible prompt format: a shuffled-diagnostic control breaks region-error correspondence and falls to RevPAR 94.30. The match is genuine but partial. A tree editor achieves stronger pooled alignment, 0.214 versus 0.266, and stronger reference-state D1, 0.328 versus 1.197, yet revenue falls to 98.91. These results show that agentic policy repair should be evaluated by whether diagnostic feedback becomes reliable closed-loop outcome, not by a single behavioral distance.