Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations

A new arXiv paper (2609.09448v1) introduces two methods, Latent Trajectory Dynamics (LTD) and the Action Representation Probe (ARP), that use a model's internal representations to predict task success in multi-turn agentic workflows. Across three interactive benchmarks (Bash, SQL, Python) and three model families (Qwen14B, Qwen7B, DeepSeek6.7B), the methods consistently outperformed surface-level generation and sequence-based calibration baselines, offering a zero-overhead reliability monitor that requires neither prompt alterations nor multi-sample rollouts.

arXiv:2609.09448v1 Announce Type: new Abstract: As agentic systems getting adopted rapidly in safety critical applications, it is vital to measure the confidence associated with the agentic actions. In comparison to the traditional machine learning systems, agentic workflows have complex failure modes with planning, tool invocation and dynamic environment interactions. In this paper, we investigate whether model's internal representations provide stronger signals of eventual task success in multi-turn agentic setups. We introduce two complementary methods: Latent Trajectory Dynamics LTD , which summarizes changes in residual-stream representations across an an interaction trajectory, and the Action Representation Probe ARP , which predicts success from representations formed at action decisions. Across three interactive benchmarks Bash, SQL, Python and three model families Qwen14B, Qwen7B, DeepSeek6.7B , our methods consistently outperform surface level generation and sequence-based calibration baselines providing a zero-overhead reliability monitor that requires neither prompt alterations nor multi-sample rollouts.