RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics

Researchers submitted RATED (Reliability-Aware Tool Retrieval via Execution Dynamics) to arXiv on 9 Oct 2026, a retrieval framework that pairs a frozen dual-encoder with contrastive fine-tuning for semantic matching and a lightweight contextual bandit module that re-ranks tool candidates using real-time execution feedback. The authors report RATED achieves a 52.2% relative MRR improvement over the ToolLLM API retriever on real-world queries and cuts fault selection rate from 86.2% to 2.9% under severe API outage scenarios with a 30% failure rate, adding less than 1 ms of latency over encode-only inference. The work introduces EcoAgent, a benchmark of 5,172 real-world tools and over 210,000 query-tool pairs, with code and dataset released on GitHub.

Computer Science Artificial Intelligence Submitted on 9 Oct 2026 Title:RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics View PDF http://arxiv.org/pdf/2610.13889v1 HTML experimental https://arxiv.org/html/2610.13889v1 Abstract:As AI agents increasingly rely on external tools, accurate retrieval is essential for reliable behavior. Existing methods optimize semantic relevance but largely ignore tool availability during execution. To address this gap, we propose RATED Reliability-Aware Tool Retrieval via Execution Dynamics , a retrieval framework that combines robust semantic matching with dynamic reliability adaptation. RATED employs a frozen dual-encoder with contrastive fine-tuning for semantic matching, and a lightweight contextual bandit module that re-ranks candidates based on real-time execution feedback. This design enables rapid adaptation to runtime tool failures without expensive model retraining. We introduce EcoAgent, a benchmark of 5,172 real-world tools and over 210,000 query-tool pairs for offline retrieval and dynamic reliability evaluation. Experimental results demonstrate that RATED achieves a 52.2% relative MRR improvement over the ToolLLM API retriever on real-world queries, and reduces fault selection rate from 86.2% to 2.9% under severe API outage scenarios with a 30% failure rate, while incurring negligible latency overhead of less than 1 ms over encode-only inference. The code and dataset are available at this https URL https://github.com/Elliotes/EcoAgent . Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .