{"slug": "rated-reliability-aware-ai-agent-tool-retrieval-via-execution-dynamics", "title": "RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics", "summary": "Researchers submitted RATED (Reliability-Aware Tool Retrieval via Execution Dynamics) to arXiv on 9 Oct 2026, a retrieval framework that pairs a frozen dual-encoder with contrastive fine-tuning for semantic matching and a lightweight contextual bandit module that re-ranks tool candidates using real-time execution feedback. The authors report RATED achieves a 52.2% relative MRR improvement over the ToolLLM API retriever on real-world queries and cuts fault selection rate from 86.2% to 2.9% under severe API outage scenarios with a 30% failure rate, adding less than 1 ms of latency over encode-only inference. The work introduces EcoAgent, a benchmark of 5,172 real-world tools and over 210,000 query-tool pairs, with code and dataset released on GitHub.", "body_md": "# Computer Science > Artificial Intelligence\n\n  [Submitted on 9 Oct 2026]\n\n# Title:RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics\n\n[View PDF](http://arxiv.org/pdf/2610.13889v1)\n\n[HTML (experimental)](https://arxiv.org/html/2610.13889v1)\n\nAbstract:As AI agents increasingly rely on external tools, accurate retrieval is essential for reliable behavior. Existing methods optimize semantic relevance but largely ignore tool availability during execution. To address this gap, we propose RATED (Reliability-Aware Tool Retrieval via Execution Dynamics), a retrieval framework that combines robust semantic matching with dynamic reliability adaptation. RATED employs a frozen dual-encoder with contrastive fine-tuning for semantic matching, and a lightweight contextual bandit module that re-ranks candidates based on real-time execution feedback. This design enables rapid adaptation to runtime tool failures without expensive model retraining. We introduce EcoAgent, a benchmark of 5,172 real-world tools and over 210,000 query-tool pairs for offline retrieval and dynamic reliability evaluation. Experimental results demonstrate that RATED achieves a 52.2% relative MRR improvement over the ToolLLM API retriever on real-world queries, and reduces fault selection rate from 86.2% to 2.9% under severe API outage scenarios with a 30% failure rate, while incurring negligible latency overhead of less than 1 ms over encode-only inference. The code and dataset are available at [this https URL](https://github.com/Elliotes/EcoAgent).\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/rated-reliability-aware-ai-agent-tool-retrieval-via-execution-dynamics", "canonical_source": "http://arxiv.org/abs/2610.13889v1", "published_at": "2026-10-12 14:21:01+00:00", "updated_at": "2026-10-12 14:48:26.629213+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "artificial-intelligence", "ai-tools", "mlops"], "entities": ["RATED", "EcoAgent", "ToolLLM", "arXiv", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/rated-reliability-aware-ai-agent-tool-retrieval-via-execution-dynamics", "markdown": "https://wpnews.pro/news/rated-reliability-aware-ai-agent-tool-retrieval-via-execution-dynamics.md", "text": "https://wpnews.pro/news/rated-reliability-aware-ai-agent-tool-retrieval-via-execution-dynamics.txt", "jsonld": "https://wpnews.pro/news/rated-reliability-aware-ai-agent-tool-retrieval-via-execution-dynamics.jsonld"}}