{"slug": "agentabstain-do-llm-agents-know-when-not-to-act", "title": "AgentAbstain: Do LLM Agents Know When Not to Act?", "summary": "Researchers introduced AgentAbstain, the first systematic evaluation framework for measuring whether LLM agents know when to abstain from acting, revealing that the best agent (Gemini 3.1 Pro) achieves only 59.5% paired accuracy across 263 paired tasks in 42 sandbox environments. The study found that abstention capability is largely independent of general task-solving ability, and identified failure modes such as post-hoc abstention where agents execute irreversible actions before recognizing abstention triggers.", "body_md": "# Computer Science > Artificial Intelligence\n\n[Submitted on 11 Jul 2026]\n\n# Title:AgentAbstain: Do LLM Agents Know When Not to Act?\n\n[View PDF](/pdf/2607.10059)\n\n[HTML (experimental)](https://arxiv.org/html/2607.10059v1)\n\nAbstract:Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain. This gap poses real risks: under ambiguity, conflicting constraints, or tool failures, agents may execute unintended and irreversible actions. To close this gap, we present the first systematic evaluation framework for agentic abstention: the calibrated ability of tool-using LLM agents to recognize when not to act. At its core, AgentAbstain is a paired-task benchmark built on an agent-native taxonomy of 8 abstention scenarios across pre-execution reasoning and runtime discovery. It contains 263 paired tasks across 42 executable sandbox environments, where each pair consists of a should-act task and a should-abstain variant produced through a controlled perturbation to the instruction, tool, or environment state. To scale this paired design and resist data contamination, we propose AbstainGen, a fully automated pipeline that synthesizes sandbox environments and generates paired tasks end-to-end, validated by deterministic replay and semantic LLM judges; fresh task instances can be regenerated on demand, and three independent annotators rate 94-98% of sampled tasks as well-designed. Across 17 frontier LLMs in 4 agent harnesses, the best agent (Gemini 3.1 Pro) achieves only 59.5% paired accuracy (correct on both the act and abstain sides of each paired task). More importantly, abstention capability is largely independent of general task-solving capability, indicating that scaling task-solving alone will not close this gap. We further identify failure modes such as post-hoc abstention, in which agents execute irreversible actions before recognizing abstention triggers. Our code and dataset are open-sourced at[this http URL].\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/agentabstain-do-llm-agents-know-when-not-to-act", "canonical_source": "https://arxiv.org/abs/2607.10059", "published_at": "2026-07-20 12:45:03+00:00", "updated_at": "2026-07-20 12:57:41.557289+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-safety", "ai-agents", "ai-research"], "entities": ["AgentAbstain", "Gemini 3.1 Pro", "AbstainGen"], "alternates": {"html": "https://wpnews.pro/news/agentabstain-do-llm-agents-know-when-not-to-act", "markdown": "https://wpnews.pro/news/agentabstain-do-llm-agents-know-when-not-to-act.md", "text": "https://wpnews.pro/news/agentabstain-do-llm-agents-know-when-not-to-act.txt", "jsonld": "https://wpnews.pro/news/agentabstain-do-llm-agents-know-when-not-to-act.jsonld"}}