{"slug": "reliable-ai-assisted-decisions-via-selective-auditing", "title": "Reliable AI-Assisted Decisions via Selective Auditing", "summary": "A paper submitted to arXiv on 22 October 2026 proposes a family of algorithms that let an AI system's decisions be accepted automatically or routed to a human auditor while keeping the fraction of unaudited AI errors below a user-defined threshold. The authors formulate the task as an online constrained optimization problem and derive methods with provable error-control guarantees and no assumptions, including one they describe as optimally efficient, asymptotically attaining the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text annotation and data labeling showed valid error control in practice while substantially reducing the need for human auditing compared with baselines.", "body_md": "# Statistics > Machine Learning\n\n  [Submitted on 22 Oct 2026]\n\n# Title:Reliable AI-Assisted Decisions via Selective Auditing\n\n[View PDF](http://arxiv.org/pdf/2610.31662v1)\n\n[HTML (experimental)](https://arxiv.org/html/2610.31662v1)\n\nAbstract:We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text annotation, and data labeling show that our methods achieve valid error control in practice while substantially reducing the need for human auditing compared to baselines.\n    \n\n### Current browse context:\n\nstat.ML\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/reliable-ai-assisted-decisions-via-selective-auditing", "canonical_source": "http://arxiv.org/abs/2610.31662v1", "published_at": "2026-10-23 14:39:06+00:00", "updated_at": "2026-10-23 15:10:00.925631+00:00", "lang": "en", "topics": ["machine-learning", "ai-safety", "ai-research", "large-language-models"], "entities": ["arXiv", "stat.ML"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/reliable-ai-assisted-decisions-via-selective-auditing", "markdown": "https://wpnews.pro/news/reliable-ai-assisted-decisions-via-selective-auditing.md", "text": "https://wpnews.pro/news/reliable-ai-assisted-decisions-via-selective-auditing.txt", "jsonld": "https://wpnews.pro/news/reliable-ai-assisted-decisions-via-selective-auditing.jsonld"}}