{"slug": "not-all-information-is-worth-asking-for-decision-informed-information-seeking", "title": "Not All Information Is Worth Asking For: Decision-Informed Information Seeking with LLMs", "summary": "A training-free procedure called Decision-Informed Value-of-information Acquisition (DIVA) reduced the regret of the best LLM baseline by 75% in synthetic scenarios where every check's reliability is given, according to arXiv paper 2610.32181v1. Across four domains and 20 LLMs, the DIVA-U variant, which accounts for an LLM's uncertain estimates of how strongly each check shifts a bad-outcome probability, had lower regret than every LLM baseline on real-world data, mainly by stopping once evidence sufficed. DIVA works by having the LLM specify a small decision model through short numeric questions and valuing each candidate check by how much its imagined outcomes improve the decision net of its cost; code is at github.com/seunghan96/diva.", "body_md": "arXiv:2610.32181v1 Announce Type: cross \nAbstract: Large language models (LLMs) are increasingly used to support decisions under incomplete information, where deciding what to look up and when to stop matters as much as the final answer. However, LLMs fall into an information trap: They pay for checks, i.e., costly pieces of evidence, that cannot change their decision or are not worth their cost, and they keep asking when the evidence is already sufficient. To this end, we propose Decision-Informed Value-of-information Acquisition (DIVA), a simple yet effective procedure for LLM information seeking based on the decision value of evidence. DIVA 1) lets the LLM specify a small decision model, i.e., how likely a bad outcome is and how each check would shift this probability, through short numeric questions and 2) values each candidate check by how much its imagined outcomes improve the decision net of its cost, stopping when no check is worth its cost. Its variant, DIVA-U, further accounts for the LLM's uncertain estimates of how strongly each check shifts this probability. DIVA is training-free and applies to any LLM. Across four domains and 20 LLMs, DIVA-U has lower regret, i.e., value lost relative to the best possible decision, than every LLM baseline on real-world data, mainly by stopping once evidence suffices, and DIVA cuts the regret of the best LLM baseline by 75% in synthetic scenarios where every check's reliability is given. Code: github.com/seunghan96/diva.", "url": "https://wpnews.pro/news/not-all-information-is-worth-asking-for-decision-informed-information-seeking", "canonical_source": "https://www.machinebrief.com/news/not-all-information-is-worth-asking-for-decision-informed-in-ev1h", "published_at": "2026-10-26 04:00:00+00:00", "updated_at": "2026-10-26 06:41:35.225801+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-agents"], "entities": ["DIVA", "DIVA-U", "arXiv", "github.com/seunghan96/diva"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/not-all-information-is-worth-asking-for-decision-informed-information-seeking", "markdown": "https://wpnews.pro/news/not-all-information-is-worth-asking-for-decision-informed-information-seeking.md", "text": "https://wpnews.pro/news/not-all-information-is-worth-asking-for-decision-informed-information-seeking.txt", "jsonld": "https://wpnews.pro/news/not-all-information-is-worth-asking-for-decision-informed-information-seeking.jsonld"}}