When to Ask, When to Answer: Strategic Clarification for Ambiguous Reference Resolution in Dialogue

A new arXiv paper (2611.05517v1) introduces BACE (Belief-Aware Critic with Epistemic tracking), a lightweight action-chooser built on a frozen language model backbone that more consistently selects the optimal number of clarifying questions than GRPO and ArCHer. The authors built a controlled test environment from the SIMMC 2.1 dataset that records how many questions each ambiguous reference needs to resolve, and used it to compare baseline LLM behavior and common post-training methods on asking clarifying questions versus answering prematurely. The work targets LLMs' documented bias toward definite single-turn answers over clarification exchanges in reference games.

arXiv:2611.05517v1 Announce Type: new Abstract: For humans, referential disambiguation is a collaborative process involving clarification questions asked when they will meaningfully change the outcome. However, large language models LLMs have been shown to be biased toward premature answering, preferring definite single-turn responses over more clarification exchanges. In this paper, we study the problem of "strategic clarification" -- asking clarification questions in a reference game when their answers will narrow the set of candidate referents. We construct a controlled test environment from the SIMMC 2.1 dataset that records how many questions each ambiguous reference needs to be successfully resolved, and examine how baseline LLM behavior and common post-training methods ask clarifying questions vs. attempt to answer the referent. We also propose BACE Belief-Aware Critic with Epistemic tracking , a lightweight action chooser method on top of a frozen LM backbone, and show that BACE more consistently chooses the optimal number of clarifying questions than methods such as GRPO and ArCHer.