Towards Proactive Detection of User-Side Implicit Conflicts in Human-LLM Dialogue

Researchers constructed UC-Bench, a human-annotated benchmark for detecting user-side implicit conflicts in Human-LLM dialogue, and found existing LLMs struggle with the task, especially when conflicts stem from implicit incompatibilities grounded in dialogue history. The team proposed SynUC, a constraint-guided synthesis method using the SPEAKING framework, and applied it to WildChat to build UC-Data, a 2,487-sample training set. On UC-Bench, Qwen3.5-4B trained on UC-Data outperformed larger general-purpose LLMs including Claude Opus 4.8 and the same backbone trained on data synthesized by existing methods.

arXiv:2609.19155v1 Announce Type: new Abstract: In Human-LLM dialogue, follow-up user utterances may implicitly conflict with earlier intents, leading the LLM to misinterpret user needs and generate inappropriate responses. A reliable dialogue system should proactively detect user-side conflicts before generating a response and seek clarification when necessary. However, prior work has largely focused on LLM-side conflicts, leaving user-side conflicts underexplored. To fill this gap, we construct UC-Bench, a human-annotated benchmark for evaluating user-side conflict detection. Preliminary experiments show that existing LLMs struggle with this task, especially when conflicts arise from implicit incompatibilities grounded in dialogue history. To improve lightweight LLMs with limited training data, we investigate data synthesis for user-side conflict detection. Existing synthesis methods do not explicitly model the implicit incompatibilities between historical and current user utterances, making it difficult to capture the evolution of conflicts and to generate reliably labeled implicit conflict samples. We propose SynUC, a constraint-guided synthesis method that represents user-side conflicts in a constraint space and uses the SPEAKING framework to guide traceable constraint transformations. Applying SynUC to WildChat, we construct UC-Data, a user-side conflict training set containing 2,487 samples. On UC-Bench, Qwen3.5-4B trained on UC-Data outperforms larger general-purpose LLMs such as Claude Opus 4.8, as well as the same backbone trained on data synthesized by existing methods.