{"slug": "dialogue-swe-bench-a-benchmark-for-dialogue-driven-coding-agents", "title": "Dialogue SWE-Bench: A Benchmark for Dialogue-Driven Coding Agents", "summary": "Researchers introduced Dialogue SWE-Bench, a benchmark for evaluating AI coding agents through dialogue with users, revealing that coding proficiency does not guarantee strong dialogue skills. The study proposes a schema-guided agent that improves dialogue capabilities by 3-14% over baselines, highlighting dialogue as a distinct performance dimension.", "body_md": "arXiv:2606.13995v1 Announce Type: new\nAbstract: AI coding agents have rapidly transformed software engineering, powering widely used interactive coding assistants. Despite their interactive real-world use, existing benchmarks evaluate them as fully-autonomous systems. In this work, we introduce Dialogue SWE-Bench, an automatic benchmark dataset for evaluating the ability of coding agents to resolve real-world software engineering problems through dialogue with a user. We design a novel, persona-grounded user simulator to support our task evaluation, and augment our task evaluation with automatic evaluations of dialogue quality. We also propose a new schema-guided agent, aimed at improving the dialogue capabilities of off-the-shelf coding agents, which improves over strong baselines by 3-14%. Our results indicate that better coding models do not always correspond to better dialogue models, suggesting that dialogue capability is a distinct and currently understudied dimension of coding agent performance.", "url": "https://wpnews.pro/news/dialogue-swe-bench-a-benchmark-for-dialogue-driven-coding-agents", "canonical_source": "https://arxiv.org/abs/2606.13995", "published_at": "2026-06-15 04:00:00+00:00", "updated_at": "2026-06-15 04:17:19.089759+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "natural-language-processing", "ai-research", "developer-tools"], "entities": ["Dialogue SWE-Bench", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/dialogue-swe-bench-a-benchmark-for-dialogue-driven-coding-agents", "markdown": "https://wpnews.pro/news/dialogue-swe-bench-a-benchmark-for-dialogue-driven-coding-agents.md", "text": "https://wpnews.pro/news/dialogue-swe-bench-a-benchmark-for-dialogue-driven-coding-agents.txt", "jsonld": "https://wpnews.pro/news/dialogue-swe-bench-a-benchmark-for-dialogue-driven-coding-agents.jsonld"}}