{"slug": "do-chatbot-llms-talk-too-much", "title": "Do Chatbot LLMs Talk Too Much?", "summary": "Researchers introduced YapBench, a benchmark with over 300 English prompts, to quantify verbosity in chatbot LLMs, measuring excess response length via the YapScore metric. Evaluating 76 assistant LLMs including ChatGPT, Claude, and Gemini, they found an order-of-magnitude spread in median excess length, with distinct failure modes such as vacuum-filling on ambiguous inputs and explanation overhead on one-line coding tasks. The benchmark and live leaderboard are released to track verbosity behavior over time.", "body_md": "# Computer Science > Machine Learning\n\n[Submitted on 2 Jan 2026]\n\n# Title:Do Chatbot LLMs Talk Too Much? The YapBench Benchmark\n\n[View PDF](/pdf/2601.00624)\n\n[HTML (experimental)](https://arxiv.org/html/2601.00624v1)\n\nAbstract:Large Language Models (LLMs) such as ChatGPT, Claude, and Gemini increasingly act as general-purpose copilots, yet they often respond with unnecessary length on simple requests, adding redundant explanations, hedging, or boilerplate that increases cognitive load and inflates token-based inference cost. Prior work suggests that preference-based post-training and LLM-judged evaluations can induce systematic length bias, where longer answers are rewarded even at comparable quality.\n\nWe introduce YapBench, a lightweight benchmark for quantifying user-visible over-generation on brevity-ideal prompts. Each item consists of a single-turn prompt, a curated minimal-sufficient baseline answer, and a category label. Our primary metric, YapScore, measures excess response length beyond the baseline in characters, enabling comparisons across models without relying on any specific tokenizer. We summarize model performance via the YapIndex, a uniformly weighted average of category-level median YapScores.\n\nYapBench contains over three hundred English prompts spanning three common brevity-ideal settings: (A) minimal or ambiguous inputs where the ideal behavior is a short clarification, (B) closed-form factual questions with short stable answers, and (C) one-line coding tasks where a single command or snippet suffices. Evaluating 76 assistant LLMs, we observe an order-of-magnitude spread in median excess length and distinct category-specific failure modes, including vacuum-filling on ambiguous inputs and explanation or formatting overhead on one-line technical requests. We release the benchmark and maintain a live leaderboard for tracking verbosity behavior over time.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/do-chatbot-llms-talk-too-much", "canonical_source": "https://arxiv.org/abs/2601.00624", "published_at": "2026-08-20 13:01:03+00:00", "updated_at": "2026-08-20 13:17:06.447923+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-research", "ai-tools"], "entities": ["YapBench", "ChatGPT", "Claude", "Gemini", "YapScore", "YapIndex"], "alternates": {"html": "https://wpnews.pro/news/do-chatbot-llms-talk-too-much", "markdown": "https://wpnews.pro/news/do-chatbot-llms-talk-too-much.md", "text": "https://wpnews.pro/news/do-chatbot-llms-talk-too-much.txt", "jsonld": "https://wpnews.pro/news/do-chatbot-llms-talk-too-much.jsonld"}}