{"slug": "where-do-llms-fall-short-in-cbt-guided-affective-reasoning", "title": "Where do LLMs Fall Short in CBT-Guided Affective Reasoning?", "summary": "Researchers found that large language models (LLMs) score up to 96% on CBT licensing exams but fail to apply CBT effectively in dialogue, defaulting to validation and reflection. A new metric, Protocol Leverage Force (F), shows that even guided prompting only shifts behavior by about 1%, revealing a gap between theoretical knowledge and practical application in affective reasoning.", "body_md": "arXiv:2607.02885v1 Announce Type: new\nAbstract: Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognitive and behavioral factors. However, out-of-the-box LLMs respond fluently and empathetically, yet collapse into validation & reflection, regardless of what the user actually needs. They know theoretical CBT (scoring up to 96% accuracy on licensing exam questions) but fail to apply it effectively. We explore this gap with a knowledge-guided framework that treats CBT dialogue as controlled affective reasoning: user narratives are decomposed into Beck's Cognitive Conceptualization structure, grounded in clinical SNOMED CT concepts validated via Natural Language Inference, and a Multiple Chain-of-Thought (MCoT) strategy selection between Validation & Reflection, Socratic Questioning, or Alternative Perspectives. To measure whether such guidance actually changes behavior, we introduce the Protocol Leverage Force (F), a behavior-level metric that captures how far an intervention shifts a model away from its default response. Across three open-weight LLMs and 14 RealCBT-derived case studies, evaluated with human experts, valence-arousal trajectories, and linguistic entrainment, F shows that simply introducing protocol definitions via single chain-of-thought prompting fails to change LLM behavior, while MCoT on these definitions guides strategy selection better. Still, the effect stays within 1% (approx. 1.2-1.3%), and all models remain biased toward Validation & Reflection. These results show CBT knowledge alone does not ensure effective application, giving the affective-computing community instrumentation to measure where LLMs fall short.", "url": "https://wpnews.pro/news/where-do-llms-fall-short-in-cbt-guided-affective-reasoning", "canonical_source": "https://arxiv.org/abs/2607.02885", "published_at": "2026-07-07 04:00:00+00:00", "updated_at": "2026-07-07 04:01:53.781562+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "ai-research"], "entities": ["Cognitive Behavioral Therapy", "SNOMED CT", "RealCBT", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/where-do-llms-fall-short-in-cbt-guided-affective-reasoning", "markdown": "https://wpnews.pro/news/where-do-llms-fall-short-in-cbt-guided-affective-reasoning.md", "text": "https://wpnews.pro/news/where-do-llms-fall-short-in-cbt-guided-affective-reasoning.txt", "jsonld": "https://wpnews.pro/news/where-do-llms-fall-short-in-cbt-guided-affective-reasoning.jsonld"}}