{"slug": "necessary-or-sufficient-evaluating-llm-explanations-with-behavioural-evidence", "title": "Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence", "summary": "A new arXiv preprint (2609.05385) evaluating eight models from the Claude, GPT, and Gemini families finds that LLM explanations of decision factors only weakly align with actual decision behavior, with mean Spearman correlations between cited factor rankings and necessity and sufficiency scores ranging from 0.349 to 0.580 across advisor recommendation and prompt monitoring tasks. The study also found that uncited factors outperformed the lowest-cited factor in up to 58.1% of cases, indicating that the top three cited factors do not reliably identify the most influential factors.", "body_md": "# Computer Science > Artificial Intelligence\n\n  [Submitted on 4 Sep 2026]\n\n# Title:Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence\n\n[View PDF](/pdf/2609.05385v1)\n\n[HTML (experimental)](https://arxiv.org/html/2609.05385v1)\n\nAbstract:LLM decision components that can operate within agent workflows often produce action-relevant recommendations or judgements together with explanations. Operators may use the named factors to monitor a system, diagnose errors, or decide when to escalate an output. Such use assumes that the explanations agree with the component's observable decision behaviour. We test two interpretations of the named factors: necessity, meaning that changing a factor would change the output, and sufficiency, meaning that retaining it while removing other changeable information would preserve the output. We evaluate these interpretations in two synthetic use cases: recommending advisors to clients and judging prompts for harmfulness or risk. Models return an output and the top three factors that most influenced it. Controlled black-box interventions estimate a necessity score for each factor by measuring how often changing it changes the output, and a sufficiency score by measuring how often retaining it preserves the output. Across eight models from the Claude, GPT, and Gemini families, the mean Spearman correlations between the cited ranking and the necessity and sufficiency scores are 0.349 and 0.354 for advisor recommendation, and 0.431 and 0.580 for prompt monitoring. Furthermore, an uncited factor scores above the lowest-scoring cited factor in 57.6% of advisor responses under necessity and 58.1% under sufficiency; the corresponding prompt-monitoring rates are 25.8% and 8.9%. The cited top three contain useful information but do not reliably identify the three factors with the strongest measured influence under necessity or sufficiency. The framework provides a black-box reliability check for explanations used in agent oversight while remaining scoped to individual LLM decisions.\n    \n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/necessary-or-sufficient-evaluating-llm-explanations-with-behavioural-evidence", "canonical_source": "http://arxiv.org/abs/2609.05385v1", "published_at": "2026-09-08 14:41:23+00:00", "updated_at": "2026-09-08 14:58:14.158001+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-safety"], "entities": ["arXiv", "Claude", "GPT", "Gemini"], "alternates": {"html": "https://wpnews.pro/news/necessary-or-sufficient-evaluating-llm-explanations-with-behavioural-evidence", "markdown": "https://wpnews.pro/news/necessary-or-sufficient-evaluating-llm-explanations-with-behavioural-evidence.md", "text": "https://wpnews.pro/news/necessary-or-sufficient-evaluating-llm-explanations-with-behavioural-evidence.txt", "jsonld": "https://wpnews.pro/news/necessary-or-sufficient-evaluating-llm-explanations-with-behavioural-evidence.jsonld"}}