{"slug": "stable-miscalibration-in-large-language-models-a-practical-view-of-high-errors", "title": "Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors", "summary": "A new arXiv paper (2608.13591v1) finds that high-confidence errors in large language models can be stable under small perturbations, indicating stable miscalibration rather than fragile inference. The study combines an output-level audit score and an internal sensitivity probe, showing that self-critical prompting reduces hidden-state sensitivity across layers in three open-weight models, but audit-defined overconfident errors are not more locally sensitive than correct answers.", "body_md": "arXiv:2608.13591v1 Announce Type: new\nAbstract: High-confidence errors in large language models are often treated as evidence of fragile internal inference. We study a different possibility: stable miscalibration, where a confident wrong answer remains locally stable under small perturbations. We combine two diagnostics: a label-aware output-level audit score that ranks domains by confidence variation and overconfident mistakes under a forced-answer baseline, and an internal sensitivity probe that measures hidden-state movement. On a multi-domain binary factual audit set, this audit score tracks where abstention-aware self-critique reduces decision loss, although direct labeled baselines rank the same gain more strongly. Internally, self-critical prompting consistently reduces hidden-state sensitivity across layers in three open-weight models. This supports prompt-induced local stabilization rather than a purely output-level abstention pattern, but it does not imply calibration: audit-defined overconfident errors are not clearly more locally sensitive than confidently correct answers, so some high-confidence errors may be stable and miscalibrated rather than simply fragile.", "url": "https://wpnews.pro/news/stable-miscalibration-in-large-language-models-a-practical-view-of-high-errors", "canonical_source": "https://arxiv.org/abs/2608.13591", "published_at": "2026-08-17 04:00:00+00:00", "updated_at": "2026-08-17 04:12:14.561621+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-research", "ai-safety"], "entities": ["arXiv"], "alternates": {"html": "https://wpnews.pro/news/stable-miscalibration-in-large-language-models-a-practical-view-of-high-errors", "markdown": "https://wpnews.pro/news/stable-miscalibration-in-large-language-models-a-practical-view-of-high-errors.md", "text": "https://wpnews.pro/news/stable-miscalibration-in-large-language-models-a-practical-view-of-high-errors.txt", "jsonld": "https://wpnews.pro/news/stable-miscalibration-in-large-language-models-a-practical-view-of-high-errors.jsonld"}}