{"slug": "confidence-estimation-for-financial-vision-language-models-in-chart-and-document", "title": "Confidence Estimation for Financial Vision-Language Models in Chart and Document Understanding", "summary": "A new arXiv study (2608.06532v1) evaluating seven confidence estimators across five open-weight LVLMs on three financial visual question-answering benchmarks finds that only trained internal probes produce thresholdable calibration, while inference-only baselines are badly overconfident. The best estimator varies by model and task, with none leading more than eight of twenty (model, condition) cells, and a bilingual contrast shows apparent language robustness is a composition artifact. Under a strict 5% error budget, deferral clears a real share of the easiest condition but almost none of the hardest, with only the grounding-aware probe lowering confidence on answers given without using the figure.", "body_md": "arXiv:2608.06532v1 Announce Type: new\nAbstract: LVLMs are increasingly used to read financial charts, tables, and documents, where a single misread figure can move a decision and the most authoritative-looking answer is sometimes one the model produced without reading the exhibit. The operational question is therefore trust, not accuracy: which answers can be acted on, and which escalated to a reviewer. We evaluate seven confidence estimators, three inference-only and four trained internal probes, across five open-weight LVLMs and four conditions from three financial visual question-answering benchmarks, one bilingual; every probe is trained only on natural images and applied to finance without adaptation, so the results measure out-of-distribution transfer. Three findings hold. First, the scarce property is calibration, not ranking: the inference baselines rank correct above incorrect answers competitively but are badly overconfident, calibration error far above what a threshold can tolerate, and only the trained probes produce a thresholdable score. Second, reliability is structured rather than global, along two axes a practitioner can read directly: the best estimator shifts with both model and task, none leading more than eight of twenty (model, condition) cells, and a controlled bilingual contrast exposes an apparent language robustness as a composition artifact that dissolves once models are read one at a time. Third, cast as deferral under an error budget, how much can be safely automated is set first by the model's competence and only narrowed by its confidence, so deferral clears a real share of the easiest condition and almost none of the hardest, near zero at a strict 5% budget. Two trained probes carry the calibration a deferral policy needs, and among them only the grounding-aware one lowers its confidence on answers a model gives without using the figure, separating detected non-grounding from a fluent guess.", "url": "https://wpnews.pro/news/confidence-estimation-for-financial-vision-language-models-in-chart-and-document", "canonical_source": "https://arxiv.org/abs/2608.06532", "published_at": "2026-08-10 04:00:00+00:00", "updated_at": "2026-08-10 04:11:21.233084+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "natural-language-processing", "ai-research"], "entities": ["arXiv"], "alternates": {"html": "https://wpnews.pro/news/confidence-estimation-for-financial-vision-language-models-in-chart-and-document", "markdown": "https://wpnews.pro/news/confidence-estimation-for-financial-vision-language-models-in-chart-and-document.md", "text": "https://wpnews.pro/news/confidence-estimation-for-financial-vision-language-models-in-chart-and-document.txt", "jsonld": "https://wpnews.pro/news/confidence-estimation-for-financial-vision-language-models-in-chart-and-document.jsonld"}}