From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifiers

A systematic evaluation of 12 instruction-tuned open-weight large language models across six benchmark causal graphs found that LLM-based causal judgments are recall-dominant, producing overly dense graphs with many false-positive edges, and that models misclassify 40.0% of indirect and 36.0% of reversed non-edges as direct edges, with 80.8% and 84.6% of these false positives receiving verbalized confidence of at least 80%. The study, posted on arXiv (2608.23660v1), concludes that LLMs are better viewed as sources of externally validated soft causal priors than as direct evidence of causal structure, and that cross-prompt and cross-model agreement offer more reliable confidence signals than logit-based or verbalized confidence.

arXiv:2608.23660v1 Announce Type: new Abstract: Large language models LLMs are increasingly used to provide prior causal knowledge for structural causal discovery, yet whether their direct-edge judgments and confidence can be trusted remains unclear. We systematically evaluate 12 instruction-tuned open-weight models across six benchmark causal graphs, five prompting strategies, and four confidence sources: verbalized, logit-based, cross-prompt agreement, and cross-model agreement. Under our language-only pairwise protocol, our evaluation yields three key findings. i LLM-based causal judgments are strongly recall-dominant: models predict overly dense graphs with many false-positive edges, while prompting mainly shifts the precision-recall trade-off rather than resolving overprediction. Gains from model scale diminish on the largest graphs and do not eliminate miscalibration. ii LLMs often capture causal relatedness without reliably identifying directness or orientation. Relative to published reference graphs, models misclassify 40.0% of indirect and 36.0% of reversed non-edges as direct edges, versus 28.2% of other non-edges. Moreover, 80.8% and 84.6% of these false positives receive verbalized confidence of at least 80%, revealing substantial overconfidence in structurally incorrect predictions. iii Conventional confidence estimates are unreliable, whereas agreement offers a more promising signal. Logit-based confidence frequently collapses near 1.0 regardless of correctness, while cross-prompt and cross-model agreement achieve better mean calibration and discrimination, though their advantages are not statistically significant after Holm correction. A benchmark-familiarity audit further identifies potential familiarity in five model-dataset pairs, all involving AsiaM. Overall, our results suggest LLMs are better viewed as sources of externally validated soft causal priors than as direct evidence of causal structure.