B-CARE: Blind Classwise Assessment of Relative Evidence for Stance Error Ranking

A new arXiv paper (2610.29350v1) introduces B-CARE, a prediction-blind framework that uses a fixed LLM verifier to assign ordinal evidence scores to candidate stance labels and rank potentially incorrect stance predictions for human review. Evaluated on four stance benchmarks and three heterogeneous base LLMs across 12 dataset-model settings and 5,206 unique test instances, B-CARE achieved a mean AUROC of 0.735 and a mean AUPRC of 0.524, capturing 41.06% of errors within the highest-risk 20% of predictions. The authors report B-CARE does not uniformly outperform strong confidence-based baselines but offers a reusable, complementary evidence signal that requires no separate error detector or risk-score calibration across models.

arXiv:2610.29350v1 Announce Type: new Abstract: Stance classifiers make errors that can be costly to identify under limited human-review budgets. Prioritizing potentially incorrect predictions remains challenging, particularly across heterogeneous base models. We introduce B-CARE Blind Classwise Assessment of Relative Evidence for Stance Error Ranking , a prediction-blind framework for identifying high-risk stance predictions. Given only a target and text, a fixed LLM verifier assigns ordinal evidence scores to all candidate stance labels without observing the base model's prediction or confidence. B-CARE constructs a comparative risk score by contrasting the strongest alternative-class evidence with the evidence supporting the predicted class. The resulting classwise evidence profile can be reused across multiple base models without training a separate error detector or calibrating risk scores. We evaluate B-CARE on four stance benchmarks and three heterogeneous base LLMs, covering 12 dataset-model settings and 5,206 unique test instances. Evaluation includes error-ranking discrimination AUROC and AUPRC , selective prediction E-AURC , and review effectiveness ErrorCapture at fixed review budgets . B-CARE achieves a mean AUROC of 0.735 and a mean AUPRC of 0.524, capturing 41.06% of errors within the highest-risk 20% of predictions. Although B-CARE does not uniformly outperform strong confidence-based baselines, it provides a reusable, complementary evidence signal for prioritizing potentially incorrect stance predictions across heterogeneous models.