{"slug": "a-canary-in-the-neural-mine-catching-an-image-classifier-s-silent-confident", "title": "A Canary in the Neural Mine: Catching an Image Classifier's Silent, Confident Failures", "summary": "A new arXiv paper (2610.33063v1) introduces CANARY, a training-free failure detector that matches or improves Maximum Softmax Probability (MSP) on 382 of 385 shift cells measured by e-AURC and 379 of 385 on AUROC across five backbones, with worst-case regression within seed noise at -0.0002 AUROC. CANARY computes a per-class kNN distance on the PCA residual of the penultimate feature, percentile-calibrates it on validation data, and fuses it with MSP through a single validation-tuned weight, with every parameter selected on in-distribution data alone. The authors report the largest gains on natural distribution shifts (ImageNet-A/R, ObjectNet) and describe a headroom law in which gains grow as baseline MSP quality falls, letting a single clean-validation number forecast the improvement.", "body_md": "arXiv:2610.33063v1 Announce Type: new \nAbstract: Deployed vision models auto-accept confident predictions, so high-confidence errors propagate silently. Maximum Softmax Probability (MSP), the standard failure predictor, has a structural blind spot: as a scalar read from the classifier head, it is first-order blind to feature variation orthogonal to its confidence gradient. Through a factorial analysis of feature-space detectors, we identify a conditional-redundancy principle explaining why they fail to complement MSP: the class-separability geometry they measure is already encoded in MSP and collapses under fusion. The signal that survives is class-conditional typicality on the off-manifold residual---and because this residual's distribution varies by backbone, from near-Gaussian to heavy-tailed, it must be estimated non-parametrically. We instantiate these findings as CANARY, a training-free detector that computes a per-class kNN distance on the PCA residual of the penultimate feature, percentile-calibrates it on validation data, and fuses it with MSP through a single validation-tuned weight. CANARY's benefit follows a headroom law: gains grow as baseline MSP quality falls, so a single clean-validation number forecasts the improvement. With every parameter selected on in-distribution data alone, CANARY matches or improves 382 of 385 shift cells (e-AURC; 379 on AUROC) across five backbones, with worst-case regression within seed noise ($-0.0002$ AUROC) and the largest gains on natural distribution shifts (ImageNet-A/R, ObjectNet)---where deployed models are most exposed.", "url": "https://wpnews.pro/news/a-canary-in-the-neural-mine-catching-an-image-classifier-s-silent-confident", "canonical_source": "https://arxiv.org/abs/2610.33063", "published_at": "2026-10-27 04:00:00+00:00", "updated_at": "2026-10-27 04:11:33.059406+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "ai-research", "ai-safety", "neural-networks"], "entities": ["arXiv", "CANARY", "Maximum Softmax Probability", "ImageNet-A", "ImageNet-R", "ObjectNet"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-canary-in-the-neural-mine-catching-an-image-classifier-s-silent-confident", "markdown": "https://wpnews.pro/news/a-canary-in-the-neural-mine-catching-an-image-classifier-s-silent-confident.md", "text": "https://wpnews.pro/news/a-canary-in-the-neural-mine-catching-an-image-classifier-s-silent-confident.txt", "jsonld": "https://wpnews.pro/news/a-canary-in-the-neural-mine-catching-an-image-classifier-s-silent-confident.jsonld"}}