{"slug": "bridging-human-and-model-attention-explainability-analysis-of-cnn-mamba-and-vit", "title": "Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation", "summary": "A systematic explainability study of Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Vision Mamba (ViM) found that CNNs produce more spatially diffuse saliency maps that correlate well with human gaze in general contexts, while ViTs show fine-grained, patch-level attention on discriminative microfeatures, according to arXiv paper 2610.12703v1. The authors quantified model attention against eye-tracking ground truth using Jensen-Shannon similarity, Chi-square similarity, and Pearson correlation across fine-grained classification, medical imaging, and emotion recognition, and report that Mamba achieves strong explainability in medical tasks but divergent intensity distributions on natural images, attaining near-unit support overlap alongside vanishing linear correlation. The study also analyzes how backbone freezing and ensemble strategies affect interpretability.", "body_md": "arXiv:2610.12703v1 Announce Type: new \nAbstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation across Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Vision Mamba (ViM). Unlike predominantly qualitative studies, we quantify the alignment between model attention and human visual perception using eye-tracking ground truth. We formalize this comparison via a standardized normalization pipeline mapping post-hoc attribution maps and gaze heatmaps onto spatial probability distributions. Alignment is evaluated across fine-grained classification, medical imaging, and emotion recognition using architecture-specific XAI methods. We measure fidelity using Jensen-Shannon similarity, Chi-square similarity, and Pearson correlation, three complementary estimators capturing support overlap, mass discrepancy, and structural co-variation. Our results reveal distinct attention dynamics: CNNs tend to exhibit a more global focus, with spatially more diffuse saliency maps that correlate well with human perception in general contexts, while ViTs demonstrate fine-grained, patch-level attention, often focused on discriminative microfeatures. The Mamba architecture shows biased behavior: it achieves strong explainability in medical tasks but exhibits divergent intensity distributions in natural images; our analytical characterization of the metrics makes this dissociation explicit, since near-unit support overlap is attained simultaneously with a vanishing linear correlation. We further analyze the effect of backbone freezing and ensemble strategies on model interpretability. The results highlight how architectural and training choices influence the alignment between model saliency and human visual attention.", "url": "https://wpnews.pro/news/bridging-human-and-model-attention-explainability-analysis-of-cnn-mamba-and-vit", "canonical_source": "https://arxiv.org/abs/2610.12703", "published_at": "2026-10-12 04:00:00+00:00", "updated_at": "2026-10-12 04:42:41.045848+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "ai-research", "neural-networks"], "entities": ["arXiv", "Convolutional Neural Networks", "Vision Transformers", "Vision Mamba", "Jensen-Shannon similarity", "Chi-square similarity", "Pearson correlation"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/bridging-human-and-model-attention-explainability-analysis-of-cnn-mamba-and-vit", "markdown": "https://wpnews.pro/news/bridging-human-and-model-attention-explainability-analysis-of-cnn-mamba-and-vit.md", "text": "https://wpnews.pro/news/bridging-human-and-model-attention-explainability-analysis-of-cnn-mamba-and-vit.txt", "jsonld": "https://wpnews.pro/news/bridging-human-and-model-attention-explainability-analysis-of-cnn-mamba-and-vit.jsonld"}}