Counterfactual Reasoning for Robust Visual Question Answering

A new training framework for Visual Question Answering (VQA) achieves 61.64% accuracy on the bias-sensitive VQA-CP v2 benchmark while maintaining 62.80% on the standard VQA v2 dataset, a generalization gap of 1.16%, according to an arXiv paper (2609.16567v1). The framework combines a three-stage curriculum for multi-objective optimization, an enhanced Batch-Contrastive loss for feature discrimination, and two regularizers — Answer-Contrastive (AC) loss and Gradient-Discrepancy (GD) loss — to counter the spurious correlations and language bias that degrade out-of-distribution generalization in modern VQA models.

arXiv:2609.16567v1 Announce Type: new Abstract: Modern Visual Question Answering VQA models often exploit spurious correlations in training data, leading to poor out-of-distribution OOD generalization due to language bias. Although counterfactual learning has shown promise, existing methods can be improved to better guide attention toward causal evidence and strengthen feature discrimination. To address this, we propose a novel training framework that enhances counterfactual contrastive learning for VQA. Our framework introduces three key contributions: 1 a three-stage curriculum for stable multi-objective optimization, 2 an enhanced Batch-Contrastive loss for more discriminative feature learning, and 3 two novel regularizers, Answer-Contrastive AC loss to refine the prediction space and Gradient-Discrepancy GD loss to enforce causal visual grounding. Our model achieves a competitive accuracy of 61.64% on the bias-sensitive VQA-CP v2 benchmark while maintaining 62.80% on the standard VQA v2 dataset, yielding a small generalization gap of 1.16%. This demonstrates a strong balance between OOD robustness and in-distribution performance.