RADC: Risk-Aware Dual Caching for Vision-Language Test-Time Adaptation

Researchers proposed RADC, a risk-aware dual caching method for test-time adaptation of vision-language models, detailed in arXiv paper 2610.06932v1. RADC adds a Semantic Foreground Cache that aggregates category-consistent spatial evidence from CLIP representations to build foreground prototypes alongside the global cache, and uses Gaussian Risk Admission to model multi-view representations as diagonal Gaussian distributions, jointly weighing class separation and feature uncertainty when admitting cache candidates. The authors report consistent state-of-the-art performance on cross-domain and out-of-distribution benchmarks.

arXiv:2610.06932v1 Announce Type: new Abstract: Cache-based test-time adaptation TTA for vision-language models is often hindered by background bias in global representations and unreliable entropy-based cache admission under representation variations. To address these limitations, we propose RADC, which enhances prototype learning through reliable dual caching. RADC introduces a Semantic Foreground Cache that aggregates category-consistent spatial evidence from CLIP representations, yielding foreground prototypes that complement the global cache while mitigating background interference. To reliably manage both caches, Gaussian Risk Admission models multi-view representations as diagonal Gaussian distributions and jointly considers class separation and feature uncertainty to prioritize reliable cache candidates. RADC integrates zero-shot logits with complementary global- and foreground-cache predictions for robust inference. Extensive experiments on cross-domain and out-of-distribution benchmarks demonstrate consistent state-of-the-art performance.