{"slug": "visual-grounding-first-multimodal-in-context-learning-follows", "title": "Visual Grounding First, Multimodal In-context Learning Follows", "summary": "A new arXiv paper (2610.30592v1) proposes ProCoRe, a method that improves multimodal in-context learning in MLLMs by targeting visual grounding through contrastive reinforcement learning on multimodal contrastive data. ProCoRe generates captions for contrastive image pairs, decomposes them into propositions, and optimizes contrastive rewards that align captions with matching images and penalize mismatched ones; without ever seeing ICL-formatted training examples, it raises average multimodal ICL classification accuracy by 5.6% and captioning ROUGE-L by 13.8% over the off-the-shelf Qwen-VL-3-8B model, outperforming ICL-data-dependent state-of-the-art methods across benchmarks. The authors also introduce PICL, a personalized multimodal ICL captioning benchmark testing whether MLLMs genuinely understand visual demonstrations rather than relying on pre-trained semantic priors.", "body_md": "arXiv:2610.30592v1 Announce Type: new \nAbstract: LLMs have shown strong in-context learning (ICL) capability, but extending it to Multimodal LLMs (MLLMs) remains challenging. Prior multimodal ICL methods often rely on ICL-specific datasets for additional training or task vector extraction, which can improve performance on the same ICL benchmark used for adaptation. However, they do not generalize well to other benchmarks and often induce forgetting in previously well-solved tasks. For this reason, we identify 'visual grounding' as the key bottleneck in multimodal ICL; MLLMs often fail to attend to task-relevant visual evidence in demonstrations, instead over-relying on textual cues and producing hallucinated outputs. To address this, we propose ProCoRe to enhance visual grounding through contrastive reinforcement learning on multimodal contrastive data for improving multimodal ICL. ProCoRe generates captions for contrastive image pairs, decomposes them into propositions, and optimizes contrastive rewards that encourage alignment with corresponding images while discouraging mismatched ones. Despite never seeing ICL-formatted training examples, ProCoRe improves average multimodal ICL classification accuracy by 5.6% and captioning ROUGE-L by 13.8% over the off-the-shelf Qwen-VL-3-8B model, outperforming ICL-data-dependent state of the arts across benchmarks. Finally, we introduce PICL, a personalized multimodal ICL captioning benchmark that tests whether MLLMs genuinely understand visual demonstrations rather than relying on pre-trained semantic priors.", "url": "https://wpnews.pro/news/visual-grounding-first-multimodal-in-context-learning-follows", "canonical_source": "https://arxiv.org/abs/2610.30592", "published_at": "2026-10-23 04:00:00+00:00", "updated_at": "2026-10-23 04:13:42.924547+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "computer-vision"], "entities": ["ProCoRe", "Qwen-VL-3-8B", "PICL", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/visual-grounding-first-multimodal-in-context-learning-follows", "markdown": "https://wpnews.pro/news/visual-grounding-first-multimodal-in-context-learning-follows.md", "text": "https://wpnews.pro/news/visual-grounding-first-multimodal-in-context-learning-follows.txt", "jsonld": "https://wpnews.pro/news/visual-grounding-first-multimodal-in-context-learning-follows.jsonld"}}