cd /news/ai-research/does-the-model-use-what-it-cites-cou… · home › topics › ai-research › article
[ARTICLE · art-171551] src=arxiv.org ↗ pub= topic=ai-research verified=true sentiment=· neutral

Does the Model Use What It Cites? Counterfactual Tests of Grounded Reasoning Chains in Small Video-Language Models

A new arXiv paper (2611.21940v1) introduces the "cite gap," a counterfactual test that blacks out the frame regions a video-language model cites in its reasoning chain and compares answer flips against masking same-size boxes elsewhere on the same frames. Applied to 2-3B video-language models, the cite gap was 0.089 on CLEVRER and 0.003 on NExT-QA; masking cited boxes flipped 2.2 times as many correct answers as the control on CLEVRER and 1.1 times as many on NExT-QA, and masking the cited entity on every frame flipped 1.9 times as many answers as its control. The authors conclude that a citation can be verifiable and relevant without the answer depending on it, and that tests of grounded reasoning should remove what a chain names, not only where it points.

recommendation

Test grounded reasoning chains by masking the cited regions and comparing answer flips against same-size control boxes; a cite gap near zero means the citations aren't load-bearing.

more in playbook: Evals and testing for LLM apps
by read1 min views1 publishedNov 18, 2026

arXiv:2611.21940v1 Announce Type: new Abstract: A grounded reasoning chain cites a frame span and a box for every step, so a reader can check where the model looked. Whether the answer depends on the cited region is a different question. We test it by intervention: the regions a model cited in its own chain are blacked out, and the share of correct answers that flip is compared with blacking out boxes of the same size elsewhere on the same frames. The paired difference, the cite gap, needs no annotation. We apply it to 2-3B video-language models trained on exact CLEVRER chains and on pseudo-labelled NExT-QA chains. With the same weights, the cite gap is 0.089 on CLEVRER and 0.003 on NExT-QA. Pooled over models, masking the cited boxes flips 2.2 times as many correct answers as the control on CLEVRER and 1.1 times as many on NExT-QA, also among the answers that depend on the video at all, while the boxes pass a grounding check on both. Masking the cited entity on every frame, instead of the cited box, flips 1.9 times as many answers as its control. The NExT-QA chains were supervised to cite two frames of entities that stay visible on most frames, so this outcome is what the supervision predicts; the point is that the usual check on grounded chains cannot see it. A citation can be verifiable and relevant without the answer depending on it, and a test of grounded reasoning should remove what a chain names, not only where it points.

── more in #ai-research 4 stories · sorted by recency
── more on @clevrer 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/does-the-model-use-w…] indexed:0 read:1min 2026-11-18 · —