cd /news/artificial-intelligence/visalloc-allocating-visual-supervisi… · home › topics › artificial-intelligence › article
[ARTICLE · art-173028] src=arxiv.org ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

VisAlloc: Allocating Visual Supervision for On-Policy Self-Distillation

A research team introduced VisAlloc, a dual-level framework for on-policy self-distillation (OPSD) that combines sample-level visual filtering with token-level counterfactual reweighting to improve visual reasoning in multimodal large language models. Across eight benchmarks, the 4B and 9B VisAlloc models reached macro-averages of 81.44 and 84.92, outperforming Vision-OPD reproductions by 0.97 and 0.52 points respectively, with code released on GitHub. The method retains only samples answered correctly from an evidence-centered positive crop but incorrectly from a spatially displaced negative crop, and uses two EMA teachers' absolute log-probability differences on the same student response to measure view sensitivity, requiring only the full-image student at inference.

recommendation

When distilling visual reasoning into MLLMs, keep only samples where a positive crop succeeds and a displaced crop fails, and add token supervision where teacher log-probabilities diverge; the source reports +0.97 and +0.52 points over Vision-OPD.

more in playbook: Fine-tuning and data
by read1 min views2 publishedNov 20, 2026

arXiv:2611.25069v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) offers a promising approach to improving visual reasoning in multimodal large language models (MLLMs). However, not all samples benefit equally from privileged visual guidance, and uniform token supervision can dilute visually grounded signals. We propose VisAlloc, a dual-level framework combining sample-level visual filtering with token-level counterfactual reweighting. A fixed paired-teacher filter retains samples answered correctly from an evidence-centered positive crop but incorrectly from a spatially displaced negative crop. At the token level, both EMA teachers score the same student-generated response, and their absolute log-probability differences measure view sensitivity. Additional supervision targets tokens admitted by sensitivity or teacher-distribution disagreement, with shared frozen-prior, template-bigram, and student--teacher divergence gates. Inverse-frequency redistribution preserves the total additional weight within each micro-batch, while the positive teacher remains the sole distillation target. Inference requires only the full-image student. Across eight benchmarks, our 4B and 9B models achieve macro-averages of 81.44 and 84.92, outperforming Vision-OPD reproductions by 0.97 and 0.52 points, respectively. Code on GitHub: https://github.com/Kaelyn01/VisAlloc

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @visalloc 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/visalloc-allocating-…] indexed:0 read:1min 2026-11-20 · —