Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning

A paper submitted to arXiv on 9 November 2026 introduces Reframe, a post-training framework for visual agents that aligns the observation state rather than only the output distribution. Reframe combines Ground-Drop-GRPO (GD-GRPO), which trains spatial and temporal specialists with evidence-aware rewards and stochastic removal of the original visual input, with Observation-Gated-MOPD (OG-MOPD), which replaces deficient tool actions with canonical ones mid-rollout and resumes distillation from the repaired observation. Across 14 image and video benchmarks, Reframe outperforms both Mix-RL and vanilla multi-teacher on-policy distillation, and ablations show GD-GRPO increases observation dependence and tool gain compared with outcome-based GRPO, with neither stage requiring a task-specific SFT cold start.

Computer Science Computer Vision and Pattern Recognition Submitted on 9 Nov 2026 Title:Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning View PDF http://arxiv.org/pdf/2611.12886v1 HTML experimental https://arxiv.org/html/2611.12886v1 Abstract:Agentic reinforcement learning RL and multi-teacher on-policy distillation MOPD offer complementary routes to post-training. For visual agents, however, tool use presents two related challenges. Models may overlook useful tool-returned observations, while poorly grounded actions can limit the evidence available for subsequent distillation. We introduce Reframe, a post-training framework that treats the observation state, rather than only the output distribution, as an object of alignment. First, Ground-Drop-GRPO GD-GRPO trains spatial and temporal specialists with evidence-aware rewards and stochastic removal of the original visual input, encouraging them to acquire grounded evidence and use the returned observations. Second, Observation-Gated-MOPD OG-MOPD replaces deficient tool actions with canonical ones mid-rollout and resumes distillation from the repaired observation. The two stages complement each other, as learning to rely on tool-returned observations can help the policy benefit from observation repair. Neither stage requires a task-specific SFT cold start. Extensive experiments over 14 image and video benchmarks demonstrate that Reframe outperforms both Mix-RL and vanilla MOPD. Ablation results show that GD-GRPO increases observation dependence and tool gain compared with outcome-based GRPO. Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .