{"slug": "beyond-visual-cot-internalized-visual-thinking-for-proactive-video-reasoning", "title": "Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning", "summary": "Researchers from Microsoft and academic institutions introduced Internalized Visual Thinking (IVT), a post-training framework that enables multimodal large language models to reason about videos without generating intermediate images at inference time. In controlled studies, IVT improved over text-only post-training across all six evaluation settings and reduced end-to-end latency by more than 5× compared with Visual CoT, suggesting explicit pixel-space generation may be unnecessary for proactive video reasoning.", "body_md": "[content type paper](/research/)published August 2026\n\nBeyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning\n\nAuthorsXiaoyu Zhu, Xinke Deng, Suresh Taddewadikar, Arnab Kumar Mondal, Zhongyu Jiang, Ian Fasel, Joerg Liebelt\n\nBeyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning\n\nAuthorsXiaoyu Zhu, Xinke Deng, Suresh Taddewadikar, Arnab Kumar Mondal, Zhongyu Jiang, Ian Fasel, Joerg Liebelt\n\nMultimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is particularly problematic for proactive video reasoning. We ask whether models can learn to think visually during training while reasoning directly at inference. We introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos. Given a partially observed video, IVT predicts latent representations of future frames together with the target textual answer, encouraging the model to capture motion, object transitions, interactions, and latent intent. At inference, IVT generates the answer directly without synthesizing or re-encoding future frames. We conduct controlled studies across target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives. IVT improves over text-only post-training across all six evaluation settings while retaining the same efficient inference pathway. Compared with Visual CoT, IVT achieves comparable or better performance and reduces end-to-end latency by more than 5×. Our findings suggest that explicit pixel-space generation at inference time, as used in Visual CoT, may not be necessary for proactive video reasoning. Predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.\n\nThe Potential of CoT for Reasoning: A Closer Look at Trace Dynamics\n\nFebruary 24, 2026[research area Methods and Algorithms](/research/?domain=Methods%20and%20Algorithms), [research area Speech and Natural Language Processing](/research/?domain=Speech%20and%20Natural%20Language%20Processing)[conference ICLR](/research/?event=ICLR)\n\nChain-of-thought (CoT) prompting is a de-facto standard technique to elicit reasoning-like responses from large language models (LLMs), allowing them to spell out individual steps before giving a final answer. While the resemblance to human-like reasoning is undeniable, the driving forces underpinning the success of CoT reasoning still remain largely unclear. In this work, we perform an in-depth analysis of CoT traces originating from…\n\nImprove Vision Language Model Chain-of-thought Reasoning\n\nJune 5, 2025[research area Computer Vision](/research/?domain=Computer%20Vision), [research area Speech and Natural Language Processing](/research/?domain=Speech%20and%20Natural%20Language%20Processing)[conference ACL](/research/?event=ACL)\n\nChain-of-thought (CoT) reasoning in vision language models (VLMs) is crucial for improving interpretability and trustworthiness. However, current training recipes often relying on datasets dominated by short annotations with minimal rationales. In this work, we show that training VLM on short answers leads to poor generalization on reasoning tasks that require more detailed explanations. To address this limitation, we propose a two-stage…", "url": "https://wpnews.pro/news/beyond-visual-cot-internalized-visual-thinking-for-proactive-video-reasoning", "canonical_source": "https://machinelearning.apple.com/research/internalized-visual-thinking", "published_at": "2026-08-24 00:00:00+00:00", "updated_at": "2026-08-24 16:15:45.992843+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "large-language-models"], "entities": ["Microsoft", "Xiaoyu Zhu", "Xinke Deng", "Suresh Taddewadikar", "Arnab Kumar Mondal", "Zhongyu Jiang", "Ian Fasel", "Joerg Liebelt"], "alternates": {"html": "https://wpnews.pro/news/beyond-visual-cot-internalized-visual-thinking-for-proactive-video-reasoning", "markdown": "https://wpnews.pro/news/beyond-visual-cot-internalized-visual-thinking-for-proactive-video-reasoning.md", "text": "https://wpnews.pro/news/beyond-visual-cot-internalized-visual-thinking-for-proactive-video-reasoning.txt", "jsonld": "https://wpnews.pro/news/beyond-visual-cot-internalized-visual-thinking-for-proactive-video-reasoning.jsonld"}}