Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding

Researchers propose Temporal Tree of Thought (T^3), a training-free framework that improves long-video understanding in Multimodal Large Language Models by constructing a hierarchical temporal tree and performing an answer-retrieve-explore loop. On VideoMME, LongVideoBench, and LVBench, T^3 boosts Qwen2.5-VL-7B by 0.5%, 4.6%, and 4.4% respectively under the same frame budget.

arXiv:2608.27871v1 Announce Type: new Abstract: Long-video understanding remains challenging for Multimodal Large Language Models MLLMs due to limited context length. Uniform sampling may miss crucial moments, while agent-based frame video understanding methods often evaluate frames independently, overlooking the temporal organization of videos. Ideally, evidence selection should mimic how humans answer questions about long videos: first locating the relevant segment from the global context, then zooming into local objects and details. We propose Temporal Tree of Thought T^3, a training-free framework for adaptive coarse-to-fine long-video understanding. T^3 constructs a question-agnostic hierarchical temporal tree via recursive temporally constrained clustering, where each node represents a contiguous segment with an informative key frame. During inference, T^3 performs an answer-retrieve-explore loop: it reasons over coarse representative frames, generates a search statement when evidence is insufficient, and expands relevant branches for finer-grained evidence. This process adaptively shifts the search target from temporal regions to specific objects and visual details to help video understanding. Experiments on VideoMME, LongVideoBench, and LVBench show that T^3 improves Qwen2.5-VL-7B by 0.5%, 4.6%, and 4.4%, respectively, under the same frame budget, demonstrating the effectiveness of structured temporal reasoning.