{"slug": "allocate-before-you-embed-adaptive-visual-input-allocation-for-video-embeddings", "title": "Allocate Before You Embed: Adaptive Visual Input Allocation for Video Embeddings", "summary": "Researchers introduced AllocEmbed, an allocate-then-embed framework that reallocates a fixed visual-input budget across more frames to improve video retrieval, achieving the best overall performance on MMEB-V2 V-QA and V-RET tasks and the LongRet benchmark among budget-matched methods. The method, which uses a lightweight allocator and Retrieval-Driven Policy Optimization (RDPO), integrates with existing retrieval systems without modifying the embedding model or downstream pipeline. Code is available at https://github.com/jinsong8/AllocEmbed.", "body_md": "arXiv:2609.01778v1 Announce Type: new\nAbstract: Large-scale video retrieval requires embedding models to encode long and diverse videos under tight visual-input and inference budgets. Existing methods typically sample a small, fixed set of frames at their original resolution, limiting temporal coverage and ignoring frame importance. Our empirical analysis shows that expanding temporal coverage improves retrieval even under a fixed visual-input budget. Gains are larger when the original per-frame resolution is preserved, highlighting the complementary roles of temporal coverage and spatial fidelity. Motivated by this finding, we propose AllocEmbed, an allocate-then-embed framework that reallocates a fixed visual-input budget across more frames. A lightweight allocator uses low-cost previews to assign frame-wise resolutions before the embedding backbone, preserving more detail where it most benefits retrieval while reducing visual cost elsewhere. We further introduce Retrieval-Driven Policy Optimization (RDPO), which learns the allocator directly from retrieval feedback using a rank-validated similarity gap and a confidence-guided efficiency incentive. Operating entirely before the backbone, AllocEmbed integrates with existing retrieval systems without modifying the embedding model or downstream pipeline. Experiments on the MMEB-V2 V-QA and V-RET tasks and our LongRet benchmark show that AllocEmbed achieves the best overall retrieval performance among the evaluated budget-matched methods and transfers across embedding backbones. Our code is publicly available at https://github.com/jinsong8/AllocEmbed.", "url": "https://wpnews.pro/news/allocate-before-you-embed-adaptive-visual-input-allocation-for-video-embeddings", "canonical_source": "https://arxiv.org/abs/2609.01778", "published_at": "2026-09-03 04:00:00+00:00", "updated_at": "2026-09-03 04:22:50.309219+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "ai-research"], "entities": ["AllocEmbed", "RDPO", "MMEB-V2", "LongRet", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/allocate-before-you-embed-adaptive-visual-input-allocation-for-video-embeddings", "markdown": "https://wpnews.pro/news/allocate-before-you-embed-adaptive-visual-input-allocation-for-video-embeddings.md", "text": "https://wpnews.pro/news/allocate-before-you-embed-adaptive-visual-input-allocation-for-video-embeddings.txt", "jsonld": "https://wpnews.pro/news/allocate-before-you-embed-adaptive-visual-input-allocation-for-video-embeddings.jsonld"}}