{"slug": "omnixref-a-unified-framework-for-multimodal-multi-reference-video-editing", "title": "OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing", "summary": "Researchers introduced OmniXRef, a unified framework for multimodal multi-reference video editing that lets users combine variable numbers of text, image, and video references to control attributes of a source video. The framework includes a reference-bank input design with R²-RoPE for generalizing to more references, an OmniXRef-Model built on a single-stream diffusion transformer that jointly processes the source video and MLLM-derived multimodal context in an interleaved sequence, and the OmniXRef-Data synthesis pipelines and OmniXRef-Bench benchmark covering Camera, Motion, Content, Style, and Visual Effects (VFX). The authors report OmniXRef-Model outperforms task-specific baselines and commercial models in comprehensive evaluations.", "body_md": "arXiv:2610.22744v1 Announce Type: new \nAbstract: Creative intent in video editing can be expressed through text instructions (e.g., \"remove the people\"), image references (e.g., subject appearance), video references (e.g., camera movement), or a combination of the above. We introduce OmniXRef, a unified framework for multimodal multi-reference video editing that allows users to combine variable numbers of text, image, and video references to control diverse attributes of a source video. We first devise a unified input design with a reference-bank interface and introduce $R^2$-RoPE to support generalization to more references. Then our model, OmniXRef-Model, uses a single-stream diffusion transformer to jointly process the source video and multimodal context from an MLLM in an interleaved sequence. To address the scarcity of training data for video-reference editing, we construct OmniXRef-Data using synthesis pipelines for different editing tasks and combinations of references. We use curriculum learning to progressively increase task complexity and resolution. To support systematic evaluation of video as a control modality, we introduce OmniXRef-Bench, a benchmark for video-reference attribute transfer across Camera, Motion, Content, Style, and Visual Effects (VFX). Comprehensive evaluations show that our OmniXRef-Model outperforms task-specific baselines and commercial models.", "url": "https://wpnews.pro/news/omnixref-a-unified-framework-for-multimodal-multi-reference-video-editing", "canonical_source": "https://arxiv.org/abs/2610.22744", "published_at": "2026-10-19 04:00:00+00:00", "updated_at": "2026-10-19 04:22:11.451317+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "computer-vision", "ai-research"], "entities": ["OmniXRef", "OmniXRef-Model", "OmniXRef-Data", "OmniXRef-Bench", "R²-RoPE", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/omnixref-a-unified-framework-for-multimodal-multi-reference-video-editing", "markdown": "https://wpnews.pro/news/omnixref-a-unified-framework-for-multimodal-multi-reference-video-editing.md", "text": "https://wpnews.pro/news/omnixref-a-unified-framework-for-multimodal-multi-reference-video-editing.txt", "jsonld": "https://wpnews.pro/news/omnixref-a-unified-framework-for-multimodal-multi-reference-video-editing.jsonld"}}