{"slug": "gen4u-unifying-video-generation-and-understanding-via-diffusion", "title": "Gen4U: Unifying Video Generation and Understanding via Diffusion", "summary": "Researchers demonstrate that state-of-the-art video diffusion models possess a highly structured latent space capable of both generation and understanding. They introduce Gen4U, a framework that repurposes frozen generative representations for tasks like video classification, depth estimation, and captioning without fine-tuning. The work unifies video generation and understanding, achieving strong perception performance while preserving generative capabilities.", "body_md": "arXiv:2607.06856v1 Announce Type: new\nAbstract: Prior work suggests that diffusion representations capture low-level geometry but struggle with high-level semantics. We demonstrate that state-of-the-art video diffusion models overcome this limitation. By systematically probing their intermediate activations using recent mutual-kNN alignment metrics, we reveal a highly structured latent space where visual representations evolve across both network depth and noise levels. We show that while moderate noise levels yield linearly separable global semantics, fine-grained details persist at lower noise levels but become spatially scattered, requiring attention mechanisms to decode. Building on these insights, we introduce Gen4U (Generation for Understanding), a framework that repurposes these generative representations with a single forward pass. Our experiments establish that frozen, large-scale video diffusion models function as highly competitive video encoders across a wide spectrum of tasks, spanning semantic and non-semantic objectives (video classification, depth estimation, camera pose estimation, image and video captioning). Bypassing fine-tuning, Gen4U unifies the generation and understanding paradigms, achieving strong perception performance while fully preserving the model's ability to generate high-quality video.", "url": "https://wpnews.pro/news/gen4u-unifying-video-generation-and-understanding-via-diffusion", "canonical_source": "https://arxiv.org/abs/2607.06856", "published_at": "2026-07-09 04:00:00+00:00", "updated_at": "2026-07-09 04:29:04.225661+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "generative-ai", "ai-research"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/gen4u-unifying-video-generation-and-understanding-via-diffusion", "markdown": "https://wpnews.pro/news/gen4u-unifying-video-generation-and-understanding-via-diffusion.md", "text": "https://wpnews.pro/news/gen4u-unifying-video-generation-and-understanding-via-diffusion.txt", "jsonld": "https://wpnews.pro/news/gen4u-unifying-video-generation-and-understanding-via-diffusion.jsonld"}}