{"slug": "video-coder-reinforcing-agentic-coding-for-video-generation", "title": "Video-Coder: Reinforcing Agentic Coding for Video Generation", "summary": "A paper submitted to arXiv on 6 Nov 2026 introduces Video-Coder, described by its authors as the first attempt to train a code-driven video generation agent that autonomously writes and executes code to build a dynamic 3D scene, called a Draft Video, supplying structure and motion signals to downstream video generators. The authors trained Video-Coder with supervised fine-tuning followed by harness-native agentic reinforcement learning using dual execution-based and video-based rewards, with an annealing strategy that gradually reduces the execution reward so the model relies more on the video reward later in training, and they released the Video-Coder-SFT-6k and Video-Coder-RL-2k datasets plus the Video-Coder Harness environment. Experiments show Video-Coder improves Minimax-H3 by approximately 7 points on PhyGenBench and 24 points on VBench-2.0, and the authors state they fully open-source their data, models, and code.", "body_md": "# Computer Science > Computer Vision and Pattern Recognition\n\n  [Submitted on 6 Nov 2026]\n\n# Title:Video-Coder: Reinforcing Agentic Coding for Video Generation\n\n[View PDF](http://arxiv.org/pdf/2611.09894v1)\n\n[HTML (experimental)](https://arxiv.org/html/2611.09894v1)\n\nAbstract:Recent years have witnessed remarkable progress in video generation in terms of visual quality and temporal consistency. However, generating videos with precise spatial structure, coherent motion dynamics, and controllable scene composition remains challenging. In this paper, we introduce Video-Coder, the first attempt to train a code-driven video generation agent, which autonomously writes and executes code to construct a dynamic 3D scene (Draft Video) that provides structure and motion signals to downstream video generators. To support this paradigm, we design a dedicated data pipeline and curate two high-quality datasets, Video-Coder-SFT-6k and Video-Coder-RL-2k, which contain diverse prompts and ground-truth reference videos. We further develop the Video- Coder Harness, a unified interactive environment integrating code execution, 3D scene rendering, and execution feedback for agent training and inference. We then train Video-Coder with SFT fol- lowed by harness-native agentic reinforcement learning with dual reward feedback, which combines execution-based and video-based rewards. Additionally, we design an annealing training strategy that gradually reduces the execution reward over time, allowing the model to rely more on the video reward in the later training stages. Experiments show that Video-Coder achieves significant gains, improving Minimax-H3 by approximately 7 points on PhyGenBench and 24 points on VBench-2.0. We hope this work can serve as an open foundation for coding agents in video generation, and we fully open-source our data, models, and code\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/video-coder-reinforcing-agentic-coding-for-video-generation", "canonical_source": "http://arxiv.org/abs/2611.09894v1", "published_at": "2026-11-09 16:16:28+00:00", "updated_at": "2026-11-09 16:50:49.651674+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "computer-vision", "ai-agents", "ai-research"], "entities": ["Video-Coder", "arXiv", "Minimax-H3", "PhyGenBench", "VBench-2.0", "Video-Coder-SFT-6k", "Video-Coder-RL-2k", "Video-Coder Harness"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/video-coder-reinforcing-agentic-coding-for-video-generation", "markdown": "https://wpnews.pro/news/video-coder-reinforcing-agentic-coding-for-video-generation.md", "text": "https://wpnews.pro/news/video-coder-reinforcing-agentic-coding-for-video-generation.txt", "jsonld": "https://wpnews.pro/news/video-coder-reinforcing-agentic-coding-for-video-generation.jsonld"}}