{"slug": "boxworld-let-world-engines-control-and-video-generators-render", "title": "BoxWorld: Let World Engines Control and Video Generators Render", "summary": "A new arXiv paper, BoxWorld (arXiv:2610.36371v1), separates interaction reasoning from generative rendering in real-time interactive video worlds by having a coding agent build a \"Box World Engine\" from user-specified rules that outputs layout boxes, persistent object attributes and dynamic status variables, which a Box-State Generative Renderer then turns into video autoregressively. The renderer is trained in three stages — object grounding from open-domain videos, state transitions from gameplay videos, and rollout distillation on self-generated histories — with both video sources annotated automatically. The authors report BoxWorld substantially outperforms end-to-end world models and remains competitive across multiple public benchmarks, and they introduce PairedStateBench, a paired benchmark for fine-grained interaction control.", "body_md": "arXiv:2610.36371v1 Announce Type: new \nAbstract: We tackle the challenge of building real-time interactive video worlds with user-defined behaviors. Such worlds must track object states, apply interaction rules, and render the resulting consequences as users act. Yet existing end-to-end video world models typically learn interaction dynamics within the generator, making it difficult to specify object-level rules and reason about their consequences. In this work, we introduce BoxWorld, which separates interaction reasoning from generative rendering. Given user-specified rules, a coding agent creates a Box World Engine that determines interaction outcomes and exposes the resulting object states through a lightweight interface of layout boxes, persistent object attributes, and dynamic status variables, leaving appearance and physical effects to generative rendering. A Box-State Generative Renderer then renders this evolving state stream into video autoregressively, enabling online control over object motion, state switches, and conditional events. Rendering this state stream is non-trivial: the renderer must respond to object-level state updates in real time while remaining coherent with the generated history. To this end, we introduce a three-stage training strategy that learns object grounding from open-domain videos and state transitions from gameplay videos, both annotated automatically, and finally performs rollout distillation on self-generated histories. We further propose PairedStateBench, a paired benchmark for fine-grained interaction control. BoxWorld substantially outperforms end-to-end world models and remains competitive across multiple public benchmarks. Project page: https://boxworld-web.github.io.", "url": "https://wpnews.pro/news/boxworld-let-world-engines-control-and-video-generators-render", "canonical_source": "https://arxiv.org/abs/2610.36371", "published_at": "2026-10-28 04:00:00+00:00", "updated_at": "2026-10-28 04:14:31.492629+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "computer-vision", "ai-research", "machine-learning"], "entities": ["BoxWorld", "Box World Engine", "Box-State Generative Renderer", "PairedStateBench", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/boxworld-let-world-engines-control-and-video-generators-render", "markdown": "https://wpnews.pro/news/boxworld-let-world-engines-control-and-video-generators-render.md", "text": "https://wpnews.pro/news/boxworld-let-world-engines-control-and-video-generators-render.txt", "jsonld": "https://wpnews.pro/news/boxworld-let-world-engines-control-and-video-generators-render.jsonld"}}