Why Video Agent models are next — Ethan He, xAI Grok Imagine
type: NewsArticle
description: Ethan He, an engineer at xAI who built Grok Imagine in three months, argued that video models derive their intelligence primarily from large language models rather than video training data, and predicted the next frontier for generative media will be video agents capable of planning, editing, and iterating across creative tasks. He suggested that the evolution of video generation will mirror AI coding's shift from one-shot output to multiturn reasoning and orchestration systems, with real-time, interactive world models depending more on language models and agents than on diffusion alone.
date published: 2026-06-01 15:41:48+00:00
date modified: 2026-06-02 20:29:06.292244+00:00
language: en
is accessible for free: true
article body: *We’re announcing AIEWF speakers this week! Take the AI Engineering Survey!* Today’s guest Ethan first joined us for the LS Paper Club as the lead on [NVIDIA Cosmos World Model](https://www.youtube.com/watch?v=og59L4JECz4&pp=ygUWbGF0ZW50c3BhY2V0diBldGhhbiBoZQ%3D%3D), but then joined xAI and built Grok Imagine in 3 months: He comes back on Latent Space with some nuclear hot takes: that **Video Models primarily get their intelligence from LLMs**, not from training on video data, and that the next frontier for truly interactive, realtime, long-horizon **world models** is to work on LLMs (perhaps [Interaction Models ](https://www.latent.space/p/ainews-thinking-machines-native-interaction)as well…) Put it this way: In the near term, the next Sora won’t be a better video model, but **a video agent**. ** Generative Media** may more closely follow **the evolution of AI coding** which went from focusing on one-shot output performance and cost, to multiturn reasoning and planning models for agents and systems that can plan, edit, test, debug, and submit PRs. At a certain point, coding models got so good that the only significant next step to improve performance was **handling the orchestration of these models.** Now as the performance of video models increases significantly across realism, consistency, & prompt adherence while becoming more cost efficient, the next evolution of video generation may also be systems that can plan, generate, edit, critique, and iterate across an entire creative task. In this episode, Ethan joins swyx and Vibhu to unpack what it actually takes to build **frontier image and video systems**: data, VAEs, diffusion transformers, audio-video alignment, inference speedups, and the hidden cost of storing and moving massive video datasets. From building ** NVIDIA’s Cosmos world model** to joining **xAI** as **was being built from zero to one,** [Grok Imagine](https://grok.com/imagine)**Ethan He** has been at the center of some of the most important work …
publisher: Web Pulse
keywords: generative-ai, computer-vision, large-language-models, ai-agents, ai-research
mentions: Ethan He
mentions: xAI
mentions: Grok Imagine
mentions: NVIDIA
mentions: Cosmos World Model
mentions: Latent Space
mentions: Sora
mentions: AI Engineering Survey
citation: CreativeWork