This is already a fairly active research area, not just a hunch: there are several 2025-2026 papers doing exactly this (FilmAgent, VideoGen-of-Thought, CineAGI, StoryAgent, AniME), all decomposing video production into specialized agents (script/storyboard, character consistency, animation, TTS, assembly) coordinated by an orchestrator. Multi-agent frameworks including FilmAgent, Kubrick, and VideoGen-of-Thought decompose production into specialized roles but fail to coordinate complex narrative, visual, and temporal constraints, so your instinct about “the missing part is the editor/coordinator” lines up with what these papers identify as the actual bottleneck too.
The character-identity and sync problems you listed are the two hardest parts in practice. One approach (CineAGI) tackles identity consistency through a decoupled character-centric pipeline using instance-level tracking, and reports meaningful gains in character and subject consistency over baselines. On the coordination side specifically, most early systems used simple waterfall pipelines (one agent’s output feeds the next) but these waterfall pipelines are susceptible to cascading errors, and newer work is moving toward a formal orchestrator that manages exploration/exploitation across the creative decisions instead of just chaining fixed steps. Worth searching arXiv for “agentic video generation” or “multi-agent video storytelling” if you want to dig into any of these, it’s a good sign this is being taken seriously as a research direction rather than just a hobbyist idea.