How to teach a robot to water the table

HumanSignal, a dataset company for physical AI, found that current video models could not convincingly render a deliberate task failure in a planned 20-second household-robot ad, despite holding character consistency across shots with Seedance 2.5 using a character turnaround and three-quarter and straight-on room references. The company attributed the difficulty to a "competence prior" in video models that pushes water toward where water should land, calling the prompt for a plausible near-miss "failure choreography," and cited Google DeepMind's paper "Video models are zero-shot learners and reasoners" on video models as general-purpose visual foundations.

Video models have gotten remarkably good at generating convincing frames. But that is a completely separate skill from understanding the three-dimensional world those frames depict: how objects occupy space, how liquids move, what a joint can and cannot do. Here's how we backed ourselves into creating a nearly impossible ad, what we did to save it, and what it says about the current state of AI world understanding. At HumanSignal we build datasets, especially novel data for physical AI. We wanted to test whether current video models could carry a real ad campaign, so we planned out a few storyboards. The idea was to create an amiable household robot character who appears at first to be succeeding, until the camera moves to reveal the task as "almost-but-not-quite". The surprise shift from success to failure is designed to be the payoff and the humor. Each spot was conceived to run about 20 seconds. You can watch the current watering-plant cut below: We unfortunately learned the hard way that the camera orbit, the task failure, and the fluid dynamics were all extremely challenging to pull off with the current generation of video models. Going in, our biggest worry was character consistency. We wanted to avoid the robot mutating in strange ways between shots. That actually turned out to be the manageable part. The fix for this is to produce a "character turnaround": multiple views of the robot, fed to the model as an image reference. To stabilize the environment as the camera moves, we also generated a three-quarter view of the room, one corner showing how the key objects relate to each other, plus a straight-on view, and fed both as references. A lot of research has been applied towards solving consistency issues, and it shows in modern models. With the turnaround and environment references in place, Seedance 2.5 held the same character across the spot admirably well. Same face, same proportions, same joint design, shot after shot. What we expected to be the hard problem turned out to be the least of our worries. The joke requires the robot to do the task wrong , convincingly, legibly, almost-but-not-quite. And that seemed to run against the model's tendency to produce task completion plausibility, with things going the way things are supposed to go. Internally, we've taken to calling this failure choreography : prompting for a deliberate, plausible-looking failure. Video models seem to have what you might call a competence prior : show them a robot, a pitcher, and a plant, and they tend toward the water landing where water should land. Asking for a near-miss means asking the model to depart from that familiar outcome. At least that's our working explanation of the behavior. So in hindsight, our little ad turned out to be a triple stress test: If large language models learn from representations of human knowledge, video models learn from representations of the physical world. Simon Willison's write-up https://simonwillison.net/2025/Sep/27/video-models-are-zero-shot-learners-and-reasoners/ of Google DeepMind's paper “Video models are zero-shot learners and reasoners” https://arxiv.org/abs/2509.20328 is a great read. It explores the potential of this training to ultimately do so much more than produce cute ads. The core argument is that video models could ultimately become general-purpose foundations for visual tasks, much as language models have for text. The Google paper is worth reading in depth. The generated video is the output , but the broader ambition is a transferable understanding of the visual world that works across domains. The paper explores “chain-of-frames” reasoning: a video model working through a problem across time, the visual analogue of chain-of-thought. Training on video alone can produce two things that feel like one thing but aren't. Architectures and objectives differ, so it isn't all simply next-frame prediction. The distinction that matters for our work is between: Looking right and behaving right are different objectives, and our attempt to create this seemingly simple ad showed how far apart they can be. The research community has started measuring this gap. The Physics-IQ paper, “Do generative video models understand physical principles?” https://arxiv.org/abs/2501.09038 , found that visual realism and physical understanding were largely unrelated in the models it tested. VideoPhy-2 https://arxiv.org/abs/2503.06800 also found shortcomings in physical commonsense, including conservation laws. Note that these studies evaluated their own model sets, not the three versions below. We were curious enough that we wanted to see it for ourselves. We came up with a simple experiment: send an identical "first frame" image and an identical action-only prompt to each of three video models, Seedance 2.5 ByteDance , Veo 3.1 Google DeepMind , and Kling 3.0 Pro Kuaishou , and compare what comes back. The prompts only described the steps to perform, rather than spelling out their physical consequences “pour the water,” not “the water level falls” , so those consequences are left to the model to infer. For this one we created a bit of ground truth: we filmed a real video of water being poured admittedly a bit clumsily from a full pint glass into an empty one, took the first frame, and asked each model to recreate the action specifically, to pour about 60% of the water across . All three failed to give us a convincing account of pouring water, and they failed in different ways. The common struggle was volume transfer: the visible levels don't change as you'd expect. The Kling example really exhibits the realism/understanding gap perfectly. To our eyes, the pour itself was the most realistic of the three, but the end result is essentially two glasses of water. The scene appears to create water from nowhere. Next we used an image of a Radio Shack Armatron https://en.wikipedia.org/wiki/Armatron , the beloved 1980s toy robot arm, with a blue ball and a plastic cup on a table, and asked each model to have the arm pick up the ball and drop it in the cup. We chose the Armatron deliberately: much of its mechanism is legible from a single still. Visible pivot points and colorful geometry make it easier to track what rotates and what doesn't. A single frame can't specify every hidden mechanical constraint, but it gives us a useful test of whether the visible geometry stays consistent. Instead, the models improvised the machine's kinematics. Wrist and arm geometry warped mid-motion to reach the cup. The outputs worked toward the right goal, but the mechanism changed along the way. The models appeared to understand the goal; they hallucinated the machine. My favorite bit: two of the models had the whole toy "hop" closer to the cup, Luxo Jr. <https://en.wikipedia.org/wiki/Luxo Jr. character style. So even in late 2026, video models are still very much a work in progress on physics. Across these saved examples, we kept finding a gap between plausible appearance and consistent behavior. The labs are working on this from several directions. Google DeepMind has been explicit about the destination: the zero-shot paper cited above, plus Genie 3 https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/ , its interactive world-model line. Both point toward broader uses for learned representations of the world. ByteDance's Seedance https://arxiv.org/abs/2604.14148 seems to have spent considerable time in post training, working to provide content creators more precise creative control. While most generative video models support 2-3 references, Seedance 2.5 now supports up to 50 image, sound, and video references. Crucially, those references can each be tagged within a prompt and given specific jobs. “This reference governs geometry and camera placement only.” “This one governs character appearance.” That turns references into a more useful control surface. While not a generative video model, OpenAI's GPT-6 Astra offers another route through its work with 3D tools. It can help create and manipulate actual scene geometry using software such as Blender, as illustrated in OpenAI's architectural visualization example https://developers.openai.com/blog/architectural-visualization-with-astra . That gives us geometry we can inspect and a camera move we can control before generating the finished video. While it's easy to smile at the failures, the rate of progress has been impressive: we re-ran a robot-arm shirt-folding task from an earlier 2025 Google Veo failure example, using the same starting frame against the current models. Two of the three, Seedance 2.5 and Veo 3.1, now produce a more credible fold, with the cloth's movement more closely connected to the arms' actions. In the earlier clip, the transition to a folded state was much less convincing. These examples aren't a field-wide measure of progress, but they're an encouraging practical improvement. All this brings us back to our ad, and how we actually got the camera pivot to work. The practice that saved our spot was to borrow spatial structure from a system that represents it explicitly. We used OpenAI's Astra to help generate a clay previsualization of the action and camera move: crude, untextured geometry standing in for the plant, the pitcher, the robot. Then we fed that previs clip to the video model as a tagged reference governing motion and camera only. The counterintuitive part: less detail was better . Blocky, simple, deliberately unfinished geometry was less likely to bleed its visual style into the final output. We wanted the model to take the choreography and ignore the clay . The result is a practical substitute for the spatial consistency we couldn't reliably get from the video model alone. We're essentially transferring an explicit layout and camera path from 3D tools into a video-generation workflow, pairing the video model's strength, high-fidelity frames, with a controllable account of how the scene moves. For now, it's an essential part of the work, especially for the scenarios that matter commercially: novel ones, where we can't rely on the model having seen the same thing before. Like a robot that waters the table. As world understanding improves inside the models, we expect previsualization and its cousins to become less necessary. What we think will replace it, for VFX houses and creative teams, is a different kind of work: less time in 3D workflows to render artifacts, more time teaching AI models about the novel characters and objects that power the story. Your unique spaceship. Your new robot character. The goal is help AI build a useful understanding, so the character moves, articulates, and behaves in predictable, believable ways across shots. That's a dataset problem much more than a prompting problem. And that kind of novel-entity training is where we expect the effects industry to lean in. Novel datasets for AI training are what HumanSignal does: for physical AI, for creative services near and dear to us, as you can tell , and beyond. If you're thinking about what your model needs to understand , as well as render, talk to the team https://humansignal.com/contact-sales/ .