An AI agent can edit a video project from one prompt and still hand back a bad result. A scene can look right when it plays from zero and wrong when someone scrubs back. An MP4 can export at the wrong size, with no audio track, or a second short. Both are cheap to catch before a person watches anything.
The fix has three parts. Make every frame a pure function of time and test the render path, check each export against an explicit delivery spec with ffprobe, and run both checks through the agent's instructions and CI so a bad render cannot be reported as done. Below is the code for each. The Node seek test, the Python gate and the CI script were run on generated clips and stub scenes. The render step uses CoAnimator's coa CLI (docs version 0.7.2). The seek rules and the gate work with any renderer that writes an MP4.
Seeking is the first thing I test, because everything else builds on it. A preview plays from zero. An exporter that starts at second 3, or a reviewer who drags the playhead backwards, lands on a state the animation never visited if the scene keeps history between calls.
Smoothing is one common way that happens. This scene eases the marker toward its target instead of jumping there.
export function createScene() {
let x = 6; // lives between calls
return function render(t, els) {
const target = 6 + t * 6.5;
x += (target - x) * 0.2; // eases toward the target, so it depends on the last call
els.marker.style.left = `${x}%`;
};
}
On a fresh page, rendering second 4 puts the marker at 11.2%. After playing the full 10 seconds at 30 fps and then seeking to second 4, it sits at about 62.5%. Playback looks smooth, while the exporter or a scrubbing reviewer gets a different frame.
The fix is to compute everything from t.
const clamp = (n, lo, hi) => Math.max(lo, Math.min(hi, n));
export function frameAt(t) {
t = clamp(t, 0, 10);
const p = clamp(t / 2, 0, 1);
return {
x: 6 + t * 6.5,
opacity: 1 - Math.pow(1 - p, 3),
caption: t < 3 ? 'Import' : t < 7 ? 'Map columns' : 'Review',
};
}
export const createScene = () => (t, els) => {
const f = frameAt(t);
els.marker.style.left = `${f.x}%`;
els.marker.style.opacity = String(f.opacity);
els.caption.textContent = f.caption;
};
The scene-state seek test exercises the render path, not just the math. It plays a scene to the end on one stage, then sweeps backwards and forwards through fixed timestamps on that same stage, and compares each result with a fresh scene rendered at the same time. The stage is a plain object, so the test runs in Node without a browser. It checks the values the scene writes, which are marker position, opacity and caption.
const { createScene } = await import(process.argv[2]);
const stage = () => ({ marker: { style: {} }, caption: {} });
const snap = (s) => JSON.stringify([s.marker.style.left, s.marker.style.opacity, s.caption.textContent]);
const freshAt = (t) => {
const s = stage();
createScene()(t, s);
return snap(s);
};
// Play to the end on one stage, then scrub backwards and forwards on that same stage
const played = stage();
const render = createScene();
for (let i = 0; i <= 300; i++) render(i / 30, played);
const times = [];
for (let t = 10; t >= 0; t -= 0.25) times.push(t); // backwards sweep
for (let t = 0; t <= 10; t += 0.7) times.push(t); // forwards sweep, off the grid
for (const t of times) {
render(t, played);
if (snap(played) !== freshAt(t)) {
console.error(`mismatch at ${t.toFixed(2)}s\n scrubbed ${snap(played)}\n fresh ${freshAt(t)}`);
process.exit(1);
}
}
console.log(`seek ok, ${times.length} timestamps`);
Saved as scene-smoothed.js and scene-pure.js, the two scenes give these results. The sweep is fixed, so the failure repeats on every run. The smoothed scene only sets left, so the other two values print as null.
$ node seek.test.mjs ./scene-smoothed.js
mismatch at 10.00s
scrubbed ["70.30666666666669%",null,null]
fresh ["19%",null,null]
$ node seek.test.mjs ./scene-pure.js
seek ok, 56 timestamps
A headless browser that compares screenshots at the same timestamps would also cover pixels and CSS. This test is the quick layer that runs without one. It assumes the scene logic sits in a module that the page and the test both import. If your renderer needs a single HTML file, keep the module as the source and inline it at build time.
Three things live outside the scene code, so check for them in review.
t.
The same rule applies on page load, where a skipped render shows up as a flash of the wrong frame. An export worker can open the animation at a later time through a ?t= parameter, and the page has to paint that frame before anything is visible. Here the stage starts hidden with #stage { visibility: hidden; } in the page's CSS, the requested frame is drawn immediately, and the stage is revealed once fonts and images are ready.
import { createScene } from './scene.js';
const duration = 10;
const stage = document.querySelector('#stage');
const els = {
marker: document.querySelector('#marker'),
caption: document.querySelector('#caption'),
};
const scene = createScene();
const startAt = Math.min(
Math.max(Number(new URLSearchParams(location.search).get('t')) || 0, 0),
duration
);
scene(startAt, els);
await Promise.all([
document.fonts.load('600 32px "Inter"'), // one entry per font the scenes use
...[...document.images].map((img) => img.decode().catch(() => {})),
]);
scene(startAt, els);
stage.style.visibility = 'visible';
I use document.fonts.load() here instead of document.fonts.ready. Per MDN, ready resolves once the document has finished the fonts it needs. Fonts load when text that uses them is rendered, so a font used only by a hidden scene may not have been requested yet. Replace the example font with each font your scenes use.
CoAnimator animations also expose window.__kcControl with seek, seekAnd, play, ``, getTime and getDuration. I route seek and seekAnd into the same render function, so preview and export share one code path. The full player contract is in the timeline reference that ships with the app.
With seeking covered, I render only the scene that changed after each edit.
coa render ./release-demo --timeline main \
--start 3 --end 7 --fps 30 --out ./import-scene-review.mp4
The end is exclusive, so 3 to 7 at 30 fps is 120 frames. coa exits with 0 on success and 1 on failure. It also checks dimensions, frame count, fps, duration and expected audio before it replaces the destination, and a failed run leaves any existing file in place. An old good MP4 can therefore remain at the path after a failed render, so I write every render to a unique name and let a nonzero exit stop the script.
That check uses the tool's own rules. A second one should use mine.
The gate takes the delivery spec as arguments, so it fails when a file is valid for the renderer and wrong for the brief. It checks size, frame rate, decoded frame count, duration and, on request, an audio stream.
#!/usr/bin/env python3
"""Fail the build when a rendered MP4 does not match the spec in the arguments."""
import argparse, json, subprocess, sys
from fractions import Fraction
p = argparse.ArgumentParser()
p.add_argument("file")
p.add_argument("--width", type=int, required=True)
p.add_argument("--height", type=int, required=True)
p.add_argument("--fps", type=float, required=True)
p.add_argument("--duration", type=float, required=True)
p.add_argument("--tolerance", type=float, default=0.1, help="seconds")
p.add_argument("--frame-tolerance", type=int, default=1, help="frames")
p.add_argument("--no-count", action="store_true", help="skip decoding, skip the frame check")
p.add_argument("--require-audio", action="store_true")
a = p.parse_args()
def number(value):
"""Parse '30000/1001' or '12.5'. Return None for 0/0, N/A or missing."""
try:
return float(Fraction(value))
except (TypeError, ValueError, ZeroDivisionError):
return None
cmd = ["ffprobe", "-v", "error"] + ([] if a.no_count else ["-count_frames"]) + [
"-show_streams", "-show_format", "-of", "json", a.file]
r = subprocess.run(cmd, capture_output=True, text=True)
if r.returncode != 0:
sys.exit(f"FAIL ffprobe cannot read {a.file}: {r.stderr.strip()}")
info = json.loads(r.stdout)
streams = info.get("streams", [])
video = next((s for s in streams if s.get("codec_type") == "video"
and not s.get("disposition", {}).get("attached_pic")), None)
audio = next((s for s in streams if s.get("codec_type") == "audio"), None)
problems = []
if video is None:
problems.append("no video stream")
else:
size = (video.get("width"), video.get("height"))
if size != (a.width, a.height):
problems.append(f"size {size[0]}x{size[1]}, expected {a.width}x{a.height}")
fps = number(video.get("avg_frame_rate"))
if fps is None:
problems.append("frame rate unavailable")
elif abs(fps - a.fps) > 0.01:
problems.append(f"fps {fps:g}, expected {a.fps:g}")
if not a.no_count:
frames = video.get("nb_read_frames", "")
expected = round(a.duration * a.fps)
if not frames.isdigit():
problems.append("frame count unavailable")
elif abs(int(frames) - expected) > a.frame_tolerance:
problems.append(f"{frames} frames, expected {expected}")
duration = number(info.get("format", {}).get("duration"))
if duration is None:
problems.append("duration unavailable")
elif abs(duration - a.duration) > a.tolerance:
problems.append(f"duration {duration:.2f}s, expected {a.duration:g}s")
if a.require_audio and audio is None:
problems.append("no audio stream")
if problems:
print("FAIL " + a.file)
for line in problems:
print(" - " + line)
sys.exit(1)
print(f"OK {a.file}")
bash
$ python3 check_export.py short.mp4 --width 1280 --height 720 --fps 30 --duration 4 --require-audio
FAIL short.mp4
- 90 frames, expected 120
- duration 3.00s, expected 4s
$ python3 check_export.py good.mp4 --width 1280 --height 720 --fps 30 --duration 4 --require-audio
OK good.mp4
Real files are rarely exact, so two settings soften the checks. Frame counts get a one frame tolerance by default. A 29.97 fps file with 600 frames lasts about 20.02 seconds. Against a 20 second spec the expected count rounds to 599, so the default tolerance passes it and --frame-tolerance 0 fails it. The --no-count flag skips the full decode that -count_frames needs, which suits long films, and it drops the frame check with it. The frame rate check reads ffprobe's average, so it assumes a constant frame rate.
A file with no audio stream fails with no audio stream. A file with a silent audio track still passes, because the gate checks that a stream exists, not what it contains. A file ffprobe cannot read, such as a text file renamed to .mp4 or a truncated export, exits with 1 and ffprobe's error. The gate does not check the container, so a valid WebM passes. Add a check on format_name if the brief requires MP4.
A check only helps if the agent runs it. The instruction file the agent reads, such as AGENTS.md or CLAUDE.md, is the place for that.
## Rendering
- Write each render to `renders/` with a timestamp in the name. Never overwrite an earlier file.
- After every render, run `python3 check_export.py <file> --width 1920 --height 1080 --fps 30 --duration 20 --require-audio`.
- If it prints FAIL, fix the cause and render again. Do not call the video done until it prints OK.
- After changing scene code, run `node seek.test.mjs ./scene.js`.
The FAIL lines say exactly what is off, such as 90 frames, expected 120, which gives the agent something concrete to act on. An instruction can still be skipped, so the same checks run in CI.
On a headless Linux runner the app still needs a display server, so the render goes through xvfb-run. The runner also needs CoAnimator, FFmpeg and FFprobe installed, because version 0.7.2 has no hosted rendering workers.
set -eu
RES=1080p W=1920 H=1080
mkdir -p renders
out="renders/review-$(date -u +%Y%m%dT%H%M%SZ).mp4"
node seek.test.mjs ./scene.js
xvfb-run -a coa render ./release-demo --timeline main \
--resolution "$RES" --fps 30 --out "$out"
python3 check_export.py "$out" \
--width "$W" --height "$H" --fps 30 --duration 20 --require-audio
Free exports are 720p, so on that plan set RES=720p, W=1280 and H=720. Swapping the render for a stub that copies a 3 second clip makes the gate fail on size, frames and duration, and the shell exits with 1. With the smoothed scene in place, the job stops at the seek test before any render starts.
The gate compares properties and never hashes. Encoded files can differ across engines, hardware and platforms, so a checksum would fail for reasons nobody cares about.
Two ffmpeg filters report things the gate does not look at. They are diagnostics, not pass or fail checks, because turning their numbers into a verdict needs a rule that depends on the project. The first reports black frames.
$ ffmpeg -i blackstart.mp4 -vf blackdetect=d=0.5:pix_th=0.10 -an -f null -
[blackdetect @ ...] black_start:0 black_end:1 black_duration:1
That clip opens with one second of black, and the filter reports it from second 0 to second 1. The second filter measures loudness.
$ ffmpeg -nostats -i good.mp4 -vn -af ebur128 -f null -
Integrated loudness:
I: -21.8 LUFS
That is a test tone. A silent track reads -70.0 LUFS, but -70.0 acts as a floor. The same tone attenuated by 40 dB reads -61.8 LUFS, and attenuated by 55 dB or more it reads -70.0. A fixed threshold would suit some content and wrongly flag other content, so choose one per project and try it on a known quiet clip before enforcing it. That is why the gate leaves loudness out.
Neither filter reads what the video says. A headline can contradict the narration, dialogue can clip, and small text can be unreadable, and all of that passes every check above. After the gate passes I watch the full MP4 with sound at the size viewers will use.
The same numbers appear in the agent's rule file and the CI script. A spec.json beside the project could hold them once. The gate would need a small change to read it, after which a change in delivery size is a single edit.
CoAnimator's agent-compatible video studio guide covers the project layout and a seek playground, and the CLI reference lists every coa render flag.