{"slug": "how-to-test-a-video-your-ai-agent-rendered-from-seek-bugs-to-the-final-mp4", "title": "How to test a video your AI agent rendered: From seek bugs to the final MP4", "summary": "A developer published a three-part method for validating video rendered by an AI agent: making every frame a pure function of time so seek and scrub operations are deterministic, gating each export against an explicit delivery spec with ffprobe, and wiring both checks into the agent's instructions and CI so a bad render cannot be reported as done. In a demonstration, a scene that eased a marker toward its target using state carried between calls rendered the marker at 11.2% on a fresh page at second 4 but at roughly 62.5% after playing the full 10 seconds and seeking back, while a pure-function rewrite passed a Node seek test that sweeps backwards and forwards across fixed timestamps and compares each result against a fresh scene.", "body_md": "An AI agent can edit a video project from one prompt and still hand back a bad result. A scene can look right when it plays from zero and wrong when someone scrubs back. An MP4 can export at the wrong size, with no audio track, or a second short. Both are cheap to catch before a person watches anything.\n\nThe fix has three parts. Make every frame a pure function of time and test the render path, check each export against an explicit delivery spec with ffprobe, and run both checks through the agent's instructions and CI so a bad render cannot be reported as done. Below is the code for each. The Node seek test, the Python gate and the CI script were run on generated clips and stub scenes. The render step uses CoAnimator's `coa` CLI (docs version 0.7.2). The seek rules and the gate work with any renderer that writes an MP4.\n\nSeeking is the first thing I test, because everything else builds on it. A preview plays from zero. An exporter that starts at second 3, or a reviewer who drags the playhead backwards, lands on a state the animation never visited if the scene keeps history between calls.\n\nSmoothing is one common way that happens. This scene eases the marker toward its target instead of jumping there.\n\n``` js\nexport function createScene() {\n  let x = 6; // lives between calls\n  return function render(t, els) {\n    const target = 6 + t * 6.5;\n    x += (target - x) * 0.2; // eases toward the target, so it depends on the last call\n    els.marker.style.left = `${x}%`;\n  };\n}\n```\n\nOn a fresh page, rendering second 4 puts the marker at 11.2%. After playing the full 10 seconds at 30 fps and then seeking to second 4, it sits at about 62.5%. Playback looks smooth, while the exporter or a scrubbing reviewer gets a different frame.\n\nThe fix is to compute everything from `t`.\n\n``` js\nconst clamp = (n, lo, hi) => Math.max(lo, Math.min(hi, n));\n\nexport function frameAt(t) {\n  t = clamp(t, 0, 10);\n  const p = clamp(t / 2, 0, 1);\n  return {\n    x: 6 + t * 6.5,\n    opacity: 1 - Math.pow(1 - p, 3),\n    caption: t < 3 ? 'Import' : t < 7 ? 'Map columns' : 'Review',\n  };\n}\n\nexport const createScene = () => (t, els) => {\n  const f = frameAt(t);\n  els.marker.style.left = `${f.x}%`;\n  els.marker.style.opacity = String(f.opacity);\n  els.caption.textContent = f.caption;\n};\n```\n\nThe scene-state seek test exercises the render path, not just the math. It plays a scene to the end on one stage, then sweeps backwards and forwards through fixed timestamps on that same stage, and compares each result with a fresh scene rendered at the same time. The stage is a plain object, so the test runs in Node without a browser. It checks the values the scene writes, which are marker position, opacity and caption.\n\n``` js\nconst { createScene } = await import(process.argv[2]);\n\nconst stage = () => ({ marker: { style: {} }, caption: {} });\nconst snap = (s) => JSON.stringify([s.marker.style.left, s.marker.style.opacity, s.caption.textContent]);\nconst freshAt = (t) => {\n  const s = stage();\n  createScene()(t, s);\n  return snap(s);\n};\n\n// Play to the end on one stage, then scrub backwards and forwards on that same stage\nconst played = stage();\nconst render = createScene();\nfor (let i = 0; i <= 300; i++) render(i / 30, played);\n\nconst times = [];\nfor (let t = 10; t >= 0; t -= 0.25) times.push(t); // backwards sweep\nfor (let t = 0; t <= 10; t += 0.7) times.push(t); // forwards sweep, off the grid\n\nfor (const t of times) {\n  render(t, played);\n  if (snap(played) !== freshAt(t)) {\n    console.error(`mismatch at ${t.toFixed(2)}s\\n  scrubbed ${snap(played)}\\n  fresh    ${freshAt(t)}`);\n    process.exit(1);\n  }\n}\nconsole.log(`seek ok, ${times.length} timestamps`);\n```\n\nSaved as `scene-smoothed.js` and `scene-pure.js`, the two scenes give these results. The sweep is fixed, so the failure repeats on every run. The smoothed scene only sets `left`, so the other two values print as null.\n\n``` bash\n$ node seek.test.mjs ./scene-smoothed.js\nmismatch at 10.00s\n  scrubbed [\"70.30666666666669%\",null,null]\n  fresh    [\"19%\",null,null]\n\n$ node seek.test.mjs ./scene-pure.js\nseek ok, 56 timestamps\n```\n\nA headless browser that compares screenshots at the same timestamps would also cover pixels and CSS. This test is the quick layer that runs without one. It assumes the scene logic sits in a module that the page and the test both import. If your renderer needs a single HTML file, keep the module as the source and inline it at build time.\n\nThree things live outside the scene code, so check for them in review.\n\n`t`.\nThe same rule applies on page load, where a skipped render shows up as a flash of the wrong frame. An export worker can open the animation at a later time through a `?t=` parameter, and the page has to paint that frame before anything is visible. Here the stage starts hidden with `#stage { visibility: hidden; }` in the page's CSS, the requested frame is drawn immediately, and the stage is revealed once fonts and images are ready.\n\n``` js\nimport { createScene } from './scene.js';\n\nconst duration = 10;\nconst stage = document.querySelector('#stage');\nconst els = {\n  marker: document.querySelector('#marker'),\n  caption: document.querySelector('#caption'),\n};\nconst scene = createScene();\nconst startAt = Math.min(\n  Math.max(Number(new URLSearchParams(location.search).get('t')) || 0, 0),\n  duration\n);\n\nscene(startAt, els);\n\nawait Promise.all([\n  document.fonts.load('600 32px \"Inter\"'), // one entry per font the scenes use\n  ...[...document.images].map((img) => img.decode().catch(() => {})),\n]);\n\nscene(startAt, els);\nstage.style.visibility = 'visible';\n```\n\nI use `document.fonts.load()` here instead of `document.fonts.ready`. Per MDN, `ready` resolves once the document has finished loading the fonts it needs. Fonts load when text that uses them is rendered, so a font used only by a hidden scene may not have been requested yet. Replace the example font with each font your scenes use.\n\nCoAnimator animations also expose `window.__kcControl` with `seek`, `seekAndPause`, `play`, `pause`, `getTime` and `getDuration`. I route `seek` and `seekAndPause` into the same render function, so preview and export share one code path. The full player contract is in the timeline reference that ships with the app.\n\nWith seeking covered, I render only the scene that changed after each edit.\n\n```\ncoa render ./release-demo --timeline main \\\n  --start 3 --end 7 --fps 30 --out ./import-scene-review.mp4\n```\n\nThe end is exclusive, so 3 to 7 at 30 fps is 120 frames. `coa` exits with 0 on success and 1 on failure. It also checks dimensions, frame count, fps, duration and expected audio before it replaces the destination, and a failed run leaves any existing file in place. An old good MP4 can therefore remain at the path after a failed render, so I write every render to a unique name and let a nonzero exit stop the script.\n\nThat check uses the tool's own rules. A second one should use mine.\n\nThe gate takes the delivery spec as arguments, so it fails when a file is valid for the renderer and wrong for the brief. It checks size, frame rate, decoded frame count, duration and, on request, an audio stream.\n\n``` bash\n#!/usr/bin/env python3\n\"\"\"Fail the build when a rendered MP4 does not match the spec in the arguments.\"\"\"\nimport argparse, json, subprocess, sys\nfrom fractions import Fraction\n\np = argparse.ArgumentParser()\np.add_argument(\"file\")\np.add_argument(\"--width\", type=int, required=True)\np.add_argument(\"--height\", type=int, required=True)\np.add_argument(\"--fps\", type=float, required=True)\np.add_argument(\"--duration\", type=float, required=True)\np.add_argument(\"--tolerance\", type=float, default=0.1, help=\"seconds\")\np.add_argument(\"--frame-tolerance\", type=int, default=1, help=\"frames\")\np.add_argument(\"--no-count\", action=\"store_true\", help=\"skip decoding, skip the frame check\")\np.add_argument(\"--require-audio\", action=\"store_true\")\na = p.parse_args()\n\ndef number(value):\n    \"\"\"Parse '30000/1001' or '12.5'. Return None for 0/0, N/A or missing.\"\"\"\n    try:\n        return float(Fraction(value))\n    except (TypeError, ValueError, ZeroDivisionError):\n        return None\n\ncmd = [\"ffprobe\", \"-v\", \"error\"] + ([] if a.no_count else [\"-count_frames\"]) + [\n    \"-show_streams\", \"-show_format\", \"-of\", \"json\", a.file]\nr = subprocess.run(cmd, capture_output=True, text=True)\nif r.returncode != 0:\n    sys.exit(f\"FAIL ffprobe cannot read {a.file}: {r.stderr.strip()}\")\n\ninfo = json.loads(r.stdout)\nstreams = info.get(\"streams\", [])\nvideo = next((s for s in streams if s.get(\"codec_type\") == \"video\"\n              and not s.get(\"disposition\", {}).get(\"attached_pic\")), None)\naudio = next((s for s in streams if s.get(\"codec_type\") == \"audio\"), None)\nproblems = []\n\nif video is None:\n    problems.append(\"no video stream\")\nelse:\n    size = (video.get(\"width\"), video.get(\"height\"))\n    if size != (a.width, a.height):\n        problems.append(f\"size {size[0]}x{size[1]}, expected {a.width}x{a.height}\")\n    fps = number(video.get(\"avg_frame_rate\"))\n    if fps is None:\n        problems.append(\"frame rate unavailable\")\n    elif abs(fps - a.fps) > 0.01:\n        problems.append(f\"fps {fps:g}, expected {a.fps:g}\")\n    if not a.no_count:\n        frames = video.get(\"nb_read_frames\", \"\")\n        expected = round(a.duration * a.fps)\n        if not frames.isdigit():\n            problems.append(\"frame count unavailable\")\n        elif abs(int(frames) - expected) > a.frame_tolerance:\n            problems.append(f\"{frames} frames, expected {expected}\")\n\nduration = number(info.get(\"format\", {}).get(\"duration\"))\nif duration is None:\n    problems.append(\"duration unavailable\")\nelif abs(duration - a.duration) > a.tolerance:\n    problems.append(f\"duration {duration:.2f}s, expected {a.duration:g}s\")\nif a.require_audio and audio is None:\n    problems.append(\"no audio stream\")\n\nif problems:\n    print(\"FAIL \" + a.file)\n    for line in problems:\n        print(\"  - \" + line)\n    sys.exit(1)\nprint(f\"OK {a.file}\")\nbash\n$ python3 check_export.py short.mp4 --width 1280 --height 720 --fps 30 --duration 4 --require-audio\nFAIL short.mp4\n  - 90 frames, expected 120\n  - duration 3.00s, expected 4s\n\n$ python3 check_export.py good.mp4 --width 1280 --height 720 --fps 30 --duration 4 --require-audio\nOK good.mp4\n```\n\nReal files are rarely exact, so two settings soften the checks. Frame counts get a one frame tolerance by default. A 29.97 fps file with 600 frames lasts about 20.02 seconds. Against a 20 second spec the expected count rounds to 599, so the default tolerance passes it and `--frame-tolerance 0` fails it. The `--no-count` flag skips the full decode that `-count_frames` needs, which suits long films, and it drops the frame check with it. The frame rate check reads ffprobe's average, so it assumes a constant frame rate.\n\nA file with no audio stream fails with `no audio stream`. A file with a silent audio track still passes, because the gate checks that a stream exists, not what it contains. A file ffprobe cannot read, such as a text file renamed to .mp4 or a truncated export, exits with 1 and ffprobe's error. The gate does not check the container, so a valid WebM passes. Add a check on `format_name` if the brief requires MP4.\n\nA check only helps if the agent runs it. The instruction file the agent reads, such as `AGENTS.md` or `CLAUDE.md`, is the place for that.\n\n```\n## Rendering\n- Write each render to `renders/` with a timestamp in the name. Never overwrite an earlier file.\n- After every render, run `python3 check_export.py <file> --width 1920 --height 1080 --fps 30 --duration 20 --require-audio`.\n- If it prints FAIL, fix the cause and render again. Do not call the video done until it prints OK.\n- After changing scene code, run `node seek.test.mjs ./scene.js`.\n```\n\nThe FAIL lines say exactly what is off, such as `90 frames, expected 120`, which gives the agent something concrete to act on. An instruction can still be skipped, so the same checks run in CI.\n\nOn a headless Linux runner the app still needs a display server, so the render goes through `xvfb-run`. The runner also needs CoAnimator, FFmpeg and FFprobe installed, because version 0.7.2 has no hosted rendering workers.\n\n```\nset -eu\nRES=1080p W=1920 H=1080\nmkdir -p renders\nout=\"renders/review-$(date -u +%Y%m%dT%H%M%SZ).mp4\"\n\nnode seek.test.mjs ./scene.js\n\nxvfb-run -a coa render ./release-demo --timeline main \\\n  --resolution \"$RES\" --fps 30 --out \"$out\"\n\npython3 check_export.py \"$out\" \\\n  --width \"$W\" --height \"$H\" --fps 30 --duration 20 --require-audio\n```\n\nFree exports are 720p, so on that plan set `RES=720p`, `W=1280` and `H=720`. Swapping the render for a stub that copies a 3 second clip makes the gate fail on size, frames and duration, and the shell exits with 1. With the smoothed scene in place, the job stops at the seek test before any render starts.\n\nThe gate compares properties and never hashes. Encoded files can differ across engines, hardware and platforms, so a checksum would fail for reasons nobody cares about.\n\nTwo ffmpeg filters report things the gate does not look at. They are diagnostics, not pass or fail checks, because turning their numbers into a verdict needs a rule that depends on the project. The first reports black frames.\n\n``` bash\n$ ffmpeg -i blackstart.mp4 -vf blackdetect=d=0.5:pix_th=0.10 -an -f null -\n[blackdetect @ ...] black_start:0 black_end:1 black_duration:1\n```\n\nThat clip opens with one second of black, and the filter reports it from second 0 to second 1. The second filter measures loudness.\n\n``` bash\n$ ffmpeg -nostats -i good.mp4 -vn -af ebur128 -f null -\n  Integrated loudness:\n    I: -21.8 LUFS\n```\n\nThat is a test tone. A silent track reads -70.0 LUFS, but -70.0 acts as a floor. The same tone attenuated by 40 dB reads -61.8 LUFS, and attenuated by 55 dB or more it reads -70.0. A fixed threshold would suit some content and wrongly flag other content, so choose one per project and try it on a known quiet clip before enforcing it. That is why the gate leaves loudness out.\n\nNeither filter reads what the video says. A headline can contradict the narration, dialogue can clip, and small text can be unreadable, and all of that passes every check above. After the gate passes I watch the full MP4 with sound at the size viewers will use.\n\nThe same numbers appear in the agent's rule file and the CI script. A `spec.json` beside the project could hold them once. The gate would need a small change to read it, after which a change in delivery size is a single edit.\n\nCoAnimator's [agent-compatible video studio guide](https://coanimator.com/blog/agent-compatible-video-studio) covers the project layout and a seek playground, and the [CLI reference](https://coanimator.com/docs/cli) lists every `coa render` flag.", "url": "https://wpnews.pro/news/how-to-test-a-video-your-ai-agent-rendered-from-seek-bugs-to-the-final-mp4", "canonical_source": "https://dev.to/sheen_417f0f3a7f4/how-to-test-a-video-your-ai-agent-rendered-from-seek-bugs-to-the-final-mp4-1d60", "published_at": "2026-11-16 06:10:51+00:00", "updated_at": "2026-11-16 06:21:13.898520+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "generative-ai"], "entities": ["CoAnimator", "ffprobe", "Node.js", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-test-a-video-your-ai-agent-rendered-from-seek-bugs-to-the-final-mp4", "markdown": "https://wpnews.pro/news/how-to-test-a-video-your-ai-agent-rendered-from-seek-bugs-to-the-final-mp4.md", "text": "https://wpnews.pro/news/how-to-test-a-video-your-ai-agent-rendered-from-seek-bugs-to-the-final-mp4.txt", "jsonld": "https://wpnews.pro/news/how-to-test-a-video-your-ai-agent-rendered-from-seek-bugs-to-the-final-mp4.jsonld"}}