{"slug": "this-video-is-about-how-this-video-was-made", "title": "This video is about how this video was made", "summary": "A developer used Claude Opus 5.5 inside Claude Code to build a self-documenting music video entirely in code, starting from the open-source PDoomVideo project. The pipeline renders every frame as a pure function of time t, drives four headless Chrome workers through a shared frame queue, and aligns narration and lyrics using ElevenLabs character-level timestamps, Whisper large-v3-turbo, and difflib alignment. The build also documents a reviewer agent scoring an early draft frame 4/10 and a portability fix swapping Chrome's Windows-only --use-angle=d3d11 flag for --use-angle=metal on macOS.", "body_md": "This video (the script, the voice timings, the source code, the storyboard, the briefs the subagents got, the grade a reviewer gave an earlier draft) is the real artifact from making *this video*.\n\nClaude Opus 5.5 built it in Claude Code, starting from an open-source project, and this post walks through the innards.\n\nIt started from [JohnHeibel/PDoomVideo](https://github.com/JohnHeibel/PDoomVideo), the source of a music video Opus 5.5 made entirely in code: p5.js, a studio page, a `render.mjs` that drives headless Chrome, and a `STORYBOARD.md` written before any chapter code.\n\nThe core idea is one rule: **every frame is a pure function of `t`**.\n\n``` js\nwindow.renderAt = async (t, type = 'image/png', q = .92) => { T = t; await redraw(); composite(t); return outC.toDataURL(type, q); };\n```\n\nNo `Math.random()`, no clock, no state carried between frames. Randomness comes from a seeded hash, the old GLSL one-liner:\n\n``` js\nconst hash = n => { const s = Math.sin(n * 127.1 + 311.7) * 43758.5453; return s - Math.floor(s); };\n```\n\nThat one constraint buys everything else: frames can render out of order, on several browsers at once, resumably. Any single frame can be regenerated on demand, which is what lets the video show *itself* at frame 1,365 while it's on frame 2,000-something.\n\nThe first real change was a portability fix. The repo launched Chrome with `--use-angle=d3d11` (Direct3D, Windows only). On a Mac that fails with \"Error creating webgl context\"; `--use-angle=metal` plus the macOS Chrome path fixed it.\n\nThe narration is `script.md`: 15 numbered lines. Words in `{braces}` are **cue words**, where a demonstration fires.\n\nEach line goes to ElevenLabs' `with-timestamps` endpoint, which returns the audio plus a start and end time **for every character**:\n\n```\n{ \"characters\": [\"T\",\"h\",\"o\",\"s\",\"e\",\" \", ...],\n  \"character_start_times_seconds\": [0.0, 0.046, 0.081, ...] }\n```\n\n`tools/voice.py` turns characters into words, finds each line's `{cue}` word, stitches the 15 clips at exact offsets, and writes `timeline.js`. The flash in the video fires on the frame where the word \"now\" is spoken:\n\n``` js\nconst cueT = n => line(n).cue ? line(n).cue.a : line(n).a;   // when line n's {cue} word is spoken\n```\n\nFor music, the same slot is filled differently. Suno's `.m4a` exports carry a hidden `mov_text` subtitle stream with every lyric line timestamped (`ffmpeg -map 0:s:0 subs.srt`); Whisper large-v3-turbo adds word times; `difflib.SequenceMatcher` aligns what Whisper heard to the real lyrics; and a comb search over the onset envelope finds the beat grid.\n\nBefore any chapter code, Opus wrote `STORYBOARD.md` (one row per script line: the shot and the demonstration on the cue word) and `ANIMATION_GUIDE.md`, the contract every builder follows: the API, the \"every example is itself\" rule, legibility, and the hand-off frames between chapters.\n\nThen it handed three chapters to three subagents in parallel, each with a short brief. The brief for chapter B is shown *in chapter B*. Each subagent verified its own work with contact sheets and reported back; those reports are in the repo too.\n\nA reviewer agent graded draft 1. The specific frame line 11 talks about got **4/10**: \"the side being graded is a placeholder, so the frame promises a review it can't show yet.\" It also caught that the HUD said `76.033 s` when the target was `76.008 s`. Those notes went back to the builders, and the final video shows the real draft-1 frame beside the fixed one, annotated with those exact notes.\n\nPartway through this project, the model couldn't see images at all: every screenshot came back `(media removed — rejected by API)`. So it checked its own frames two other ways:\n\nThe video shows both, using its own frames.\n\n`render.mjs` opens four headless Chrome workers. Each pulls the next frame index from a shared queue, calls `renderAt(i / 30)`, and writes a JPEG. ffmpeg then stitches the frames to the narration with `libx264`.\n\nOne real bug from the music-video build: with a subtitle stream as an input, `ffmpeg -shortest` never finishes. The encode sat at 0% CPU for 20 minutes. The fix is an explicit `-t <duration>`.\n\nThe last shot is a DEV page with a player inside it, and on the word \"inside\" the player shows the video recursively. That isn't faked: the renderer draws its own output canvas back into the player, so it's a real Droste loop.\n\nOne honest wrinkle: the video was rendered before this post existed, so that page is labelled \"DRAFT · the real post comes later.\" You're reading the real one now.\n\n*Related reading on DEV:* [HTML/CSS Animation to Video (MP4): the Headless, Deterministic Way](https://dev.to/dsplce-co/htmlcss-animation-to-video-mp4-the-headless-deterministic-way-incl-claude-4a2) · [I asked an agent to make a product video. It wrote HTML and rendered an MP4](https://dev.to/dhseadev/i-asked-an-agent-to-make-a-product-video-it-wrote-html-and-rendered-an-mp4-dota-companion-1nia) · [How to make AI videos: production diary of a one-minute film](https://dev.to/hiper2d/how-to-make-ai-videos-production-diary-of-a-one-minute-film-450)\n\n*The video above is draft 2: 3,224 frames at 30 fps, 107.5 s. The 4/10 was the reviewer's grade on draft 1.*", "url": "https://wpnews.pro/news/this-video-is-about-how-this-video-was-made", "canonical_source": "https://dev.to/peter/this-video-is-about-how-this-video-was-made-42hl", "published_at": "2026-09-24 19:54:35+00:00", "updated_at": "2026-09-24 20:29:19.308787+00:00", "lang": "en", "topics": ["generative-ai", "ai-agents", "ai-tools", "developer-tools"], "entities": ["Claude Opus 5.5", "Claude Code", "PDoomVideo", "ElevenLabs", "Suno", "Whisper large-v3-turbo", "ffmpeg", "p5.js"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/this-video-is-about-how-this-video-was-made", "markdown": "https://wpnews.pro/news/this-video-is-about-how-this-video-was-made.md", "text": "https://wpnews.pro/news/this-video-is-about-how-this-video-was-made.txt", "jsonld": "https://wpnews.pro/news/this-video-is-about-how-this-video-was-made.jsonld"}}