# The 14 Open-Source Repos Powering Code-Driven Video in 2026: How We Built 3D AJ the Cat > Source: > Published: 2026-11-04 05:00:00+00:00 [AI & MCP Agents](https://codingcat.dev/topic/ai) # The 14 Open-Source Repos Powering Code-Driven Video in 2026: How We Built 3D AJ the Cat Every AI engineering timeline this month is flooded with launch films, kinetic typography reels, and 3D product videos captioned `"one prompt in Opus 5.5"`. If you actually believed a 20-word vibe prompt inside a standard chat window produced those 60fps showreels, you probably tried it yourself and got a centered title over a purple gradient with drifting CSS animations. A recent viral breakdown by `@0xnicc0` and `@seeconvm` pulled back the curtain on what those viral videos are actually hiding: **foundation models cannot output an MP4 file directly**. Every viral code-rendered film is a deterministic program written by an agent with shell access, driven by a strict `window.seek(t)` frame contract, closed-form spring physics, beat-locked audio (`beats.json`), and a contact-sheet critique loop powered by **14 free open-source repositories**. On **CodingCat.dev**, we took that exact 14-repo stack and wired it directly into our production site and local RTX 5090 video studio: 1. We built our canonical **3D Vinyl-Toy AJ the Cat GLB model** (`aj-3d-mascot.glb` ) using**Three.js** ,**Blender** , and**Blender MCP** (`ahujasid/mcp-for-blender` ), complete with a live**2D Vector vs. 3D GLB toggle** on[`/mascot-lab`](https://codingcat.dev/mascot-lab) and our homepage Hero. 2. We upgraded AJ's runtime (`AjAnimatedMascot.astro` and`Aj3dMascotViewer.astro` ) with a shared**deterministic `window.seek(t)` motion engine** (`src/lib/aj-motion-engine.ts` ) using**closed-form harmonic springs** and**seeded noise** . 3. We wired **120 BPM audio-reactive lip-sync** into our**Podcasts** (`/podcasts` and`/podcast/[slug]` ) and automated our 1080p**Course Factory** with**faster-whisper** ,**FFmpeg** , and headless**Chrome DevTools MCP** . Let's break down the mental model behind deterministic code-driven video, see the Architecture Diagrams, rank all 14 repositories in an **Alex Patterson Architectural Tier List**, and walk through the exact TypeScript and Python code powering 3D AJ. ## 1. The Mental Model: Why window.seek(t) Beats Non-Deterministic Timers When you ask an LLM to write a web animation without a render contract, it defaults to `requestAnimationFrame`, `setTimeout`, CSS transitions, and `Math.random()`. That works fine for a casual decorative hover effect, but it falls apart the moment a headless browser tries to capture 900 frames for a 15-second 60fps video: - If frame 240 takes 45ms to paint instead of 16.6ms, timer-based animations drift out of sync with the audio track. - If a headline lands 10 frames too early at second `4.0` , you cannot jump straight to frame 240 to inspect it without re-simulating frames`0` through`239` . - If particle positions use `Math.random()` , rendering frame 300 twice produces two different image hashes, breaking loop continuity and automated visual diffs. The solution is a strict **Frame Function Contract** enforced via your agent rules (`CLAUDE.md` or `GEMINI.md`): 1. **`window.seek(t)` Paints Frame `t`** : Nothing else drives the screen. Passing` window.seek(4.0)` renders the exact visual state at`4.000` seconds deterministically. 2. **Closed-Form Springs (`spring(t - t_i)`)** : Instead of accumulating velocity state frame-by-frame in a physics loop, every movement is evaluated as an analytical function of elapsed time since impulse`t_i` . 3. **Seeded Hash Noise Only** : Replace`Math.random()` with a deterministic 32-bit integer hash (`seededNoise1D(seed, t)` ) so frame`300` hashes identically on every run. 4. **The FFmpeg Contact-Sheet Critic** : Before the agent shows you an MP4, it uses**FFmpeg** to extract a 2-fps contact sheet and a 12-frame strip around the fastest motion, inspects its own rendered frames via**Chrome DevTools MCP** , scores them from 1 to 10, and fixes the worst 3 defects automatically. | Architectural Piece | What Goes Wrong Without It | What Fixes It in Production | |---|---|---| | Frame Function | Renders drift under GPU load; every edit requires replaying from t = 0. | window.seek(t) + pure analytical functions of time + zero CSS timers. | | Closed-Form Springs | Fixed linear/cubic-bezier easing feels robotic; iterative physics breaks random frame access. | track(t) = v0 + Σ (v_i - v_(i-1)) * spring(t - t_i) with analytical damping. | | The Agent Rig | Chat-only models never see or hear their own broken layouts or overlapping text. | Agent shell + Headless Chromium / Chrome DevTools MCP + FFmpeg. | | Audio & Beat Clock | Voiceovers and music float on top of visuals with off-beat state transitions. | Chatterbox / F5-TTS + faster-whisper word timestamps + librosa beats.json. | | The Contact-Sheet Critic | The first render ships with clipped text, dead beats, or loop stutters. | 6-across FFmpeg contact sheet + 360px mobile readability test + automated self-critique. | ## 2. Alex Patterson's Architectural Tier List: All 14 Code-Driven Video Repos Ranked Instead of treating all 14 repositories from `@0xnicc0`'s list as interchangeable, here is how they rank in our production **CodingCat.dev** stack based on determinism, GPU efficiency, MCP agent ergonomics, and how heavily we rely on them to build **3D AJ** and our **Courses**: ### Alex's 2026 Code-Driven Video & 3D Web Tier List (All 14 Repos Ranked) Ranked from S-Tier to C-Tier across deterministic window.seek(t) support, GPU performance, MCP agent integration, and how we use them for 3D AJ. #### Three.js (mrdoob/three.js) Browser 3D & WebGPU The undisputed king of browser 3D. Zero plugin install for users, native glTF 2.0 .glb support, and instant seek(t) scrubbing. #### FFmpeg (FFmpeg/FFmpeg) Media Compositor & Critic Non-negotiable infrastructure. Every frame pipeline, audio normalizer, and contact-sheet critic ends in FFmpeg. #### faster-whisper (SYSTRAN/faster-whisper) Word-Level Audio Alignment 4x faster than stock Whisper with lower VRAM via CTranslate2. Gives agents an exact millisecond clock for every spoken word. #### Blender MCP (ahujasid/mcp-for-blender) Agentic 3D Scene Control Turns Blender from a manual GUI into an agent-programmable 3D compiler. Essential for automated mesh inspection and rigging. #### Blender (blender/blender) 3D Modeling, Rigging & Baking The open-source 3D standard. Combined with Python bpy, it eliminates manual 3D artist bottlenecks. #### Remotion (remotion-dev/remotion) React Video Composition Best-in-class DX when your team already builds in React and Tailwind CSS. Deterministic frame rendering out of the box. #### Remotion Skills (remotion-dev/skills) Agent Procedural Memory Crucial context upgrade. Without this skill, LLMs write brittle React effects instead of pure useCurrentFrame() math. #### Chrome DevTools MCP (ChromeDevTools/chrome-devtools-mcp) Live DOM & Frame Inspection Closes the blind-agent gap. Your agent sees the exact DOM, computed styles, and pixel output in real time. #### GSAP (greensock/GSAP) Timeline & UI Motion Battle-tested timeline control. Just remember to pause auto-ticking and drive tl.time(t) from window.seek(t) in render mode. #### HyperFrames (heygen-com/hyperframes) Agent-Native HTML-to-Video Purpose-built for coding agents that prefer writing raw HTML/Canvas over full React bundles. Fast and lightweight. #### Chatterbox (resemble-ai/chatterbox) Open-Source Voice Synthesis Impressive open-source expressive voice cloning with emotion control. Great local alternative to paid ElevenLabs seats. #### ComfyUI (Comfy-Org/ComfyUI) Node-Based Diffusion & 3D Gen Unmatched local GPU control for image-to-3D plates and texture maps, though JSON workflow graphs require disciplined versioning. #### Impeccable (pbakaus/impeccable) Automated UI & Design Critic Great automated design gate to stop agents from shipping generic purple-gradient SaaS clichés. #### yt-dlp (yt-dlp/yt-dlp) Reference Media Ingestion Indispensable utility for reference grammar extraction, though strictly an ingestion CLI rather than a rendering engine. ## 3. Deep Dive: How We Built and Upgraded 3D AJ the Cat Across CodingCat.dev Before this upgrade, AJ the Cat lived in two separate worlds on **CodingCat.dev**: 1. **Static 3D Turnaround Renders** : High-resolution Pixar-style vinyl-toy studio JPEGs (`assets/AJ-3D-Headphones-Front.jpg` ,`Left.jpg` ,`Right.jpg` ) used in YouTube thumbnails and video bumpers. 2. **2D Articulated Vector SVG (`AjAnimatedMascot.astro`)** : A 57-path inline SVG rig with 14 bone pivot groups (`#aj-head-group` ,`#aj-arm-right` ,`#aj-tail` , etc.) driven by CSS`@keyframes` and random`setTimeout` blinks. While the 2D SVG rig had zero external JavaScript dependencies, its CSS `@keyframes` and `Math.random()` blink timers meant **it was not frame-seekable** for headless video rendering, and visitors could not orbit or inspect a real 3D mesh in the browser. Here is the exact 4-stage architecture we implemented to unify **2D Vector AJ** and **3D GLB AJ** under a single deterministic motion engine. ### Stage 1: Programmatic 3D Vinyl-Toy GLB Generation (scripts/build-aj-3d-glb.mjs) & Blender MCP Verification To ensure 3D AJ matches every single anatomical law in our Design System (`09-assets.md`) while keeping the binary payload compact (` 756 KB`) and 100% deterministic, we built `scripts/build-aj-3d-glb.mjs` and paired it with our headless **Blender MCP** verification script (`scripts/blender-aj-3d-pipeline.py`). Every anatomical feature of AJ is mapped to a named glTF 2.0 node so both **Blender** (`bpy.data.objects`) and **Three.js** (` gltf.scene.getObjectByName`) can target the exact same bones: - `AjRoot` ->`AjPedestal` +`AjBodyRoot` - `AjTorso` (Matte-black`#0B0B0E` PBR fur with roughness`0.62` + dark charcoal`#2D2D30` oval belly patch) - `AjCollar` +`AjHexTag` (Soft Lavender`#B4A7D6` collar with hexagonal gold & Royal Purple`"AJ"` tag) - `AjHeadGroup` ->`AjEarLeft` ,`AjEarRight` ,`AjWhiskersLeft` ,`AjWhiskersRight` - `AjEyeLeftPupil` &`AjEyeRightPupil` (Electric Lime Green`#C5E86C` eyes with vertical black pupils and dual white catchlights) - `AjJawMouth` (Articulated jaw and pink tongue viseme group for 120 BPM audio-reactive lip-sync) - `AjHeadphonesGroup` (`CodingCat.dev Podcast` charcoal headband + Royal Purple`#7C3AED` & Soft Lavender`#B4A7D6` earcups fitting directly over his cat ears) ### Stage 2: Closed-Form Springs & Deterministic window.seek(t) (src/lib/aj-motion-engine.ts) In `@0xnicc0` and `@seeconvm`'s guide, the secret to lifelike motion that remains 100% seekable at any arbitrary frame `t` is **stacking closed-form harmonic springs**: Because `closedFormSpring(dt, omega, zeta)` is a pure analytical equation using `Math.exp(-zeta * omega * dt)` and `Math.cos(omegaD * dt)`, evaluating frame `812` (`t = 13.533s`) requires **zero knowledge of frames `0` through `811`**. Here is the production motion engine we shipped in `apps/site/src/lib/aj-motion-engine.ts`: Both `` (2D SVG) and `` (3D Three.js GLB) subscribe to `window.seek(t)` (`cc:seek` custom event) and call `computeAjPoseAtTime(t, state)`. You can test this live right now on [`/mascot-lab`](https://codingcat.dev/mascot-lab) using the **` window.seek(t)` frame scrubber** or by opening DevTools and running `window.seek(4.25)`. ### Stage 3: Zero-LCP-Penalty 3D Three.js Viewer (Aj3dMascotViewer.astro) A common mistake when adding **Three.js** to a content site is mounting a heavy WebGL canvas above the fold that blocks Largest Contentful Paint (LCP). On **CodingCat.dev**, we kept our zero-JS 57-path 2D SVG rig as the instant-paint SSR default and added an interactive **2D SVG / 3D GLB toggle** in the Homepage Hero (`AnimatedHero.astro`) and on [`/mascot-lab`](https://codingcat.dev/mascot-lab). When you switch to `3D GLB`, `Aj3dMascotViewer.astro` loads `/models/aj-3d-mascot.glb` (`756 KB`), lights AJ with ACES Filmic tone mapping plus Royal Purple (`#7C3AED`) and Electric Lime (`#C5E86C`) spot rim lights, and binds cursor coordinates to `AjEyeLeftPupil` and `AjEyeRightPupil`. ### Stage 4: 120 BPM Audio-Reactive Lip-Sync on Podcast Pages & Video Studio In Section 6 of the Opus 5.5 Motion Design guide (*"Springs and Sound, or Why Cheap Motion Feels Cheap"*), sound and picture must share the exact same clock. We wired `PodcastListen.astro` and `/podcasts` to `window.setAjAudioAmplitude(amp, bpm)`: - Whenever you play an episode in `PodcastListen.astro` (or click**Test AJ 120 BPM Beat Sync** on`/podcasts` ), AJ's jaw viseme (`#aj-talking-frame` in 2D SVG and`AjJawMouth` in 3D GLB), head nod, and ear bounce lock to the audio envelope at 120 BPM. - In our local RTX 5090 `tools/video-studio/synthesize-voiceover.py` pipeline,**faster-whisper** extracts word-level timestamps into`storyboard.json` so every rendered Course Lesson video aligns captions, diagram transitions, and AJ reactions to the millisecond. **AJ Pro-Tip Callout: Never Use `Math.random()` or Unpaused Timelines in Headless Video Renders** When capturing frames with Playwright or Headless Chrome, even a single `Math.random()` inside a blink timer or particle emitter will cause a visual stutter at your video's loop point. Always derive blinks and noise from `hashUint32(seed + Math.floor(t * 60))` and drive every animation from `window.seek(t)`. ## 4. Full Course Roadmap: 5 Flagship Stack Courses + 14 Deep-Dive Tool Courses To help you master this entire code-driven media and 3D web stack without paying `$1,200` per launch video, we have prepped **19 new Courses** (5 Flagship Architectural Stack Courses + 14 dedicated Tool-by-Tool Courses, totaling **95 Lessons**) in the **CodingCat.dev** Course Factory (`/courses`): ### The 5 Flagship Stack Courses 1. **Flagship 1: Programmatic Video with React, Remotion & Agent Skills** (`Remotion` +`Remotion Skills` ) 2. **Flagship 2: Agentic 3D Web Pipelines with Blender, Blender MCP & Three.js** (`Blender` +`Blender MCP` +`Three.js` ) 3. **Flagship 3: Deterministic HTML Motion & Visual Critique with HyperFrames, GSAP, Impeccable & Chrome DevTools MCP** (`HyperFrames` +`GSAP` +`Impeccable` +`Chrome DevTools MCP` ) 4. **Flagship 4: Local GPU Audio & Diffusion Pipelines with Chatterbox, faster-whisper & ComfyUI** (`Chatterbox` +`faster-whisper` +`ComfyUI` ) 5. **Flagship 5: Automated Media Engineering & Reference Extraction with FFmpeg & yt-dlp** (`FFmpeg` +`yt-dlp` ) ### The 14 Dedicated Tool-by-Tool Courses (1 Per GitHub Repo) | Course # | Course Title & Slug (/course/[slug]) | Primary GitHub Repo | Core Production Deliverable | |---|---|---|---| | 01 | Remotion in Production: React Video Composition (/course/remotion-react-video-production) | remotion-dev/remotion | Multi-aspect-ratio 60fps launch video rendered from React + Tailwind CSS. | | 02 | Remotion Skills for AI Coding Agents (/course/remotion-skills-ai-agents) | remotion-dev/skills | Custom SKILL.md harness that teaches Claude Code & Antigravity deterministic video rules. | | 03 | Blender 5.0 for Web Developers: Headless Python & GLB (/course/blender-headless-python-web-devs) | blender/blender | Automated bpy script that builds, lights, and exports a <1MB .glb mascot. | | 04 | Blender MCP: Driving 3D Scenes with AI Agents (/course/blender-mcp-agentic-3d-scenes) | ahujasid/mcp-for-blender | Live TCP socket MCP server letting an agent model, rig, and render 3D AJ. | | 05 | Three.js & WebGPU in Production: Interactive 3D Mascots (/course/threejs-webgpu-3d-mascots) | mrdoob/three.js | Lazy-loaded with orbital camera, rim lights, and seek(t). | | 06 | GSAP Deterministic Timelines & SVG Morphing (/course/gsap-deterministic-timelines-svg) | greensock/GSAP | Frame-locked tl.time(t) UI morphs and 57-path SVG bone animations. | | 07 | HyperFrames: Agent-Native HTML-to-Video Rendering (/course/hyperframes-html-to-video-agents) | heygen-com/hyperframes | Zero-React standalone HTML/Canvas video compiler driven by window.seek(t). | | 08 | faster-whisper on Local GPU: Word-Level Captions & Lip-Sync (/course/faster-whisper-word-level-captions) | SYSTRAN/faster-whisper | Sub-20ms word alignment pipeline driving dynamic captions and mascot visemes. | | 09 | Chatterbox TTS: Zero-Shot Local Voice Cloning (/course/chatterbox-local-voice-cloning) | resemble-ai/chatterbox | Expressive local voiceover synthesizer with phonetic code identifier rules. | | 10 | FFmpeg for AI Video Pipelines & Contact-Sheet Critics (/course/ffmpeg-ai-video-contact-sheet-critics) | FFmpeg/FFmpeg | -14 LUFS EBU R128 audio normalization, 6-across contact sheets, and H.264/AV1 mastering. | | 11 | Chrome DevTools MCP: Self-Healing Visual Agent Loops (/course/chrome-devtools-mcp-visual-agent-loops) | ChromeDevTools/chrome-devtools-mcp | Automated DOM/WebGL inspection loop that catches overlapping text before export. | | 12 | Impeccable: Automated Design & Motion Polish Gates (/course/impeccable-automated-design-critique) | pbakaus/impeccable | Pre-export design critic enforcing W3C design tokens and banning generic AI gradients. | | 13 | ComfyUI for 3D Turnarounds & Generate-Then-Trace (/course/comfyui-3d-turnarounds-video-workflows) | Comfy-Org/ComfyUI | Reproducible API workflow for multi-view mascot sheets and base motion plates. | | 14 | yt-dlp & Librosa: Reference Grammar & Beat Extraction (/course/yt-dlp-librosa-reference-beat-extraction) | yt-dlp/yt-dlp | Automated reference shotlist extractor (style_guide.md) and 120 BPM beats.json generator. | ## 5. Verdict: Purrfect or Pass? - **Purrfect For** : Developer educators, SaaS founders, and frontend engineers who want agency-grade launch films, interactive 3D mascots (like**3D AJ** ), and automated video courses rendered deterministically from Git-versioned TypeScript and Python code for`$0` in licensing fees. - **Pass If** : You need photorealistic live-action human cinema with complex cloth physics (where diffusion video models like Veo 3.1 still belong) or you are unwilling to write a structured`brief.md` and`window.seek(t)` render contract. Head over to [`/mascot-lab`](https://codingcat.dev/mascot-lab) right now to toggle between **2D Vector AJ** and **3D Three.js GLB AJ**, scrub the **`window.seek(t)`** timeline, download `aj-3d-mascot.glb`, and explore all 19 new Courses on [`/courses`](https://codingcat.dev/courses)! ## how this post connects together. Follow AJ from first principles to production-ready architecture. ## Watch the Related Video Deep Dive: Cloudflare Agents, Durable Objects & Stateful Workflows in TypeScript Build stateful, long-running AI agents and resilient multi-step workflows using the Cloudflare Agents SDK, Durable Objects with embedded SQLite, WebSockets hibernation, and Workflows. [View Full Page →](https://codingcat.dev/course/cloudflare-agents-durable-workflows-typescript)