AI & MCP Agents
Every AI engineering timeline this month is flooded with launch films, kinetic typography reels, and 3D product videos captioned "one prompt in Opus 5.5". If you actually believed a 20-word vibe prompt inside a standard chat window produced those 60fps showreels, you probably tried it yourself and got a centered title over a purple gradient with drifting CSS animations.
A recent viral breakdown by @0xnicc0 and @seeconvm pulled back the curtain on what those viral videos are actually hiding: foundation models cannot output an MP4 file directly. Every viral code-rendered film is a deterministic program written by an agent with shell access, driven by a strict window.seek(t) frame contract, closed-form spring physics, beat-locked audio (beats.json), and a contact-sheet critique loop powered by 14 free open-source repositories.
On CodingCat.dev, we took that exact 14-repo stack and wired it directly into our production site and local RTX 5090 video studio:
- We built our canonical 3D Vinyl-Toy AJ the Cat GLB model (
aj-3d-mascot.glb) usingThree.js ,Blender , andBlender MCP (ahujasid/mcp-for-blender), complete with a live2D Vector vs. 3D GLB toggle on/mascot-laband our homepage Hero. - We upgraded AJ's runtime (
AjAnimatedMascot.astroandAj3dMascotViewer.astro) with a shareddeterministicwindow.seek(t)motion engine (src/lib/aj-motion-engine.ts) usingclosed-form harmonic springs andseeded noise . - We wired 120 BPM audio-reactive lip-sync into ourPodcasts (
/podcastsand/podcast/[slug]) and automated our 1080pCourse Factory withfaster-whisper ,FFmpeg , and headlessChrome DevTools MCP .
Let's break down the mental model behind deterministic code-driven video, see the Architecture Diagrams, rank all 14 repositories in an Alex Patterson Architectural Tier List, and walk through the exact TypeScript and Python code powering 3D AJ.
1. The Mental Model: Why window.seek(t) Beats Non-Deterministic Timers #
When you ask an LLM to write a web animation without a render contract, it defaults to requestAnimationFrame, setTimeout, CSS transitions, and Math.random(). That works fine for a casual decorative hover effect, but it falls apart the moment a headless browser tries to capture 900 frames for a 15-second 60fps video:
- If frame 240 takes 45ms to paint instead of 16.6ms, timer-based animations drift out of sync with the audio track.
- If a headline lands 10 frames too early at second
4.0, you cannot jump straight to frame 240 to inspect it without re-simulating frames0through239. - If particle positions use
Math.random(), rendering frame 300 twice produces two different image hashes, breaking loop continuity and automated visual diffs.
The solution is a strict Frame Function Contract enforced via your agent rules (CLAUDE.md or GEMINI.md):
window.seek(t)Paints Framet: Nothing else drives the screen. Passingwindow.seek(4.0)renders the exact visual state at4.000seconds deterministically.- Closed-Form Springs (
spring(t - t_i)) : Instead of accumulating velocity state frame-by-frame in a physics loop, every movement is evaluated as an analytical function of elapsed time since impulset_i. - Seeded Hash Noise Only : Replace
Math.random()with a deterministic 32-bit integer hash (seededNoise1D(seed, t)) so frame300hashes identically on every run. - The FFmpeg Contact-Sheet Critic : Before the agent shows you an MP4, it usesFFmpeg to extract a 2-fps contact sheet and a 12-frame strip around the fastest motion, inspects its own rendered frames viaChrome DevTools MCP , scores them from 1 to 10, and fixes the worst 3 defects automatically.
| Architectural Piece | What Goes Wrong Without It | What Fixes It in Production |
|---|---|---|
| Frame Function | Renders drift under GPU load; every edit requires replaying from t = 0. | window.seek(t) + pure analytical functions of time + zero CSS timers. |
| Closed-Form Springs | Fixed linear/cubic-bezier easing feels robotic; iterative physics breaks random frame access. | track(t) = v0 + Σ (v_i - v_(i-1)) * spring(t - t_i) with analytical damping. |
| The Agent Rig | Chat-only models never see or hear their own broken layouts or overlapping text. | Agent shell + Headless Chromium / Chrome DevTools MCP + FFmpeg. |
| Audio & Beat Clock | Voiceovers and music float on top of visuals with off-beat state transitions. | Chatterbox / F5-TTS + faster-whisper word timestamps + librosa beats.json. |
| The Contact-Sheet Critic | The first render ships with clipped text, dead beats, or loop stutters. | 6-across FFmpeg contact sheet + 360px mobile readability test + automated self-critique. |
2. Alex Patterson's Architectural Tier List: All 14 Code-Driven Video Repos Ranked #
Instead of treating all 14 repositories from @0xnicc0's list as interchangeable, here is how they rank in our production CodingCat.dev stack based on determinism, GPU efficiency, MCP agent ergonomics, and how heavily we rely on them to build 3D AJ and our Courses:
Alex's 2026 Code-Driven Video & 3D Web Tier List (All 14 Repos Ranked)
Ranked from S-Tier to C-Tier across deterministic window.seek(t) support, GPU performance, MCP agent integration, and how we use them for 3D AJ.
Three.js (mrdoob/three.js)
Browser 3D & WebGPU The undisputed king of browser 3D. Zero plugin install for users, native glTF 2.0 .glb support, and instant seek(t) scrubbing.
FFmpeg (FFmpeg/FFmpeg)
Media Compositor & Critic Non-negotiable infrastructure. Every frame pipeline, audio normalizer, and contact-sheet critic ends in FFmpeg.
faster-whisper (SYSTRAN/faster-whisper)
Word-Level Audio Alignment 4x faster than stock Whisper with lower VRAM via CTranslate2. Gives agents an exact millisecond clock for every spoken word.
Blender MCP (ahujasid/mcp-for-blender)
Agentic 3D Scene Control Turns Blender from a manual GUI into an agent-programmable 3D compiler. Essential for automated mesh inspection and rigging.
Blender (blender/blender)
3D Modeling, Rigging & Baking The open-source 3D standard. Combined with Python bpy, it eliminates manual 3D artist bottlenecks.
Remotion (remotion-dev/remotion)
React Video Composition Best-in-class DX when your team already builds in React and Tailwind CSS. Deterministic frame rendering out of the box.
Remotion Skills (remotion-dev/skills)
Agent Procedural Memory Crucial context upgrade. Without this skill, LLMs write brittle React effects instead of pure useCurrentFrame() math.
Chrome DevTools MCP (ChromeDevTools/chrome-devtools-mcp)
Live DOM & Frame Inspection Closes the blind-agent gap. Your agent sees the exact DOM, computed styles, and pixel output in real time.
GSAP (greensock/GSAP)
Timeline & UI Motion
Battle-tested timeline control. Just remember to auto-ticking and drive tl.time(t) from window.seek(t) in render mode.
#### HyperFrames (heygen-com/hyperframes)
Agent-Native HTML-to-Video
Purpose-built for coding agents that prefer writing raw HTML/Canvas over full React bundles. Fast and lightweight.
Chatterbox (resemble-ai/chatterbox)
Open-Source Voice Synthesis Impressive open-source expressive voice cloning with emotion control. Great local alternative to paid ElevenLabs seats.
ComfyUI (Comfy-Org/ComfyUI)
Node-Based Diffusion & 3D Gen Unmatched local GPU control for image-to-3D plates and texture maps, though JSON workflow graphs require disciplined versioning.
Impeccable (pbakaus/impeccable)
Automated UI & Design Critic Great automated design gate to stop agents from shipping generic purple-gradient SaaS clichés.
yt-dlp (yt-dlp/yt-dlp)
Reference Media Ingestion Indispensable utility for reference grammar extraction, though strictly an ingestion CLI rather than a rendering engine.
3. Deep Dive: How We Built and Upgraded 3D AJ the Cat Across CodingCat.dev #
Before this upgrade, AJ the Cat lived in two separate worlds on CodingCat.dev:
- Static 3D Turnaround Renders : High-resolution Pixar-style vinyl-toy studio JPEGs (
assets/AJ-3D-Headphones-Front.jpg,Left.jpg,Right.jpg) used in YouTube thumbnails and video bumpers. - 2D Articulated Vector SVG (
AjAnimatedMascot.astro) : A 57-path inline SVG rig with 14 bone pivot groups (#aj-head-group,#aj-arm-right,#aj-tail, etc.) driven by CSS@keyframesand randomsetTimeoutblinks.
While the 2D SVG rig had zero external JavaScript dependencies, its CSS @keyframes and Math.random() blink timers meant it was not frame-seekable for headless video rendering, and visitors could not orbit or inspect a real 3D mesh in the browser.
Here is the exact 4-stage architecture we implemented to unify 2D Vector AJ and 3D GLB AJ under a single deterministic motion engine.
Stage 1: Programmatic 3D Vinyl-Toy GLB Generation (scripts/build-aj-3d-glb.mjs) & Blender MCP Verification
To ensure 3D AJ matches every single anatomical law in our Design System (09-assets.md) while keeping the binary payload compact ( 756 KB) and 100% deterministic, we built scripts/build-aj-3d-glb.mjs and paired it with our headless Blender MCP verification script (scripts/blender-aj-3d-pipeline.py).
Every anatomical feature of AJ is mapped to a named glTF 2.0 node so both Blender (bpy.data.objects) and Three.js ( gltf.scene.getObjectByName) can target the exact same bones:
AjRoot->AjPedestal+AjBodyRootAjTorso(Matte-black#0B0B0EPBR fur with roughness0.62+ dark charcoal#2D2D30oval belly patch)AjCollar+AjHexTag(Soft Lavender#B4A7D6collar with hexagonal gold & Royal Purple"AJ"tag)AjHeadGroup->AjEarLeft,AjEarRight,AjWhiskersLeft,AjWhiskersRightAjEyeLeftPupil&AjEyeRightPupil(Electric Lime Green#C5E86Ceyes with vertical black pupils and dual white catchlights)AjJawMouth(Articulated jaw and pink tongue viseme group for 120 BPM audio-reactive lip-sync)AjHeadphonesGroup(CodingCat.dev Podcastcharcoal headband + Royal Purple#7C3AED& Soft Lavender#B4A7D6earcups fitting directly over his cat ears)
Stage 2: Closed-Form Springs & Deterministic window.seek(t) (src/lib/aj-motion-engine.ts)
In @0xnicc0 and @seeconvm's guide, the secret to lifelike motion that remains 100% seekable at any arbitrary frame t is stacking closed-form harmonic springs:
Because closedFormSpring(dt, omega, zeta) is a pure analytical equation using Math.exp(-zeta * omega * dt) and Math.cos(omegaD * dt), evaluating frame 812 (t = 13.533s) requires zero knowledge of frames 0 through 811.
Here is the production motion engine we shipped in apps/site/src/lib/aj-motion-engine.ts:
Both <AjAnimatedMascot /> (2D SVG) and <Aj3dMascotViewer /> (3D Three.js GLB) subscribe to window.seek(t) (cc:seek custom event) and call computeAjPoseAtTime(t, state). You can test this live right now on /mascot-lab using the window.seek(t) frame scrubber or by opening DevTools and running window.seek(4.25).
Stage 3: Zero-LCP-Penalty 3D Three.js Viewer (Aj3dMascotViewer.astro)
A common mistake when adding Three.js to a content site is mounting a heavy WebGL canvas above the fold that blocks Largest Contentful Paint (LCP).
On CodingCat.dev, we kept our zero-JS 57-path 2D SVG rig as the instant-paint SSR default and added an interactive 2D SVG / 3D GLB toggle in the Homepage Hero (AnimatedHero.astro) and on /mascot-lab. When you switch to 3D GLB, Aj3dMascotViewer.astro loads /models/aj-3d-mascot.glb (756 KB), lights AJ with ACES Filmic tone mapping plus Royal Purple (#7C3AED) and Electric Lime (#C5E86C) spot rim lights, and binds cursor coordinates to AjEyeLeftPupil and AjEyeRightPupil.
Stage 4: 120 BPM Audio-Reactive Lip-Sync on Podcast Pages & Video Studio
In Section 6 of the Opus 5.5 Motion Design guide ("Springs and Sound, or Why Cheap Motion Feels Cheap"), sound and picture must share the exact same clock. We wired PodcastListen.astro and /podcasts to window.setAjAudioAmplitude(amp, bpm):
- Whenever you play an episode in
PodcastListen.astro(or clickTest AJ 120 BPM Beat Sync on/podcasts), AJ's jaw viseme (#aj-talking-framein 2D SVG andAjJawMouthin 3D GLB), head nod, and ear bounce lock to the audio envelope at 120 BPM. - In our local RTX 5090
tools/video-studio/synthesize-voiceover.pypipeline,faster-whisper extracts word-level timestamps intostoryboard.jsonso every rendered Course Lesson video aligns captions, diagram transitions, and AJ reactions to the millisecond.
AJ Pro-Tip Callout: Never Use Math.random() or Und Timelines in Headless Video Renders When capturing frames with Playwright or Headless Chrome, even a single Math.random() inside a blink timer or particle emitter will cause a visual stutter at your video's loop point. Always derive blinks and noise from hashUint32(seed + Math.floor(t * 60)) and drive every animation from window.seek(t).
4. Full Course Roadmap: 5 Flagship Stack Courses + 14 Deep-Dive Tool Courses #
To help you master this entire code-driven media and 3D web stack without paying $1,200 per launch video, we have prepped 19 new Courses (5 Flagship Architectural Stack Courses + 14 dedicated Tool-by-Tool Courses, totaling 95 Lessons) in the CodingCat.dev Course Factory (/courses):
The 5 Flagship Stack Courses
- Flagship 1: Programmatic Video with React, Remotion & Agent Skills (
Remotion+Remotion Skills) - Flagship 2: Agentic 3D Web Pipelines with Blender, Blender MCP & Three.js (
Blender+Blender MCP+Three.js) - Flagship 3: Deterministic HTML Motion & Visual Critique with HyperFrames, GSAP, Impeccable & Chrome DevTools MCP (
HyperFrames+GSAP+Impeccable+Chrome DevTools MCP) - Flagship 4: Local GPU Audio & Diffusion Pipelines with Chatterbox, faster-whisper & ComfyUI (
Chatterbox+faster-whisper+ComfyUI) - Flagship 5: Automated Media Engineering & Reference Extraction with FFmpeg & yt-dlp (
FFmpeg+yt-dlp)
The 14 Dedicated Tool-by-Tool Courses (1 Per GitHub Repo)
| Course # | Course Title & Slug (/course/[slug]) | Primary GitHub Repo | Core Production Deliverable |
|---|---|---|---|
| 01 | Remotion in Production: React Video Composition (/course/remotion-react-video-production) | remotion-dev/remotion | Multi-aspect-ratio 60fps launch video rendered from React + Tailwind CSS. |
| 02 | Remotion Skills for AI Coding Agents (/course/remotion-skills-ai-agents) | remotion-dev/skills | Custom SKILL.md harness that teaches Claude Code & Antigravity deterministic video rules. |
| 03 | Blender 5.0 for Web Developers: Headless Python & GLB (/course/blender-headless-python-web-devs) | blender/blender | Automated bpy script that builds, lights, and exports a <1MB .glb mascot. |
| 04 | Blender MCP: Driving 3D Scenes with AI Agents (/course/blender-mcp-agentic-3d-scenes) | ahujasid/mcp-for-blender | Live TCP socket MCP server letting an agent model, rig, and render 3D AJ. |
| 05 | Three.js & WebGPU in Production: Interactive 3D Mascots (/course/threejs-webgpu-3d-mascots) | mrdoob/three.js | Lazy-loaded <Aj3dMascotViewer /> with orbital camera, rim lights, and seek(t). |
| 06 | GSAP Deterministic Timelines & SVG Morphing (/course/gsap-deterministic-timelines-svg) | greensock/GSAP | Frame-locked tl.time(t) UI morphs and 57-path SVG bone animations. |
| 07 | HyperFrames: Agent-Native HTML-to-Video Rendering (/course/hyperframes-html-to-video-agents) | heygen-com/hyperframes | Zero-React standalone HTML/Canvas video compiler driven by window.seek(t). |
| 08 | faster-whisper on Local GPU: Word-Level Captions & Lip-Sync (/course/faster-whisper-word-level-captions) | SYSTRAN/faster-whisper | Sub-20ms word alignment pipeline driving dynamic captions and mascot visemes. |
| 09 | Chatterbox TTS: Zero-Shot Local Voice Cloning (/course/chatterbox-local-voice-cloning) | resemble-ai/chatterbox | Expressive local voiceover synthesizer with phonetic code identifier rules. |
| 10 | FFmpeg for AI Video Pipelines & Contact-Sheet Critics (/course/ffmpeg-ai-video-contact-sheet-critics) | FFmpeg/FFmpeg | -14 LUFS EBU R128 audio normalization, 6-across contact sheets, and H.264/AV1 mastering. |
| 11 | Chrome DevTools MCP: Self-Healing Visual Agent Loops (/course/chrome-devtools-mcp-visual-agent-loops) | ChromeDevTools/chrome-devtools-mcp | Automated DOM/WebGL inspection loop that catches overlapping text before export. |
| 12 | Impeccable: Automated Design & Motion Polish Gates (/course/impeccable-automated-design-critique) | pbakaus/impeccable | Pre-export design critic enforcing W3C design tokens and banning generic AI gradients. |
| 13 | ComfyUI for 3D Turnarounds & Generate-Then-Trace (/course/comfyui-3d-turnarounds-video-workflows) | Comfy-Org/ComfyUI | Reproducible API workflow for multi-view mascot sheets and base motion plates. |
| 14 | yt-dlp & Librosa: Reference Grammar & Beat Extraction (/course/yt-dlp-librosa-reference-beat-extraction) | yt-dlp/yt-dlp | Automated reference shotlist extractor (style_guide.md) and 120 BPM beats.json generator. |
5. Verdict: Purrfect or Pass? #
- Purrfect For : Developer educators, SaaS founders, and frontend engineers who want agency-grade launch films, interactive 3D mascots (like3D AJ ), and automated video courses rendered deterministically from Git-versioned TypeScript and Python code for
$0in licensing fees. - Pass If : You need photorealistic live-action human cinema with complex cloth physics (where diffusion video models like Veo 3.1 still belong) or you are unwilling to write a structured
brief.mdandwindow.seek(t)render contract.
Head over to /mascot-lab right now to toggle between 2D Vector AJ and 3D Three.js GLB AJ, scrub the window.seek(t) timeline, download aj-3d-mascot.glb, and explore all 19 new Courses on /courses!
how this post connects together. #
Follow AJ from first principles to production-ready architecture.
Watch the Related Video Deep Dive: Cloudflare Agents, Durable Objects & Stateful Workflows in TypeScript #
Build stateful, long-running AI agents and resilient multi-step workflows using the Cloudflare Agents SDK, Durable Objects with embedded SQLite, WebSockets hibernation, and Workflows.