{"slug": "gitframes", "title": "Gitframes", "summary": "Gatewai-dev released gitframes, an npm package in beta that lets AI coding agents render video from TypeScript compositions directly on GPU hardware via Dawn/WebGPU/Metal/Vulkan, without a headless browser or screenshot pipeline. Gitframes bundles Photoshop-grade compositing and VFX (50+ GPU shaders), After Effects-style motion and keyframing, and Blender-style 3D scenes into one package, using roughly 200–400 MB RAM per worker versus 2–4 GB for Chromium, and installs via `/plugin install gitframes` in Claude Code or `npx skills add gatewai-dev/gitframes` for other agents. The package positions itself against Remotion (React/Chromium) and Hyperframes (Canvas2D/SVG) for programmatic video generation, and its APIs may change between releases while in beta.", "body_md": "**Photoshop-, After Effects-, and Blender-class video tools as one npm package that AI agents drive with code.**\n\n **⚠️ Beta:** gitframes is under active development. APIs may change between releases and some features may be incomplete or unstable.\n\nGitframes is built for coding agents. It packs the work people usually split across three desktop apps (Photoshop-grade compositing and VFX, After Effects-style motion, typography and keyframing, and Blender-style 3D scenes, cameras and models) into one lightweight npm package. Your agent writes a TypeScript composition, checks frames, and renders an MP4, and nobody has to install or license a multi-gigabyte creative suite.\n\n**Code-first video as pure software engineering** — no headless browser, no DOM reflow, no screenshot pipeline.\nRenders directly on GPU hardware via Dawn / WebGPU / Metal / Vulkan in Node.js and modern WebGPU browsers.\n\nEvery frame of these films is rendered by gitframes from TypeScript in [`examples/`](https://github.com/gatewai-dev/gitframes/blob/main/examples). Click a still to watch it on YouTube.\n\n|  [**Gitframes launch**](https://youtu.be/w6IQNrhJJek)<sub>[`22_gitframes_launch`](https://github.com/gatewai-dev/gitframes/blob/main/examples/22_gitframes_launch)</sub> |  [**Full Circle**](https://youtu.be/YeIpp4xf_j8)<sub>[`21_full_circle`](https://github.com/gatewai-dev/gitframes/blob/main/examples/21_full_circle)</sub> |  [**Dancer showcase**](https://youtu.be/R5Zug49FTTQ)<sub>[`19_gitframes_film`](https://github.com/gatewai-dev/gitframes/blob/main/examples/19_gitframes_film)</sub> | \n\nNote\n\n**Using an AI coding agent?** Install the gitframes skills in one line.\n\n**Claude Code**\n\n```\n/plugin install gitframes\n```\n\n**Any other agent** (Codex, Cursor, Hermes, Gemini CLI, Copilot, and more)\n\n```\nnpx skills add gatewai-dev/gitframes\n```\n\nSee [Agent Skills & Plugins](#agent-skills--plugins) for details.\n\nModern automated video generation is usually constrained by the architectures of general-purpose web browsers: process overhead, non-deterministic DOM layout reflows, and slow screenshot capture. Gitframes treats **video composition as software engineering**:\n\n|  | Pillar | What it means | \n|---|---|---|\n| 🚀 | **Zero Headless-Browser Overhead** | No Puppeteer, no Chromium IPC, no `page.screenshot()` . Gitframes talks straight to native GPU devices via Dawn/WebGPU and hardware-encodes with`@napi-rs/webcodecs` . | \n| 🎯 | **Deterministic Frame-Accurate Clock** | Absolute frame clocks, discrete sample points, and frame-accurate audio BeatGrids. No floating timers, no drift, no dropped frames. | \n| 🔠 | **Analytic, Resolution-Independent Type** | The Slug algorithm evaluates glyph contours per-pixel in WGSL — no texture atlases, no scaling artifacts, razor-sharp from 10 px to 10,000 px. | \n| 🎨 | **Photoshop-Grade Tonal & Spatial VFX** | 50+ modular GPU shaders: Curves, Levels, Selective Color, 3D LUTs, Halftone, Film Grain, Unsharp Mask, Mesh Warp, and Screen-Space Relighting. | \n| 🧊 | **Unified 3D & 2D Depth Compositing** | Nest 2D flex/box trees inside 3D homography planes, multiplane rigs, and meshes (OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF), with PBR glass and SSAO. | \n| 🔊 | **Built-in Procedural Audio DSP** | Multi-track soundtracks, deterministic procedural transition SFX (whoosh, impact, riser), and reactive signals that drive visuals from audio. | \n| 👁️ | **On-Device Neural Vision** | Object tracking, instance segmentation, multi-person pose, and person mattes from Apache-2.0 ONNX models — feeding reactive signals without a round trip to disk. | \n| ☁️ | **Cloud-Native & CI/CD Ready** | ~200–400 MB RAM per worker (vs. 2–4 GB for Chromium), ideal for serverless GPU render clusters (AWS G4/G5, Modal, RunPod, Kubernetes). | \n\nDevelopers generating video programmatically commonly weigh **Remotion** (React/Chromium) or **Hyperframes** (Canvas2D/SVG web animation). The matrix below compares the fundamental engineering dimensions.\n\n| Capability / Dimension | **Gitframes** | **Remotion** | **Hyperframes** | \n|---|---|---|---|\n| **Underlying Engine** | **Native WebGPU** (WGSL compute & render pipelines via Dawn / Metal / Vulkan) | **Chromium / Puppeteer** (React DOM, HTML/CSS layout) | **Canvas2D / WebGL / SVG** (browser or Node Skia) | \n| **Rendering Architecture** | Direct hardware framebuffer rendering & hardware video encoding ( `@napi-rs/webcodecs` ) | Spawns headless Chrome; captures frames via CDP / `page.screenshot()` | Software or hardware 2D canvas context | \n| **Throughput** | **60–120+ FPS** (real-time to faster-than-real-time GPU execution) | **5–20 FPS** (DOM reflow, IPC, rasterization) | **20–40 FPS** (CPU draw commands / JS) | \n| **Memory Footprint** | **~200–400 MB** per render (zero browser) | **1.5–4.0 GB+** per worker (Chromium + V8 DOM heap) | **~500 MB–1 GB** (Skia/Canvas bindings) | \n| **Typography Engine** | **Slug GPU** — analytic Bézier evaluation in WGSL, infinite zoom, After Effects selectors | Browser DOM text (CSS fonts, rasterized, blurry under 3D transforms) | Canvas2D / path text (CPU-rasterized glyphs) | \n| **2D VFX & Post-Processing** | **50+ WebGPU shaders** (Curves, Levels, Selective Color, 3D LUT, Film Grain, Halftone, Liquify, PBR Glass, Relight) | CSS Filters or custom WebGL canvas wrappers | Basic Canvas2D composites and 2D filters | \n| **3D Graphics & Depth** | **Native 3D scene graph** — LookAt/Turntable camera, multiplane, skinning (OBJ/FBX/glTF), SSAO, PCSS, DoF | None built-in (embed Three.js/Fiber inside React DOM) | Minimal 2.5D layers; no unified mesh pipeline | \n| **Motion Blur & Physics** | Physical 180° shutter velocity buffers in MRT + closed-form spring kinematics | CSS transitions / JS interpolation; synthetic blur hacks | Frame interpolation or manual multipass | \n| **Audio Engine & DSP** | Native audio DSP & procedural SFX (multi-track mixing, beat grids, reactive signals) | `<Audio>` playback; basic volume curves | Basic static audio playback | \n| **Charts & Data Viz** | **`Layer.chart`** — line, area, bar, scatter, candlestick, pie and donut charts built from native vector nodes, with staggered reveal animations | DOM chart libraries (Recharts, Chart.js) | Custom canvas draw operations | \n| **AI & Computer Vision** | **On-device ONNX vision** — COCO-80 detection + instance masks (RTMDet-Ins), COCO-17 pose (RTMO), person mattes (Selfie Segmenter); WebGPU tensor conditioning (Canny, depth-to-normals, optical flow, deflicker) | External pre-rendered assets; no native GPU tensor conditioning | External pre-rendered assets | \n| **Headless Verification** | **FrameGrid contact sheets** , single-frame snapshots, Skia MSE pixel-invariant assertions | Playwright/Puppeteer visual snapshots | Manual frame inspection / canvas diffing | \n| **Docker / Cloud Portability** | **Compact** (~500 MB slim image with native GPU/Vulkan drivers) | **Heavy** (~2–3 GB with Chromium, fonts, X11/Mesa) | Moderate container size | \n\nTraditional text relies on CPU rasterization or low-res SDF atlases that soften under 3D camera sweeps. Gitframes integrates the **Slug algorithm** ([`SlugPipeline`](https://github.com/gatewai-dev/gitframes/blob/main/packages/webgpu-renderers/src/slug/slug-pipeline.ts)):\n\n- **Analytic GPU evaluation** — WGSL fragment shaders solve exact cubic/quadratic Béziers per-pixel. Glyphs stay sharp at 10 px or 10,000 px with zero CPU re-rasterization.\n- **After Effects–parity animators** — range selectors (`square` ,`ramp_up` ,`ramp_down` ,`triangle` ,`smooth` ),`easeHigh` /`easeLow` curves, and seeded PRNG character shuffling ([`TextAnimator`](https://github.com/gatewai-dev/gitframes/blob/main/packages/gitframes/src/index.ts) ).\n- **Human typing cadence** — weighted punctuation delays (commas 3×, sentence ends 5.5×, newlines 7×) and trailing scramble resolution ([`TypewriterAnimator`](https://github.com/gatewai-dev/gitframes/blob/main/packages/gitframes/src/index.ts) ).\n- **3D volumetric formations** — map text onto cylindrical drums, logarithmic vortex spirals, and double-helix ribbons with surface-normal banking ([`evaluateVolumetricFormation`](https://github.com/gatewai-dev/gitframes/blob/main/packages/gitframes/src/index.ts) ).\n- **Dynamic leading & skew** — area-preserving unimodular shear and accordion line-leading anchored to baseline, center, or top.\n\nA comprehensive suite of professional image/video shader nodes in [`nodes/`](https://github.com/gatewai-dev/gitframes/blob/main/nodes) and [` packages/webgpu-renderers`](https://github.com/gatewai-dev/gitframes/blob/main/packages/webgpu-renderers):\n\n- **Tonal grading** — Curves (RGB/R/G/B spline), Levels (black/white point, gamma, output), Shadows/Highlights, Selective Color (CMYK gamut isolation), 3D Cube LUT ([`ApplyLUT`](https://github.com/gatewai-dev/gitframes/blob/main/packages/gitframes/src/effects) ).\n- **Stylization & grain** — Film Grain (Gaussian emulsion with spatial seed variation), Halftone (mono/RGB/CMYK, adjustable dot shape & angle), Gradient Map, High Pass.\n- **Optics & lens** — Bilateral Gaussian Blur, Unsharp Mask, Vignette, Refraction Caustics, PBR Glassmorphism with chromatic dispersion ([`PBRGlass`](https://github.com/gatewai-dev/gitframes/blob/main/packages/gitframes/src/effects) ).\n- **Distortion & warping** — Displacement Maps, Liquify, Mesh Warp, Corner Pin homography.\n\n- **Calibrated camera rig** — LookAt and Turntable cameras ([`Camera3D`](https://github.com/gatewai-dev/gitframes/blob/main/packages/webgpu-renderers/src/math3d/camera3d.ts) ) calibrated so`z = 0` matches 2D canvas pixel coordinates 1:1.\n- **3D layout primitives** —[`Layer3D.cube`](https://github.com/gatewai-dev/gitframes/blob/main/packages/gitframes/src/shapes3d.ts) ,`carousel` ,`prism` ,`plane` ,`grid` with unified depth-buffer testing.\n- **Zero-dependency model parsers** — OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF.\n- **Skeletal animation & shading** — 128-bone Linear Blend Skinning, Blinn-Phong & PBR multi-light shading, PCSS/Poisson contact shadows, SSAO, and optical DoF.\n- **Physical motion blur** — 180° shutter motion blur with per-vertex velocity vectors packed into`rg16float` MRT buffers.\n\n- **Soundtrack layers** —`.audio` media nodes with frame-exact lifecycle control.\n- **Procedural SFX** — deterministic CPU-synthesized whooshes, impacts, risers, downshifters, and glitches placed on the bar/beat grid (`renderSfx` ,`mixSfxInto` ,`softLimit` ).\n- **Multi-track mixing** — master tracks headlessly with`mixAudioTracks` and`encodeStereoWav` .\n- **Reactive signals** — drive transforms, scale, borders, or shader uniforms from tempo signals (`Signal.builder` ) or audio analysis.\n\n[`Layer.chart`](https://github.com/gatewai-dev/gitframes/blob/main/packages/gitframes/src/chart.ts) builds line, area, bar (grouped or stacked), scatter, candlestick, pie and donut charts. [d3](https://d3js.org) computes the scales, ticks and geometry; every bar, line, slice and label is an ordinary box, path or text node:\n\n- Labels use the composition's registered fonts and the same GPU text renderer as the rest of the film.\n- A built-in reveal draws lines on, grows bars from the baseline and staggers points and slices (`animate: { start, duration, stagger, ease }` , or`animate: false` ).\n- The chart is one box, so it positions, animates, grades and tilts into 3D like any other layer.\n\n```\nLayer.chart(\n  {\n    type: \"bar\",\n    width: 900,\n    height: 480,\n    categories: [\"Q1\", \"Q2\", \"Q3\", \"Q4\"],\n    series: [\n      { name: \"Revenue\", data: [12, 19, 24, 31] },\n      { name: \"Costs\", data: [8, 11, 13, 15] },\n    ],\n    yAxis: { format: \"$,.0f\" },\n    animate: { start: 10, duration: 30 },\n  },\n  { position: \"absolute\", x: 120, y: 200 },\n);\n```\n\n[`@gitframes/vision`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision) runs ONNX models via `onnxruntime-node` (CPU) or `onnxruntime-web` (WebGPU) and wires every result into the same reactive signal surface the rest of Gitframes consumes.\n\nTip\n\n**Lazy by construction.** `VisionRunner.create()`, `comp.withVision(...)` and `VisionNode.attach(...)` perform **zero I/O** — no downloads, no sessions, no file probes. A model is fetched the first time a task actually runs. To warm up ahead of time, call `await runner.preload([\"detect\", \"pose\"])` (or `await vision.ready()` on an attached node).\n\nEvery model is **Apache-2.0**, pinned to an immutable Hugging Face revision, and verified by SHA-256 after download.\n\n| Task | Option | Model | Output | \n|---|---|---|---|\n| **Detect** | `enableDetection` | RTMDet-Ins `t/s/m` (OpenMMLab) | COCO-80 boxes + scores, tracked over time | \n| **Segment** | `enableSegmentation` | RTMDet-Ins (same forward pass as detect) | Soft per-instance masks, frame-aligned | \n| **Pose** | `enablePose` | RTMO `t/s/m` (OpenMMLab) | 17 COCO keypoints + visibility per person | \n| **Matte** | `enableMatte` | MediaPipe Selfie Segmenter (Google) | Fast person-vs-background alpha for portrait / webcam framing | \n\n- **Variants** —`variant: \"t\" | \"s\" | \"m\"` (default`\"s\"` ; ~24 / 43 / 116 MB for RTMDet-Ins). Tune`confidence` and a COCO`classes` filter per composition. On CPU, a 2K frame takes roughly 200–340 ms to detect + segment, ~120 ms for pose and ~20 ms for the matte with`\"s\"` .\n- **One pass, two tasks** — detection and segmentation share a single RTMDet-Ins inference per frame.\n- **Picking a matte** — the Selfie Segmenter is tuned for a person filling much of the frame: it misses distant figures and can report \"person\" on close-ups with nobody in them. For anything else, cut out with instance masks (`matteSource: \"instance\"` , the default).\n- **Whole-subject cutouts** —`mask` /`matte` /`crop` modes merge every comparably sized instance that overlaps the main subject, so a flowing dress or a held instrument stays attached to the person, while a tunnel or window framing them does not.\n- **One-frame delay** — vision reads each layer's previous rendered frame, so results trail the plate by one frame and frame 0 has none. Verify vision layers with the exported video or consecutive frames, not frame grids.\n- **Model cache & mirrors** — models are cached atomically (temp + rename) in`$GITFRAMES_MODELS_DIR` (default`~/.cache/gitframes/models` ). Point`baseUrl` or`GITFRAMES_MODELS_BASE_URL` at your own mirror for air-gapped or CI renders.\n\n- **OpenPose-style skeleton textures** — rasterize COCO-17 keypoints into a VRAM conditioning texture ([`PoseSkeletonRenderer`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/gpu/pose-skeleton-renderer.ts) ).\n- **GPU segmentation texture pool** — reusable silhouette textures ([`SegmentationTexturePool`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/gpu/segmentation-texture-pool.ts) ).\n\n- **Multi-object tracker** ([`TemporalObjectTracker`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/tracking/temporal-object-tracker.ts) ) assigns stable`trackId` s via IoU association, with configurable`minHits` ,`positionSmoothing` , and velocity-based**coasting** for up to`maxMissedFrames` (default 15) so a transient miss holds the track instead of flashing.\n- **Pose↔track matching** ([`pose-track-matcher`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/tracking/pose-track-matcher.ts) ) binds keypoints to the right track by id, then by spatial IoU fallback.\n- **One-shot sequence analysis** —`comp.analyzeVisionSequence(src, { tasks, categories })` decodes frames through the mediabunny pipeline, tracks them, and returns a**zod-serializable** report (per-track frame ranges, mean speed, sampled center paths, per-class presence/confidence, mean mask coverage, model download bytes/timing) ([`analyzeSequence`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/analysis/analyze-sequence.ts) ).\n\nEvery tracked entity is exposed as reactive `ProgrammaticSignal` s that animate layers and shader uniforms:\n\n| Group | Highlights | \n|---|---|\n| `objects` | `get(trackId)` ,`byCategory(cat, rank)` ,`primary` ,`count` ,`hasCategory` ,`detectedCategories` | \n| `objects.*.bounds` | `x/y/width/height` ,`screenX/screenY/screenWidth/screenHeight` ,`aspectRatio` ,`area` | \n| `objects.*.anchors` | 9 anchors (corners, edges, center) ready for pinning | \n| `objects.*.kinematics` | `vx` ,`vy` ,`speed` ,`acceleration` ,`headingRad/Deg` | \n| `objects.*.pose` | All 17 COCO keypoints, plus `hasPose` ,`wristSpeed` ,`handRaised` ,`bodyTiltAngle` | \n| `masks` | `get(trackId)` ,`subject` ,`count` ; per-mask`area` ,`coverage` ,`solidity` ,`bboxFill` | \n| `segmentation` | `subject` ,`humanSilhouette` ,`instanceMasks` ,`matte.coverage` , GPU`stencilTexture` | \n| `classes` | Per-class `count` ,`maxConfidence` ,`present` ,`primary` , plus a detection`histogram` | \n| Tensors | `poseLandmarksTensor [17,3]` ,`objectsTensor [16,8]` ,`masksTensor [16,2]` ,`histogramTensor [80]` | \n\nProject normalized landmarks to screen space with a configurable camera FOV, then bind any node to a track or landmark ([`SpatialLandmarkTransformer`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/spatial/camera-space-transformer.ts), [`spatial-pin`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/spatial/spatial-pin.ts)):\n\n- `pinToObject(track, { anchor, offsetX/Y/Z, matchWidth, matchHeight, smoothFrames, hideWhenLost })`\n- `pinToLandmark(coord, { offsetX/Y/Z })`\n\n- **Subject Sandwich** —`comp.addSubjectSandwich({ source, behind, feather, fit })` cuts the foreground subject out and places typography/graphics behind them.\n- **Smart Reframing** —`comp.addSmartFraming({ source, target, targetAspect, damping, leadHeadroom })` auto-crops 16:9 → 9:16 while tracking`target` .\n- **Subject Outline** —`comp.addSubjectOutline(vision.segmentation.subject, { source, color, width, blur })` strokes the segmented boundary as an audio-reactive contour glow.\n- **Tracked Region Blur** —`layer.blurRegion(track, { strength })` blurs faces, plates, or any detected class.\n- **Node modes** —`passthrough` ,`mask` ,`matte` ,`crop` ,`skeleton` ,`boxes` ,`tracking` ; pick the cutout alpha with`matteSource: \"instance\" | \"selfie\"` , and optionally`keyBackground` to grow the subject into connected foreground.\n\n- **Runtime config is zod-validated** and available from a**zod-only entry** (`@gitframes/vision/schemas` ) so the hot path stays zod-free. Unknown or removed options are rejected, not silently ignored.\n- **`vision.summary(frame)`** returns a deterministic, serializable snapshot (objects, classes, masks) safe to call inside a frame hook.\n- **Clear failures** — a model that is the wrong size, fails its checksum, or lacks an expected output raises an error naming the model and its source.\n- **Browser entry** —`@gitframes/vision/web` re-exports the engine plus`createWebGPUProvider()` /`hasWebGPU()` ;`onnxruntime-web` is an optional lazy peer.\n\n- **Pixel-sampling invariant assertions** — test compositions in Vitest with`skia-canvas` to verify shader math, font coverage, and Mean Squared Error (MSE) temporal deltas.\n- **FrameGrid contact sheets** —`comp.renderFrameGrid(...)` outputs sequential-frame contact sheets for instant review of easing, kinetic type, and transitions.\n\n- **Runs your composition, not a video** —`startPreview({ entry, export })` serves a localhost WebGPU player that loads the composition's own module and renders every frame live in the browser. Nothing is streamed: the server only hands over the bundle, the project's assets, and the soundtrack mixed by the export engine.\n- **Timeline, waveform & frame stepping** — play/pause, scrub, step frame by frame, and read resolution, FPS, duration, and audio status at a glance.\n- **One stable URL per project** — the port is derived from the working directory, so re-running the preview replaces the running server and any open tab reloads into the new version by itself. Close the tab and the server shuts down about five seconds later.\n- **Shown where you are** —`startPreview` serves the page and returns its URL instead of opening a browser, so an agent can show it in its own pane (Claude Code, Codex); pass`open: true` to open the system browser.\n\n``` js\nimport { startPreview } from \"gitframes\";\n\nconst session = await startPreview(\n  { entry: new URL(\"./film.ts\", import.meta.url), export: \"buildFilm\" },\n  { title: \"gitframes film\" },\n);\nconsole.log(`Preview at ${session.url}`);\nawait session.closed; // serves until its tab closes or a newer preview takes over\n```\n\nManaged with `pnpm` workspaces and `turbo`:\n\n```\ngitframes/\n├── packages/\n│   ├── gitframes/              # Unified SDK (Composition, Layer, LayerAnimation, Signal, effects)\n│   ├── core/                   # Core AST, Effect base class, VirtualMediaData, vision types\n│   ├── compositions/           # Layout engine, Flex/Box AST compiler, timeline evaluator\n│   ├── webgpu-renderers/       # WGSL shaders, Slug text engine, 3D renderer, camera, lights, materials\n│   ├── tensor-webgpu/          # WebGPU compute pipelines (Canny, depth-to-normals, flow, deflicker, landmarks)\n│   ├── vision/                 # ONNX vision engine: detect, segment, pose, matte, tracking, signals\n│   ├── renderer/               # Headless Node.js WebGPU renderer via Dawn, WebCodecs, skia-canvas\n│   ├── renderers/              # Higher-level render orchestration\n│   ├── media/                  # Media decoding / encoding adapters\n│   ├── node-sdk/               # Node renderer contracts and result schemas\n│   ├── server-utils/           # Server infrastructure, storage, asset caches\n│   └── client-utils/           # Shared browser utilities\n├── nodes/                      # 58+ specialized domain nodes (VFX, audio, layout, node-vision)\n├── apps/\n│   └── renderer-service/       # Production HTTP / gRPC rendering microservice container\n├── examples/                   # Reference compositions and films\n├── plugins/gitframes/          # Agent plugin: skills only (setup, compose, effects, render)\n└── scripts/                    # Build, release, and plugin validation tooling\npnpm add gitframes\n```\n\n**Requirements:** Node.js ≥ 22. Gitframes uses native GPU acceleration via Dawn / WebGPU or Vulkan.\n\n``` js\nimport { Composition, Layer, LayerAnimation } from \"gitframes\";\n\n// 1. Initialize a 1080p60 composition\nconst comp = new Composition({\n  width: 1920,\n  height: 1080,\n  fps: 60,\n  durationFrames: 180, // 3 seconds\n  backgroundColor: \"#090a0f\",\n  fonts: [\"assets/fonts/Inter.ttf\", \"assets/fonts/SpaceGrotesk.ttf\"],\n});\n\n// 2. Define physical snap-overshoot animations\nconst cardEntrance = LayerAnimation.create()\n  .fadeIn(0, 20, \"power2.out\")\n  .fromTo(\"y\", 60, 0, { start: 0, end: 35, ease: \"back.out(1.5)\" })\n  .fromTo(\"scale\", 0.92, 1.0, { start: 0, end: 35, ease: \"back.out(1.2)\" });\n\n// 3. Assemble a responsive flex-layout card\nconst heroCard = Layer.box({\n  width: 720,\n  height: 380,\n  background: \"#141721\",\n  borderRadius: 24,\n  borderColor: \"#262b3d\",\n  borderWidth: 1.5,\n  padding: 32,\n  children: [\n    Layer.flex({\n      dir: \"column\",\n      gap: 16,\n      children: [\n        Layer.text(\"GITFRAMES ENGINE\", {\n          fontSize: 16,\n          fontWeight: 700,\n          fill: \"#6366f1\",\n          letterSpacing: 2.0,\n        }),\n        Layer.text(\"Next-Gen WebGPU Motion\", {\n          fontSize: 48,\n          fontWeight: 700,\n          fill: \"#f8fafc\",\n          fontFamily: \"SpaceGrotesk\",\n        }),\n        Layer.text(\"Direct hardware video composition without headless browser overhead.\", {\n          fontSize: 20,\n          fill: \"#94a3b8\",\n          lineHeight: 28,\n        }),\n      ],\n    }),\n  ],\n}).animate(cardEntrance);\n\ncomp.add(heroCard);\njs\nimport { Composition, Layer, Layer3D, CameraAnimation, Light } from \"gitframes\";\n\nconst comp = new Composition({ width: 1920, height: 1080, fps: 60, durationFrames: 300 });\n\n// 1. LookAt 3D camera with a continuous orbit\nconst cameraAnim = CameraAnimation.camera().orbit({\n  azimuth: { from: -30, to: 30 },\n  elevation: { from: 15, to: 15 },\n  radius: { to: 1200 },\n  start: 0,\n  end: 300,\n});\n\ncomp.add(\n  Layer.camera({ x: 960, y: 540, z: -1000, targetX: 960, targetY: 540, targetZ: 0 }).animate(cameraAnim)\n);\n\n// 2. Studio lighting\ncomp.add(Light.ambient(\"#ffffff\", 0.4));\ncomp.add(Light.directional({ color: \"#e0e7ff\", intensity: 1.2, x: 500, y: -800, z: -600 }));\n\n// 3. 3D model with skeletal animation\ncomp.add(\n  Layer.glb(\"assets/models/character.glb\", {\n    x: 960,\n    y: 640,\n    z: 0,\n    scale: 2.5,\n    material: \"lit\",\n    loop: true,\n  })\n);\n\n// 4. 3D prism layout carousel\ncomp.add(\n  Layer3D.carousel({\n    radius: 400,\n    items: [\n      Layer.box({ width: 280, height: 180, background: \"#1e293b\", borderRadius: 16 }),\n      Layer.box({ width: 280, height: 180, background: \"#334155\", borderRadius: 16 }),\n      Layer.box({ width: 280, height: 180, background: \"#0f172a\", borderRadius: 16 }),\n    ],\n  })\n);\njs\nimport { Composition, Layer, LayerAnimation, Signal, renderSfx, mixSfxInto, softLimit } from \"gitframes\";\n\nconst comp = new Composition({ width: 1920, height: 1080, fps: 60 });\nconst totalFrames = 240;\n\n// 1. Soundtrack layer\ncomp.addAudio(Layer.audio(\"assets/score.mp3\", { volume: 0.9, durationFrames: totalFrames }));\n\n// 2. Frame-accurate procedural SFX on the beat grid\nconst bed: [Float32Array, Float32Array] = [\n  new Float32Array(Math.ceil((totalFrames / 60) * 48000)),\n  new Float32Array(Math.ceil((totalFrames / 60) * 48000)),\n];\nmixSfxInto(bed, [\n  renderSfx({ type: \"whoosh\", atBar: 0.79, volume: 0.5 }, { sampleRate: 48000, secondsPerBar: 2.0, seed: 1 }),\n  renderSfx({ type: \"impact\", atBar: 1.0, volume: 0.8 }, { sampleRate: 48000, secondsPerBar: 2.0, seed: 2 }),\n]);\nsoftLimit(bed);\n\n// 3. Tempo signal (120 BPM = 2 Hz)\nconst beatPulse = Signal.builder({ type: \"sawtooth\", frequency: 2, amplitude: 0.08, offset: 1.0 });\n\n// 4. Bind it to visuals\nconst reactiveCard = Layer.box({ width: 400, height: 250, background: \"#1c202e\", borderRadius: 20 })\n  .animate(\n    LayerAnimation.create()\n      .signal(\"scale\", beatPulse, { multiplier: 1.0, offset: 0.0 })\n      .fromTo(\"opacity\", 0, 1, { start: 0, end: 15, ease: \"power2.out\" }),\n  );\n\ncomp.add(reactiveCard);\njs\nimport { Composition, FilmGrain, Vignette, ColorBalance } from \"gitframes\";\n\nconst comp = new Composition({ width: 1920, height: 1080, fps: 60 });\n\n// Whole-composition cinematic grade + film emulsion\ncomp.apply(new Vignette({ strength: 0.28, radius: 0.85 }));\ncomp.apply(new FilmGrain({ strength: 0.06, size: 1.5, animated: true }));\ncomp.apply(\n  new ColorBalance({\n    shadows: { cyanRed: 0, magentaGreen: 2, yellowBlue: 6 },\n    highlights: { cyanRed: 4, magentaGreen: 1, yellowBlue: -2 },\n  }),\n);\njs\nimport { Composition, Layer, Vignette } from \"gitframes\";\n\nconst comp = new Composition({ width: 1920, height: 1080, fps: 30 });\n\n// Run vision on the whole composition. Models download lazily on first use.\nconst vision = comp.withVision({\n  enableDetection: true,\n  enableSegmentation: true,\n  enablePose: true,\n  variant: \"s\",\n  confidence: 0.35,\n});\n\n// Pin a caption to the primary tracked subject (smoothing + auto-hide when lost)\ncomp.add(\n  Layer.text(\"SUBJECT 01\", { fontSize: 40, fill: \"#f8fafc\" }).pinToObject(\n    vision.objects.primary,\n    { anchor: \"topCenter\", offsetY: -48, smoothFrames: 5, hideWhenLost: true },\n  ),\n);\n\n// Drive a shader uniform from a reactive signal — here, subject mask coverage\ncomp.add(\n  Layer.box({ width: 1920, height: 1080, background: \"#000000\" }).withEffect(\n    new Vignette({ strength: vision.segmentation.subject.coverage, radius: 0.9 }),\n  ),\n);\n\n// Or use the one-liners for the common editorial moves:\n// comp.addSubjectSandwich({ source: \"assets/dancer.mp4\", behind: [headline], feather: 4 });\n// comp.addSmartFraming({ source: \"assets/action.mp4\", target: vision.objects.primary, targetAspect: 9 / 16 });\n// comp.addSubjectOutline(vision.segmentation.subject, { source: \"assets/character.mp4\", color: \"#FF5A1F\", width: 6 });\n\n// Inspect a source before authoring: one-shot, ffmpeg-free, zod-serializable report\nconst report = await comp.analyzeVisionSequence(\"assets/street.mp4\", {\n  tasks: [\"detect\", \"pose\"],\n  categories: [\"person\"],\n});\nconsole.log(report.tracks.map((t) => `${t.category}#${t.trackId} ${t.frames.join(\"–\")}`));\n```\n\n**Standalone runner (no composition):**\n\n``` js\nimport { VisionRunner } from \"@gitframes/vision\";\n\nconst runner = VisionRunner.create({ variant: \"s\", confidence: 0.3 }); // zero I/O\nconst frame = { data: rgba, width: 1920, height: 1080 };\nconst boxes = await runner.detect(frame); // downloads RTMDet-Ins on first call\nconst { masks } = await runner.segment(frame); // same forward pass, no second inference\nconst { people } = await runner.pose(frame); // RTMO, COCO-17 keypoints\nrunner.close();\n```\n\n**In the browser (WebGPU EP):**\n\n``` js\nimport { VisionRunner, createWebGPUProvider, hasWebGPU } from \"@gitframes/vision/web\";\n\nif (hasWebGPU()) {\n  const runner = VisionRunner.create({ provider: createWebGPUProvider() });\n}\njs\nimport { buildMyComposition } from \"./my-composition.js\";\n\nconst comp = await buildMyComposition();\n\n// 1. Single frame to a PNG buffer for visual inspection\nconst frameBuffer = await comp.renderFrame({ frame: 45 });\n\n// 2. Contact-sheet grid of 12 sequential frames\nconst gridBuffer = await comp.renderFrameGrid({\n  startFrame: 0,\n  endFrame: 120,\n  stepFrames: 10,\n  cellWidth: 320,\n  showLabels: true,\n});\n\n// 3. Final hardware-encoded MP4 with mixed audio\nconst { filePath } = await comp.renderVideo({\n  outputPath: \"output/final-product-film.mp4\",\n  quality: \"high\",\n  concurrency: 4,\n});\n\nconsole.log(`Video rendered successfully to: ${filePath}`);\n```\n\n1. **Design tokens & theme contracts** — define a centralized`THEME` for colors, type, radii, and spacing. Never hardcode magic hex values or ad-hoc margins.\n2. **WebGPU premultiplied-alpha invariant** — fragment shaders outputting premultiplied alpha (`color * opacity * alpha` ) must use`srcFactor: \"one\"` in their blend state (`{ srcFactor: \"one\", dstFactor: \"one-minus-src-alpha\", operation: \"add\" }` ). Never use`srcFactor: \"src-alpha\"` for premultiplied output — squaring alpha darkens fades into murky gray.\n3. **Carrier match cuts** — carry a visual element (badge, card, cursor, container) across scene boundaries with continuous velocity and position to avoid jarring cuts.\n4. **Physical easing vocabulary** —`back.out(1.4–1.7)` for snap-overshoot entrances,`spring` /`expo.out` for decelerating motion,`power2.in` for exits. Reserve`linear` for infinite spinners and time counters.\n5. **Headless invariant verification** — verify shader transforms, glyph coverage, and temporal MSE deltas with`skia-canvas` pixel sampling in Vitest before shipping.\n\nGitframes ships agent skills that teach Claude, Codex, and other coding agents how to write, render, and check compositions. The plugin (`gitframes`) is listed in Anthropic's official plugin directory and contains **only skills** — no MCP servers, hooks, or commands. Every other agent gets the same skills through the [`skills`](https://skills.sh) CLI.\n\n| Skill | Use it for | \n|---|---|\n| `gitframes` | Starting a project: install from npm, scaffold a composition and render script, first verified render | \n| `gitframes-compose` | Compositions, layer trees, layout, animation and easing, beat grids, film structure | \n| `gitframes-effects` | Effect classes, the unified section architecture, premultiplied-alpha invariants, vision conditioning | \n| `gitframes-render` | Headless rendering, FrameGrid inspection, pixel probes, MP4 delivery checks | \n\nOnce installed, skills load automatically when a task matches (e.g. *\"add a film-grain pass to this scene\"* or *\"render a frame grid of intro.ts\"*).\n\nThe plugin is instructions only. It bundles no executables, MCP servers, hooks, or package launchers, and it sends no data anywhere. The skills tell your agent to add the [`gitframes`](https://www.npmjs.com/package/gitframes) npm package to your project and how to use it. When that code uses on-device vision, the SDK downloads the pinned model weights from Hugging Face on first use (see [On-Device Vision](#6-on-device-vision--tracking)). Nothing else leaves your machine.\n\n```\n/plugin install gitframes\n```\n\nOr from your shell:\n\n```\nclaude plugin install gitframes@claude-plugins-official\n```\n\nIt installs from Anthropic's official marketplace, which Claude Code adds for you, so there is no marketplace step, and plugins from it update automatically. Afterwards, restart Claude Code or run `/reload-plugins`. `/plugin` commands need an interactive `claude` terminal; in the desktop app's Code tab, use the shell form or **+ > Plugins > Add plugin** and pick **Gitframes**.\n\nAdd `--scope project` to the shell form to record the plugin in `.claude/settings.json` for the whole team.\n\n**Enable it for everyone in your repo.** Commit this to `.claude/settings.json`; Claude Code prompts teammates to install it when they trust the folder:\n\n```\n{\n  \"enabledPlugins\": {\n    \"gitframes@claude-plugins-official\": true\n  }\n}\n```\n\n**Straight from this repository** (tracks `main` instead of the directory release):\n\n```\n/plugin marketplace add gatewai-dev/gitframes\n/plugin install gitframes@gitframes-plugins\n```\n\nThe [`skills`](https://skills.sh) CLI installs the skills into any of 70+ agents, including Codex, Cursor, Hermes, Gemini CLI, GitHub Copilot, Windsurf, OpenCode, and Goose:\n\n```\nnpx skills add gatewai-dev/gitframes\n```\n\nIt detects the agents on your machine and asks where to install. To choose them yourself, pass `-a` once per agent, add `-g` to install for your user instead of this project, and `-y` to skip the prompts:\n\n```\nnpx skills add gatewai-dev/gitframes -a codex -a cursor -a hermes-agent -g -y\n```\n\nKeep them current with `npx skills update`, and remove them with `npx skills remove`.\n\nOr copy the folders by hand: put `plugins/gitframes/skills/<name>/` into `.claude/skills/`, `.agents/skills/`, or `~/.agents/skills/`. VS Code / Copilot / Cursor / Kiro can load the portable [` plugin.json`](https://github.com/gatewai-dev/gitframes/blob/main/plugins/gitframes/plugin.json) through their plugin UI.\n\nThe plugin lives in [`plugins/gitframes/`](https://github.com/gatewai-dev/gitframes/blob/main/plugins/gitframes) so installs carry only the skills; users get the engine from npm. Two manifests there describe it: [`plugin.json`](https://github.com/gatewai-dev/gitframes/blob/main/plugins/gitframes/plugin.json) (portable [Agent Plugins 1.0](https://agent-plugins.org), which also carries the OpenAI listing metadata) and [`.claude-plugin/plugin.json`](https://github.com/gatewai-dev/gitframes/blob/main/plugins/gitframes/.claude-plugin/plugin.json). The marketplace catalog is [`.claude-plugin/marketplace.json`](https://github.com/gatewai-dev/gitframes/blob/main/.claude-plugin/marketplace.json). The portable field set is closed — client-specific fields go in that client's manifest, not in `plugin.json`. The `version` in both follows the `gitframes` package: `pnpm run version:packages` syncs it after `changeset version` (or run `pnpm run sync:plugin-version` on its own), since clients use it to decide when to update.\n\nInside this repository, Claude Code and other agents pick up skills through the symlinks in `.agents/skills/` and `.claude/skills/`. Skills live only under `plugins/gitframes/skills/`; never copy them elsewhere. `pnpm run check:plugins` validates manifests, skill frontmatter, marketplace catalogs, symlinks, and the generated effects catalog. `pnpm run sync:effects-catalog` regenerates the `gitframes-effects` catalog after any `Effect` class change.\n\nThe [`examples/`](https://github.com/gatewai-dev/gitframes/blob/main/examples) directory holds production-grade reference compositions:\n\n| Example | What it demonstrates | \n|---|---|\n| [`19_gitframes_film`](https://github.com/gatewai-dev/gitframes/blob/main/examples/19_gitframes_film) | The 30-second master brand film — full pipeline, audio, VFX, 3D | \n| [`21_full_circle`](https://github.com/gatewai-dev/gitframes/blob/main/examples/21_full_circle) | Multi-scene narrative composition | \n| [`22_gitframes_launch`](https://github.com/gatewai-dev/gitframes/blob/main/examples/22_gitframes_launch) | Launch/product-motion composition | \n\nGitframes uses `pnpm` (10+) and `turbo` for orchestration.\n\n```\n# Install\npnpm install\n\n# Build all packages\npnpm build\n\n# Run conformance tests\npnpm test\n\n# Check the vision models end to end (downloads ~380 MB of weights once)\npnpm --filter @gitframes/vision test:models\n\n# Render a specific showcase example\npnpm --filter @gitframes/example-21-full-circle render\n\n# Render the master brand film\ncd examples/19_gitframes_film && pnpm render\n```\n\nAn optimized [`Dockerfile.renderer`](https://github.com/gatewai-dev/gitframes/blob/main/Dockerfile.renderer) deploys the renderer service into cloud GPU clusters:\n\n```\ndocker build -t gitframes-renderer -f Dockerfile.renderer .\n```\n\n- **Discord:** ask questions, share renders, and follow development at[discord.gg/cbqMGGme5](https://discord.gg/cbqMGGme5) .\n- **YouTube:** watch films made with gitframes on[@gatewai.studio](https://www.youtube.com/@gatewai.studio) .\n\nGitframes is open-source software licensed under [Apache-2.0](https://github.com/gatewai-dev/gitframes/blob/main/LICENCE). The vision models it downloads on demand — RTMDet-Ins and RTMO (OpenMMLab) and the Selfie Segmenter (Google) — are also Apache-2.0; see [`registry.ts`](https://github.com/gatewai-dev/gitframes/blob/main/packages/vision/src/model/registry.ts) for exact sources and checksums.", "url": "https://wpnews.pro/news/gitframes", "canonical_source": "https://github.com/gatewai-dev/gitframes", "published_at": "2026-10-05 12:59:14+00:00", "updated_at": "2026-10-05 13:21:10.148215+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "generative-ai"], "entities": ["gitframes", "gatewai-dev", "Remotion", "Hyperframes", "Claude Code", "Dawn", "WebGPU", "npm"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/gitframes", "markdown": "https://wpnews.pro/news/gitframes.md", "text": "https://wpnews.pro/news/gitframes.txt", "jsonld": "https://wpnews.pro/news/gitframes.jsonld"}}