{"slug": "does-a-tiny-desk-ai-clip-keep-the-reference-s-cuts-and-camera-moves-measuring", "title": "Does a Tiny Desk AI Clip Keep the Reference's Cuts and Camera Moves? Measuring Shot Timing With NumPy Phase Correlation (Tested)", "summary": "A developer built a NumPy and ffmpeg-only tool that uses phase correlation to measure hard cuts and global camera motion in short vertical video clips, then applied it to eight public sample files from the Tiny Desk Outstanding photo-to-video studio to test its claim that generated clips preserve the source's length, camera and soundtrack. The script decodes at 12 fps and 144×256 grayscale, estimating pan and tilt from the phase-correlation peak while flagging cuts where the peak falls below 0.06.", "body_md": "\"Does the AI version keep the same camera moves as the original?\" is one of the first things people ask about photo-to-video meme generators. The [Tiny Desk Outstanding](https://tinydeskoutstanding.video/) studio makes a strong claim here. Its about page says a photo \"does not change the length, the camera, or the song already in the clip\", and every scene page calls the source \"a 15-second vertical clip with the original soundtrack\".\n\nThat's a testable claim. The home page has eight public sample files: the \"Whole Band reference video\" and seven generated or alternate cuts. So I measured two things in each one: **where the hard cuts are** and **how much the whole picture moves between frames**. I used NumPy and ffmpeg only, no OpenCV. Everything below was run on the box on November 14, 2026 (Asia/Shanghai), and the outputs are pasted as they came out.\n\nWhen a camera pans, most of the frame shifts by the same amount. When you shift an image, its Fourier transform keeps the same magnitude and only the phase changes. If you divide the cross-power spectrum of two frames by its own magnitude and take the inverse FFT, you get one sharp peak at the shift. That peak's height also tells you something useful:\n\nSo the same 10 lines give you a pan/tilt estimate and a cut detector.\n\n`camera_motion.py` decodes the clip at 12 fps, 144×256 grayscale (aspect forced to 9:16 so clips of different sizes compare), and runs phase correlation between consecutive frames:\n\n``` bash\n#!/usr/bin/env python3\n\"\"\"Global camera motion + cut map for a short vertical clip (NumPy + ffmpeg only).\n\nusage: python3 camera_motion.py clip.mp4 [more.mp4 ...] [--fps 12] [--csv out.csv]\n\"\"\"\nimport argparse, csv, subprocess, sys\nimport numpy as np\n\nW, H = 144, 256  # analysis size (9:16); small is enough for global motion\n\ndef frames(path, fps):\n    cmd = [\"ffmpeg\", \"-v\", \"error\", \"-i\", path, \"-vf\",\n           f\"fps={fps},scale={W}:{H}:force_original_aspect_ratio=disable,format=gray\",\n           \"-f\", \"rawvideo\", \"-\"]\n    raw = subprocess.run(cmd, capture_output=True, check=True).stdout\n    return np.frombuffer(raw, np.uint8).reshape(-1, H, W).astype(np.float32)\n\nWIN = np.outer(np.hanning(H), np.hanning(W)).astype(np.float32)\n\ndef phase_corr(a, b):\n    \"\"\"Return (dx, dy, peak) so that b is roughly a shifted by (dx, dy).\"\"\"\n    A = np.fft.fft2((a - a.mean()) * WIN)\n    B = np.fft.fft2((b - b.mean()) * WIN)\n    R = np.conj(A) * B\n    R /= np.abs(R) + 1e-9\n    r = np.fft.ifft2(R).real\n    y, x = np.unravel_index(np.argmax(r), r.shape)\n    if y > H // 2: y -= H\n    if x > W // 2: x -= W\n    return float(x), float(y), float(r.max())\n\ndef analyse(path, fps, cut_peak=0.06):\n    f = frames(path, fps)\n    rows = []\n    for i in range(1, len(f)):\n        dx, dy, peak = phase_corr(f[i - 1], f[i])\n        rows.append({\"t\": round(i / fps, 3), \"dx\": dx, \"dy\": dy,\n                     \"peak\": round(peak, 3), \"cut\": peak < cut_peak})\n    return rows\n\ndef summary(rows):\n    cuts = [r[\"t\"] for r in rows if r[\"cut\"]]\n    # merge cut flags closer than 0.25 s (one cut can flag two pairs)\n    merged = []\n    for t in cuts:\n        if not merged or t - merged[-1] > 0.25:\n            merged.append(t)\n    still = [r for r in rows if not r[\"cut\"]]\n    pan = sum(abs(r[\"dx\"]) for r in still)\n    tilt = sum(abs(r[\"dy\"]) for r in still)\n    moving = sum(1 for r in still if abs(r[\"dx\"]) + abs(r[\"dy\"]) >= 1) / max(len(still), 1)\n    return merged, pan, tilt, moving\n\nif __name__ == \"__main__\":\n    ap = argparse.ArgumentParser()\n    ap.add_argument(\"clips\", nargs=\"+\")\n    ap.add_argument(\"--fps\", type=float, default=12)\n    ap.add_argument(\"--csv\")\n    a = ap.parse_args()\n    out = []\n    print(f\"{'clip':<14}{'cuts (s)':<34}{'pan px':>8}{'tilt px':>8}{'moving':>8}\")\n    for c in a.clips:\n        rows = analyse(c, a.fps)\n        cuts, pan, tilt, moving = summary(rows)\n        name = c.rsplit(\"/\", 1)[-1][:12]\n        print(f\"{name:<14}{', '.join(f'{t:.2f}' for t in cuts):<34}{pan:>8.0f}{tilt:>8.0f}{moving:>8.0%}\")\n        out += [{\"clip\": name, **r} for r in rows]\n    if a.csv:\n        with open(a.csv, \"w\", newline=\"\") as fh:\n            w = csv.DictWriter(fh, fieldnames=list(out[0]))\n            w.writeheader(); w.writerows(out)\n```\n\nBefore pointing it at real clips, I cut two windows out of the same blurred noise texture with a known offset, and also compared the frame with unrelated noise:\n\n``` python\nimport numpy as np, camera_motion as cm\n# smooth random texture, then cut two windows offset by a known shift\nrng = np.random.default_rng(0)\nbig = rng.random((cm.H + 40, cm.W + 40)).astype(np.float32)\nfor _ in range(3):  # cheap blur so it looks like an image, not noise\n    big = (big + np.roll(big, 1, 0) + np.roll(big, 1, 1) + np.roll(big, -1, 0) + np.roll(big, -1, 1)) / 5\na = big[20:20 + cm.H, 20:20 + cm.W]\nfor dx, dy in [(3, 0), (-2, 5), (7, -4)]:\n    b = big[20 - dy:20 - dy + cm.H, 20 - dx:20 - dx + cm.W]\n    x, y, peak = cm.phase_corr(a, b)\n    print(\"true\", (dx, dy), \"measured\", (x, y), \"peak\", round(peak, 3))\nother = rng.random((cm.H, cm.W)).astype(np.float32)\nprint(\"unrelated frame peak\", round(cm.phase_corr(a, other)[2], 3))\ntrue (3, 0) measured (3.0, 0.0) peak 0.977\ntrue (-2, 5) measured (-2.0, 5.0) peak 0.972\ntrue (7, -4) measured (7.0, -4.0) peak 0.936\nunrelated frame peak 0.03\n```\n\nThe shift comes back exactly, matching frames peak at 0.94 to 0.98, and unrelated frames peak at 0.03. In the real clips, cuts landed at 0.027 to 0.045 and ordinary frames had a median peak of 0.23 to 0.31, so a 0.06 threshold sits in the gap.\n\n```\npython3 camera_motion.py samples/*.mp4 --csv motion.csv\nclip          cuts (s)                            pan px tilt px  moving\n0299078c3f43  5.33, 11.67, 15.00                     152     160     57%\n632c0129d45a  5.33, 11.67, 15.00                     248     212     71%\n8256cd873b7e  5.25, 11.42, 14.75                     225     203     60%\n875d2458181e                                         102      60     94%\n947f5eb2a447  1.50, 7.00, 13.67, 17.33               257     258     59%\ndb033545db00  1.58, 4.75, 7.83                       160     145     64%\ndb09b1c54359  5.17, 11.17                            260     164     74%\nf7ff52686113  1.50, 6.92, 10.08, 13.17, 16.50        273     274     62%\n```\n\n(`pan px` / `tilt px` add up the absolute per-frame shift at 144×256 within shots, and `moving` is the share of frame pairs that shifted by at least 1 px.) `db09b1c54359` is the reference. `0299…`, `632c…` and `8256…` are the three full recasts (`632c…` is the cat version the home page also shows). The others are alternate or partial cuts the site labels as such (a 10-second take, a longer cut, a 4-second close-up).\n\nTo line the recasts up against the reference shot by shot, `shot_compare.py` splits each clip at its cuts and reports the mean shift per frame in each shot:\n\n``` bash\n#!/usr/bin/env python3\n\"\"\"Compare each clip's shot list and per-shot camera motion with a reference.\n\nusage: python3 shot_compare.py reference.mp4 clip.mp4 [clip.mp4 ...]\n\"\"\"\nimport sys\nimport numpy as np\nfrom camera_motion import analyse, summary\n\nFPS = 12\n\ndef shots(path):\n    rows = analyse(path, FPS)\n    cuts, *_ = summary(rows)\n    end = round((len(rows) + 1) / FPS, 2)\n    edges = [0.0] + cuts + [end]\n    out = []\n    for a, b in zip(edges, edges[1:]):\n        seg = [r for r in rows if a < r[\"t\"] < b and not r[\"cut\"]]\n        mag = np.mean([np.hypot(r[\"dx\"], r[\"dy\"]) for r in seg]) if seg else 0.0\n        out.append((a, b, mag))\n    return out\n\ndef fmt(s):\n    return \" | \".join(f\"{a:5.2f}-{b:5.2f}s {m:4.2f}px/f\" for a, b, m in s)\n\nref = shots(sys.argv[1])\nprint(\"reference \", fmt(ref))\nfor clip in sys.argv[2:]:\n    s = shots(clip)\n    name = clip.rsplit(\"/\", 1)[-1][:10]\n    print(f\"{name:<10} \", fmt(s))\n    n = min(len(ref), len(s))\n    drift = [s[i][1] - ref[i][1] for i in range(n - 1)]  # cut-time differences\n    print(f\"{'':<11} shots {len(s)} vs {len(ref)}; cut drift vs reference: \"\n          + \", \".join(f\"{d:+.2f}s\" for d in drift))\npython3 shot_compare.py samples/db09b1c5…mp4 samples/0299078c…mp4 samples/632c0129…mp4 samples/8256cd87…mp4\nreference   0.00- 5.17s 0.94px/f |  5.17-11.17s 2.59px/f | 11.17-14.50s 2.58px/f\n0299078c3f   0.00- 5.33s 0.22px/f |  5.33-11.67s 2.36px/f | 11.67-15.00s 1.42px/f | 15.00-17.58s 0.41px/f\n            shots 4 vs 3; cut drift vs reference: +0.17s, +0.50s\n632c0129d4   0.00- 5.33s 0.68px/f |  5.33-11.67s 2.91px/f | 11.67-15.00s 2.50px/f | 15.00-17.67s 0.74px/f\n            shots 4 vs 3; cut drift vs reference: +0.17s, +0.50s\n8256cd873b   0.00- 5.25s 0.13px/f |  5.25-11.42s 3.37px/f | 11.42-14.75s 1.87px/f | 14.75-17.42s 0.66px/f\n            shots 4 vs 3; cut drift vs reference: +0.08s, +0.25s\n```\n\nCaveats: phase correlation measures the single dominant shift, so a laughing head in a close-up counts as \"motion\" even if the camera is still. It doesn't detect zoom or rotation. Use it to compare clips against each other, not as an absolute camera tracker. A 12 fps analysis can't place a cut more precisely than about 0.08 s.\n\nIf you plan to cut your own clip on the beat, add a slow-motion replay, or line up captions with the laugh, **don't hard-code the reference's timestamps**. Run `camera_motion.py` on *your* file and use its cut list. The shots are in the same order, but a recast can be up to half a second late and about 3 seconds longer.\n\nThe scene choices (Just Me for the close-up, Back Row for the two standing singers in the opening shot, Whole Band for all five seats) are compared on the [scenes page](https://tinydeskoutstanding.video/templates). Every scene uses the same reference, so these shot timings apply to all three.", "url": "https://wpnews.pro/news/does-a-tiny-desk-ai-clip-keep-the-reference-s-cuts-and-camera-moves-measuring", "canonical_source": "https://dev.to/lpgit/does-a-tiny-desk-ai-clip-keep-the-references-cuts-and-camera-moves-measuring-shot-timing-with-2i14", "published_at": "2026-11-14 04:13:42+00:00", "updated_at": "2026-11-14 04:20:38.201862+00:00", "lang": "en", "topics": ["computer-vision", "ai-tools", "developer-tools"], "entities": ["Tiny Desk Outstanding", "NumPy", "ffmpeg"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/does-a-tiny-desk-ai-clip-keep-the-reference-s-cuts-and-camera-moves-measuring", "markdown": "https://wpnews.pro/news/does-a-tiny-desk-ai-clip-keep-the-reference-s-cuts-and-camera-moves-measuring.md", "text": "https://wpnews.pro/news/does-a-tiny-desk-ai-clip-keep-the-reference-s-cuts-and-camera-moves-measuring.txt", "jsonld": "https://wpnews.pro/news/does-a-tiny-desk-ai-clip-keep-the-reference-s-cuts-and-camera-moves-measuring.jsonld"}}