# Does a Tiny Desk AI Clip Keep the Reference's Cuts and Camera Moves? Measuring Shot Timing With NumPy Phase Correlation (Tested)

> Source: <https://dev.to/lpgit/does-a-tiny-desk-ai-clip-keep-the-references-cuts-and-camera-moves-measuring-shot-timing-with-2i14>
> Published: 2026-11-14 04:13:42+00:00

"Does the AI version keep the same camera moves as the original?" is one of the first things people ask about photo-to-video meme generators. The [Tiny Desk Outstanding](https://tinydeskoutstanding.video/) studio makes a strong claim here. Its about page says a photo "does not change the length, the camera, or the song already in the clip", and every scene page calls the source "a 15-second vertical clip with the original soundtrack".

That's a testable claim. The home page has eight public sample files: the "Whole Band reference video" and seven generated or alternate cuts. So I measured two things in each one: **where the hard cuts are** and **how much the whole picture moves between frames**. I used NumPy and ffmpeg only, no OpenCV. Everything below was run on the box on November 14, 2026 (Asia/Shanghai), and the outputs are pasted as they came out.

When a camera pans, most of the frame shifts by the same amount. When you shift an image, its Fourier transform keeps the same magnitude and only the phase changes. If you divide the cross-power spectrum of two frames by its own magnitude and take the inverse FFT, you get one sharp peak at the shift. That peak's height also tells you something useful:

So the same 10 lines give you a pan/tilt estimate and a cut detector.

`camera_motion.py` decodes the clip at 12 fps, 144×256 grayscale (aspect forced to 9:16 so clips of different sizes compare), and runs phase correlation between consecutive frames:

``` bash
#!/usr/bin/env python3
"""Global camera motion + cut map for a short vertical clip (NumPy + ffmpeg only).

usage: python3 camera_motion.py clip.mp4 [more.mp4 ...] [--fps 12] [--csv out.csv]
"""
import argparse, csv, subprocess, sys
import numpy as np

W, H = 144, 256  # analysis size (9:16); small is enough for global motion

def frames(path, fps):
    cmd = ["ffmpeg", "-v", "error", "-i", path, "-vf",
           f"fps={fps},scale={W}:{H}:force_original_aspect_ratio=disable,format=gray",
           "-f", "rawvideo", "-"]
    raw = subprocess.run(cmd, capture_output=True, check=True).stdout
    return np.frombuffer(raw, np.uint8).reshape(-1, H, W).astype(np.float32)

WIN = np.outer(np.hanning(H), np.hanning(W)).astype(np.float32)

def phase_corr(a, b):
    """Return (dx, dy, peak) so that b is roughly a shifted by (dx, dy)."""
    A = np.fft.fft2((a - a.mean()) * WIN)
    B = np.fft.fft2((b - b.mean()) * WIN)
    R = np.conj(A) * B
    R /= np.abs(R) + 1e-9
    r = np.fft.ifft2(R).real
    y, x = np.unravel_index(np.argmax(r), r.shape)
    if y > H // 2: y -= H
    if x > W // 2: x -= W
    return float(x), float(y), float(r.max())

def analyse(path, fps, cut_peak=0.06):
    f = frames(path, fps)
    rows = []
    for i in range(1, len(f)):
        dx, dy, peak = phase_corr(f[i - 1], f[i])
        rows.append({"t": round(i / fps, 3), "dx": dx, "dy": dy,
                     "peak": round(peak, 3), "cut": peak < cut_peak})
    return rows

def summary(rows):
    cuts = [r["t"] for r in rows if r["cut"]]
    # merge cut flags closer than 0.25 s (one cut can flag two pairs)
    merged = []
    for t in cuts:
        if not merged or t - merged[-1] > 0.25:
            merged.append(t)
    still = [r for r in rows if not r["cut"]]
    pan = sum(abs(r["dx"]) for r in still)
    tilt = sum(abs(r["dy"]) for r in still)
    moving = sum(1 for r in still if abs(r["dx"]) + abs(r["dy"]) >= 1) / max(len(still), 1)
    return merged, pan, tilt, moving

if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("clips", nargs="+")
    ap.add_argument("--fps", type=float, default=12)
    ap.add_argument("--csv")
    a = ap.parse_args()
    out = []
    print(f"{'clip':<14}{'cuts (s)':<34}{'pan px':>8}{'tilt px':>8}{'moving':>8}")
    for c in a.clips:
        rows = analyse(c, a.fps)
        cuts, pan, tilt, moving = summary(rows)
        name = c.rsplit("/", 1)[-1][:12]
        print(f"{name:<14}{', '.join(f'{t:.2f}' for t in cuts):<34}{pan:>8.0f}{tilt:>8.0f}{moving:>8.0%}")
        out += [{"clip": name, **r} for r in rows]
    if a.csv:
        with open(a.csv, "w", newline="") as fh:
            w = csv.DictWriter(fh, fieldnames=list(out[0]))
            w.writeheader(); w.writerows(out)
```

Before pointing it at real clips, I cut two windows out of the same blurred noise texture with a known offset, and also compared the frame with unrelated noise:

``` python
import numpy as np, camera_motion as cm
# smooth random texture, then cut two windows offset by a known shift
rng = np.random.default_rng(0)
big = rng.random((cm.H + 40, cm.W + 40)).astype(np.float32)
for _ in range(3):  # cheap blur so it looks like an image, not noise
    big = (big + np.roll(big, 1, 0) + np.roll(big, 1, 1) + np.roll(big, -1, 0) + np.roll(big, -1, 1)) / 5
a = big[20:20 + cm.H, 20:20 + cm.W]
for dx, dy in [(3, 0), (-2, 5), (7, -4)]:
    b = big[20 - dy:20 - dy + cm.H, 20 - dx:20 - dx + cm.W]
    x, y, peak = cm.phase_corr(a, b)
    print("true", (dx, dy), "measured", (x, y), "peak", round(peak, 3))
other = rng.random((cm.H, cm.W)).astype(np.float32)
print("unrelated frame peak", round(cm.phase_corr(a, other)[2], 3))
true (3, 0) measured (3.0, 0.0) peak 0.977
true (-2, 5) measured (-2.0, 5.0) peak 0.972
true (7, -4) measured (7.0, -4.0) peak 0.936
unrelated frame peak 0.03
```

The shift comes back exactly, matching frames peak at 0.94 to 0.98, and unrelated frames peak at 0.03. In the real clips, cuts landed at 0.027 to 0.045 and ordinary frames had a median peak of 0.23 to 0.31, so a 0.06 threshold sits in the gap.

```
python3 camera_motion.py samples/*.mp4 --csv motion.csv
clip          cuts (s)                            pan px tilt px  moving
0299078c3f43  5.33, 11.67, 15.00                     152     160     57%
632c0129d45a  5.33, 11.67, 15.00                     248     212     71%
8256cd873b7e  5.25, 11.42, 14.75                     225     203     60%
875d2458181e                                         102      60     94%
947f5eb2a447  1.50, 7.00, 13.67, 17.33               257     258     59%
db033545db00  1.58, 4.75, 7.83                       160     145     64%
db09b1c54359  5.17, 11.17                            260     164     74%
f7ff52686113  1.50, 6.92, 10.08, 13.17, 16.50        273     274     62%
```

(`pan px` / `tilt px` add up the absolute per-frame shift at 144×256 within shots, and `moving` is the share of frame pairs that shifted by at least 1 px.) `db09b1c54359` is the reference. `0299…`, `632c…` and `8256…` are the three full recasts (`632c…` is the cat version the home page also shows). The others are alternate or partial cuts the site labels as such (a 10-second take, a longer cut, a 4-second close-up).

To line the recasts up against the reference shot by shot, `shot_compare.py` splits each clip at its cuts and reports the mean shift per frame in each shot:

``` bash
#!/usr/bin/env python3
"""Compare each clip's shot list and per-shot camera motion with a reference.

usage: python3 shot_compare.py reference.mp4 clip.mp4 [clip.mp4 ...]
"""
import sys
import numpy as np
from camera_motion import analyse, summary

FPS = 12

def shots(path):
    rows = analyse(path, FPS)
    cuts, *_ = summary(rows)
    end = round((len(rows) + 1) / FPS, 2)
    edges = [0.0] + cuts + [end]
    out = []
    for a, b in zip(edges, edges[1:]):
        seg = [r for r in rows if a < r["t"] < b and not r["cut"]]
        mag = np.mean([np.hypot(r["dx"], r["dy"]) for r in seg]) if seg else 0.0
        out.append((a, b, mag))
    return out

def fmt(s):
    return " | ".join(f"{a:5.2f}-{b:5.2f}s {m:4.2f}px/f" for a, b, m in s)

ref = shots(sys.argv[1])
print("reference ", fmt(ref))
for clip in sys.argv[2:]:
    s = shots(clip)
    name = clip.rsplit("/", 1)[-1][:10]
    print(f"{name:<10} ", fmt(s))
    n = min(len(ref), len(s))
    drift = [s[i][1] - ref[i][1] for i in range(n - 1)]  # cut-time differences
    print(f"{'':<11} shots {len(s)} vs {len(ref)}; cut drift vs reference: "
          + ", ".join(f"{d:+.2f}s" for d in drift))
python3 shot_compare.py samples/db09b1c5…mp4 samples/0299078c…mp4 samples/632c0129…mp4 samples/8256cd87…mp4
reference   0.00- 5.17s 0.94px/f |  5.17-11.17s 2.59px/f | 11.17-14.50s 2.58px/f
0299078c3f   0.00- 5.33s 0.22px/f |  5.33-11.67s 2.36px/f | 11.67-15.00s 1.42px/f | 15.00-17.58s 0.41px/f
            shots 4 vs 3; cut drift vs reference: +0.17s, +0.50s
632c0129d4   0.00- 5.33s 0.68px/f |  5.33-11.67s 2.91px/f | 11.67-15.00s 2.50px/f | 15.00-17.67s 0.74px/f
            shots 4 vs 3; cut drift vs reference: +0.17s, +0.50s
8256cd873b   0.00- 5.25s 0.13px/f |  5.25-11.42s 3.37px/f | 11.42-14.75s 1.87px/f | 14.75-17.42s 0.66px/f
            shots 4 vs 3; cut drift vs reference: +0.08s, +0.25s
```

Caveats: phase correlation measures the single dominant shift, so a laughing head in a close-up counts as "motion" even if the camera is still. It doesn't detect zoom or rotation. Use it to compare clips against each other, not as an absolute camera tracker. A 12 fps analysis can't place a cut more precisely than about 0.08 s.

If you plan to cut your own clip on the beat, add a slow-motion replay, or line up captions with the laugh, **don't hard-code the reference's timestamps**. Run `camera_motion.py` on *your* file and use its cut list. The shots are in the same order, but a recast can be up to half a second late and about 3 seconds longer.

The scene choices (Just Me for the close-up, Back Row for the two standing singers in the opening shot, Whole Band for all five seats) are compared on the [scenes page](https://tinydeskoutstanding.video/templates). Every scene uses the same reference, so these shot timings apply to all three.