Tiny Desk AI video in ffmpeg: from five AI edits to one 571-frame file

The Tiny Desk AI video trend turns five source shots into one mock concert that runs 19 seconds using edge-detected outlines. The ffmpeg work is three jobs: turn each shot into a line drawing, trim and join five AI edits at 30 fps into 571 frames, and lay NPR's live recording back under it. Here's the one trap: trim by seconds, never by frame count, because the two models hand back different frame rates and lengths.

Here are the commands in the order we run them, with the numbers we got. You can run them directly in your shell next to this post.

ffmpeg -i shot.mp4 -vf "scale=720:1280:flags=lanczos,gblur=sigma=2,edgedetect=low=0.1:high=0.25,format=gray" -r 24 -an lines.mp4
ffmpeg -i shot.mp4 -vf "scale=720:1280:flags=lanczos,unsharp=7:7:1.5,gblur=sigma=1.2,edgedetect=low=0.05:high=0.15,format=gray" -r 24 -an lines.mp4

We send line drawings as the reference files for the AI models. The model never sees filmed footage. It gets an edge-detected line drawing of each shot: grey outlines on black, 720×1280, 9:16, muted. No face, outfit or colour of the original people survives the filter.

Band shots use gblur=sigma=2,edgedetect=low=0.1:high=0.25,format=gray. That's a light blur first so fabric texture doesn't turn into lines, then edge detection, then grey.

The two close-ups are soft in the AI clip, so the band filter lit only about 0.4–0.9% of their pixels. The sharper chain uses unsharp=7:7:1.5,gblur=sigma=1.2,edgedetect=low=0.05:high=0.15,format=gray. It sharpens first and lowers the edge thresholds.

That brings them to about 2%, which gives the model enough outline to follow the head turns and the laugh. We scale drawings to 720×1280 with lanczos, encode them at 24 fps (-r 24), and mute them with -an.

The model gets a drawing, not the video, because stripping colour and clothes leaves only motion.

For the Wan 3.0 route, we also black out 22 objects across the five shots on every frame. These include caps, headphones and headsets, chest logos, a printed slogan, a hood, and an out-of-focus person in a corner. We use keyframed masks that keep the lines of the eyes, nose and mouth.

Between 62% and 81% of the lit pixels stay. If you leave anything in the drawing, it ends up in the video, which is why the masks exist.

The shot layout traces the viral 19-second motion source shot for shot. TikTok creator @theburnerfromhe11 posted College Football's Week 8 All-Loss Tiny Desk Concert on 2026-10-18; it passed 2.3 million views.

Our motion source is that viral AI clip, not NPR's footage, so our video matches it shot for shot and second for second. That clip is vertical 9:16, 720×1280, 24 fps, and 457 frames (19.042 seconds). Hard cuts after frames 33, 161, 312 and 393 give five shots.

The lead singer's in shots 1 and 3, the band (keyboard player and two backup singers) is in shots 2 and 5, and the sax player's only in shot 4. One to five photos fill those five spots.

The resolution picks the engine: 480p and 720p run on Wan 3.0, while 1080p runs on Seedance 2.5. Each edit gets its own shot's photos and drawing, with the model's sound turned off.

Sending the two band shots as one drawing keeps the same three people on both sides of the cut. We pad Seedance drawings with their last frame to hit those whole seconds. You can grab the copy-ready Tiny Desk AI prompts for every shot and role on our blog.

AI models return different lengths and frame rates from the exact same drawings. Seedance segments come back at 24 fps and exactly one frame longer than you asked for. That means 4 s comes back as 4.0417 s, 6 s as 6.0417 s, and 7 s as 7.0417 s, so there's always one extra frame at 24 fps.

Wan edits come back at 30 fps and follow the drawing's own length. Shot 1 is 33 frames (1.375 s), but its Wan drawing is 48 frames (2.0 s). That stays clear of the shortest clip the model handles well, and we keep only the first 1.375 s in the join.

The 8 s Wan band edit holds shot 2 (128 frames) and shot 5 (64 frames) back to back. Wan drew no visible cut at 5.333 s, so the motion runs straight through. We take shot 2 from 0 s and shot 5 from 5.333 s by time, feeding the band file to ffmpeg twice as input 1 and input 4.

So a frame count means different things in different files (24 fps source, 24 fps Seedance, 30 fps Wan); seconds mean the same thing in all of them. Every trim in our join is a duration in seconds.

FIT="scale=480:854:force_original_aspect_ratio=decrease,pad=480:854:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p"
ffmpeg -y \
  -i shot1.mp4 -i band.mp4 -i shot3.mp4 -i shot4.mp4 -i band.mp4 -i song.m4a \
  -filter_complex "\
[0:v]trim=duration=1.375,setpts=PTS-STARTPTS,$FIT[s0];\
[1:v]trim=duration=5.333,setpts=PTS-STARTPTS,$FIT[s1];\
[2:v]trim=duration=6.292,setpts=PTS-STARTPTS,$FIT[s2];\
[3:v]trim=duration=3.375,setpts=PTS-STARTPTS,$FIT[s3];\
[4:v]trim=start=5.333:duration=2.667,setpts=PTS-STARTPTS,$FIT[s4];\
[s0][s1][s2][s3][s4]concat=n=5:v=1:a=0[v]" \
  -map "[v]" -map 5:a:0 \
  -c:v libx264 -preset veryfast -crf 20 -c:a aac -b:a 128k \
  -shortest -movflags +faststart tiny-desk.mp4

One ffmpeg filter graph does the whole join with one re-encode.

setpts=PTS-STARTPTS. setsar=1. format=yuv420p. n=5:v=1:a=0.[v] and the song from the sixth input 5:a:0.-preset veryfast -crf 20.-movflags +faststart so playback starts before the file's fully downloaded. We force the output to 30 fps, so every finished file is 571 frames and 19.033 seconds, at either 480×854 or 720×1280. We probe the first shot to set that target size.

The block above is the Wan route at 480p. The Seedance route runs the same graph on its five segment files, each trimmed from 0 s to the same shot lengths.

A join gets 3 tries and 75 seconds per try. Locally, our join finished 5.45 seconds after the last shot downloaded.

This is the file the join makes.

NPR's live recording replaces the silent model outputs. We always turn the model's own audio off (audio: false on Wan, generate_audio: false on Seedance).

What you'll hear is NPR's live recording from Charlie Wilson's 2023 NPR Tiny Desk concert, posted on YouTube on 2023-06-05. The closing number is "Outstanding", The Gap Band's 1982 song, running at about 98 BPM. We take roughly 23:45.886 to 24:04.928 of their upload (19.042 s), lay it under the joined video from 0 s, and add a 0.3 s fade-out at the end.

We found that span by audio cross-correlation (sliding one soundtrack along the other and scoring how well they line up at each offset) against the AI clip's soundtrack. The match peak stood 10.2 times above the next best, so there's one clear answer, not a shortlist.

ffmpeg -i npr-upload.m4a -ss 23:45.886 -t 19.042 -vn \
  -af "afade=t=out:st=18.742:d=0.3" -c:a aac -b:a 128k song.m4a
ffprobe -v error -count_frames -select_streams v:0 \
  -show_entries stream=nb_read_frames,width,height,r_frame_rate -of compact tiny-desk.mp4
ffmpeg -i tiny-desk.mp4 -vf freezedetect=d=0.3 -map 0:v -f null - 2>&1 | grep freeze_start

You check the output before you ship it. Reading every frame with ffprobe -count_frames confirms the real count: 571 on a finished file.

The freezedetect filter flags any frozen video lasting 0.3 s or more. We found no freeze of 0.3 s or more in any shipped test file across 25 test renders we ran on 26 and 27 October 2026.

Input portrait photos need clean geometry before the video models see them. Our free Tiny Desk AI Photo Prep Space runs the photo steps right in your browser, without storing anything:

If you want to skip the terminal, Seadanse's Tiny Desk AI effect turns 1 to 5 photos into the 19-second 9:16 video with the original "Outstanding" sound in a few minutes. The drawings and join described above are the exact ones the site runs today. A 720p video costs $9.90 (one $9.90 credit pack, no subscription), with no sign-up needed to order.

Every multi-shot AI video assembly follows the same timing principles in ffmpeg.

-shortest. Disclosure: we build Seadanse, which sells this Tiny Desk AI effect. The numbers here come from 25 test renders we ran on 26 and 27 October 2026, and every person in our test photos is AI-generated.