{"slug": "tiny-desk-ai-video-in-ffmpeg-from-five-ai-edits-to-one-571-frame-file", "title": "Tiny Desk AI video in ffmpeg: from five AI edits to one 571-frame file", "summary": "A developer built a 19-second, 571-frame Tiny Desk-style AI concert video by feeding edge-detected line drawings of five source shots to Wan 3.0 and Seedance 2.5, then reassembling the AI edits in ffmpeg at 30 fps with NPR's live recording underneath. The pipeline uses separate ffmpeg filter chains for band shots and soft close-ups, keyframed masks that black out 22 objects across the five shots, and trims by seconds rather than frame count because the two models return different frame rates and lengths.", "body_md": "The Tiny Desk AI video trend turns five source shots into one mock concert that runs 19 seconds using edge-detected outlines. The ffmpeg work is three jobs: turn each shot into a line drawing, trim and join five AI edits at 30 fps into 571 frames, and lay NPR's live recording back under it. Here's the one trap: trim by seconds, never by frame count, because the two models hand back different frame rates and lengths.\n\nHere are the commands in the order we run them, with the numbers we got. You can run them directly in your shell next to this post.\n\n```\n# band shots\nffmpeg -i shot.mp4 -vf \"scale=720:1280:flags=lanczos,gblur=sigma=2,edgedetect=low=0.1:high=0.25,format=gray\" -r 24 -an lines.mp4\n# soft close-ups\nffmpeg -i shot.mp4 -vf \"scale=720:1280:flags=lanczos,unsharp=7:7:1.5,gblur=sigma=1.2,edgedetect=low=0.05:high=0.15,format=gray\" -r 24 -an lines.mp4\n```\n\nWe send line drawings as the reference files for the AI models. The model never sees filmed footage. It gets an edge-detected line drawing of each shot: grey outlines on black, 720×1280, 9:16, muted. No face, outfit or colour of the original people survives the filter.\n\nBand shots use `gblur=sigma=2,edgedetect=low=0.1:high=0.25,format=gray`. That's a light blur first so fabric texture doesn't turn into lines, then edge detection, then grey.\n\nThe two close-ups are soft in the AI clip, so the band filter lit only about 0.4–0.9% of their pixels. The sharper chain uses `unsharp=7:7:1.5,gblur=sigma=1.2,edgedetect=low=0.05:high=0.15,format=gray`. It sharpens first and lowers the edge thresholds.\n\nThat brings them to about 2%, which gives the model enough outline to follow the head turns and the laugh. We scale drawings to 720×1280 with lanczos, encode them at 24 fps (`-r 24`), and mute them with `-an`.\n\nThe model gets a drawing, not the video, because stripping colour and clothes leaves only motion.\n\nFor the Wan 3.0 route, we also black out 22 objects across the five shots on every frame. These include caps, headphones and headsets, chest logos, a printed slogan, a hood, and an out-of-focus person in a corner. We use keyframed masks that keep the lines of the eyes, nose and mouth.\n\nBetween 62% and 81% of the lit pixels stay. If you leave anything in the drawing, it ends up in the video, which is why the masks exist.\n\nThe shot layout traces the viral 19-second motion source shot for shot. TikTok creator @theburnerfromhe11 posted [College Football's Week 8 All-Loss Tiny Desk Concert](https://www.tiktok.com/@theburnerfromhe11/video/7698160711830129934) on 2026-10-18; it passed 2.3 million views.\n\nOur motion source is that viral AI clip, not NPR's footage, so our video matches it shot for shot and second for second. That clip is vertical 9:16, 720×1280, 24 fps, and 457 frames (19.042 seconds). Hard cuts after frames 33, 161, 312 and 393 give five shots.\n\nThe lead singer's in shots 1 and 3, the band (keyboard player and two backup singers) is in shots 2 and 5, and the sax player's only in shot 4. One to five photos fill those five spots.\n\n| Shot | Time (s) | Frames | Who is on screen | \n|---|---|---|---|\n| 1 | 0.000–1.375 | 33 | Lead singer, close-up in profile | \n| 2 | 1.375–6.708 | 128 | Keyboard player front centre, two backup singers behind | \n| 3 | 6.708–13.000 | 151 | Lead singer, laughing close-up | \n| 4 | 13.000–16.375 | 81 | Sax solo | \n| 5 | 16.375–19.042 | 64 | The band again | \n\nThe resolution picks the engine: 480p and 720p run on Wan 3.0, while 1080p runs on Seedance 2.5. Each edit gets its own shot's photos and drawing, with the model's sound turned off.\n\nSending the two band shots as one drawing keeps the same three people on both sides of the cut. We pad Seedance drawings with their last frame to hit those whole seconds. You can grab the [copy-ready Tiny Desk AI prompts for every shot](https://seadanse.com/blog/tiny-desk-ai-prompt) and role on our blog.\n\n| On the same drawings, 480p | Wan 3.0 | Seedance 2.5 | What it means for you | \n|---|---|---|---|\n| Jobs per video | 4 | 5 | Wan lets you send both band shots as one clip | \n| Seconds sent | 19.667 (the drawings' own lengths) | 25 (whole seconds, padded) | Seedance renders more footage than you keep | \n| Frame rate back | 30 fps | 24 fps | The join forces 30 fps either way | \n| Length back | The drawing's length | Asked length plus one frame | Trim by time, never by count | \n\nAI models return different lengths and frame rates from the exact same drawings. Seedance segments come back at 24 fps and exactly one frame longer than you asked for. That means 4 s comes back as 4.0417 s, 6 s as 6.0417 s, and 7 s as 7.0417 s, so there's always one extra frame at 24 fps.\n\nWan edits come back at 30 fps and follow the drawing's own length. Shot 1 is 33 frames (1.375 s), but its Wan drawing is 48 frames (2.0 s). That stays clear of the shortest clip the model handles well, and we keep only the first 1.375 s in the join.\n\nThe 8 s Wan band edit holds shot 2 (128 frames) and shot 5 (64 frames) back to back. Wan drew no visible cut at 5.333 s, so the motion runs straight through. We take shot 2 from 0 s and shot 5 from 5.333 s by time, feeding the band file to ffmpeg twice as input 1 and input 4.\n\nSo a frame count means different things in different files (24 fps source, 24 fps Seedance, 30 fps Wan); seconds mean the same thing in all of them. Every trim in our join is a duration in seconds.\n\n```\n# Wan route, 480p: four edits, the 8 s band edit used twice (shot 2 from 0 s, shot 5 from 5.333 s)\nFIT=\"scale=480:854:force_original_aspect_ratio=decrease,pad=480:854:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p\"\nffmpeg -y \\\n  -i shot1.mp4 -i band.mp4 -i shot3.mp4 -i shot4.mp4 -i band.mp4 -i song.m4a \\\n  -filter_complex \"\\\n[0:v]trim=duration=1.375,setpts=PTS-STARTPTS,$FIT[s0];\\\n[1:v]trim=duration=5.333,setpts=PTS-STARTPTS,$FIT[s1];\\\n[2:v]trim=duration=6.292,setpts=PTS-STARTPTS,$FIT[s2];\\\n[3:v]trim=duration=3.375,setpts=PTS-STARTPTS,$FIT[s3];\\\n[4:v]trim=start=5.333:duration=2.667,setpts=PTS-STARTPTS,$FIT[s4];\\\n[s0][s1][s2][s3][s4]concat=n=5:v=1:a=0[v]\" \\\n  -map \"[v]\" -map 5:a:0 \\\n  -c:v libx264 -preset veryfast -crf 20 -c:a aac -b:a 128k \\\n  -shortest -movflags +faststart tiny-desk.mp4\n```\n\nOne ffmpeg filter graph does the whole join with one re-encode.\n\n`setpts=PTS-STARTPTS`.` setsar=1`.` format=yuv420p`.` n=5:v=1:a=0`.`[v]` and the song from the sixth input `5:a:0`.`-preset veryfast -crf 20`.`-movflags +faststart` so playback starts before the file's fully downloaded.\nWe force the output to 30 fps, so every finished file is 571 frames and 19.033 seconds, at either 480×854 or 720×1280. We probe the first shot to set that target size.\n\nThe block above is the Wan route at 480p. The Seedance route runs the same graph on its five segment files, each trimmed from 0 s to the same shot lengths.\n\nA join gets 3 tries and 75 seconds per try. Locally, our join finished 5.45 seconds after the last shot downloaded.\n\nThis is the file the join makes.\n\nNPR's live recording replaces the silent model outputs. We always turn the model's own audio off (`audio: false` on Wan, `generate_audio: false` on Seedance).\n\nWhat you'll hear is NPR's live recording from [Charlie Wilson's 2023 NPR Tiny Desk concert](https://www.youtube.com/watch?v=J3jUEB7qmt8), posted on YouTube on 2023-06-05. The closing number is \"Outstanding\", The Gap Band's 1982 song, running at about 98 BPM. We take roughly 23:45.886 to 24:04.928 of their upload (19.042 s), lay it under the joined video from 0 s, and add a 0.3 s fade-out at the end.\n\nWe found that span by audio cross-correlation (sliding one soundtrack along the other and scoring how well they line up at each offset) against the AI clip's soundtrack. The match peak stood 10.2 times above the next best, so there's one clear answer, not a shortlist.\n\n```\n# NPR's live recording, 23:45.886 for 19.042 s, 0.3 s fade-out at the end\nffmpeg -i npr-upload.m4a -ss 23:45.886 -t 19.042 -vn \\\n  -af \"afade=t=out:st=18.742:d=0.3\" -c:a aac -b:a 128k song.m4a\n# real frame count (571 on a finished file)\nffprobe -v error -count_frames -select_streams v:0 \\\n  -show_entries stream=nb_read_frames,width,height,r_frame_rate -of compact tiny-desk.mp4\n# any freeze of 0.3 s or more prints a freeze_start line\nffmpeg -i tiny-desk.mp4 -vf freezedetect=d=0.3 -map 0:v -f null - 2>&1 | grep freeze_start\n```\n\nYou check the output before you ship it. Reading every frame with `ffprobe -count_frames` confirms the real count: 571 on a finished file.\n\nThe freezedetect filter flags any frozen video lasting 0.3 s or more. We found no freeze of 0.3 s or more in any shipped test file across 25 test renders we ran on 26 and 27 October 2026.\n\nInput portrait photos need clean geometry before the video models see them. Our free [Tiny Desk AI Photo Prep Space](https://huggingface.co/spaces/codeugar/tiny-desk-photo-prep) runs the photo steps right in your browser, without storing anything:\n\nIf you want to skip the terminal, [Seadanse's Tiny Desk AI effect](https://seadanse.com/video-effects/tiny-desk-ai) turns 1 to 5 photos into the 19-second 9:16 video with the original \"Outstanding\" sound in a few minutes. The drawings and join described above are the exact ones the site runs today. A 720p video costs $9.90 (one $9.90 credit pack, no subscription), with no sign-up needed to order.\n\nEvery multi-shot AI video assembly follows the same timing principles in ffmpeg.\n\n`-shortest`.\n*Disclosure: we build Seadanse, which sells this Tiny Desk AI effect. The numbers here come from 25 test renders we ran on 26 and 27 October 2026, and every person in our test photos is AI-generated.*", "url": "https://wpnews.pro/news/tiny-desk-ai-video-in-ffmpeg-from-five-ai-edits-to-one-571-frame-file", "canonical_source": "https://dev.to/codesugar_lin_037a57b06a4/tiny-desk-ai-video-in-ffmpeg-from-five-ai-edits-to-one-571-frame-file-58e8", "published_at": "2026-10-27 15:30:58+00:00", "updated_at": "2026-10-27 15:42:14.168237+00:00", "lang": "en", "topics": ["ai-tools", "generative-ai", "ai-products"], "entities": ["ffmpeg", "Wan 3.0", "Seedance 2.5", "NPR", "TikTok", "@theburnerfromhe11"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tiny-desk-ai-video-in-ffmpeg-from-five-ai-edits-to-one-571-frame-file", "markdown": "https://wpnews.pro/news/tiny-desk-ai-video-in-ffmpeg-from-five-ai-edits-to-one-571-frame-file.md", "text": "https://wpnews.pro/news/tiny-desk-ai-video-in-ffmpeg-from-five-ai-edits-to-one-571-frame-file.txt", "jsonld": "https://wpnews.pro/news/tiny-desk-ai-video-in-ffmpeg-from-five-ai-edits-to-one-571-frame-file.jsonld"}}