Automatic beat-synced video editing.
Feed it a song and raw footage. It analyzes the beats, maps the energy, tags every scene, and cuts a ready-to-post edit — zero manual editing.
preview_flash-montage-1x1.mp4 #
|
preview_sealife-beat-edit-9x16.mp4 #
|
Full-quality versions of both demos live in examples/.
Run the four stages yourself. Same inputs, same edit, every time.
The repo ships a skill at .claude/skills/beat-sync-edit/SKILL.md
. Open this folder in Claude Code and describe the edit you want:
"cut my footage to the chorus, white flash on every drop, space the cuts out so the shots breathe, then make it 9:16"
Claude runs the same pipeline and layers ffmpeg effects at beat-accurate timestamps — white flashes, stretch punches, hue shifts, RGB split, speed ramps, shakes, color grades. The full effects cookbook lives in the skill file.
In conversation you can also dictate:
Aspect ratio— 9:16 vertical, 1:1 square, stacked over-under, or leave it 16:9** Cut density**— "a cut on every beat" vs "spaced out, let the shots breathe"** Hue / tint by reference**— point Claude at any other video on your machine ("grade it like this one") and it will sample frames from the reference and build a matching color grade; with the Claude-in-Chrome extension it can even study a look from a video on the web
song.mp3 ──► beat_map.py ──► beats, energy curve, peaks/valleys (JSON)
clips.mp4 ──► clip_tag.py ──► scenes tagged by motion/energy/brightness (JSON)
both ──► plan_edit.py ──► edit decision list: which clip hits which beat
EDL ──► render_edit.py ──► final MP4 (extract → concat → mux audio)
| Stage | What it does |
|---|---|
beat_map.py |
|
| librosa audio analysis — beat timestamps, tempo, energy over time, peaks/valleys, best highlight segments | |
clip_tag.py |
|
PySceneDetect scene detection + per-clip motion/energy/brightness scoring; --thumbs exports a labeled contact sheet |
|
plan_edit.py |
|
| the matcher — high-energy clips land on energy peaks, calm clips on valleys, with anti-repetition, source spacing, and black-frame filtering | |
render_edit.py |
|
ffmpeg assembly; --overunder emits a stacked 960×1080 variant |
|
vertical_style.py |
|
| fits 16:9 output into 9:16 with a stylized squeeze + grade for TikTok / Reels / Shorts | |
flash_montage.py |
|
the white-flash montage template — 1:1 edits from stills and/or video clips with fadewhite transitions, punch-ins, and handheld sway (see below) |
pip install -r requirements.txt # ffmpeg must be on PATH (brew install ffmpeg)
python3 beat_map.py song.mp3
python3 clip_tag.py footage.mp4 --thumbs
python3 plan_edit.py song_beatmap.json footage_clips.json --html
python3 render_edit.py footage_edl.json -a song.mp3 -v footage.mp4
All creative control lives in plan_edit.py
:
| Flag | Effect |
|---|---|
--beat-stride N |
|
Cut density. 1 = a cut on every beat (default). 2 + keeps every Nth beat — fewer, longer cuts that breathe, for slow or contemplative footage |
|
--full |
|
| Edit the whole song instead of the best-scoring segment | |
--segment 2 |
|
| Use the second (or third) best highlight segment | |
--exclude '1,6,0-30' |
|
| Drop clips by id or source time range | |
--lead '2:18,5' |
|
| Force the opening cuts, in order | |
--pin '8=138,peak2=22' |
|
| Pin a clip to a specific beat, peak, or valley |
A useful convention when comparing densities: render both and suffix them _beat-full
and _beat-thinned
.
The square demo above is a montage: each scene holds for a couple of beats (about 2.8 seconds in the demo), then blooms into the next through a white flash. You make one by describing it.
Open this folder in Claude Code, point it at your song and a folder of clips or photos, and say what you want:
"make a white-flash montage from these six clips — punch in on each animal, flashes about a second long"
"use still frames instead of the moving clips, and give them a little handheld shake"
"quicker flashes, and end it on a fade to white"
Claude picks the moments from each clip, frames each subject, times the scenes to your song, and renders the finished 1:1 video. Scenes can be video clips or still photos — stills get a subtle handheld sway so they don't look frozen — and the flash length is whatever you ask for.
Prefer the command line? python3 flash_montage.py --seq seq.json --audio song.wav --out montage.mp4
— the seq.json
format is documented at the top of the script.
- Python 3.10+
- ffmpeg on PATH
pip install -r requirements.txt
— librosa, PySceneDetect, OpenCV, NumPy