{"slug": "build-a-local-python-pipeline-that-turns-a-long-talk-into-six-9-16-shorts", "title": "Build a local Python pipeline that turns a long talk into six 9:16 Shorts", "summary": "A developer published a local Python pipeline that converts long talks, podcasts and interviews into six vertical 9:16 Shorts, combining faster-whisper word-level transcription, a heuristic or LLM-assisted clip scorer, OpenCV face tracking and FFmpeg rendering. The pipeline scores candidate clips on seven checks, trims silence, burns in word-by-word captions and hook titles, and supports an --analyze mode that exports a timestamped transcript for an LLM to select clips. The writeup notes CPU-only runs with the base Whisper model are sufficient for a first pass and warns that only media the user has rights to should be processed.", "body_md": "If you cut Shorts from podcasts or interviews, most of the clock time is not creative. It is the same checklist: find a 30-second stretch that stands on its own, drop the dead air, reframe to 9:16, put the face in the middle, burn in captions, add a hook title, maybe a zoom and a sound on the punch word. That checklist is regular enough to automate on your own machine.\n\nThis article walks through a local Python pipeline that does that. No cloud editor, no monthly seat. The blocks are ordinary: `faster-whisper` for a word-timed transcript, a small scorer (or an LLM that only sees text) to pick clips, OpenCV to track a face, and FFmpeg to render. I will be honest about what this replaces and what it does not.\n\nA useful default is six clips, each 20–60 seconds (30–45 is the sweet spot), already in 9:16, with silence trimmed, a vertical frame, word-by-word captions, a hook title in the first couple of seconds, light zooms, optional B-roll and sound effects, a progress bar, and a clean end that can loop.\n\nSupport two inputs, because the media is different:\n\nThe rest of this article is talk mode. Lifestyle is a second renderer on the same command, not a second architecture.\n\nA hard rule: only process video you or your client have the rights to. `yt-dlp` can fetch a URL, but downloading someone else's upload and reposting it breaks copyright and platform rules, automation or not.\n\nKeep one entry point:\n\n```\n# heuristic selection, then render six Shorts\npython clip.py podcast.mp4 --lang en\n\n# German audio, more accurate captions (slower)\npython clip.py interview.mp4 --lang de --model medium\n\n# export a transcript for an LLM to pick clips, then render that file\npython clip.py podcast.mp4 --analyze\npython clip.py podcast.mp4 --clips work/podcast/clips.json --only 2 5\n```\n\nUseful flags: `--fast` (FFmpeg `veryfast`), `--no-sfx`, `--no-broll`, `--music default` for a quiet bed with ducking, and `--crop` for a full-bleed vertical crop. Cropping is fashionable; it also chops hands and slides. A safer default is to fit the whole landscape frame onto 9:16 and use a blurred copy of the same frame as background.\n\nYou need Python 3.12, FFmpeg and ffprobe on `PATH`, `faster-whisper`, OpenCV, NumPy, Pillow, SciPy, `soundfile`, and `librosa`. A GPU helps Whisper, but a CPU-only run with `--model base` and `--fast` is enough for a first pass.\n\nCaptions and clip boundaries both need *word* times, not paragraph times. `faster-whisper` (the CTranslate2 port of Whisper) writes `{word, start, end}` locally, including German with `--lang de`. From that list you can group sentences on punctuation, treat long gaps as silence, and later highlight the active caption word.\n\nWrite work files into `work/<slug>/`. A replayable transcript lets you re-render clip 3 without transcribing again.\n\nA Short has to survive without the rest of the episode. Score seven checks from 0–10 and skip anything with a mean under 6:\n\nStart on the first word of the hook sentence. End on the last word of the payoff (keep the laugh if it comes right after). Do not let clips overlap.\n\nA first version does not need an LLM. Slide a sentence window across the 20–60 s range. Boost hook words in the first sentence, questions, and emotion or controversy terms (English and German). Penalize openings that start mid-thought (`And`, `So`, `Und`, `Also`). Prefer a clean sentence end and a duration near 38 s. Measure RMS energy with FFmpeg (`f32le` mono at 8 kHz) and reward variance, plus a mild words-per-second bonus. Sort by score and greedily keep non-overlapping winners.\n\nThat already beats \"cut every 45 seconds from the start.\" It still misses quiet jokes and can over-rank a segment that merely contains the word \"secret.\"\n\nThe better, still cheap path is `--analyze`: export a timestamped sentence list plus a short briefing, and let Claude or ChatGPT apply the seven checks and write a `clips.json` (`start`, `end`, a hook title of at most six words, keywords to highlight, optional speaker name, optional B-roll search terms). The model never sees the pixels, only text. A 20-minute episode is roughly 5,000–8,000 tokens. Then render exactly that file.\n\nFor each output frame, detect the face (OpenCV's YuNet model is small and MIT-licensed), aim a 9:16 window at it, and *smooth the window over time*. Without a low-pass, every head nod becomes a camera bump.\n\nTwo framing policies:\n\nIf two faces are in frame, prefer a window where one person is clearly speaking, or start from a file that is already vertical.\n\nOnce you have `(start, end)` and the word list, the renderer is bookkeeping: cut the source, drop pauses above a threshold (around 0.35 s works), overlay word-by-word captions with the current word emphasized, show a hook title for about 2.5 seconds, punch-in zoom on three to eight keywords, optionally lay B-roll on those hits, add light SFX, duck a music bed, and draw a progress bar with a last-frame hold so the clip can loop.\n\nFor B-roll, a local folder named by keyword is the cleanest option. If you fall back to a stock API, write a credits file and put required CC BY attribution in the description.\n\nGenerating ASS or title/drawtext filters from the word timestamps is more reliable than typography frame-by-frame in Python, except for motion-graphics layers you pre-render with Pillow.\n\nWatch every file before you post. The pipeline will caption a mis-heard name, start a cut on a breath, or zoom on the wrong word. A two-minute review saves a bad upload.\n\nLifestyle mode, briefly: skip Whisper, detect the beat with `librosa`, lay clips on the downbeats, and add ramps plus flash, glitch, zoom, or whip-pan transitions. This will not beat a careful CapCut or After Effects pass on tracking and music feel. Treat it as a rough cut.\n\nUse client-cleared or your own recordings. Running locally means no subscription for the core path. Optional API keys (Pexels, Pixabay, or an LLM) are extras. Budget about fifteen minutes on Windows for Python, FFmpeg, a venv, and `pip install`. Linux and macOS should work with Python 3.12 and FFmpeg; I have only treated Windows as the tested baseline.\n\nA pipeline like this can take over something like 70–80% of the repetitive cutting. It does not replace a good editor, and it will not invent a hook that is not in the tape. The win is that you spend your time on selection and a last pass, not on clicking the same five effects in a timeline.\n\nI packaged this exact flow as a paid kit (the Auto-Clipping-System): the Python source, a German step-by-step guide, a Windows installer script, the clip-scoring rules, prompts for Claude, bundled CC0 sounds, OFL fonts, and demo clips. It is 29 € once, no subscription. If you want the boxed version instead of wiring the stages yourself, it is here: [https://whop.com/kelvaro-creator-kits/auto-clipping-system](https://whop.com/kelvaro-creator-kits/auto-clipping-system)\n\n*Disclosure: that kit is my own paid product. No guarantee of views or revenue. Please preview every clip before you publish it.*", "url": "https://wpnews.pro/news/build-a-local-python-pipeline-that-turns-a-long-talk-into-six-9-16-shorts", "canonical_source": "https://dev.to/kelvaro/build-a-local-python-pipeline-that-turns-a-long-talk-into-six-916-shorts-mec", "published_at": "2026-11-11 19:50:01+00:00", "updated_at": "2026-11-11 19:52:44.502967+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "generative-ai", "ai-agents"], "entities": ["faster-whisper", "FFmpeg", "OpenCV", "Whisper", "yt-dlp", "Claude", "ChatGPT", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/build-a-local-python-pipeline-that-turns-a-long-talk-into-six-9-16-shorts", "markdown": "https://wpnews.pro/news/build-a-local-python-pipeline-that-turns-a-long-talk-into-six-9-16-shorts.md", "text": "https://wpnews.pro/news/build-a-local-python-pipeline-that-turns-a-long-talk-into-six-9-16-shorts.txt", "jsonld": "https://wpnews.pro/news/build-a-local-python-pipeline-that-turns-a-long-talk-into-six-9-16-shorts.jsonld"}}