{"slug": "show-hn-audio-tldr-summarize-any-video-or-podcast-locally-with-whisper", "title": "Show HN: Audio-tldr – Summarize any video or podcast locally with Whisper", "summary": "Audio-tldr, an open-source agent skill by AugustusW, lets users summarize any video, podcast, or audio file locally with Whisper, producing 3–7 key takeaways and a summary without uploading audio to the cloud. The skill, which works in Claude Code and Codex, caches transcripts by content hash so re-summarizing from a different angle is instant and free, addressing the cost and privacy issues of cloud transcription. It supports YouTube, yt-dlp URLs, local files, and includes features like language auto-detection, digest templates, and optional frame extraction.", "body_md": "Any video, audio, or podcast → key takeaways. Transcribed locally, cached forever.\n\nEnglish | [繁體中文](/AugustusW/audio-tldr-skill/blob/main/README.zh-TW.md)\n\nAn agent skill — open [SKILL.md standard](https://developers.openai.com/codex/skills), works in\n[Claude Code](https://claude.com/claude-code) **and** [Codex](https://developers.openai.com/codex/skills) —\nthat turns long-form media into **3–7 key\ntakeaways + a summary**. Transcription runs locally with whisper and is cached by content hash —\nwhile a cache entry exists, the same source is **not transcribed again** (unless you `--force`\n\n).\nAsk for a different angle later and it re-digests from cache in seconds.\n\nFirst-run transcription time depends on your hardware, model, and backend — after that, the cache answers.\n\nWatching a 90-minute talk to extract 5 useful points is a bad trade. Sending audio to a cloud API costs money and leaks content. And summarizing the same episode twice — because the first summary had the wrong focus — means paying the transcription cost all over again.\n\n```\nWithout audio-tldr                    With audio-tldr\n──────────────────                    ───────────────\nwatch the whole video                 paste the URL\ntake notes by hand                    get takeaways + summary\n\"summarize it differently…\"           re-digest from cache, instant\nre-upload, re-transcribe, re-pay      transcribe once, reuse from cache\n```\n\n- ✓ YouTube, podcasts, and any yt-dlp-supported URL — or local audio/video files\n- ✓ Local media pipeline: download, transcription, cache all run on your machine — audio is never uploaded (see\n[Privacy](#privacy)) - ✓ Content-hash cache: re-summarizing (any angle) reuses the transcript while the entry exists\n- ✓ Whisper backend auto-detection: mlx-whisper / faster-whisper / whisper.cpp / openai-whisper\n- ✓ Language auto-detection; optional Simplified→Traditional Chinese conversion (OpenCC)\n- ✓ Cache management built in: list, clear one, clear all, opt-in retention\n- ✓ Timeline for long content (> 20 min)\n- ✓ Digests saved to an output folder as Markdown or HTML — transcripts stay in the cache\n- ✓ Conversational digest prompt: no request stated? The agent asks in plain text, listing the template menu\n- ✓ Digest templates: meeting minutes, key summary, analysis report — or save your own reusable format\n- ✓ Translation at the digest layer: digests in any language, or a faithful full-transcript translation\n- ✓ Optional preferences file for standing habits — zero setup required\n- ✓ Interpreter auto-selection: backend installed in another Python (e.g. Homebrew) is found and used automatically;\n`--doctor`\n\ndiagnoses the environment - ✓ Apple Podcasts fallback built in: when yt-dlp's extractor fails, episodes resolve via the iTunes lookup API — cache identity stays on your original link; a show link (no episode id) automatically uses the latest episode\n- ✓ Opt-in frame extraction for video sources: scene-detection slide capture, or stills at exact timestamps — video fetched at ≤720p and deleted after extraction; frames share the transcript's cache entry\n- ✓ Install by copy, as a Claude Code plugin,\n**or** into Codex (open SKILL.md standard)\n\n**Option A — copy the skill (simplest):**\n\n```\ngit clone https://github.com/AugustusW/audio-tldr-skill.git\ncp -r audio-tldr-skill/skills/audio-tldr ~/.claude/skills/\n```\n\nInvoke with `/audio-tldr`\n\n, or just ask Claude to summarize a video — it auto-triggers.\n\n**Option B — install as a plugin:**\n\n```\n/plugin marketplace add AugustusW/audio-tldr-skill\n/plugin install audio-tldr@audio-tldr-skill\n```\n\nInvoke with `/audio-tldr:audio-tldr`\n\n. Both options can coexist — plugin skills are namespaced.\n\n**Option C — Codex CLI / ChatGPT app:**\n\nThe skill follows the open SKILL.md standard, so it works in Codex as-is. Copy the skill folder into Codex's skills directory:\n\n```\ngit clone https://github.com/AugustusW/audio-tldr-skill.git\ncp -r audio-tldr-skill/skills/audio-tldr ~/.codex/skills/audio-tldr        # personal\n# or, per-project: cp -r audio-tldr-skill/skills/audio-tldr <repo>/.codex/skills/audio-tldr\n```\n\nInvoke it with a `$audio-tldr`\n\nmention, or let Codex pick it implicitly when you ask to\nsummarize audio/video. The transcript cache (`~/.cache/audio-tldr/`\n\n) and the preferences file\n(`~/.config/audio-tldr/preferences.md`\n\n) are shared with Claude Code — transcribe once, digest\nanywhere.\n\nThe media pipeline — download, transcription, cache — runs entirely on your machine.\n\n| Requirement | Why | Install |\n|---|---|---|\n| Python 3.9+ | runs the transcription script | usually preinstalled |\n`yt-dlp` |\ndownload audio from URLs | `pip install yt-dlp` or `brew install yt-dlp` |\n`ffmpeg` |\naudio extraction/conversion | `brew install ffmpeg` / `apt install ffmpeg` |\nOne whisper backend |\nspeech-to-text | table below |\n\nWhisper backends, in the order the skill auto-detects them:\n\n| Backend | Best for | Install | Default model |\n|---|---|---|---|\n|\n\n`pip install mlx-whisper`\n\n`large-v3-turbo`\n\n[faster-whisper](https://pypi.org/project/faster-whisper/)`pip install faster-whisper`\n\n`large-v3-turbo`\n\n[whisper.cpp](https://github.com/ggerganov/whisper.cpp)`brew install whisper-cpp`\n\n+ set `AUDIO_TLDR_WHISPER_CPP_MODEL`\n\n[openai-whisper](https://pypi.org/project/openai-whisper/)`pip install openai-whisper`\n\n`large-v3-turbo`\n\nLocal files don't need `yt-dlp`\n\n— only a whisper backend.\n\nFor URL sources, make sure you have the right to download and process the content, and comply with the source site's terms and your local copyright law.\n\nThe default is `large-v3-turbo`\n\non every backend (whisper.cpp excepted — its model is the\n`AUDIO_TLDR_WHISPER_CPP_MODEL`\n\nfile). On CPU-only machines this favors quality over speed —\ndrop to `small`\n\nif transcription is too slow. Override per run with `--model`\n\n, or persistently\nwith `AUDIO_TLDR_MODEL`\n\n(the flag wins). Bare names are mapped per backend (mlx gets the\n`mlx-community/whisper-`\n\nprefix automatically; a full HF repo path is used as-is):\n\n| Situation | Suggested model |\n|---|---|\n| CPU / quick tests | `small` |\n| General Chinese summaries | `medium` |\n| Names, jargon, accuracy-critical | `large-v3` |\n| Capable GPU, speed + quality | `large-v3` or `large-v3-turbo` |\n\n```\npython3 scripts/transcribe.py --model small \"<source>\"   # per run\n$env:AUDIO_TLDR_MODEL = \"large-v3\"    # persistent; PowerShell (bash/zsh: export AUDIO_TLDR_MODEL=large-v3)\n```\n\n**Optional — Traditional Chinese:** whisper often emits Simplified Chinese. `pip install opencc`\n\nand Chinese transcripts are converted to Taiwan Traditional automatically — including\ncommon-phrase localization (`s2twp`\n\n, e.g. 軟件→軟體) — plus the model is biased toward\nTraditional vocabulary. Not installed → transcripts are left as-is.\n\nWindows is supported by the underlying Python stack, but the full flow has **not yet been\nverified on Windows** — reports welcome. Install with PowerShell:\n\n```\n# prerequisites (winget shown; Chocolatey: choco install ffmpeg yt-dlp)\nwinget install Gyan.FFmpeg\nwinget install yt-dlp.yt-dlp\npy -3 -m pip install faster-whisper      # recommended backend on Windows\n\n# install the skill (manual copy)\ngit clone https://github.com/AugustusW/audio-tldr-skill.git\n$skillsDir = \"$env:USERPROFILE\\.claude\\skills\"\nNew-Item -ItemType Directory -Force -Path $skillsDir | Out-Null\nCopy-Item -Recurse -Force \"audio-tldr-skill\\skills\\audio-tldr\" $skillsDir\n```\n\nManual copy does not auto-update, and `-Force`\n\noverwrites an existing `audio-tldr`\n\nfolder —\nprefer the plugin install if you want managed versions.\n\n**Python command**— if`python3`\n\nisn't recognized, use`python`\n\nor the py launcher (`py -3`\n\n); the skill tells Claude to fall back automatically, but substitute accordingly when running the script yourself.**Skill path**— Claude Code on Windows reads skills from`%USERPROFILE%\\.claude\\skills\\`\n\n(plugin install works identically to macOS/Linux).**GPU (optional)**— faster-whisper runs on CPU out of the box. NVIDIA acceleration goes through CTranslate2; check that a CUDA device is visible with`py -3 -c \"import ctranslate2; print(ctranslate2.get_cuda_device_count())\"`\n\n. Non-zero means CTranslate2 can see the GPU — it does**not** guarantee the CUDA runtime, cuBLAS/cuDNN DLLs, and GPU model loading all work; run one short real transcription to confirm. Required CUDA/cuDNN versions: see the[faster-whisper README](https://github.com/SYSTRAN/faster-whisper#gpu).- mlx-whisper is Apple-Silicon-only. whisper.cpp on Windows needs a\n`whisper-cli.exe`\n\non PATH plus`AUDIO_TLDR_WHISPER_CPP_MODEL`\n\n.\n\n```\n> summarize https://www.youtube.com/watch?v=xxxx\n> give me the key points from this podcast: https://podcasts.apple.com/...\n> /audio-tldr ~/Downloads/meeting-recording.m4a\n> summarize this talk for a beginner — action items only: https://youtu.be/xxxx\n> (later) same video, but focus only on what they said about pricing\n```\n\nState your needs in the request — focus, audience, format, length, language — and the digest follows them instead of the default takeaways+summary structure. The last one re-uses the cached transcript — instant, no re-transcription.\n\nPairs well with— once the digest is written, \"share this as a doc\" turns it into a link you can hand to anyone (append, extend, or revoke it later).[sharedoc-mcp]\n\n```\n> summarize this talk and grab the slides: https://youtu.be/xxxx\n> screenshots at 1:30 and 12:05 from ~/Videos/keynote.mp4\n```\n\nAsking for screenshots / slides runs `scripts/frames.py`\n\n: ffmpeg scene detection captures\nframes where the picture visibly changes (`--threshold`\n\n, `--min-gap`\n\n, `--max-frames`\n\n), or\n`--at 90,12:05`\n\nextracts stills at exact timestamps (e.g. to illustrate a digest timeline).\nOff by default — plain summarize requests never download video. For URLs the video is\nfetched at ≤720p and deleted right after extraction (`--keep-video`\n\nkeeps it); local files\nare used in place and never modified. Frames and their `manifest.json`\n\nlive in the same\ncache entry as the transcript, so repeated requests are instant. Pure logic is covered by\nautomated tests; real-video extraction is verified manually per release.\n\nTwo phases, deliberately separated:\n\n**Transcribe**(`scripts/transcribe.py`\n\n) — resolves a cache key (normalized URL or file content hash), returns instantly on a hit; otherwise downloads via yt-dlp, transcribes with the best available whisper backend, and caches`transcript.txt`\n\n+`meta.json`\n\nunder`~/.cache/audio-tldr/<sha256>/`\n\n.**Digest**— the agent reads the cached transcript and produces takeaways, a summary, and (for long content) an approximate timeline. If your request didn't say how to digest, it asks first — in plain conversational text, never a clickable menu, so it works over plain-text messaging channels too. Every digest is also saved to the output folder (default`./audio-tldr-output/`\n\n) as`<title>-<date>-<style>.md`\n\n(or`.html`\n\n). Re-digesting with a different focus skips phase 1 entirely.\n\nBe precise about what stays local and what doesn't:\n\n-\n**Your audio/video never leaves the machine.** No third-party transcription service is used, and the scripts in this repo contain no telemetry. Network access still happens where you'd expect: yt-dlp fetches URL sources from the source site, and whisper backends may download their model on first use (dependency behavior is governed by those projects). -\n**The digest phase sends the transcript text (never the audio) to the model**, inside your own Claude session — exactly like asking Claude to read any local file. -\n**Cached transcripts are unencrypted plaintext, kept indefinitely by default**, under`~/.cache/audio-tldr/`\n\n. After processing sensitive content,`--clear`\n\nthat entry, or configure a retention period. -\n**Digests persist in the output folder**(default`./audio-tldr-output/`\n\n, relative to your working directory) — including full-transcript translations, which carry essentially the whole transcript. The output folder has no clearing or retention mechanism; delete files manually, and add the folder to`.gitignore`\n\nif you run the skill inside a git-tracked directory. -\n**Phase 1 only (sensitive recordings):** transcribe without ever handing the text to Claude — run the script yourself; stdout is metadata JSON only, and the transcript stays at the returned`transcript_path`\n\nuntil you delete it:\n\n```\n# macOS/Linux\npython3 ~/.claude/skills/audio-tldr/scripts/transcribe.py \"/path/to/recording.m4a\"\n# Windows PowerShell\npy -3 \"$env:USERPROFILE\\.claude\\skills\\audio-tldr\\scripts\\transcribe.py\" \"C:\\path\\to\\recording.m4a\"\n```\n\nCreate `~/.config/audio-tldr/preferences.md`\n\nto set standing habits — every field is optional\nand everything works without the file:\n\n```\noutput_dir: ~/Documents/audio-digests\ntimeline: off\nauto_delete_audio: off\noutput_format: html\nmodel: large-v3\n```\n\n| field | default | meaning |\n|---|---|---|\n`output_dir` |\n`./audio-tldr-output` |\nwhere digest files are saved |\n`timeline` |\n`on` |\ninclude a timeline section in digests when content warrants it |\n`auto_delete_audio` |\n`on` |\ndelete downloaded audio after transcription; `off` keeps the mp3 in the cache entry |\n`output_format` |\n`md` |\ndigest file format, `md` or `html` ; a per-request choice always wins |\n`model` |\n`large-v3-turbo` |\nwhisper model for transcription (passed as `--model` ); a per-request choice always wins |\n`digest_model` |\n(platform default) | model for the digest subagent — unset = platform default (Claude Code: `sonnet` ; Codex: `GPT-5.6 Terra` ); a model name pins it; `off` = digest inline on the current agent (typically pricier) |\n\nThe file is read by the agent (Claude Code and Codex share it) — the install never asks you to set it up, and defaults apply whenever it's absent.\n\nThree built-in templates shape the digest output — name one (\"digest this as meeting minutes\") or pick from the menu when asked:\n\n| Template | What you get |\n|---|---|\n`meeting-minutes` |\nMeta, topics discussed, decisions, action items, open questions |\n`key-summary` |\nKey takeaways, one-paragraph summary, optional timeline (the default) |\n`analysis-report` |\nArguments with evidence, data points, perspectives, implications |\n\nThe timeline in `key-summary`\n\nappears only when the `timeline`\n\npreference is not\n`off`\n\n, the source runs over 20 minutes, and the transcript has clear topic shifts.\n\n**Build your own:** drop a markdown file in `~/.config/audio-tldr/templates/`\n\n—\nfrontmatter (`name`\n\n, `description`\n\n) plus section instructions; same name overrides a\nbuilt-in, a new name becomes a new menu option. Easiest start: copy a built-in from\n`skills/audio-tldr/templates/`\n\nand edit (e.g. change the timeline threshold). You can\nalso just describe a format in conversation — the skill offers to save it for reuse.\nYour templates live outside the skill folder, so skill updates never touch them.\n\n**Cheaper digests:** on platforms with subagents the digest runs on a cheaper model\nby default (Claude Code: `sonnet`\n\n; Codex: `GPT-5.6 Terra`\n\n) — see the `digest_model`\n\npreference to pin a model or turn this off (`off`\n\n= digest inline).\n\nThe cache is **kept forever by default** — nothing is auto-deleted unless you opt in.\n\nAsk Claude, or run `scripts/transcribe.py`\n\ndirectly:\n\n| Command | What it does |\n|---|---|\n`--cache-info` |\nlist cached transcripts + sizes (JSON) |\n`--clear \"<source>\"` |\ndelete one entry |\n`--clear-all --yes` |\ndelete everything |\n`--set-retention <days>` |\nauto-prune entries older than N days (`off` = keep forever) |\n`--force` |\nre-transcribe one source, ignoring cache |\n`--keep-audio` |\nkeep the downloaded mp3 in the cache entry (default deletes it after transcription) |\n`--doctor` |\nJSON environment diagnosis: Python path/version, backend & tool visibility, other interpreters that have a backend, MLX Metal availability |\n\nEnvironment variables:\n\n| Variable | Purpose |\n|---|---|\n`AUDIO_TLDR_MODEL` |\noverride the whisper model for the active backend (`--model` beats it) |\n`AUDIO_TLDR_WHISPER_CPP_MODEL` |\npath to a ggml model file (enables the whisper.cpp backend) |\n`AUDIO_TLDR_ZH_CONVERT` |\nChinese conversion: `off` , or an OpenCC config (default `s2twp` — Taiwan Traditional incl. common phrases) |\n`AUDIO_TLDR_PYTHON` |\npin the Python interpreter the script runs under (wins over auto-probing). Useful when your whisper backend lives in a non-default Python (e.g. Homebrew 3.12) |\n\n```\ngit clone https://github.com/AugustusW/audio-tldr-skill.git\ncd audio-tldr-skill\npython3 -m pytest tests/   # 93 unit tests, no network or model needed\n```\n\nVersioning: every release bumps `version`\n\nin `.claude-plugin/plugin.json`\n\n**and**\n`.claude-plugin/marketplace.json`\n\n(kept identical), adds a [CHANGELOG](/AugustusW/audio-tldr-skill/blob/main/CHANGELOG.md) entry,\nand is published as a git tag + [GitHub Release](https://github.com/AugustusW/audio-tldr-skill/releases).\n**To get update notifications**: Watch this repo (Custom → Releases), or — if you installed as a\nClaude Code plugin — run `/plugin`\n\nand update from the marketplace (it compares the version above).\nManual-copy installs have no auto-update: re-copy the skill folder after a new release.\nYour preferences, custom templates (`~/.config/audio-tldr/`\n\n), and cache\n(`~/.cache/audio-tldr/`\n\n) all live outside the skill folder — updating never touches them.\n\nv0.4.0 ([CHANGELOG](/AugustusW/audio-tldr-skill/blob/main/CHANGELOG.md)) — core logic is covered by 63 offline unit tests (yt-dlp,\nwhisper backends, cache, and OpenCC are mocked; no network or models needed). The full flow has\nbeen manually verified (2026-07-19: real YouTube download, transcription, cached re-digest,\nChinese conversion, `--keep-audio`\n\n, output-folder digests in md/html, transcript translation,\ninterpreter auto-selection from `/usr/bin/python3`\n\n, and the Apple Podcasts fallback end-to-end —\na real 53-min episode resolved via iTunes lookup, transcribed, and cache-hit on the original\nApple URL) on:\n\n| Component | Verified version |\n|---|---|\n| macOS | 26.5.1 (Apple M4 Pro) |\n| Python | 3.12.13 |\n| mlx-whisper | 0.4.3 |\n| ffmpeg | 8.1 |\n| yt-dlp | 2026.06.09 |\n\nNewer dependency versions may behave differently. Not yet covered by automated tests: real downloads, the other three backends, and Windows. Codex support follows the open SKILL.md standard; the transcription core was verified end-to-end inside Codex on 2026-07-19 (a real 53-min podcast downloaded, transcribed, and cache-hit, including the interpreter auto-selection path). Digest-layer features (output folder, translation, preferences) have so far been exercised in Claude Code only. Possible next: SRT export, speaker diarization. Issues and PRs welcome.\n\nMIT. See [LICENSE](/AugustusW/audio-tldr-skill/blob/main/LICENSE).\n\nLong content is worth hearing once — by your machine, not by you.", "url": "https://wpnews.pro/news/show-hn-audio-tldr-summarize-any-video-or-podcast-locally-with-whisper", "canonical_source": "https://github.com/AugustusW/audio-tldr-skill", "published_at": "2026-08-05 13:28:54+00:00", "updated_at": "2026-08-05 13:37:46.091173+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "ai-agents", "natural-language-processing", "developer-tools"], "entities": ["AugustusW", "audio-tldr", "Whisper", "Claude Code", "Codex", "OpenAI", "yt-dlp", "Apple Podcasts"], "alternates": {"html": "https://wpnews.pro/news/show-hn-audio-tldr-summarize-any-video-or-podcast-locally-with-whisper", "markdown": "https://wpnews.pro/news/show-hn-audio-tldr-summarize-any-video-or-podcast-locally-with-whisper.md", "text": "https://wpnews.pro/news/show-hn-audio-tldr-summarize-any-video-or-podcast-locally-with-whisper.txt", "jsonld": "https://wpnews.pro/news/show-hn-audio-tldr-summarize-any-video-or-podcast-locally-with-whisper.jsonld"}}