Summarize is an open-source tool I built that turns any video (URL or file) into notes you can actually use.
Not “a summary.” Notes. The kind you keep.
It exists because there’s too much content and too little time.
What it helps with:
- A long podcast, lecture, or conference talk compressed into something you can skim in five minutes.
- A Reddit post, X video, or Instagram reel where there is one idea buried under two minutes of fluff.
- A video you already watched but want to remember without rewatching.
Under the hood, Summarize is UX sugar (Streamlit, CLI, HTTP API, Docker, even a Raycast extension) over a simple pipeline: get a transcript YouTube captions when they exist; otherwise Whisper), chunk it, send the chunks to an LLM with a prompt from prompts.json.
Sounds simple. It is simple.
{
"endpoint": "/v1/chat/completions",
"content": "Summarize this post"
}
But the felt experience of getting a clean summary is completely different from doing the thing yourself. The gap between “I could do this manually” and “I actually do this regularly” is where tools like this live.
Origin story: Colab #
In October 2023 I wanted to summarize videos without watching them. So I opened a Colab notebook, threw in a couple of parallel calls to OpenAI’s Whisper, and used free Colab compute.
Then I found summarize.tech: ten dollars a month is a lot for something that is mostly automation over scraping. I figured I could do it better, faster, and basically for free (not just for YouTube, but for any URL or VTT file).
In my free time the architecture matured the way good software does:
Not by design, but by accumulated embarrassment.
The monolithic Colab notebook became a modular CLI. Then a Streamlit app. Then a Docker-deployable service. Then an HTTP API. Then a Raycast extension and an agent skill, because once you have a clean main config everything else is just another front door.
For non-YouTube sources (TikTok, Instagram, X, Reddit, Facebook) download goes through a fallback chain: yt-dlp first, then Cobalt as a local Docker sidecar when yt-dlp fails. YouTube is special-cased: captions when available, audio download + Whisper when they’re not. Local files, Google Drive, Dropbox work too. Same pipeline, different front ends.
The tool speaks any OpenAI-compatible LLM endpoint. I’ve run it against Gemini, Groq, Ollama, OpenRouter, Perplexity, DeepSeek, NVIDIA’s API, and plain OpenAI. You put providers in summarizer.yaml
, keys in .env
, and you’re done. Switching models is a one-line change. That’s the whole point of not hardcoding a vendor.
There’s also a transcript cache. Re-summarize the same video with a different prompt or model and you skip transcription entirely. Sounds minor until you notice you do that constantly.
NotebookLM ? #
For a while I stopped using Summarize and just used NotebookLM. I’ve always been fascinated by it.
Then the lack of control got to me. No real chunking. Weak support for anything outside YouTube. No “use this model with this prompt and this cost.” So I came back.
This is free, in the way that matters: you pay for compute, not for a product wrapping compute. Even with Whisper on Groq you’re looking at roughly $0.04 per hour of audio and only when you actually need transcription. YouTube captions are free. Open-source models at 20B or even 120B are commodities now. Cents per summary. Sometimes fractions of cents.
Groq deserves a mention. Their LPUs transcribe at roughly 25× real-time. A one-hour podcast finishes in under three minutes. That’s an order of magnitude faster than running Whisper locally on a GPU, and about 3× cheaper than OpenAI’s Whisper API. The API is OpenAI-compatible, so switching is boring in the best way.
Whispering #
I added a --speed
flag that speeds up the audio before Whisper sees it. The idea is almost dumb: feed Whisper a 1-hour file at 1.5x and it only processes 40 minutes. You pay less. It finishes faster. ffmpeg chains atempo
filters so any positive speed works, not just the 0.5–2.0 range of a single stage.
The tradeoff is accuracy. At 1.5x the degradation is barely noticeable. At 2x it starts to show, especially with fast speakers or accents. There’s no rigorous published benchmark. It’s one of those tricks people poke at in Whisper’s GitHub issues. I tested it on my own stuff and decided the imprecision was worth the savings. Your mileage may vary.
If you want to go deeper: faster-whisper , whisper.cpp and Whisper Turbo / small / tiny etc.
Eyes #
And if you don’t want transcription at all: --visual
sends the video straight to a vision-capable model. Same prompts. Same config. Different path.
No captions. No Whisper. No “what was said.” You get notes from the image stream itself, not from a speech-to-text approximation of it. The model is forced to watch scenes, on-screen text, gestures, cuts, the stuff a transcript throws away.
Visual mode is for when the payload is on the screen: slides, demos, whiteboards, TikToks where half the meaning is text overlay, talks where the diagram matters more than the filler words around it.
Transcriptions are still good (and faster and cheaper) enough for talk-heavy content.
Under the hood it’s still the same pipeline, just with different inputs. Long videos get split into segments (visual-chunk-seconds
, optional overlap) so you stay under model duration and payload limits.
You can compress before sending, pass a YouTube URL through when the provider supports it, or fall back to base64 of a local/downloaded file. Speed preprocessing works here too when you’re on the download path. Chunk notes get stitched the same way transcript chunks do.
It costs more than captions + a text model. It also sees things captions never will. Use it when that trade is real; keep Whisper or Youtube captions for everything else.
Summarize Metabolize
Ok but is it actually useful?
Do I use it?
Let’s talk about my real use cases, not the README ones. Let's take two of my fav prompts :
{
"Distill Wisdom": "Analyze the transcript and extract key insights and wisdom including a concise title that reflects the content.\n\n**TITLE**\n\n** IDEAS**\n- ...\n\n** QUOTES**\n- ...\n\n** REFERENCES**\n- ...\n\n- **Formatting Guidelines**:\n - **Title**: Start with the title in bold (`** TITLE**`).\n - **Categories**: Use bold for category headers (`** IDEAS**`, `** QUOTES**`, `** REFERENCES**`).\n - **Bullet Points**: Use hyphens (`-`) for each bullet point.\n - **Omit Empty Categories**: Do not include a category if there are no relevant items.\n - **No Additional Text**: Do not add any introductory phrases, explanations, or headers using `#`.\n - **Strict Template Adherence**: Follow the template exactly as shown above without deviations.\n\nHere is the text:\n{text}"
}
These two prompts do most of the work for me. Distill Wisdom is for when I don’t want a generic summary. I want the parts that survive after the fluff dies: ideas, lines, references, the bits worth keeping.
The other is more structural:
{
"Questions and answers": "Analyze the input text and generate the essential questions that, when answered, capture the main points and core meaning of the text. Do not add any introductory phrases, explanations! Just start with the questions and answers. Mark each question with **bold syntax** and don't number them. When formulating your questions: a. Address the central theme or argument b. Identify key supporting ideas c. Highlight important facts or evidence d. Reveal the purpose or perspective e. Explore any significant implications or conclusions. 3.) Answer all of your generated questions one-by-one in detail.\nHere is the text:\n{text}"
}
Questions and answers forces the material into a shape my brain likes: what is the point, what supports it, what is worth asking directly.
One is for extraction. The other is for understanding. Together they cover almost everything I actually do.
These and other templates live in prompts.json. Add your own with a {text}
placeholder and it shows up in the CLI and the web UI.
One style worth calling out: Mermaid Diagram. Instead of prose, the LLM (if capable enough to avoid syntax errors) returns Mermaid graph syntax, and the UI renders it as an interactive diagram per chunk.
Perplexity bitches #
And now Perplexity. Perplexity is an answer engine with live web search and inline citations. Their Sonar models can search while summarizing, fact-check claims against current sources, and return numbered references for what they say.
So: take a YouTube video, split it into chunks, send them to Perplexity’s API. What comes back isn’t just a summary. It’s a summary with sources. Claims verified or flagged. References linked.
Pair that with the Fact Checker prompt and you get something NotebookLM doesn’t really give you for now: a local-first pipeline that can argue with the video, not only restate it.
Since Summarize works with any OpenAI-compatible endpoint, wiring Perplexity is a provider entry in summarizer.yaml
. Citations get appended when the response includes them. I think that’s pretty cool.
An alternative option over Perplexity could be Grok's ability to use X search: not just the open web, but live X. Double-check a claim the video made, see what people are actually saying about it right now, catch the discourse the polished web results sanded off.
Install if you want to play:
pipx install martino-summarize
summarizer --init-config
echo "GROQ_API_KEY=your_key_here" > .env
summarizer --source "https://www.youtube.com/watch?v=arj7oStGLkU"
Or Docker: docker pull ghcr.io/martinopiaggi/summarize:latest
and docker compose up -d
Docs: summarize.martino.im
Code: github.com/martinopiaggi/summarize