I record a lot of lectures and meetings, but I almost never watch them again.
What I actually want afterward is something I can read: a summary of what was discussed, the main topics, and any books or papers that were mentioned.
So I started building vtd (video-to-doc), an open-source CLI that turns audio and video recordings into structured Markdown or PDF documents.
At some point, I thought it would be useful to have the AI identify books and academic papers mentioned in a recording.
That sounded like a fairly straightforward feature.
Then I tested it with a real two-hour lecture.
My local model (qwen3.6:35b running through Ollama) suggested 27 references. I could only verify 7 of them against Open Library and Crossref.
That test led me to add a catalog verification step before a reference can appear in the final document.
The basic idea is simple. Give it a recording:
vtd lecture.mp4
Without enabling AI, vtd uses whisper.cpp to transcribe the recording, splits it into sections based on s, extracts frames from the video, and generates a document combining everything.
It works locally, without an API key.
If you add --with-ai, a language model generates notes that include:
You can use a hosted model or a local one through Ollama.
I use Ollama to run the model locally.
I added a --references flag.
When enabled, the model identifies two types of works:
For each one, it proposes an author, title, and publication year.
But instead of trusting that information, vtd searches two public catalogs:
To accept a reference, vtd checks that:
For example, Vygotsky and Vygotskiĭ can still match.
The final citation uses the catalog’s metadata.
To organize the bibliography, vtd checks the relevant transcript section for the author’s capitalized surname or a title of at least four words. It lists verified works with transcript evidence under Works referred to in the recording and other verified suggestions under Further reading.
I added a reproducible example to the repository.
I used President Kennedy's 1962 speech at Rice University, an 18-minute public-domain recording.
Here's the command:
vtd rice-speech.ogg \
--language en \
--whisper-model large-v3-turbo \
--with-ai \
--provider ollama \
--ai-model qwen3.6:35b \
--content-type lecture \
--references \
--pdf
I ran it on a Mac with an M4 Pro and 48 GB of memory.
The results:
| Step | Time |
|---|---|
| Processing without AI | 36 seconds |
| AI notes, references, and PDF | 101 additional seconds |
The Whisper model was already downloaded.
These timings are from a single run.
One of the eight proposed works matched a catalog record and appeared in the bibliography as further reading.
Works referred to in the recording
None detected.
Further reading (not mentioned in the recording)
The project is open source and MIT-licensed:
GitHub: https://github.com/Mariotutal/video-to-doc
If you work with recorded lectures, meetings, or presentations, feel free to give it a try.
I’d be interested to hear how you use it with your own recordings. Share your experience in the comments or open an issue on GitHub.