{"slug": "my-local-llm-suggested-27-references-only-7-passed-verification", "title": "My local LLM suggested 27 references. Only 7 passed verification.", "summary": "A developer built vtd (video-to-doc), an open-source MIT-licensed CLI that converts audio and video recordings into structured Markdown or PDF documents using whisper.cpp for transcription and an optional local or hosted LLM for notes. Testing a reference-extraction feature on a two-hour lecture, the developer's local qwen3.6:35b model via Ollama proposed 27 references, of which only 7 could be verified against Open Library and Crossref, prompting a catalog-verification step that checks author, title, and year before any citation appears. In a reproducible example on Kennedy's 1962 Rice University speech, processing took 36 seconds without AI and 101 additional seconds with AI notes, references, and PDF output, with one of eight proposed works matching a catalog record.", "body_md": "I record a lot of lectures and meetings, but I almost never watch them again.\n\nWhat I actually want afterward is something I can read: a summary of what was discussed, the main topics, and any books or papers that were mentioned.\n\nSo I started building [vtd (video-to-doc)](https://github.com/Mariotutal/video-to-doc), an open-source CLI that turns audio and video recordings into structured Markdown or PDF documents.\n\nAt some point, I thought it would be useful to have the AI identify books and academic papers mentioned in a recording.\n\nThat sounded like a fairly straightforward feature.\n\nThen I tested it with a real two-hour lecture.\n\nMy local model (`qwen3.6:35b` running through Ollama) suggested **27 references. I could only verify 7 of them** against Open Library and Crossref.\n\nThat test led me to add a catalog verification step before a reference can appear in the final document.\n\nThe basic idea is simple. Give it a recording:\n\n```\nvtd lecture.mp4\n```\n\nWithout enabling AI, vtd uses `whisper.cpp` to transcribe the recording, splits it into sections based on pauses, extracts frames from the video, and generates a document combining everything.\n\nIt works locally, without an API key.\n\nIf you add `--with-ai`, a language model generates notes that include:\n\nYou can use a hosted model or a local one through Ollama.\n\nI use Ollama to run the model locally.\n\nI added a `--references` flag.\n\nWhen enabled, the model identifies two types of works:\n\nFor each one, it proposes an author, title, and publication year.\n\nBut instead of trusting that information, vtd searches two public catalogs:\n\nTo accept a reference, vtd checks that:\n\nFor example, `Vygotsky` and `Vygotskiĭ` can still match.\n\nThe final citation uses **the catalog’s metadata**.\n\nTo organize the bibliography, vtd checks the relevant transcript section for the author’s capitalized surname or a title of at least four words. It lists verified works with transcript evidence under **Works referred to in the recording** and other verified suggestions under **Further reading**.\n\nI added a reproducible example to the repository.\n\nI used President Kennedy's 1962 speech at Rice University, an 18-minute public-domain recording.\n\nHere's the command:\n\n```\nvtd rice-speech.ogg \\\n  --language en \\\n  --whisper-model large-v3-turbo \\\n  --with-ai \\\n  --provider ollama \\\n  --ai-model qwen3.6:35b \\\n  --content-type lecture \\\n  --references \\\n  --pdf\n```\n\nI ran it on a Mac with an M4 Pro and 48 GB of memory.\n\nThe results:\n\n| Step | Time | \n|---|---|\n| Processing without AI | 36 seconds | \n| AI notes, references, and PDF | 101 additional seconds | \n\nThe Whisper model was already downloaded.\n\nThese timings are from a single run.\n\nOne of the eight proposed works matched a catalog record and appeared in the bibliography as further reading.\n\n**Works referred to in the recording**\n\n*None detected.*\n\n**Further reading (not mentioned in the recording)**\n\nThe project is open source and MIT-licensed:\n\n**GitHub:** [https://github.com/Mariotutal/video-to-doc](https://github.com/Mariotutal/video-to-doc)\n\nIf you work with recorded lectures, meetings, or presentations, feel free to give it a try.\n\nI’d be interested to hear how you use it with your own recordings. Share your experience in the comments or open an issue on GitHub.", "url": "https://wpnews.pro/news/my-local-llm-suggested-27-references-only-7-passed-verification", "canonical_source": "https://dev.to/drakenmario/my-local-llm-suggested-27-references-only-7-passed-verification-4nm9", "published_at": "2026-11-18 22:06:56+00:00", "updated_at": "2026-11-18 22:11:50.522552+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-agents", "natural-language-processing"], "entities": ["vtd (video-to-doc)", "Ollama", "qwen3.6:35b", "whisper.cpp", "Open Library", "Crossref", "GitHub"], "also_reported_by": [], "takeaway": "Verifying LLM-proposed book and paper citations against Open Library and Crossref cut a 27-reference hallucination set down to 7 confirmed works, so route every generated reference through catalog lookup and use the catalog's metadata in the final citation.", "recommendation": {"advice": "Before publishing LLM-suggested references, match each author, title, and year against Open Library or Crossref and cite the catalog's metadata; in the source, only 7 of 27 proposals verified.", "playbook": "output-verification", "playbook_url": "https://wpnews.pro/playbooks/output-verification"}, "alternates": {"html": "https://wpnews.pro/news/my-local-llm-suggested-27-references-only-7-passed-verification", "markdown": "https://wpnews.pro/news/my-local-llm-suggested-27-references-only-7-passed-verification.md", "text": "https://wpnews.pro/news/my-local-llm-suggested-27-references-only-7-passed-verification.txt", "jsonld": "https://wpnews.pro/news/my-local-llm-suggested-27-references-only-7-passed-verification.jsonld"}}