{"slug": "lantern-archive-find-a-remembered-story-in-its-original-words", "title": "Lantern Archive: find a remembered story in its original words", "summary": "A developer built Lantern Archive, an offline semantic search tool that uses Xenova's quantized all-MiniLM-L6-v2 model via Transformers.js to match natural-language questions to passages in letters, journals and notes while returning the original text with file and line references. The tool splits documents into passages with character offsets, embeds each into a 384-dimensional vector, and selects up to three passages by cosine similarity above a threshold, verifying source SHA-256 hashes before each search. In a recorded demo it indexed twelve passages in 23 ms and answered three queries in 1.6–3.8 ms each on macOS arm64 with fetch disabled, and nine core tests plus a real-model paraphrase test pass.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge: Ghost in the Machine](https://dev.to/challenges/hacktoberfest-week4-2026-10-26).*\n\nLantern Archive is a small offline search tool for letters, journals and notes.\n\nYou can remember the meaning of a story without remembering the words that\n\nwould find it. Lantern uses an open embedding model to connect a question to\n\nthe relevant passage, then shows the original words and where to find them.\n\nIn the included fictional archive, “Who fixed the broken timepiece?” finds\n\nElsie's repair of the station clock. “What gave the homemade loaf its citrus\n\nfragrance?” finds the letter about orange peel rubbed into sugar.\n\nThe “ghost” is a trace already present in the source: a small detail waiting to\n\nbe found again. Every result includes its file and line numbers so the reader\n\ncan return to the full story.\n\n[Watch the 48-second walkthrough](https://github.com/soundcake/lantern-archive-hacktoberfest-20261029/blob/main/demo/walkthrough.mp4) · [Read the exact search results](https://github.com/soundcake/lantern-archive-hacktoberfest-20261029/blob/main/demo/transcript.txt)\n\nThe walkthrough shows three searches against four fictional letters. Its\n\ncaptions come from an actual local inference run. The saved JSON and transcript\n\nare included alongside the video in the repository's `demo/` folder.\n\nTo run the same demonstration:\n\n```\nnpm ci\nnpm run setup:model\nnpm run demo\n```\n\nSetup downloads the pinned model once. The demonstration then disables fetch\n\nrequests while the model loads, indexes the archive and searches it. This run\n\nrecorded zero attempted requests.\n\n[Source code and setup instructions](https://github.com/soundcake/lantern-archive-hacktoberfest-20261029)\n\nThe repository contains the CLI, source-indexing logic, a model setup script,\n\nfictional example letters, tests and the recorded demonstration. The app and\n\nexamples use the MIT license.\n\n```\nnode src/cli.mjs index examples/letters memories.lantern.json\nnode src/cli.mjs ask memories.lantern.json \"Who fixed the broken timepiece?\"\n```\n\nLantern runs [Xenova's quantized all-MiniLM-L6-v2 model](https://huggingface.co/Xenova/all-MiniLM-L6-v2)\n\nthrough [Transformers.js](https://github.com/huggingface/transformers.js).\n\nThe model builds on the [Sentence Transformers all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)\n\nsentence-embedding work. The model weights and Transformers.js use Apache-2.0.\n\nThe code pins both the model revision and the library version.\n\nDocuments are split into passages with their original character offsets and\n\nline numbers. Each passage becomes a 384-dimensional vector. A question becomes\n\nanother vector, and cosine similarity selects up to three passages above a\n\nthreshold. Returned text comes directly from those source passages.\n\nSource SHA-256 hashes are checked before each search. If a document changes,\n\nLantern asks for a rebuilt index, keeping quotations and line references tied\n\nto the current document. An explicit no-match result handles questions whose\n\nscores fall below the threshold.\n\nNine core tests pass, covering quotation accuracy, line references, changed\n\nsources, passage splitting, ranking, no-match behavior and index handling. A\n\nseparate real-model test passes four paraphrase searches with fetch disabled.\n\nThe demonstration indexed twelve passages in 23 ms and answered its three\n\nqueries in 1.6–3.8 ms each on macOS arm64; the saved report records the runtime\n\nand measurements for this small archive.\n\nA personal archive can outlast the service used to create it. Open weights let\n\nLantern search locally, without a hosted inference account or a subscription.\n\nThe model, its revision and the retrieval code are available for inspection.\n\nSomeone can repeat the same experiment, change the threshold or replace the\n\nmodel as their archive grows.\n\nThat matters for the theme: keeping a story accessible includes keeping the\n\ntools needed to find it accessible. Lantern brings an old passage back into\n\nview, with its original wording intact.", "url": "https://wpnews.pro/news/lantern-archive-find-a-remembered-story-in-its-original-words", "canonical_source": "https://dev.to/lukasz_snopkiewicz_0d1bab/lantern-archive-find-a-remembered-story-in-its-original-words-324o", "published_at": "2026-10-29 22:36:41+00:00", "updated_at": "2026-10-29 22:47:50.399796+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "natural-language-processing", "ai-search", "ai-agents"], "entities": ["Lantern Archive", "Xenova", "all-MiniLM-L6-v2", "Transformers.js", "Hugging Face", "Sentence Transformers", "Hacktoberfest"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/lantern-archive-find-a-remembered-story-in-its-original-words", "markdown": "https://wpnews.pro/news/lantern-archive-find-a-remembered-story-in-its-original-words.md", "text": "https://wpnews.pro/news/lantern-archive-find-a-remembered-story-in-its-original-words.txt", "jsonld": "https://wpnews.pro/news/lantern-archive-find-a-remembered-story-in-its-original-words.jsonld"}}