I let a 4B model watch 321 stock clips overnight so I never have to scroll again

A developer built nightshelf, an open-source command-line tool that indexes a local stock-video library by meaning using a locally run Gemma 3 4B vision model via Ollama. The tool extracts three frames per clip with ffmpeg, generates structured JSON descriptions, embeds them, and stores results in SQLite with FTS5 for full-text search, processing 321 clips (7.6 GB) in 2,874 seconds with zero failures at 8.9 seconds per clip on a Mac without a GPU server. Released under Apache 2.0 on Java 21 with 169 tests, it reads the clip library and existing catalog read-only and keeps all processing on the local network.

This is a submission for the Hacktoberfest Open-Source AI Challenge, Week 3: Slow and Steady https://dev.to/challenges/hacktoberfest-week3-2026-10-19 . I make short videos, and all of them draw from one shared folder of stock clips: 321 files from Pixabay and Pexels, 7.6 GB, about two and a half hours of footage. Every file is named with a number. pixabay/4926.mp4 could be anything. There is a catalog next to the files, with tags, but the tags are the search queries I typed when I downloaded each clip. They tell me what I was looking for that day, not what the clip contains. So when I need "a single candle on a dark background", the honest method is to open the folder and scrub through thumbnails. I have done that more times than I want to admit. nightshelf is a small command-line tool that looks at the library once, unattended, and writes down what is in every clip. For each video it pulls three evenly spaced frames, asks Gemma 3 the 4B vision model, running locally through Ollama to describe them, embeds the description, and stores everything in a SQLite file. After that I just type what I want: bash $ nightshelf search "a candle flame" --limit 3 1. pixabay/4926 score 0.0328 sim 0.83 10.6s A single candle burns brightly against a dark background, casting a warm, flickering light… status: prayers-nightwatch=approved | used: prayers-nightwatch 2. pixabay/4933 score 0.0320 sim 0.80 33.1s A single candle with a flickering flame burns brightly against a dark background… 3. pixabay/51749 score 0.0320 sim 0.82 20.0s A single lit candle burns brightly in a dark room, casting a warm glow against a black background… The tool also reads my existing catalog read-only, it never writes to it , so it knows which clips I already approved, rejected, or used for a given project. --status unused means "something I have not used yet and did not reject". The real run on my library: 321 clips described and embedded, 0 failures, 2,874 seconds in total, 8.9 seconds per clip , on my Mac, with no GPU server involved. The video about 80 seconds, a real terminal on my real library shows the overnight run with its one-line-per-clip progress, the stats of the full run, a few searches, and an export of a shortlist. These are real terminal outputs from my library the path of my clip folder is masked : Overnight indexer that makes a local video clip library searchable by meaning. A stock-clip library grows until nobody remembers what is in it, and file names like clip 0412.mp4 say nothing. nightshelf looks at a few frames of every clip with an open-weight vision model running on your own machine, writes a short description, stores it in a local SQLite database, and lets you search the library from the command line "rainy city at night", "someone walking a dog on a beach" . Nothing leaves your network, the library is only ever read and the whole library is processed unattended, overnight. php flowchart LR L "Clip library<br/ read-only" -- S "scan + sync<br/ new / changed / removed " C "catalog.json<br/ read-only" -. "filters, status, uses" .- Q S -- F "ffmpeg<br/ N frames per clip" F -- V "Vision model<br/ Ollama " V -- D "SQLite index<br/ descriptions + FTS5" D -- E "Embedding model<br/ Ollama " E -- D D -- Q "search / export<br/ full-text … Apache 2.0, Java 21, no framework, 169 tests. The README has a quick start jar or Docker , every command, and a configuration table. The pipeline is deliberately boring: clip library read-only → scan + sync → ffmpeg 3 frames → Gemma 3 via Ollama → SQLite descriptions + FTS5 → embeddings → search / export One structured answer per clip. I ask Gemma for JSON matching a schema: a description of at most two sentences, a list of subjects, a setting, a time of day day, night, dusk, dawn, unknown and a mood. Temperature is 0, and 0.3 on the retry after a malformed answer, so a retry does not repeat the same bad output. Everything the model returns is treated as untrusted: the JSON is validated, lengths are capped, control and bidirectional characters are stripped, and the prompt tells the model that any text visible in a frame is content to describe, never an instruction. Hybrid search without a vector database. 321 vectors do not need a server. Search runs SQLite FTS5 for the words and a plain cosine similarity in Java over the embeddings nomic-embed-text , also through Ollama , then merges the two rankings with reciprocal rank fusion k = 60 . If Ollama is not reachable at search time, it falls back to full-text only and says so in one line. SQLite, on purpose. One machine, one writer, a few hundred rows. The database uses WAL, foreign keys and versioned migrations, every statement is parameterized, and it refuses to open on a network filesystem, because SQLite locking on SMB or NFS is not something I want to trust. The clip library is mounted read-only, in Docker as well. Slow and steady, as a design rule. The indexer handles one clip at a time and commits each result in its own transaction. A SIGTERM finishes the current clip, prints a summary and exits cleanly. A test simulates a hard kill and checks that the next run redoes only the unfinished clips. Ten failures in a row stop the run, because Ollama is probably down. And if the library drive is unmounted, the scan sees zero clips, so a guard refuses to flag more than half of the index as removed. An unplugged disk should never look like a deletion. Things that did not work the first time. Almost every description began with "The video shows…". A sentence in the prompt reduced it, a display-time cleanup removed the rest, and the stored text and embeddings are untouched. The query "spring flowers in daylight" matched nearly everything because of the word "in", so stop words are now removed from the full-text query. User text is always split into words and quoted before it reaches FTS5, so operators like NEAR or are plain text. My library is not secret, but it is mine: 7.6 GB, plus a record of every choice I made about it. A hosted vision API would mean sending frames from every clip to someone else's servers, and paying again every time I add a clip. With open weights, none of that applies: gemma3:4b . Nobody can change it under me, and swapping models is two environment variables VISION MODEL , EMBED MODEL . There is a real trade-off: a larger hosted model would probably describe some clips better. For this job, a 4B model that I can run unattended on my own computer was enough, and I would rather have an index I fully control. I waited once, for 48 minutes, while doing something else. Now finding a clip takes a second instead of a scroll through hundreds of thumbnails, and export turns a query into a shortlist file Markdown, CSV or JSON that I can keep next to a project. The tool never copies or moves a clip. It has limits, and they are worth stating. Concrete visual queries work well: "a candle flame" returns candles, with similarity between 0.80 and 0.83 for the top three. Abstract ones do not: "hope after a storm" returned wind turbines, raindrops on a window and a rainy night street, with similarity between 0.58 and 0.63, and I would not trust it. Three frames per clip can also miss fast action. For my use, the approach is good enough, and I know where it stops. Best Use of Gemma. Gemma 3 4B, vision runs locally through Ollama and produces the description of each of the 321 clips: structured output with a JSON schema, a retry strategy, and sanitized results. It is not fine-tuned.