This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
I learn from AI engineering talks on YouTube, and I keep my own notes on the ideas that matter. The talks win. One channel I follow posted 77 videos in 30 days, 30.6 hours of video, and many of them say the same thing under different names.
lecture-gate watches the channels for me, slowly and steadily. When a talk is uploaded, a small server turns it into a card: a one-line conclusion, a short summary, and claims, each with a word-for-word quote and the second it was said. The cards grow into one map of concepts, so ten talks about one idea become one page. Every morning, when I open my laptop, a local model adds only what is new to my notes. My notes never leave the laptop.
So far: 81 talks on the map, 388 claims, and 346 of their quotes (89%) found word for word in the transcript, ignoring case, punctuation and "uh". Only those are ever shown. This checks that the speaker said it, not that the claim is right.
Before, a new talk would pop up and I'd think: I'll watch it later. Then I got busy with other things and forgot, and the ones I did watch I had to sum up and copy into my notes by hand. Now that is the server's and the laptop's job.
I used it this week. On October 21 the server took five uploads from push notifications without me: four became cards, and one narrated in Korean was skipped on purpose. One of them was a GEICO talk, and that day my Agent Harness note went from 42 lines to 44: my own words stayed as they were, and one new point arrived below them, with its quote and a link to the second it was said.
My Note → Agent Harness on the laptop page, before the merge (left) and after (right). Two real screenshots.
What it saves. On October 22 the laptop page held 4 new talks: 1 hour 52 minutes of video, and about 7 minutes to read. The 7 minutes is the page's own estimate at 200 words a minute; I did not time myself. I still choose which talks to watch in full.
New AI talks from a few YouTube channels become checked cards, the cards grow into one map of concepts and every morning the new points are added to my own notes on the laptop, by a local model.
server (DigitalOcean Droplet) laptop
YouTube push ─► Mastra workflow on Temporal log in / unlock
1 transcript SerpApi (cached) sync.ts pull Atlas ─► data/, map/
2 card Gemma 4 31B + checks (quotes, format) merge.ts embeddinggemma + gemma4:12b (Ollama)
3 map bge-m3 ─► Atlas vector search ─► Gemma ─► notes/ (append only, own git) + 00 Morning.md
decides, a second call checks ask.ts questions over cards and notes
─► MongoDB Atlas
nightly 03:00 UTC: verify.ts turns repeated mistakes into rules (Backboard) that the card and map prompts read
Personal notes never leave the laptop: notes/ is gitignored and only Ollama reads it.
Node 22. Install with the lockfile and without…
My first try was the obvious one: give a small model the tools and let it run the job. Gemma 4 E4B, as a Mastra agent with two tools, called the transcript tool once, never called the notes tool, and then asked me for the video ID it had just used. So no model chooses the steps. The pipeline is one fixed Mastra workflow on Temporal, each model call has one job, and code checks every answer before it is kept.
Every agent is named after its job:
| agent | model | job |
|---|---|---|
| Card Writer | Gemma 4 31B | writes the card |
| Quote Checker | none, plain code | keeps a quote only if it is in the transcript word for word |
| Concept Matcher | Gemma 4 31B + bge-m3 | same idea as an earlier talk, or new? |
| Category Keeper | Gemma 4 31B | files each talk in a section |
| Rule Tester | Gemma 4 31B | keeps a rule only if runs with it are as good or better |
| Note Merger | Gemma 4 12B + EmbeddingGemma | adds new points to my notes, on the laptop |
| Highlighter | Gemma 4 12B | marks key phrases, on the laptop |
| Notes Assistant | Gemma 4 12B + EmbeddingGemma | answers my questions from the talks and my notes, on the laptop |
Gemma 4 31B and bge-m3 run on DigitalOcean's serverless inference. The laptop agents run in Ollama.
While building it, I kept asking one question: what is the worst thing this could do while I sleep? Each answer got a check, and each check has met the real thing.
| What if… | What catches it | What happened |
|---|---|---|
| a card quotes something the speaker never said? | The Quote Checker keeps a quote only if it is in the transcript word for word. | 346 of 388 quotes found; only those are ever shown. |
| the merge changes or deletes my own words? | The merge only appends. After every merge a git check reports any existing line that was removed or changed. A claim my note already has word for word is marked known before the model is asked. | Oct 21: my Agent Harness note went from 42 to 44 lines, my words untouched. Merging the same 4 talks a second time: 23 of 23 known, 0 appended. |
| the server dies in the middle of a talk? | Temporal: every step has a 5-minute timeout and up to 3 attempts. | I killed the worker during the map step; only that step ran again, with the same result. On Oct 22, for real: the card step timed out once, and Attempt 2 finished it. |
| two different ideas are merged because they share a word? | Atlas vector search gives 6 candidates, code accepts only an answer that names one, and a second, strict call checks every join. | "AI Factory" (data centers) was pulled out of "Software Factory". |
| something fails and nobody notices? | Sentry records each failed step as an issue. | LECTURE-GATE-4: the map step failed with EACCES , and the laptop had 77 cards while the server had 80. |
The one that surprised me. On its first test the Quote Checker rejected 2 of 4 real quotes. The model was not the problem: auto-captions contain "uh"s, and the model had dropped them. Fillers, case and punctuation are now ignored on both sides. A check that is too strict looks safe, but it throws away true quotes.
Two smaller ones were Gemma's own. On one real card it wrote the chapter summaries in Korean; a code check now finds non-English fields and sends the card back with "Translate these fields into English … Change nothing else.", which fixed it 3 times in 3 (making the card again from scratch: 1 in 3). And it borrowed the 4 example concept names from my prompt, 18 times across all cards; after I removed them, the share of concept names in the speaker's own words went from 95% to 99%.
The real vectors in Atlas lecture_gate.mentions: the GEICO talk’s 5 concepts, each stored as 1024 numbers from bge-m3 (first 6 shown). The vec index above compares them by cosine.
I picked the card model on a test set of 10 talks (8 to 46 minutes, talks and interviews). With my first prompt, Gemma 4 31B found 90% of its quotes and gpt-oss-120b 71%, at $0.0018 against $0.0021 per card. After that, the prompt changed only when the same test set said so:
From v2 to v7, quotes found rose from 88% to 96%. v8, live since October 22, adds the summary and one claim per 5 minutes of talk; quotes found fell to 94.2%.
Could the laptop make the cards itself? On its CPU (no GPU), one card each for two talks:
| model in Ollama | quotes found | time per card |
|---|---|---|
| Gemma 4 E2B | 8/9 | 3.7–5.6 min |
| Gemma 4 E4B | 10/10 | 8.0–10.0 min |
| Gemma 4 12B | 8/8 | 20.4–21.8 min |
| Gemma 3 4B | 1/7 | 9.7–11.3 min |
| Qwen3 8B | 9/10 | 19.2–33.7 min |
That ranks the models; it proves nothing. Minutes per card is too slow, so cards are made in the cloud and the laptop gets the private job. For that job, E4B handled 17 claims in 86 s against 290 s for 12B, but appended 4 claims my notes already said. A merge that repeats my notes is worse than a slow one, so the Note Merger runs 12B.
Every failed check is saved as a lesson. Showing the agent its own past mistakes did not help: the Category Keeper failed all 4 rounds the same way. So the Rule Tester turns lessons into a candidate rule and runs the task with and without it: 3 runs each for a category rule; for a card rule, one first attempt on the talks the lessons came from plus 2 test talks, where at least as many cards must pass and verbatim quotes may drop by no more than 10 points. A rule that does as well or better goes to Backboard, where the prompts read it. Only the rule reaches the model, never the lessons behind it.
Backboard now holds 7 rules. The newest was added on the morning of October 22: "Do not group terms where one describes a high-level goal or purpose and the other describes a specific technical implementation or process."
To test it, I force-killed the worker 19 seconds into the map step, then started it again. Transcript and card did not run again. The map step's second attempt started about 5 minutes after it was scheduled and finished in 31 s, with the same result as the all-at-once run: "concepts 4/4 same; category same". The price is the wait: @mastra/temporal has no heartbeat option, so Temporal only notices a dead worker when the timeout runs out.
Top: the Mastra workflow definition (src/mastra/index.ts), 5-minute timeout and 3 attempts. Bottom: the same three steps as Temporal recorded them for the GEICO talk.
The October 22 retry from the table, as Temporal recorded it: the second attempt finished in under 3 minutes, and the whole run in 9, while I was doing something else.
Every model call is a Sentry gen_ai.chat span with the model name, time and token counts. In 14 days Sentry recorded 977 model calls, across every model I compared, from Gemma 4 E2B on my laptop to gpt-oss-120b in the cloud. It prices each card call from its tokens (under $0.01), and the output column stays empty: prompts and outputs are never sent.
LECTURE-GATE-4 from the table was my own mistake: I had regenerated the map files as root, but the services run as an unprivileged user. A chown and a re-run as the service user fixed it.
The base is cheap and boring. A $6 DigitalOcean Droplet runs the server and uses 833 of 961 MiB at idle. YouTube blocked scripted transcript fetches from my home and from the Droplet, so transcripts come from SerpApi, cached on disk because the free plan has 250 searches a month.
SerpApi search log: each row is one talk's transcript request (youtube_video_transcript). The top row is the newest talk ( Ve2bAarPA2I), October 22, 19:01 UTC. 105 of the month's 250 searches used.
"Apps $1.48" on that bill is another project of mine on the same account. lecture-gate is GenAI Serverless Inference $0.83 + Droplets $0.02.
The Note Merger reads each new claim next to the note it belongs to (EmbeddingGemma finds the note by meaning) and answers "known", "new" or "unrelated". "Known" counts only if the model copies the note sentence that already says it, and code checks that sentence is there. On the laptop page my notes live under My Note in the sidebar.
Points that none of my notes takes go to a note for their section of the map, so each section grows into a note of its own, without repeats. When a talk raises a question, the Notes Assistant answers only from the 8 closest claims and up to 2 notes, citing [C1] or [N1]; a citation that matches no source is not linked, and the page flags it. The page runs on 127.0.0.1 only.
The demo's narration was made with ElevenLabs (voice "Lara"). Only the narration script was sent; my notes never go to it.
What I want to try next, to catch more errors and make it faster: more harness (a code check behind every model decision, which also skips a model call that takes minutes on my CPU), more nightly Rule Tester runs so a rule is kept on measured help and not only "no harm", the same test set for the laptop agents with more runs per model, and a cost comparison of cloud against laptop, per agent.
Open worked better here because of my notes. A hosted model would need my notes on someone else's server. With open weights, the model that reads them runs on my laptop, and the laptop side is built to need no internet after the morning pull from Atlas: the merge, the highlights and every question go to Gemma in Ollama on 127.0.0.1.
Open models work for me while I sleep. Talks arrive at any hour; the server turns them into cards, and every night the Rule Tester checks its new rules, without me. Nothing in that loop is specific to lectures: the same watch, check and merge could follow other sources I learn from.
My notes stay mine. They are not only about lectures: they hold what I study on my own, my schedule, and the questions I haven't figured out yet, my own map of what I know and what I don't. Only public talks go to the cloud, where they become the shared concept map below.
The concept map from public talks, at 72 lectures. Each dot is a concept, a line joins two concepts named in the same lecture, and the largest dot, Agent Harness, spans 11 lectures.
I learn from my own collection. Because the notes and the cards sit on my laptop, I can ask the Notes Assistant right away in the chat, and it answers from the talks and my notes, citing both. Asking and reading the sources is how I build up the core ideas.
The same family, in two places. Open weights let me split the work: Gemma 4 31B in the cloud for public talks, Gemma 4 12B on my CPU for private notes. Each was chosen by the measurements above. The cloud side has cost $0.85 so far, so it runs every day without rationing.
Where a closed model might be better: a large hosted model might write smoother cards with fewer retries. It would still need the same checks.