This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
My professor emails on Monday: assignment due Friday. On Tuesday: "correction, because of the holiday it's now Thursday."
An email summariser gives me two accurate summaries and zero help. I still have to notice they're about the same assignment, remember which one wins, and move the date in my head.
Inbox Sloth does that part for me. It reads a mailbox with Gemma running on my own laptop, turns emails into commitments (what, by when, which email said so), and notices when a later email changes an earlier one:
Submit Probability Assignment 3: Thu 22 Oct, 11:59 PM. Changed (was Fri 23 Oct).
Evidence: "Please submit Probability Assignment 3 by Thursday 22 October, 11:59 PM instead."
I open one page and see what to do first, what's due this week, what clashes, and which dates moved and are worth double-checking.
Run it yourself in a few minutes (the demo command runs the real model on 18 sample emails and scores it against ground truth):
ollama pull gemma3:1b
cd backend && pip install -r requirements.txt
python -m pytest -q # 132 tests, no Ollama needed
python -m app.cli demo # real model run, scored against ground truth
uvicorn app.main:app # http://localhost:8000
https://github.com/newton-lebniz/inbox-sloth
Stack: gemma3:1b via Ollama (JSON-schema constrained output, temperature 0), FastAPI, SQLite, plain HTML/JS, pytest. It runs on a CPU-only laptop with 16 GB RAM.
The key design decision: the model is a noisy witness. The model reads; code decides.
email -> Gemma (local, JSON-schema output) -> checks in plain code -> commitment tracker -> plan
Valid model answers are cached by (model, prompt, email), so re-analysing all 18 emails after a code change takes about a second instead of three minutes. That is what made fast iteration possible on a CPU.
On Day 1, my evaluator said: 10 of 12 commitments found, 6 of 12 dates right.
Then I read the rows. It was comparing "Pay semester fee" with "Submit data structures exam". My matcher accepted the best title match even when the best was bad, so the score looked better than the output. After fixing the evaluator and re-scoring the same real output:
| Day 1 (real gemma3:1b run) | reported | actual |
|---|---|---|
| Commitments found | 10/12 | 6/12 |
| Dates correct | 6/12 | 5/12 |
| False positives | 2 | 6 |
What the real run showed: for 4 of the 14 actionable emails, including the deadline correction, no commitment came out. It invented tasks for informational emails ("Khichdi and papad dinner" from a mess-menu notice). It rewrote dates, and a quiz became "Attend class".
So the interesting problem wasn't prompting. It was building something that stays correct when the model doesn't. With the checks above, same prompt:
| Day 1 (re-scored) | After the code checks (real Gemma run) | |
|---|---|---|
| Found | 6/12 | 12/12 |
| Dates correct | 5/12 | 12/12 |
| Kinds correct | 5/12 | 12/12 |
An ablation (on recorded Gemma answers) shows where it comes from: without the rule safety net, 10/12 found and 9/12 dates; without the cue check, the false positives come back.
Email is the most private text most people have: grades, client names, medical appointments, offers. I didn't want to ship that to a third-party API just to find out when my lab report is due.
SLOTH_MODEL=gemma3:4b) changes the model. That mattered, because a 1B model made mistakes and the architecture had to survive them.
The saving is the part I used to do by hand: noticing that Tuesday's email overrides Monday's, and re-checking my own list. The honest limit: I haven't run it on a real mailbox for a week, only on 18 synthetic emails, so I can't claim hours saved. What I can show is that on those emails it found every commitment and applied the date correction, and that for every task it shows the sentence it came from, so checking it takes seconds.
What I learned: (1) a metric you wrote in ten minutes is a hypothesis, and mine flattered me until I read the rows; (2) let the model read and let code decide; (3) make every judgment visible. The evidence sentence, the ranking reasons and the ignored list made the tool trustworthy and also debuggable.
Best Use of Gemma