# TiffinTally: a WhatsApp-to-prep-sheet tool for tiffin cooks, and an honest look at its evals > Source: > Published: 2026-10-18 08:36:16+00:00 This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 2, theme: Side Hustle. Home tiffin cooks take orders on WhatsApp, usually late in the evening, in Hinglish: "Kal lunch 2, dinner skip." "Aaj raat 1 thali, bina pyaaz lahsun." "Roommate left, cancel one thali from tomorrow." I ran those three through the shipped rules (`heuristic_extract`, sender Ravi) on 17 October 2026. The first is a lunch of 2 and a dinner cancel. The second is one dinner with both `no onion` and `no garlic`. The third does not name a meal, so it is held for review (` unscoped_decrement`) instead of cancelling every lunch. Someone has to turn that chat into a count of plates, a list of special requests and a record of who owes what. A missed cancellation wastes groceries; a missed extra order loses a customer. You paste a WhatsApp export. TiffinTally: Parses the chat and extracts order events (meal, quantity, new or cancel, dietary notes, payments). Applies cutoff times and re-statements ("kal lunch 2" followed by "kal lunch 3 kar do"). Builds a kitchen prep sheet and a dues ledger. A payment mentioned in chat stays a claim until the cook confirms it. Masks phone numbers, addresses and names before any message goes to a hosted model. I have not put it in front of a cook yet, so I make no claim about real users. Most of my time went into finding out what I could not claim. What is measured today is only the offline regex rules, on the 50 rows of the frozen synthetic test set. Strict match (meal, quantity, action, mods, payment, secondary customer) is correct on 35 of those 50 rows, with a Wilson 95 percent interval of 56.2 to 80.9 points. Lenient match (meal, quantity, action) is correct on 47 of 50, interval 83.8 to 97.9. The table with accuracy percentages, intervals and the source file for every number is `RESULTS.md`, generated by `scripts/publish_results.py` from saved per-row outputs. What is not measured: the base model, a few-shot model, my fine-tuned adapter and a hosted Gemma run. All are marked NOT RUN in `RESULTS.md`. I do not claim any model beats the rules, and I have not run Gemma at all, so no Gemma number exists anywhere in this repository. Why I trust even those numbers only a little. The data is synthetic. The labels come from the templates that generated the text. The test split shares sibling templates with the training split. I documented this in `docs/KNOWN_LEAKS.md`, and the intervals do not correct for it. I had earlier reported model accuracy figures. The per-row outputs were never saved and the adapter cannot be re-scored, so I retracted them. The old logs are kept under `evals/retracted/`. An audit found five regex patterns I had fitted to test rows. I removed them, and the regex score went down. A case that used to fail. The rules used to read "don't send lunch" as one new lunch order. They now cancel it, same as the Hinglish line: ``` python from tiffintally.extract.base import heuristic_extract heuristic_extract("don't send lunch", "Ravi") # action=cancel, meal=lunch, qty=0 heuristic_extract("kal lunch mat bhejna", "Ravi") # action=cancel, meal=lunch, qty=0 ``` Both lines are what that command prints today, 17 October 2026. I have not measured how any model does on English negation, so I make no claim either way. A meal that is not named ("cancel one thali") is still sent to review rather than guessed. Live: [https://tiffintally-lbn2.onrender.com](https://tiffintally-lbn2.onrender.com). It runs in demo mode, so nothing a visitor pastes is written to disk, and with no model key configured the app answers with its on-device rules and the page says so. The stills and the video below are a local recording of this commit, on that same rules path. The names in the paste are made up. The recording does not press "Confirm received" on the payment claim. Captioned recording, about three minutes, no audio: [tiffintally-demo.mp4](https://github.com/Vedant817/tiffintally/blob/main/docs/demo/tiffintally-demo.mp4) ([download](https://raw.githubusercontent.com/Vedant817/tiffintally/main/docs/demo/tiffintally-demo.mp4), [captions](https://github.com/Vedant817/tiffintally/blob/main/docs/demo/tiffintally-demo.srt)). Run it locally: ``` git clone https://github.com/Vedant817/tiffintally.git cd tiffintally uv sync --all-extras --dev uv run tiffintally serve ``` Open [http://127.0.0.1:8000](http://127.0.0.1:8000), paste an export and read the prep sheet. With no keys set, the app uses the offline rules and the page says so ("Rules (regex) only"). When a model answers, the page names the model that actually answered. When it falls back, it says that too. `/benchmarks` runs a metric scorer over four hand-written sentences. It needs gold labels, so it demonstrates the scorer rather than correcting live output, and one of the four sentences does not pass. A home-kitchen tool that turns a pasted WhatsApp export into a prep sheet and a dues ledger. Paste an export, select the delivery date and meal price, then review uncertain or late messages. The ledger follows the same daily policy as the sheet. Payments in chat remain claims until the cook confirms receipt. Opening and closing balances can be carried between chats using CSV. Exporting separate chats remains manual work; no WhatsApp integration or first cook session is claimed. **Live:** [https://tiffintally-lbn2.onrender.com](https://tiffintally-lbn2.onrender.com) The deployed instance runs in demo mode: sheets, reviews and payment confirmations stay in memory for your browser session and are never written to disk, so anything you paste into it is discarded when you close the tab. No model key is configured there, so it answers with the local rules path and says so in the UI rather than pretending a hosted model replied. Stack: FastAPI, Pydantic, uv, pytest. On 18 October 2026, `EXTRACTOR_BACKEND=off uv run --frozen pytest -q` in this tree reported 368 passed and 4 expected failures (`xfailed`). That number is from the local run. The GitHub Actions result for the commit is the check on the pull request, not this sentence. Extraction backends, chosen by environment variables: Offline regex rules (default, no network). Gemma through Google AI Studio when `GOOGLE_API_KEY` is set, default id `gemma-4-26b-a4b-it`. No run of it has been measured, so I claim nothing about its accuracy. A Qwen2.5-3B LoRA adapter on Modal. The Qwen base carries a research licence, so it is non-commercial; I treat it as an experiment. The adapter is not published and not measured in this release. If it is ever distributed, its licence is separate from this repository's MIT licence, and "Built with Qwen" attribution is required. Any OpenAI-compatible endpoint, such as local Ollama. It is wired in but I have not measured it. The default hosted id used to be a Gemma 2 9B that the API had retired. It answered 404, which my client treats as terminal, so every call silently fell back to the rules while the page still named Gemma. The id is fixed, the parser now skips the "thinking" part Gemma 4 puts before its JSON, and a failed call shows the rules label plus the HTTP error instead of a model name. Model output is untrusted. Messages are numbered in the batch and replies are matched by index. A message the model skips falls back to the rules and is flagged for review. Quantities are bounded and prompt-injection text is flagged. Privacy. Nothing is written to disk by default. The opt-in SQLite mode (`TIFFINTALLY_PERSIST=1`) is experimental: names come back as tokens after a restart, and the cutoff, date, price and corrections are not restored. While it is on, one pseudonymizer per upload replaces every sender name, every roommate named in a message, phone numbers, email addresses and flat or building numbers with sequential tokens, in the stored lines and in the customer, secondary customer and payment note columns. A street name with no flat number in front of it is not matched, so street names can remain. A test pastes an export and reads the SQLite file (every table, plus the WAL and shared-memory bytes) and every file written under `data/` for that session, including `data/private/`. That is the disk the test covers. It does not say that no name can ever reach disk. Reconciliation and the ledger are plain deterministic code, so counts and money never depend on a model. Why open matters here The rules path runs on a laptop with no internet, so a cook's customer list never has to leave the kitchen. An open-weight model behind a local endpoint extends that. If a hosted model is used, the masked text goes to that provider and spends the owner's quota. The deployed demo has no model key, so it uses the on-device rules; the blueprint documents `EXTRACTOR_BACKEND=off` to make that explicit. I am not claiming that value is set in the Render dashboard. Open weights also let me swap models without rewriting the app. I have not measured how much any of this saves. What is next A small set of real, human-written cook messages with hand labels. Until that exists, nothing here says how the app performs on real chats. The kitchen board keeps one corner radius, one accent, and no drop shadow, so it reads as a docket. That is the same point as [@kiwibreaksme on why generated UIs look off](https://dev.to/kiwibreaksme/why-ai-generated-uis-look-off-and-the-one-principle-that-fixes-it-4j20) and [Steven Gonsalvez on the landing-page default](https://dev.to/stevengonsalvez/impeccable-stop-your-ai-making-everything-look-like-a-saas-landing-page-2a4d). Leaving the chat on the laptop unless a model key is set is the same bet as [Srijan on not sending the data to a cloud API](https://dev.to/srijan_bhai/stop-leaking-data-to-cloud-apis-self-hosting-deepseek-locally-with-ollama-4701). This entry is for the overall Side Hustle prize, and for Best Use of Render. Best Use of Render The app is deployed and live at [https://tiffintally-lbn2.onrender.com](https://tiffintally-lbn2.onrender.com) on Render's free tier, from a `render.yaml` blueprint in the repository: `runtime: python`, dependencies resolved from the committed lockfile with `uv sync --frozen --no-dev`, a `/health` endpoint, and demo mode with `TIFFINTALLY_PERSIST=0`. Platform health checks are off on the free plan (` healthCheckPath` is empty in `render.yaml`). Every pull request also gets its own deployment from `.github/workflows/preview.yml`, which waits until the preview answers on `/health` before it reports ready. That is the whole claim: a rules-only, nothing-persisted demo on a free instance. No hosted model key is configured there, and I am not claiming a model result from it.