This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
ShopSnap turns WhatsApp order screenshots into a daily sales report. It is made for small shops that take orders over chat.
Lots of small sellers run their whole business inside WhatsApp. A customer writes "2 kg rice and 1 dozen eggs" and the owner replies with a price. By evening the orders are spread over dozens of chats. Who ordered what? Who paid? Who still owes money? Most owners answer that by scrolling back through every chat or by trusting their memory.
Here is how ShopSnap handles it:
Chats that show only a saved name go to a Pending list. You attach a phone number once and ShopSnap remembers that name from then on.
Who would pay for it: any shop owner who sells over chat and keeps the books by hand. The app is free and MIT licensed today. The version I would sell is a packaged desktop app with the setup done for the shop owner. That part is future work.
Upload and review. Drop screenshots then check and fix every order before saving.
Pending. Chats that show only a name wait here until you attach a phone number.
Daily report. Total sales and who has not paid and a total for each customer.
All screenshots use fake demo data.
Turn WhatsApp order screenshots into a daily sales report. Built for small shops.
The AI runs on your own PC. Customer numbers and sales never leave it
Built for the Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle.
The repository was started on October 15, 2026, inside the challenge window.
Demo video: Watch on YouTube
DEV post: ShopSnap: WhatsApp order screenshots to a daily sales report, with AI that stays on your PC
Upload and review. Drop screenshots, then check and fix every order before saving.
Daily report. Total sales, who has not paid, and per-customer totals.
Stack: Next.js (App Router and TypeScript) with shadcn/ui and Tailwind. SQLite through better-sqlite3. Ollama runs the models. zod checks the JSON. libphonenumber-js handles phone numbers and sharp resizes the images.
Open models, both run locally through Ollama:
gemma3:4b reads the screenshot and writes out the chat as text.qwen2.5:3b takes that text and extracts the orders as JSON.
The pipeline:
screenshot -> resize -> vision model (text) -> text model (JSON)
-> code checks -> you review -> SQLite -> report
Two small models on a normal PC will make mistakes. So I did not build the app around trusting them. I built it around checking them. The model proposes and plain code decides:
I tested the pipeline on 10 fake screenshots with bun run eval:
| Measure | Result |
|---|---|
| Screenshots | 10 (fake chats) |
| Orders | 11 |
| Fields correct | 90 of 107 (84.1%) |
| Screenshots with every field right | 8 of 10 |
| Orders with a wrong field | 3 of 11 |
| Wrong orders the code checks did not flag | 2 |
| Average time per screenshot | about 26 s on my PC |
One screenshot did most of the damage. Chat 07 holds two orders. The model returned one and misread its amounts. That single chat caused 15 of the 17 wrong fields. The other miss was in chat 09 where the model misspelled a name and guessed "unpaid" when the status should have been "unknown".
Two wrong orders got past the code checks. One of them is the order the model never returned. Nothing can validate an order that is not there.
What I learned: a 4B model is weakest on chats with more than one order. The checks are good at catching wrong numbers but they cannot catch a missing order. A forced re-read can find an order the first read missed. The review screen is the last safety net before anything is saved.
Real chats are messier than my fake ones so expect lower numbers. That is why the review screen is not optional.
A shop's chats hold customer phone numbers and daily sales. That is private data. With a closed API every screenshot would leave the shop's PC and travel to someone else's server. Many owners would never accept that and I would not blame them.
Open weights changed three things for ShopSnap:
VISION_MODEL or TEXT_MODEL and you are done. It also works offline after the first download.
The honest trade is accuracy. Small local models are weaker than the biggest hosted ones. I answered that with code checks and a human review step instead of a bigger model.
Both models keep their own licenses. Gemma 3 is under the Gemma Terms of Use and Qwen2.5 3B is under the Qwen Research License. The README tells anyone to read both before using ShopSnap in a paid setting.