Voice Order Taker

A developer built Voice Order Taker, a local Streamlit app that transcribes WhatsApp voice notes in Urdu, Roman Urdu, Pashto and English and extracts them into a structured order sheet using Whisper and Gemma running locally through Ollama. The app constrains the model's output to a fixed JSON schema with a worked example and temperature 0, and converts relative date words like "kal" into real dates, keeping customer addresses on the user's own machine with no cloud or API costs. The developer notes that a small local model is less accurate than a large cloud model on messy mixed-language speech, so the app always presents an editable table for the seller to confirm.

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle https://dev.to/challenges/hacktoberfest-week2-2026-10-12 https://voice-order-taker-jvh7zuvem9z6qbfuqfcvpm.streamlit.app/ https://voice-order-taker-jvh7zuvem9z6qbfuqfcvpm.streamlit.app/ Voice Order Taker is a local app that turns WhatsApp voice notes into a clean order sheet. In Pakistan, a lot of home businesses run on voice notes. A baker, a tiffin cook or a small clothing seller gets messages like "do kilo chocolate cake aur teen dozen samosay kal shaam tak, house 12 Jinnah Road", in Urdu, Roman Urdu, Pashto or a mix with English. The seller replays each note, then writes the order down by hand. With 20 or more orders a week, that is hours of work, and one misheard quantity means a wrong delivery. With my app, the seller uploads the voice notes and the app: The person who would pay for this is a home seller handling a steady stream of orders. They don't need a big business tool. They need something that saves an hour a day and doesn't send their customers' addresses to a stranger's server. Turn WhatsApp voice notes in Urdu, Roman Urdu, Pashto or English into a clean order sheet and a daily prep list. Everything runs on your own computer. No cloud, no API key, no per-order cost. Built for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle . Home bakers, tiffin services and small clothing sellers take most of their orders as WhatsApp voice notes. Someone says "do kilo chocolate cake aur teen dozen samosay kal shaam tak, house 12 Jinnah Road", and the seller has to replay it and write it down by hand. With 20 or more orders a week, this takes hours and mistakes happen. .ogg , .opus , .m4a , .mp3 , .wav . Everything runs on a normal laptop with no cloud service. The most important part was making a small model reliable. A free-form prompt gave me bad results, so I forced the output into a fixed JSON schema and added a worked example inside the prompt: class OrderLine BaseModel : customer: str = "" item: str = "" quantity: float = 1 unit: str = "" delivery date: str = "" address: str = "" notes: str = "" class OrderList BaseModel : orders: list OrderLine r = ollama.chat model=model, format=OrderList.model json schema , options={"temperature": 0, "num ctx": 2048}, messages= {"role": "system", "content": system}, {"role": "user", "content": transcript} , The prompt also converts words like "kal" tomorrow and "parson" day after tomorrow into real dates, and I give Whisper a short list of dish names as an initial prompt so it recognises local food words. What went wrong along the way. I built this on a laptop with a nearly full C: drive and limited RAM, which turned out to be good testing for a "runs on cheap hardware" project: av<17 . Add your real numbers here, for example: "I tested X voice notes. N came out fully correct, M needed one edit, and K failed. Average time per note was about Y seconds on CPU." For this kind of customer, open models are not a nice extra. They are what makes the product possible. The trade-off is honest too: a small local model is less accurate than a large cloud model on messy mixed-language speech, so the app always shows an editable table and the seller confirms the result before acting on it. Add your DevRelay or Entire session link here, or delete this section. Who they were for example, a home baker, a relative who runs a food business , what voice notes they tried, what they said, and whether they would pay. Even one real quote helps. If you did not get a user, delete this section. Best Use of Gemma: the extraction step runs on Gemma locally through Ollama, so all order data stays on the user's machine.