Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥
Last night, my father sent four voice notes to our family WhatsApp group. He was planning a trip to Shantiniketan, setting a budget, and — most importantly — updating us on my grandfather's health after a scare that initially sounded like a heart problem.
I was asleep.
By morning, I would have had to play all four notes, remember who said what, and mentally assemble a to-do list. Instead, I played a 48-second audio briefing that caught me up on all of it — in a warm voice, with a receipt for every claim.
This is the story of Amar Khobor (আমার খবর) — a briefing tool born the night a speech-to-text model mistook "heart problem" for "samosa at the market."
Here is what Whisper-large-v3 produced from my father's voice note about Dadu:
আমি হেবে ছিলাম হাটে সমসা কিন্তু দাদু ভললো গ্যাসে সমসা
And here is what Sarvam's Saaras produced from the exact same audio:
আমি ভেবেছিলাম হার্টের সমস্যা, কিন্তু দাদু বলল গ্যাসের সমস্যা।
Whisper wrote "trouble at the market" (হাটে সমসা). My father said "heart problem" (হার্টের সমস্যা). One is a snack. The other is a family medical emergency.
That single mistranscription is why Amar Khobor exists. Family WhatsApp groups don't sleep. Overnight, voice notes pile up in Bengali, Hindi, English, Benglish, and Hinglish — and catching up means playing every note, remembering who said what, and mentally assembling a to-do list. Every morning. Every family member. Nobody has time for that.
Amar Khobor turns a WhatsApp group export into a warm, sub-minute audio briefing plus structured, verifiable show notes.
The digest arrives as two things at once:
Baba → Dadu's gastric issue, not heart → health → urgent.
Here's what it produced from my family's actual messages that night:
"Shobai kemon acho? Amar Khobor is here with a little update from Baba. Baba is already dreaming about our next trip! He's suggesting Shantiniketan—staying in a nice lodge and visiting the Sonajhuri Haat. He also mentioned a budget of around 2000, though he says we might spend a bit more if we enjoy some shopping there. So do think about it, okay? On a more serious note, there was a little scare with Dadu. Baba thought it might be a heart issue, but Dadu mentioned it's just gas. Edkat went to check on him, and Baba is also heading there. Thankfully, Dadu is doing much better now and has already had some khichdi. Baba will be taking him to see Dr. Sen tomorrow morning. So please, don't tension too much and everyone get some good sleep. We'll catch up in the morning. Shuvo ratri!"
Fifty-six messages. Four voice notes. Forty-five seconds of audio you cannot skim.
Morning-me would replay all four notes to find the budget and the health update, then scroll the chat to check who said what. The briefing is 48 seconds, and the receipts are already attached.
The saving isn't the seconds — it's that audio has no Ctrl-F.
Sloths don't catch up faster. They let the night do the work.
I didn't want to pick a transcription provider based on someone else's benchmark table. So I built a bake-off harness that scores Saaras and Whisper-large-v3 on my own family's audio.
One honest caveat first: the WER ground truth in test_data/manifest.yaml is the Saaras transcript (with one speaker name corrected from the chat log). So Saaras's 0.00% WER is an upper bound, not an independent accuracy score — Saaras agrees with itself by construction. The evidence that survived a human check is the diff below.
The clip that matters — bn_003, verified by ear:
| Transcript | |
|---|---|
| What my father said | আমি ভেবেছিলাম হার্টের সমস্যা ... ("I thought it was aheart problem ...") |
| What Saaras wrote | আমি ভেবেছিলাম হার্টের সমস্যা ... ✓ correct |
| What Whisper wrote | আমি হেবে ছিলাম হাটে সমসা ... ✗ "trouble at the market" |
Full per-clip results, including the noise-guard test: test_data/eval_results.json · demo/stt_comparison.md
| Metric | Sarvam Saaras ( saaras:v4 ) |
Groq Whisper ( whisper-large-v3 ) |
|---|---|---|
| Monolingual WER ↓ | 0.00% (upper bound — see caveat) | 67.13% |
| Noise hallucinations | 0 | 1 |
| Average latency ↓ | 499 ms | 814 ms |
That Whisper hallucination is real: on a one-second clip of near-silence, it produced the word "you" from nothing. Saaras produced nothing — correctly.
Why this matters for the product: Saaras is purpose-built for Indic languages and code-mixed speech, and it's what runs in production. Whisper is the open-weights baseline I benchmark against — inspectable, reproducible, and swappable.
Gemma 4 did something I didn't expect: it occasionally emitted its entire chain-of-thought — planning, drafting, self-correcting — before the JSON digest. Sometimes it wrapped the JSON in an array instead of returning a bare object.
My first extractor took the first { to the last }. That grabbed a { from mid-reasoning and produced invalid JSON. The pipeline crashed on a response that contained a perfectly good digest — just buried in chatter.
The fix was a tolerant backward extractor: find the last occurrence of "spoken_script", backtrack to the { that opens that object, and slice to the closing }. Now the parse succeeds even when Gemma rambles — and the prompt is hardened to discourage the rambling in the first place.
idx = content.rfind('"spoken_script"')
start = content.rfind('{', 0, idx)
end = content.rfind('}')
cleaned = content[start:end+1]
This turned a flaky LLM response into a deterministic parse — committed, with tests, in pipeline/summarize.py.
Why filter audio before STT? Voice notes are the expensive part. An energy-based VAD pass tags silence and background noise before any API call, so a forwarded chain of muted clips costs nothing. Forwarded messages are also excluded from summarization — nobody needs a briefing about a chain-forwarded good-morning graphic.
Why one LLM call for both outputs? The spoken script and the show notes describe the same events. Generating them in a single structured Gemma call keeps them consistent — if the script says Dadu needs a doctor, the show-notes card for it exists too.
Why a 202-and-poll architecture? A real overnight export takes a minute to process. POST /digests/run returns 202 Accepted immediately, FastAPI BackgroundTasks does the work, and the frontend polls every 2 seconds. No Celery, no Redis, no message broker — this is a family tool, not a distributed system.
Why ephemeral memory by default? Family voice notes are some of the most private audio that exists. Audio and briefings live strictly in-memory during the session and vanish on server restart. MongoDB Atlas is an opt-in adapter, never a requirement.
I didn't recruit beta testers — I used my own family's real WhatsApp group. Baba sent real voice notes about a real trip and a real health scare. The pipeline transcribed them with Saaras, Gemma summarized them, and ElevenLabs narrated the briefing in a warm Bengali-speaking voice.
The show-notes card for the health update read: Baba → Dadu's gastric issue, not heart → health → urgent. That's the receipt. Not a vague summary — a claim, tied to a sender, tied to a timestamp, flagged as needing attention.
This is a tool for family voice notes — some of the most private audio that exists. That constraint shaped every choice:
The honest asymmetry: Saaras — the model that wins my bake-off — is closed. I can't inspect or self-host it. I picked it for Bengali accuracy and kept the provider swap one environment variable away, which is the only honest position for a model you don't own.
render.yaml provisions an always-on Starter instance; the free tier would add a cold start after 15 minutes idle.Dockerfile and Sarvam's Saaras isn't a prize track, but it's central to the open-models story: it's the production STT that made the "samosa vs heart problem" comparison possible in the first place.
Built for the Hacktoberfest Open-Source AI Challenge, Week 3: Slow and Steady. Speech-to-text by Sarvam AI and Groq; briefing voice by ElevenLabs.