{"slug": "second-voice-facing-the-conversations-that-haunt-you-with-gemma-running-offline", "title": "Second Voice: Facing the Conversations That Haunt You, with Gemma Running Offline on Your Phone", "summary": "A developer built Second Voice, an Expo/React Native app that runs Google's open-weight Gemma 3 1B model (Q4_K_M GGUF, ~806 MB) entirely on a phone via llama.rn to help users rehearse difficult conversations, with nothing uploaded and no account or analytics. The app uses JSON-Schema-constrained decoding to force the model's output into a fixed rehearsal structure, then validates responses and issues repair prompts at lower temperature for up to three retries when drafts are duplicated or drift into medical, legal, therapy or hostile advice. When the model isn't downloaded, it falls back to six hand-written scenarios, each labelled with what produced it.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine](https://dev.to/challenges/hacktoberfest-week4-2026-10-26)*\n\nEveryone has a ghost like this.\n\nIt's the conversation you keep not having. Asking your manager for the raise you've earned. Telling your mom that five calls a day during work is too many. Apologizing to the friend whose birthday dinner you skipped. Texting your landlord that rent will be four days late.\n\nIt doesn't make noise. It just comes back: in the shower, on the commute, at 2 AM, rehearsing itself in your head with worse and worse wording.\n\n**Second Voice** is a private rehearsal room for those conversations. You describe the situation in your own words, by voice or text, and an open-weight model running **on your phone** helps you find a clear, kind way to say it:\n\nThen you can **practice it out loud**. Second Voice reads the draft back to you, you record yourself, and it gives gentle notes on pacing, filler words, clarity, and confidence.\n\nI took the theme literally: the ghost in the machine is the conversation that keeps haunting you, and the machine is a phone that can finally help you put it to rest, without anyone else ever hearing it.\n\n**A private, offline rehearsal room for the conversations you keep avoiding.**\n\nDescribe the conversation that's been haunting you — asking for a raise, setting a boundary with a parent, apologizing to a friend, telling your landlord the rent will be late. Second Voice runs **Google Gemma 3 (1B, open-weight) on your phone** to help you find a clear, kind way to say it, then lets you practice out loud.\n\nNothing is uploaded. There is no account, no analytics, and nothing is ever sent on your behalf.\n\nSecond Voice is a communication preparation tool. It does not provide therapy, counseling, medical, or legal advice.\n\n**Stack:** Expo (SDK 58) + React Native + TypeScript, Expo Router, Zustand, SQLite.\n\n**The open pieces doing the work:**\n\n| Piece | What it does | \n|---|---|\n| **Gemma 3 1B Instruct** (Q4_K_M GGUF, ~806 MB) | Generates the whole rehearsal | \n| **llama.rn** (llama.cpp for React Native) | Runs Gemma on the phone, GPU-offloaded where available | \n| **JSON-Schema constrained decoding** | Forces Gemma's output to match the rehearsal shape | \n| **On-device speech recognition** | Turns voice notes into text with `requiresOnDeviceRecognition: true` | \n\n``` php\nflowchart LR\n  A[Voice note or text] -->|on-device STT| B[Your words]\n  B --> C[Prompt: audience · goal · tone]\n  C --> D[Gemma 3 1B via llama.rn<br/>JSON-schema constrained]\n  D --> E{Validate + safety check}\n  E -->|ok| F[Rehearsal]\n  E -->|invalid| G[Repair prompt, retry]\n  G --> D\n  E -->|no model| H[Built-in examples, clearly labelled]\n  F --> I[(Local SQLite)]\n```\n\nA few design decisions for getting a 1B model to behave:\n\n**1. Make bad JSON impossible.** Small models are good at tone and weak at bookkeeping. Ask them nicely for JSON and you risk trailing commentary or missing keys. llama.rn supports grammar-constrained decoding from a JSON Schema, so the model *can't* emit anything that doesn't fit `{ situationSummary, whatYouWantToSay, howTheyMayHearIt[], safeOpeningLine, drafts{calm,direct,firm}, objections[] }`.\n\n**2. Validate anyway, then repair.** A valid shape can still be bad content: three identical drafts, an empty objection, or a model drifting into advice it shouldn't give. Every response goes through a validator that checks the shape, requires distinct drafts, and rejects medical, legal, therapy, or hostile wording. If it fails, the model gets a *repair prompt* listing exactly what was wrong, at a lower temperature, up to three tries.\n\n**3. Never fake it.** If the model isn't downloaded yet, Second Voice falls back to six hand-written scenarios (salary raise, late rent, a boundary with a parent, an apology, a recruiter follow-up, saying no to weekend work). Every rehearsal is labelled with what produced it: *\"Prepared on this device by Gemma\"* or *\"uses built-in examples.\"* Showing sample output as AI output felt wrong for an app about honesty.\n\n**4. One interface, swappable engines.** The UI only knows `generateRehearsal(input)`. Gemma, the examples, or a future ExecuTorch runtime all sit behind it.\n\nOn my vivo V2225 in airplane mode, Gemma produces a full rehearsal (summary, concerns, opening line, three drafts, and objections) in about **48 seconds**. That's slow for a chatbot and fine for a rehearsal: it's roughly the time it takes to take a breath and decide you're actually going to have this conversation.\n\nBecause of what this app is built to hear. Think about the kind of thing someone might type into it:\n\n*\"I'm scared to ask for a raise because my family depends on this job.\"*\n\n*\"My dad and I haven't really talked since the funeral.\"*\n\nSentences like these are some of the most private a person can write. Sending them to a cloud API means trusting a company's servers, logs, retention policy, and future business model with them. With an open-weight model, I didn't have to ask anyone to make that trade:\n\nA closed API would have been easier to start with. It would also have made the core promise, *your words stay on this device*, a lie.\n\nSecond Voice never sends anything for you. It doesn't tell you what to feel, and it isn't therapy. It helps you prepare, and then you decide what to say, and when.", "url": "https://wpnews.pro/news/second-voice-facing-the-conversations-that-haunt-you-with-gemma-running-offline", "canonical_source": "https://dev.to/thilagavathi1/second-voice-facing-the-conversations-that-haunt-you-with-gemma-running-offline-on-your-phone-5a83", "published_at": "2026-10-31 04:39:45+00:00", "updated_at": "2026-10-31 04:49:27.267781+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-products", "generative-ai"], "entities": ["Google", "Gemma 3", "llama.rn", "llama.cpp", "Expo", "React Native", "Second Voice"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/second-voice-facing-the-conversations-that-haunt-you-with-gemma-running-offline", "markdown": "https://wpnews.pro/news/second-voice-facing-the-conversations-that-haunt-you-with-gemma-running-offline.md", "text": "https://wpnews.pro/news/second-voice-facing-the-conversations-that-haunt-you-with-gemma-running-offline.txt", "jsonld": "https://wpnews.pro/news/second-voice-facing-the-conversations-that-haunt-you-with-gemma-running-offline.jsonld"}}