{"slug": "missed-calls-missed-revenue-the-local-first-phone-assistant-i-m-building", "title": "Missed calls, missed revenue: the local-first phone assistant I'm building", "summary": "A developer building a local-first phone assistant for their solo consulting practice is tackling the problem of missed calls and fragmented client communication. The system uses locally running AI models—Whisper for transcription, bge-m3 for embeddings, and Gemma for image analysis—to process calls, emails, and messages, with the goal of generating a daily summary of contacts, follow-ups, and open questions. The developer is currently designing the output format and entity resolution to link records across channels.", "body_md": "Previous posts in this series covered the infrastructure: [running three AI models sequentially on one server](https://dev.to/hannune/running-three-ai-models-on-one-local-server-when-your-vram-doesnt-cover-all-of-them-b7g) and [why I run speech-to-text locally instead of calling a cloud API](https://dev.to/hannune/why-i-run-speech-to-text-locally-instead-of-calling-a-cloud-api-59j7). This one is about the business problem I'm building toward.\n\nI run a solo consulting practice. Client communication arrives over three channels: phone calls (including voicemails I check hours later), messaging apps, and email. None of these talk to each other.\n\nAfter a busy stretch, I found myself doing a manual audit: scrolling back through four or five apps, trying to reconstruct what commitments I'd made, who I hadn't responded to, what questions were still open. It's time-consuming and I kept missing things.\n\nThe expensive miss is a phone call. Someone calls, leaves a voicemail, doesn't follow up in email. If I'm mid-task when the call comes in and don't process the voicemail until the next day, that lead has likely moved on. Industry data suggests small businesses miss 40–60% of incoming calls. I don't know exactly what my number is, but I've been that business.\n\nThe tools I evaluated each handle one channel. Transcription services for calls. Email summary tools for inbox. Nothing I found reads all three and produces a single output. The integration gap is where the problem actually lives.\n\nThe goal is a daily summary: here's who contacted you, here's what they wanted, here's what's still open, here are the likely follow-ups.\n\nThe infrastructure pieces are working individually. Whisper running locally handles call transcription—audio stays on my server, never goes to an external API. bge-m3 handles embeddings for semantic search. Gemma handles images and screenshots. All running sequentially on one box.\n\nWhat I'm building is the layer on top: intake → process → report.\n\nTwo problems I've been figuring out this week:\n\nBefore building the generator, I spent time designing the output format. A daily report isn't useful if it's just a list of transcripts. The items that actually need to surface:\n\nThe hard part wasn't picking the categories—it was making each one specific enough to actually implement. \"Show me missed follow-ups\" is ambiguous. \"Flag any contact where they reached out and I have no outbound reply within 48 hours\" is implementable. I went through the format item by item and rewrote vague categories as concrete conditions.\n\nIf someone calls Monday and emails Tuesday, those land in two separate systems with no shared ID. To produce a per-contact summary, I need to link them.\n\nThis is an entity resolution problem—the same class of problem I work on in my main client work, building ER pipelines for corporate data. The structure is identical: match records that don't share a unique identifier.\n\nFor comms data, the linking fields are usually phone number or email address. Complications:\n\n`john.smith@company.com`\n\nand `jsmith@company.com`\n\n)My current sketch: normalize phone numbers (strip formatting, expand country codes), match email with tolerance for common alias patterns, and handle name variants as a fallback signal rather than a primary match key. I haven't implemented this yet—this is still the design phase.\n\n**Working:** Local transcription (Whisper), local embeddings (bge-m3), local VLM (Gemma). All running sequentially on one server. Audio, documents, and images stay local.\n\n**Building:** The pipeline that takes call transcripts + email content + message content and produces the daily report. The output format is designed. The entity resolution approach is sketched. Nothing is integrated end to end.\n\nThe gap between \"pieces working individually\" and \"system producing something I'd actually use daily\" is what I'm working through now. Next concrete step: get the report generator to produce output I'd actually want to read, not just a proof of concept that technically runs.\n\n*Previously in this series: Running three AI models on one local server | Why local STT instead of a cloud API*", "url": "https://wpnews.pro/news/missed-calls-missed-revenue-the-local-first-phone-assistant-i-m-building", "canonical_source": "https://dev.to/hannune/missed-calls-missed-revenue-the-local-first-phone-assistant-im-building-2086", "published_at": "2026-08-22 02:34:47+00:00", "updated_at": "2026-08-22 03:13:13.493990+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-products", "developer-tools"], "entities": ["Whisper", "bge-m3", "Gemma"], "alternates": {"html": "https://wpnews.pro/news/missed-calls-missed-revenue-the-local-first-phone-assistant-i-m-building", "markdown": "https://wpnews.pro/news/missed-calls-missed-revenue-the-local-first-phone-assistant-i-m-building.md", "text": "https://wpnews.pro/news/missed-calls-missed-revenue-the-local-first-phone-assistant-i-m-building.txt", "jsonld": "https://wpnews.pro/news/missed-calls-missed-revenue-the-local-first-phone-assistant-i-m-building.jsonld"}}