# Missed calls, missed revenue: the local-first phone assistant I'm building > Source: > Published: 2026-08-22 02:34:47+00:00 Previous posts in this series covered the infrastructure: [running three AI models sequentially on one server](https://dev.to/hannune/running-three-ai-models-on-one-local-server-when-your-vram-doesnt-cover-all-of-them-b7g) and [why I run speech-to-text locally instead of calling a cloud API](https://dev.to/hannune/why-i-run-speech-to-text-locally-instead-of-calling-a-cloud-api-59j7). This one is about the business problem I'm building toward. I run a solo consulting practice. Client communication arrives over three channels: phone calls (including voicemails I check hours later), messaging apps, and email. None of these talk to each other. After a busy stretch, I found myself doing a manual audit: scrolling back through four or five apps, trying to reconstruct what commitments I'd made, who I hadn't responded to, what questions were still open. It's time-consuming and I kept missing things. The expensive miss is a phone call. Someone calls, leaves a voicemail, doesn't follow up in email. If I'm mid-task when the call comes in and don't process the voicemail until the next day, that lead has likely moved on. Industry data suggests small businesses miss 40–60% of incoming calls. I don't know exactly what my number is, but I've been that business. The tools I evaluated each handle one channel. Transcription services for calls. Email summary tools for inbox. Nothing I found reads all three and produces a single output. The integration gap is where the problem actually lives. The goal is a daily summary: here's who contacted you, here's what they wanted, here's what's still open, here are the likely follow-ups. The infrastructure pieces are working individually. Whisper running locally handles call transcription—audio stays on my server, never goes to an external API. bge-m3 handles embeddings for semantic search. Gemma handles images and screenshots. All running sequentially on one box. What I'm building is the layer on top: intake → process → report. Two problems I've been figuring out this week: Before building the generator, I spent time designing the output format. A daily report isn't useful if it's just a list of transcripts. The items that actually need to surface: The hard part wasn't picking the categories—it was making each one specific enough to actually implement. "Show me missed follow-ups" is ambiguous. "Flag any contact where they reached out and I have no outbound reply within 48 hours" is implementable. I went through the format item by item and rewrote vague categories as concrete conditions. If someone calls Monday and emails Tuesday, those land in two separate systems with no shared ID. To produce a per-contact summary, I need to link them. This is an entity resolution problem—the same class of problem I work on in my main client work, building ER pipelines for corporate data. The structure is identical: match records that don't share a unique identifier. For comms data, the linking fields are usually phone number or email address. Complications: `john.smith@company.com` and `jsmith@company.com` )My current sketch: normalize phone numbers (strip formatting, expand country codes), match email with tolerance for common alias patterns, and handle name variants as a fallback signal rather than a primary match key. I haven't implemented this yet—this is still the design phase. **Working:** Local transcription (Whisper), local embeddings (bge-m3), local VLM (Gemma). All running sequentially on one server. Audio, documents, and images stay local. **Building:** The pipeline that takes call transcripts + email content + message content and produces the daily report. The output format is designed. The entity resolution approach is sketched. Nothing is integrated end to end. The gap between "pieces working individually" and "system producing something I'd actually use daily" is what I'm working through now. Next concrete step: get the report generator to produce output I'd actually want to read, not just a proof of concept that technically runs. *Previously in this series: Running three AI models on one local server | Why local STT instead of a cloud API*