{"slug": "from-phone-call-to-formatted-email-in-80-lines-of-python-ai-voice-memo-cleanup", "title": "From Phone Call to Formatted Email in 80 Lines of Python — AI Voice Memo Cleanup with Telnyx", "summary": "Telnyx has released an 80-line Python Flask webhook that turns a phone call into a formatted email, using its unified API for voice, AI inference, and messaging. The demo answers a call, gathers a spoken memo, cleans up the transcript with AI, and sends the result as an email, all with a single API key. The project showcases a state-machine webhook handler that processes Telnyx events to orchestrate the flow.", "body_md": "AI Voice Memo to Email — an 80-line Flask webhook that answers a phone call, gathers a spoken memo, runs it through AI Inference to clean up grammar and extract structure, and delivers a formatted email. One API key for voice, AI, and messaging. No third-party services.\n\nVoice memos are the fastest way to capture a thought — you speak, you're done. But what you get is a rambling audio blob that nobody (including you) wants to read later. The raw transcript is worse: no punctuation, false starts, filler words, and no structure. You still have to manually clean it up before it's useful as an email, a status update, or a meeting summary.\n\nThe existing solutions split the problem across multiple services. A transcription service converts audio to text. An LLM API cleans up the text. An email service sends the result. Three vendors, three API keys, three bills, three points of failure.\n\nThe AI Voice Memo to Email example does all of it on one network — Telnyx Call Control handles the phone call, Telnyx AI Inference cleans up the transcript, and Telnyx Messaging delivers the email. One API key. One Flask file. About 80 lines of Python.\n\nYou call a Telnyx number. The app answers, speaks a greeting, and starts listening. You dictate your memo — a status update, a meeting summary, a bug report, whatever — and press `#`\n\nwhen you're done. The app sends the transcript to AI Inference with a prompt that returns structured JSON: a subject line, a formatted body, and a list of action items. The app sends that as an email to your default address and confirms back on the call: \"Memo saved and emailed. Subject: [inferred subject]. Goodbye!\"\n\n| Step | Event | Action |\n|---|---|---|\n| 1 |\n`call.initiated` (incoming) |\nAnswer the call, create session |\n| 2 | `call.answered` |\nTTS: \"Voice memo. Speak your memo after the tone. Press pound when finished.\" |\n| 3 | `call.speak.ended` |\nStart speech gather (120s timeout, `#` terminates) |\n| 4 | `call.gather.ended` |\nSend transcript to AI Inference → get structured JSON → send email → TTS confirmation |\n| 5 | `call.hangup` |\nClean up session |\n\nThe memo is also stored in memory and accessible via `GET /memos`\n\n— so even if email delivery isn't configured, the formatted memo is still retrievable.\n\nEverything lives in one Flask file. No database, no Redis, no Celery. Call state is tracked in an in-memory dict keyed by `call_control_id`\n\n. Memos are stored in a list. A background thread cleans up expired sessions every 5 minutes (1-hour TTL).\n\n```\nCaller dials your Telnyx number\n        ↓\nTelnyx sends call.initiated webhook → /webhooks/voice\n        ↓\napp calls answer() → creates session in active_calls[ccid]\n        ↓\nTelnyx sends call.answered → app calls speak() with greeting\n        ↓\nTelnyx sends call.speak.ended → app calls gather(input_type=\"speech\", terminating_digit=\"#\")\n        ↓\nCaller dictates memo, presses #\n        ↓\nTelnyx sends call.gather.ended with speech transcript\n        ↓\napp sends transcript to AI Inference → gets JSON {subject, body, action_items}\n        ↓\napp sends email via Telnyx Messaging API\n        ↓\napp calls speak() with confirmation: \"Memo saved and emailed. Subject: X. Goodbye!\"\n        ↓\nTelnyx sends call.hangup → app removes session\n```\n\nThe webhook handler is a state machine driven by Telnyx events. Each event triggers the next action:\n\n``` python\n@app.route(\"/webhooks/voice\", methods=[\"POST\"])\ndef handle_voice():\n    # Verify the Telnyx Ed25519 signature before trusting the event.\n    try:\n        client.webhooks.unwrap(request.get_data(as_text=True), headers=dict(request.headers))\n    except Exception:\n        return jsonify({\"error\": \"invalid signature\"}), 401\n\n    payload = request.get_json()\n    event_type = payload.get(\"data\", {}).get(\"event_type\")\n    data = payload.get(\"data\", {})\n    p = data.get(\"payload\", {})\n    ccid = p.get(\"call_control_id\")\n\n    if event_type == \"call.initiated\" and p.get(\"direction\") == \"incoming\":\n        active_calls[ccid] = {\"caller\": p.get(\"from\"), \"raw_text\": [], \"start\": time.time()}\n        client.calls.actions.answer(ccid)\n        return jsonify({\"status\": \"answering\"}), 200\n\n    elif event_type == \"call.answered\":\n        client.calls.actions.speak(ccid,\n            payload=\"Voice memo. Speak your memo after the tone. Press pound when finished.\",\n            voice=\"female\", language_code=\"en-US\")\n        return jsonify({\"status\": \"greeting\"}), 200\n\n    elif event_type == \"call.speak.ended\":\n        client.calls.actions.gather(ccid,\n            input_type=\"speech\", end_silence_timeout_secs=5, timeout_secs=120,\n            language_code=\"en-US\", terminating_digit=\"#\")\n        return jsonify({\"status\": \"recording\"}), 200\n\n    elif event_type == \"call.gather.ended\":\n        call = active_calls.get(ccid)\n        speech = p.get(\"speech\", {}).get(\"result\", \"\")\n        if call and speech:\n            call[\"raw_text\"].append(speech)\n            # ... AI cleanup + email + confirmation\n        return jsonify({\"status\": \"processed\"}), 200\n\n    elif event_type == \"call.hangup\":\n        active_calls.pop(ccid, None)\n        return jsonify({\"status\": \"ended\"}), 200\n```\n\nThe state machine has five transitions, one per event. The `call.initiated`\n\nhandler checks `direction == \"incoming\"`\n\nto avoid processing outbound call legs. The `call.speak.ended`\n\nhandler is what advances from greeting to gathering — Telnyx fires this event when TTS playback finishes, so you know the caller has heard the greeting before the gather starts.\n\nThe gather uses `end_silence_timeout_secs=5`\n\n— if the caller stops speaking for 5 seconds, the gather ends automatically. The `timeout_secs=120`\n\ncaps the total gather at 2 minutes. The `terminating_digit=\"#\"`\n\nlets the caller explicitly signal \"I'm done\" by pressing pound.\n\nThe core of the app is a single inference call that turns rambling speech into structured JSON:\n\n``` python\ndef call_inference(messages, max_tokens=400):\n    resp = requests.post(INFERENCE_URL,\n        headers={\"Authorization\": f\"Bearer {TELNYX_API_KEY}\", \"Content-Type\": \"application/json\"},\n        json={\"model\": AI_MODEL, \"messages\": messages,\n              \"max_tokens\": max_tokens, \"temperature\": 0.3},\n        timeout=15)\n    resp.raise_for_status()\n    return resp.json()[\"choices\"][0][\"message\"][\"content\"]\n```\n\nThe system prompt asks for three fields — subject, body, and action_items:\n\n```\nformatted = call_inference([\n    {\"role\": \"system\", \"content\":\n        \"Clean up this voice memo into a well-formatted email. \"\n        \"Fix grammar, add structure (paragraphs, bullets if needed). \"\n        \"Return JSON: subject (string, inferred from content), \"\n        \"body (string, the formatted memo), \"\n        \"action_items (list of strings).\"},\n    {\"role\": \"user\", \"content\": speech}\n])\nmemo = json.loads(formatted)\n```\n\nTemperature is 0.3 — low enough that the same memo produces roughly the same output every time, but high enough that the AI can infer a reasonable subject line from the content. The `max_tokens=400`\n\ncap is sufficient for a typical voice memo.\n\nIf the AI response isn't valid JSON, the `except`\n\nblock saves the raw speech and speaks a simpler confirmation — the caller still gets their memo saved, just without the email:\n\n```\nexcept Exception:\n    memos.append({\"raw\": speech, \"caller\": call[\"caller\"],\n                  \"timestamp\": time.strftime(\"%Y-%m-%dT%H:%M:%SZ\")})\n    client.calls.actions.speak(ccid, payload=\"Memo saved. Goodbye!\",\n        voice=\"female\", language_code=\"en-US\")\n```\n\nGraceful degradation — the call is never wasted. If AI fails, the raw transcript is preserved. If email fails, the formatted memo is preserved. The caller always gets a confirmation.\n\nAfter the memo is formatted, the app sends it as an email through the Telnyx Messaging API:\n\n``` python\ndef send_email(to, subject, body):\n    try:\n        requests.post(\"https://api.telnyx.com/v2/messages\",\n            headers={\"Authorization\": f\"Bearer {TELNYX_API_KEY}\",\n                     \"Content-Type\": \"application/json\"},\n            json={\"from\": {\"email_address\": f\"memo@{MEMO_NUMBER.replace('+','')}.telnyx.com\"},\n                  \"to\": [{\"email_address\": to}],\n                  \"subject\": subject, \"body\": body, \"type\": \"email\"},\n            timeout=15)\n    except Exception as e:\n        app.logger.error(\"Email send failed: %s\", e)\n```\n\nThe same `TELNYX_API_KEY`\n\nthat answers the call and runs the AI inference also sends the email — one key, one bill, one network. The email send is wrapped in a try/except because email delivery may require additional Telnyx setup. If it fails, the memo is still saved and retrievable via `GET /memos`\n\n.\n\nEvery Telnyx webhook is signed with an Ed25519 key. The app verifies the signature before processing the event:\n\n```\ntry:\n    client.webhooks.unwrap(request.get_data(as_text=True), headers=dict(request.headers))\nexcept Exception:\n    return jsonify({\"error\": \"invalid signature\"}), 401\n```\n\nThe `webhooks.unwrap()`\n\nmethod from the Telnyx Python SDK handles the Ed25519 verification internally — it reads the `telnyx-signature-ed25519`\n\nand `telnyx-timestamp`\n\nheaders, reconstructs the signed payload, and verifies the signature against the public key. The raw body is verified, not the parsed JSON — because JSON parsing is not canonical, and the signature would fail.\n\n```\ngit clone https://github.com/team-telnyx/telnyx-code-examples.git\ncd telnyx-code-examples/ai-voice-memo-to-email-python\ncp .env.example .env   # add TELNYX_API_KEY, MEMO_NUMBER, DEFAULT_EMAIL\npip install -r requirements.txt\npython app.py           # starts on http://localhost:5000\n```\n\nThen:\n\n```\nngrok http 5000\n```\n\nConfigure your Call Control Application webhook URL to `https://<id>.ngrok.io/webhooks/voice`\n\nin the [Telnyx Portal](https://portal.telnyx.com/call-control/applications).\n\nCall your Telnyx number. Speak your memo. Press `#`\n\n. Check your email.\n\nCheck saved memos:\n\n```\ncurl http://localhost:5000/memos | python3 -m json.tool\n```\n\n**Key links:**", "url": "https://wpnews.pro/news/from-phone-call-to-formatted-email-in-80-lines-of-python-ai-voice-memo-cleanup", "canonical_source": "https://dev.to/harpreetseehra/from-phone-call-to-formatted-email-in-80-lines-of-python-ai-voice-memo-cleanup-with-telnyx-32ga", "published_at": "2026-08-05 10:44:52+00:00", "updated_at": "2026-08-05 10:49:33.246084+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "natural-language-processing", "ai-products"], "entities": ["Telnyx", "Flask", "Python", "AI Inference", "Messaging API", "Call Control"], "alternates": {"html": "https://wpnews.pro/news/from-phone-call-to-formatted-email-in-80-lines-of-python-ai-voice-memo-cleanup", "markdown": "https://wpnews.pro/news/from-phone-call-to-formatted-email-in-80-lines-of-python-ai-voice-memo-cleanup.md", "text": "https://wpnews.pro/news/from-phone-call-to-formatted-email-in-80-lines-of-python-ai-voice-memo-cleanup.txt", "jsonld": "https://wpnews.pro/news/from-phone-call-to-formatted-email-in-80-lines-of-python-ai-voice-memo-cleanup.jsonld"}}