{"slug": "let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes", "title": "Let any AI agent answer WhatsApp voice notes in 10 minutes", "summary": "A developer at denegra-ai built agent-voice-bridge, an open-source Apache 2.0 FastAPI gateway that connects any HTTP-based AI agent to WhatsApp voice notes, handling speech-to-text and text-to-speech so agents receive plain JSON and reply with text. The bridge supports both Twilio and Meta's WhatsApp Cloud API, and the writeup walks through a Docker-based setup with ngrok, a configure wizard, and a ten-line echo agent for testing.", "body_md": "Many people on WhatsApp don't type. They hold the mic button and talk. If your AI agent only understands text, those users get nothing back.\n\nIn this tutorial you'll connect an AI agent to WhatsApp so that:\n\nThe agent can be anything that answers an HTTP POST: a LangGraph graph, a CrewAI flow, an n8n workflow or a ten-line FastAPI app. We'll use the last one so you can see every moving part, then swap in your real agent at the end.\n\nThis is about voice notes, the recorded messages WhatsApp users send with the mic button, not live WhatsApp calls.\n\nThe glue is [agent-voice-bridge](https://github.com/denegra-ai/agent-voice-bridge), a small open-source FastAPI gateway (Apache 2.0) we wrote for this. We looked for an existing project that combines all three: any agent over HTTP, both Meta and Twilio, and voice note in → voice note out. We didn't find one, so we built it.\n\n```\nWhatsApp user ──► Twilio ──► agent-voice-bridge ──► your agent\n                                │    ▲                  │\n                    speech-to-text  text-to-speech      │\n                                │    │                  │\nWhatsApp user ◄── Twilio ◄──────┴────┴──── reply_text ◄─┘\n```\n\n`{\"reply_text\": \"...\"}`.\nYour agent never sees a Twilio or Meta payload. The same agent also works on Meta's WhatsApp Cloud API without changes.\n\n`cloudflared`\nThis is the whole agent. It just echoes what it heard, which is perfect for checking the pipes.\n\n``` python\n# echo_agent.py\nfrom fastapi import FastAPI\n\napp = FastAPI()\n\n@app.post(\"/agent\")\nasync def agent(msg: dict) -> dict:\n    return {\"reply_text\": f\"You said ({msg['message_type']}): {msg['content']}\"}\npip install fastapi uvicorn\nuvicorn echo_agent:app --port 9000\n```\n\nThe bridge will send it a body like this:\n\n```\n{\n  \"sender_id\": \"whatsapp:+905551112233\",\n  \"message_type\": \"voice\",\n  \"content\": \"What time does the store open tomorrow?\",\n  \"channel\": \"twilio\"\n}\n```\n\n`sender_id` is stable per person, so use it as your conversation or session key.\n\nIn a second terminal:\n\n```\nngrok http 8000\n```\n\nCopy the `https://…ngrok-free.app` URL. That's your public URL.\n\n`join two-words`. Send that phrase from your WhatsApp to the sandbox number.\n\n```\ngit clone https://github.com/denegra-ai/agent-voice-bridge.git\ncd agent-voice-bridge\npython3 scripts/configure.py\n```\n\nThe setup wizard asks only what your choices need:\n\n`http://host.docker.internal:9000/agent` (the bridge runs in Docker, your echo agent on your laptop)\nKeys are typed hidden and checked against the real service before anything is saved. The wizard writes `.env` and prints the webhook URL you need next.\n\n```\ndocker compose up -d --build\ncurl http://localhost:8000/health\n# {\"status\":\"ok\",\"version\":\"0.1.1\",\"channels\":[\"twilio\"]}\n```\n\nOn the same Twilio page, open **Sandbox settings** and set **When a message comes in** to:\n\n```\nhttps://<your-ngrok-url>/webhook/twilio\n```\n\nMethod `POST`, then **Save**.\n\nSend a text to the sandbox number. You get:\n\nYou said (text): hello\n\nNow hold the mic button and say something. A few seconds later you get a **voice note** back that reads out \"You said (voice): …\" with your transcript.\n\nPoint `AGENT_WEBHOOK_URL` at anything that accepts the JSON above and returns `reply_text`. For example, a LangGraph graph with a checkpointer:\n\n``` python\nfrom fastapi import FastAPI\nfrom my_agent import graph  # your compiled LangGraph graph\n\napp = FastAPI()\n\n@app.post(\"/agent\")\nasync def agent(msg: dict) -> dict:\n    result = await graph.ainvoke(\n        {\"messages\": [(\"user\", msg[\"content\"])]},\n        config={\"configurable\": {\"thread_id\": msg[\"sender_id\"]}},\n    )\n    return {\"reply_text\": result[\"messages\"][-1].content}\n```\n\nIn n8n it's a **Webhook** node (respond using \"Respond to Webhook\") and a **Respond to Webhook** node that returns `{\"reply_text\": \"...\"}` at the end of your workflow.\n\nIf your agent needs a token, set `AGENT_AUTH_TOKEN` and the bridge sends it as `Authorization: Bearer …`.\n\n`X-Twilio-Signature` and Meta's `X-Hub-Signature-256` are checked. Message content is never logged. Generated audio sits at an unguessable URL and is deleted after an hour.`STT_PROVIDER=deepgram` or `TTS_PROVIDER=elevenlabs` in `.env`.\nA note on what's been tested: we've run the full voice round trip on a real phone through the Twilio sandbox, once with Azure OpenAI for speech and once with Deepgram (speech-to-text) and ElevenLabs (text-to-speech). The plain OpenAI option uses the same code as Azure OpenAI but hasn't had its own phone test yet. Meta's Cloud API path is verified up to the signed webhook and the agent call; we haven't yet done a full round trip on a phone through Meta. If you try it, we'd love to hear how it goes in the [issues](https://github.com/denegra-ai/agent-voice-bridge/issues).\n\nThe repo is here: **[https://github.com/denegra-ai/agent-voice-bridge](https://github.com/denegra-ai/agent-voice-bridge)**. Stars, issues and PRs are welcome.\n\n*agent-voice-bridge is open source and backed by [Denegra](https://denegra.ai).*", "url": "https://wpnews.pro/news/let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes", "canonical_source": "https://dev.to/denegraai/let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes-2mei", "published_at": "2026-11-05 21:17:56+00:00", "updated_at": "2026-11-05 21:23:22.226719+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "natural-language-processing", "ai-products"], "entities": ["denegra-ai", "agent-voice-bridge", "WhatsApp", "Twilio", "Meta", "FastAPI", "LangGraph", "n8n"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes", "markdown": "https://wpnews.pro/news/let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes.md", "text": "https://wpnews.pro/news/let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes.txt", "jsonld": "https://wpnews.pro/news/let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes.jsonld"}}