# Let any AI agent answer WhatsApp voice notes in 10 minutes

> Source: <https://dev.to/denegraai/let-any-ai-agent-answer-whatsapp-voice-notes-in-10-minutes-2mei>
> Published: 2026-11-05 21:17:56+00:00

Many people on WhatsApp don't type. They hold the mic button and talk. If your AI agent only understands text, those users get nothing back.

In this tutorial you'll connect an AI agent to WhatsApp so that:

The agent can be anything that answers an HTTP POST: a LangGraph graph, a CrewAI flow, an n8n workflow or a ten-line FastAPI app. We'll use the last one so you can see every moving part, then swap in your real agent at the end.

This is about voice notes, the recorded messages WhatsApp users send with the mic button, not live WhatsApp calls.

The glue is [agent-voice-bridge](https://github.com/denegra-ai/agent-voice-bridge), a small open-source FastAPI gateway (Apache 2.0) we wrote for this. We looked for an existing project that combines all three: any agent over HTTP, both Meta and Twilio, and voice note in → voice note out. We didn't find one, so we built it.

```
WhatsApp user ──► Twilio ──► agent-voice-bridge ──► your agent
                                │    ▲                  │
                    speech-to-text  text-to-speech      │
                                │    │                  │
WhatsApp user ◄── Twilio ◄──────┴────┴──── reply_text ◄─┘
```

`{"reply_text": "..."}`.
Your agent never sees a Twilio or Meta payload. The same agent also works on Meta's WhatsApp Cloud API without changes.

`cloudflared`
This is the whole agent. It just echoes what it heard, which is perfect for checking the pipes.

``` python
# echo_agent.py
from fastapi import FastAPI

app = FastAPI()

@app.post("/agent")
async def agent(msg: dict) -> dict:
    return {"reply_text": f"You said ({msg['message_type']}): {msg['content']}"}
pip install fastapi uvicorn
uvicorn echo_agent:app --port 9000
```

The bridge will send it a body like this:

```
{
  "sender_id": "whatsapp:+905551112233",
  "message_type": "voice",
  "content": "What time does the store open tomorrow?",
  "channel": "twilio"
}
```

`sender_id` is stable per person, so use it as your conversation or session key.

In a second terminal:

```
ngrok http 8000
```

Copy the `https://…ngrok-free.app` URL. That's your public URL.

`join two-words`. Send that phrase from your WhatsApp to the sandbox number.

```
git clone https://github.com/denegra-ai/agent-voice-bridge.git
cd agent-voice-bridge
python3 scripts/configure.py
```

The setup wizard asks only what your choices need:

`http://host.docker.internal:9000/agent` (the bridge runs in Docker, your echo agent on your laptop)
Keys are typed hidden and checked against the real service before anything is saved. The wizard writes `.env` and prints the webhook URL you need next.

```
docker compose up -d --build
curl http://localhost:8000/health
# {"status":"ok","version":"0.1.1","channels":["twilio"]}
```

On the same Twilio page, open **Sandbox settings** and set **When a message comes in** to:

```
https://<your-ngrok-url>/webhook/twilio
```

Method `POST`, then **Save**.

Send a text to the sandbox number. You get:

You said (text): hello

Now hold the mic button and say something. A few seconds later you get a **voice note** back that reads out "You said (voice): …" with your transcript.

Point `AGENT_WEBHOOK_URL` at anything that accepts the JSON above and returns `reply_text`. For example, a LangGraph graph with a checkpointer:

``` python
from fastapi import FastAPI
from my_agent import graph  # your compiled LangGraph graph

app = FastAPI()

@app.post("/agent")
async def agent(msg: dict) -> dict:
    result = await graph.ainvoke(
        {"messages": [("user", msg["content"])]},
        config={"configurable": {"thread_id": msg["sender_id"]}},
    )
    return {"reply_text": result["messages"][-1].content}
```

In n8n it's a **Webhook** node (respond using "Respond to Webhook") and a **Respond to Webhook** node that returns `{"reply_text": "..."}` at the end of your workflow.

If your agent needs a token, set `AGENT_AUTH_TOKEN` and the bridge sends it as `Authorization: Bearer …`.

`X-Twilio-Signature` and Meta's `X-Hub-Signature-256` are checked. Message content is never logged. Generated audio sits at an unguessable URL and is deleted after an hour.`STT_PROVIDER=deepgram` or `TTS_PROVIDER=elevenlabs` in `.env`.
A note on what's been tested: we've run the full voice round trip on a real phone through the Twilio sandbox, once with Azure OpenAI for speech and once with Deepgram (speech-to-text) and ElevenLabs (text-to-speech). The plain OpenAI option uses the same code as Azure OpenAI but hasn't had its own phone test yet. Meta's Cloud API path is verified up to the signed webhook and the agent call; we haven't yet done a full round trip on a phone through Meta. If you try it, we'd love to hear how it goes in the [issues](https://github.com/denegra-ai/agent-voice-bridge/issues).

The repo is here: **[https://github.com/denegra-ai/agent-voice-bridge](https://github.com/denegra-ai/agent-voice-bridge)**. Stars, issues and PRs are welcome.

*agent-voice-bridge is open source and backed by [Denegra](https://denegra.ai).*