# Building a WhatsApp AI Agent with Gemini Using Gemini as Your Copilot

> Source: <https://dev.to/devpato/-building-a-whatsapp-ai-agent-with-gemini-using-gemini-as-your-copilot-41>
> Published: 2026-07-29 17:19:11+00:00

**What you'll build:** A WhatsApp number that replies with an AI agent powered by Google Gemini. You'll use **Gemini** (via the **Gemini CLI** or **Gemini Code Assist** in your IDE) as your pair-programming copilot to scaffold, debug, and extend the code, so Gemini is both the agent's runtime brain *and* the assistant building it.

```
WhatsApp user
     │  message
     ▼
Meta WhatsApp Cloud API  ──webhook POST──►  Your server (Flask)
     ▲                                            │
     │  send reply (Graph API)                    ▼
     └──────────────────────────────────  Gemini API (the "brain")
```

Two moving parts:

**Two hats for Gemini:** at *runtime*, the Gemini API generates replies to WhatsApp users. While *building*, you use Gemini as your coding copilot — the **Gemini CLI** (`npm install -g @google/gemini-cli`

, then run `gemini`

) in your terminal, or **Gemini Code Assist** inside VS Code / JetBrains. You describe what you want in plain English, and it writes, explains, and fixes the code below.

`npm install -g @google/gemini-cli`

) or

Ask Gemini:"Explain what a Gemini API key can access and how to store it safely as an environment variable."

Ask Gemini:"Walk me through creating a permanent WhatsApp access token with a System User in Meta Business Suite."

```
mkdir whatsapp-gemini-agent && cd whatsapp-gemini-agent
python -m venv .venv && source .venv/bin/activate
pip install flask google-genai requests python-dotenv
```

Create a `.env`

file:

```
GEMINI_API_KEY=your_gemini_key
WHATSAPP_TOKEN=your_whatsapp_access_token
WHATSAPP_PHONE_NUMBER_ID=your_phone_number_id
VERIFY_TOKEN=pick_any_random_string
```

Ask Gemini:"Generate a .gitignore for a Python project and make sure .env is excluded."

Meta requires two things from your endpoint:

`/webhook`

— a one-time verification handshake.`/webhook`

— where inbound messages arrive.Create `app.py`

:

``` python
import os
import requests
from flask import Flask, request
from google import genai
from dotenv import load_dotenv

load_dotenv()

GEMINI_API_KEY = os.environ["GEMINI_API_KEY"]
WHATSAPP_TOKEN = os.environ["WHATSAPP_TOKEN"]
PHONE_NUMBER_ID = os.environ["WHATSAPP_PHONE_NUMBER_ID"]
VERIFY_TOKEN = os.environ["VERIFY_TOKEN"]

app = Flask(__name__)
client = genai.Client(api_key=GEMINI_API_KEY)

SYSTEM_PROMPT = (
    "You are a friendly, concise WhatsApp assistant. "
    "Keep replies short and clear — this is a chat app, not email."
)

def ask_gemini(user_text: str) -> str:
    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=user_text,
        config=genai.types.GenerateContentConfig(system_instruction=SYSTEM_PROMPT),
    )
    return response.text

def send_whatsapp_message(to: str, body: str) -> None:
    url = f"https://graph.facebook.com/v22.0/{PHONE_NUMBER_ID}/messages"
    headers = {"Authorization": f"Bearer {WHATSAPP_TOKEN}"}
    payload = {
        "messaging_product": "whatsapp",
        "to": to,
        "type": "text",
        "text": {"body": body},
    }
    requests.post(url, headers=headers, json=payload, timeout=20)

@app.get("/webhook")
def verify():
    # Meta's verification handshake
    if (request.args.get("hub.mode") == "subscribe"
            and request.args.get("hub.verify_token") == VERIFY_TOKEN):
        return request.args.get("hub.challenge"), 200
    return "Forbidden", 403

@app.post("/webhook")
def incoming():
    data = request.get_json()
    try:
        change = data["entry"][0]["changes"][0]["value"]
        message = change["messages"][0]          # inbound message
        sender = message["from"]                  # user's phone number
        text = message["text"]["body"]            # message text

        reply = ask_gemini(text)
        send_whatsapp_message(sender, reply)
    except (KeyError, IndexError):
        # Status updates and non-text messages land here — ignore them
        pass
    return "OK", 200

if __name__ == "__main__":
    app.run(port=5000)
```

Ask Gemini:"Paste this file and explain each function line by line, then suggest error handling I'm missing."

Run the server, then tunnel it:

```
python app.py            # terminal 1
ngrok http 5000          # terminal 2  → copy the https URL
```

In Meta's WhatsApp → **Configuration**:

`https://<your-ngrok-id>.ngrok.io/webhook`

`VERIFY_TOKEN`

from your `.env`

Ask Gemini:"My webhook verification is returning 403. Here's my code and the ngrok logs — what's wrong?"

Send a WhatsApp message from your registered number to the test number. Within a second or two you should get a Gemini-generated reply.

If nothing comes back, ask Claude to help you read the logs:

Ask Gemini:"The webhook receives a POST but no reply is sent. Here's the JSON payload and my server log — trace where it breaks."

A chatbot answers. An **agent** takes actions. Gemini supports **function calling** — you declare tools, and the model decides when to call them.

Example: give the agent a `check_reservation`

tool.

``` php
def check_reservation(confirmation_code: str) -> dict:
    # Replace with a real lookup (DB, internal API, etc.)
    return {"code": confirmation_code, "status": "confirmed", "checkin": "2026-08-14"}

def ask_gemini_agent(user_text: str) -> str:
    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=user_text,
        config=genai.types.GenerateContentConfig(
            system_instruction=SYSTEM_PROMPT,
            tools=[check_reservation],   # SDK auto-generates the schema from the function
        ),
    )
    return response.text
```

The Gemini SDK reads the function's signature and docstring to build the tool schema, calls it when the user asks about a reservation, and folds the result into its reply.

Ask Gemini:"Add a second tool that cancels a reservation, and add conversation memory so the agent remembers earlier messages in the same chat."

Good next tools to build with Claude's help:

`contents`

.`X-Hub-Signature-256`

header`200`

fast and process Gemini calls in a

Ask Gemini:"Write the`X-Hub-Signature-256`

verification middleware for my Flask app, and refactor the Gemini call to run in a background thread so the webhook returns 200 immediately."

`gemini "review app.py for security issues"`

).| Piece | Service | Docs |
|---|---|---|
| Inbound + outbound messages | WhatsApp Cloud API | developers.facebook.com/docs/whatsapp/cloud-api |
| Agent reasoning + tools | Gemini API | ai.google.dev/gemini-api/docs |
| Your coding copilot | Gemini CLI / Code Assist | github.com/google-gemini/gemini-cli · codeassist.google |

*Model note: gemini-2.5-flash is fast and cheap for chat; switch to gemini-2.5-pro for harder reasoning. Check Google's model list for the latest IDs.*