Building a WhatsApp AI Agent with Gemini Using Gemini as Your Copilot

A developer built a WhatsApp AI agent powered by Google Gemini, using Gemini itself as a coding copilot to scaffold, debug, and extend the code. The agent receives WhatsApp messages via Meta's Cloud API, processes them through the Gemini API, and sends back replies. The project demonstrates using Gemini both as the runtime brain and as an assistant during development.

What you'll build: A WhatsApp number that replies with an AI agent powered by Google Gemini. You'll use Gemini via the Gemini CLI or Gemini Code Assist in your IDE as your pair-programming copilot to scaffold, debug, and extend the code, so Gemini is both the agent's runtime brain and the assistant building it. WhatsApp user │ message ▼ Meta WhatsApp Cloud API ──webhook POST──► Your server Flask ▲ │ │ send reply Graph API ▼ └────────────────────────────────── Gemini API the "brain" Two moving parts: Two hats for Gemini: at runtime , the Gemini API generates replies to WhatsApp users. While building , you use Gemini as your coding copilot — the Gemini CLI npm install -g @google/gemini-cli , then run gemini in your terminal, or Gemini Code Assist inside VS Code / JetBrains. You describe what you want in plain English, and it writes, explains, and fixes the code below. npm install -g @google/gemini-cli or Ask Gemini:"Explain what a Gemini API key can access and how to store it safely as an environment variable." Ask Gemini:"Walk me through creating a permanent WhatsApp access token with a System User in Meta Business Suite." mkdir whatsapp-gemini-agent && cd whatsapp-gemini-agent python -m venv .venv && source .venv/bin/activate pip install flask google-genai requests python-dotenv Create a .env file: GEMINI API KEY=your gemini key WHATSAPP TOKEN=your whatsapp access token WHATSAPP PHONE NUMBER ID=your phone number id VERIFY TOKEN=pick any random string Ask Gemini:"Generate a .gitignore for a Python project and make sure .env is excluded." Meta requires two things from your endpoint: /webhook — a one-time verification handshake. /webhook — where inbound messages arrive.Create app.py : python import os import requests from flask import Flask, request from google import genai from dotenv import load dotenv load dotenv GEMINI API KEY = os.environ "GEMINI API KEY" WHATSAPP TOKEN = os.environ "WHATSAPP TOKEN" PHONE NUMBER ID = os.environ "WHATSAPP PHONE NUMBER ID" VERIFY TOKEN = os.environ "VERIFY TOKEN" app = Flask name client = genai.Client api key=GEMINI API KEY SYSTEM PROMPT = "You are a friendly, concise WhatsApp assistant. " "Keep replies short and clear — this is a chat app, not email." def ask gemini user text: str - str: response = client.models.generate content model="gemini-2.5-flash", contents=user text, config=genai.types.GenerateContentConfig system instruction=SYSTEM PROMPT , return response.text def send whatsapp message to: str, body: str - None: url = f"https://graph.facebook.com/v22.0/{PHONE NUMBER ID}/messages" headers = {"Authorization": f"Bearer {WHATSAPP TOKEN}"} payload = { "messaging product": "whatsapp", "to": to, "type": "text", "text": {"body": body}, } requests.post url, headers=headers, json=payload, timeout=20 @app.get "/webhook" def verify : Meta's verification handshake if request.args.get "hub.mode" == "subscribe" and request.args.get "hub.verify token" == VERIFY TOKEN : return request.args.get "hub.challenge" , 200 return "Forbidden", 403 @app.post "/webhook" def incoming : data = request.get json try: change = data "entry" 0 "changes" 0 "value" message = change "messages" 0 inbound message sender = message "from" user's phone number text = message "text" "body" message text reply = ask gemini text send whatsapp message sender, reply except KeyError, IndexError : Status updates and non-text messages land here — ignore them pass return "OK", 200 if name == " main ": app.run port=5000 Ask Gemini:"Paste this file and explain each function line by line, then suggest error handling I'm missing." Run the server, then tunnel it: python app.py terminal 1 ngrok http 5000 terminal 2 → copy the https URL In Meta's WhatsApp → Configuration : https://<your-ngrok-id .ngrok.io/webhook VERIFY TOKEN from your .env Ask Gemini:"My webhook verification is returning 403. Here's my code and the ngrok logs — what's wrong?" Send a WhatsApp message from your registered number to the test number. Within a second or two you should get a Gemini-generated reply. If nothing comes back, ask Claude to help you read the logs: Ask Gemini:"The webhook receives a POST but no reply is sent. Here's the JSON payload and my server log — trace where it breaks." A chatbot answers. An agent takes actions. Gemini supports function calling — you declare tools, and the model decides when to call them. Example: give the agent a check reservation tool. php def check reservation confirmation code: str - dict: Replace with a real lookup DB, internal API, etc. return {"code": confirmation code, "status": "confirmed", "checkin": "2026-08-14"} def ask gemini agent user text: str - str: response = client.models.generate content model="gemini-2.5-flash", contents=user text, config=genai.types.GenerateContentConfig system instruction=SYSTEM PROMPT, tools= check reservation , SDK auto-generates the schema from the function , return response.text The Gemini SDK reads the function's signature and docstring to build the tool schema, calls it when the user asks about a reservation, and folds the result into its reply. Ask Gemini:"Add a second tool that cancels a reservation, and add conversation memory so the agent remembers earlier messages in the same chat." Good next tools to build with Claude's help: contents . X-Hub-Signature-256 header 200 fast and process Gemini calls in a Ask Gemini:"Write the X-Hub-Signature-256 verification middleware for my Flask app, and refactor the Gemini call to run in a background thread so the webhook returns 200 immediately." gemini "review app.py for security issues" .| Piece | Service | Docs | |---|---|---| | Inbound + outbound messages | WhatsApp Cloud API | developers.facebook.com/docs/whatsapp/cloud-api | | Agent reasoning + tools | Gemini API | ai.google.dev/gemini-api/docs | | Your coding copilot | Gemini CLI / Code Assist | github.com/google-gemini/gemini-cli · codeassist.google | Model note: gemini-2.5-flash is fast and cheap for chat; switch to gemini-2.5-pro for harder reasoning. Check Google's model list for the latest IDs.