{"slug": "i-built-a-voice-based-daily-reflection-companion-under-15-minutes-using-agora", "title": "I Built a Voice-Based Daily Reflection Companion under 15 Minutes Using Agora Agents SDK", "summary": "A developer built a voice-based daily reflection companion called Compass in under 15 minutes using the Agora Agents SDK, which wraps Agora's real-time communications infrastructure into a Python, TypeScript, or Go library. The agent chains Deepgram STT, an OpenAI GPT-4o-mini LLM, and MiniMax TTS, and handles mid-sentence interruptions out of the box without custom interruption-handling code. The developer said the biggest surprise was that when they interrupted the agent mid-sentence, \"she just stopped and listened.", "body_md": "I expected to spend a weekend on working on my vision. Instead, I had a voice agent asking me \"How was your day?\" in under 15 minutes. What surprised me most wasn't the speed - it was that when I interrupted the voice AI agent I named \"Compass\" mid-sentence to change my answer, she just stopped and listened. No stuttering, no doubled audio, no ghost speech finishing in the background. It just worked, out of the box, without a single line of interruption-handling code on my end.\n\nIf you've tried to build a voice agent from scratch, you know the pipeline looks deceptively simple on a whiteboard: speech in, text to LLM, speech out. The reality is messier.\n\nYou need a WebRTC or WebSocket layer to stream audio in real time. You need to integrate STT (and handle partial transcripts), stream tokens to TTS, manage token refresh, handle network retries, detect when the user starts speaking mid-sentence, and somehow prevent the agent from continuing its TTS output while the user is already replying. That's before you write a single line of actual product logic.\n\nThe Agora Agents SDK removes every item on that list. It's built on top of Agora's existing RTC infrastructure, which is the same real-time communications network that powers video calling for hundreds of millions of users. The SDK wraps that into a Python (or TypeScript or Go) library where you describe *what* your agent should do, not *how* the audio pipeline should work.\n\n**Daily Reflection Companion**\n\na voice AI named Compass that guides users through a structured end-of-day reflection: a daily check-in, a meaningful moment exploration, a gratitude round, a forward intention, and then an AI-generated written summary of the conversation.\n\n**Install**\n\n```\npip install agora-agents fastapi uvicorn[standard] python-dotenv openai\n```\n\nThat's it. The SDK installs in under 10 seconds. Imports are under `agora_agent` (note: no hyphen at import time).\n\n**Credentials You'll Need**\n\nCopy your `.env.example` to `.env`:\n\n```\nAGORA_APP_ID=your_app_id\nAGORA_APP_CERTIFICATE=your_certificate\nOPENAI_API_KEY=sk-…\nDEEPGRAM_API_KEY=your_deepgram_key\n```\n\n**Note:** The Agora console's Conversational AI toggle is not prominently labelled. I spent about 4 minutes finding it - it lives under *Project Settings → Features*. Once enabled, everything worked on the first try.\n\n**The builder chain**\n\nThe entire STT → LLM → TTS pipeline is configured in one fluent chain. Here's the real code from [agent.py](//agent.py):\n\n``` python\nfrom agora_agent import (\n  Agent, Agora, Area,\n  DeepgramSTT, OpenAI as AgoraOpenAI, MiniMaxTTS,\n  expires_in_hours,\n)\n\nclient = Agora(area=Area.US, app_id=app_id, app_certificate=app_certificate)\n\nagent = (\n  Agent(client=client, turn_detection={\"language\": \"en-US\"})\n  .with_stt(DeepgramSTT(model=\"nova-3\", language=\"en\"))\n  .with_llm(\n    AgoraOpenAI(\n    model=\"gpt-4o-mini\",\n    system_messages=[{\"role\": \"system\", \"content\": REFLECTION_SYSTEM_PROMPT}],\n    greeting_message=\"Hello! I'm Compass. How was your day today?\",\n    failure_message=\"I didn't quite catch that. Could you say that again?\",\n    max_history=50,\n    params={\"max_tokens\": 150, \"temperature\": 0.75},\n    )\n  )\n  .with_tts(MiniMaxTTS(model=\"speech_2_6_turbo\", voice_id=\"English_captivating_female1\"))\n)\n```\n\nEvery line is intentional:\n\n`turn_detection={\"language\": \"en-US\"}` - tells the VAD (Voice Activity Detection) which language's speech patterns to use for end-of-turn detection. This directly affects how quickly the agent recognises you've finished speaking.`max_history=50` - the agent's LLM receives up to 50 turns of conversation as context. Critical for a reflection agent that needs to remember what was said early in the session.`max_tokens=150` - voice replies need to be short. Capping at 150 tokens enforces this at the model level, not just the prompt.`greeting_message` - the agent speaks this immediately when the session starts, without waiting for the user to speak first. No extra `session.say()` call needed.\nThe reflection flow is driven entirely by the LLM system prompt. No explicit state machine, no conditional logic in the server code. The prompt instructs the agent to move through five phases naturally:\n\n```\nPHASE 1 - DAILY CHECK-IN: Ask one meaningful follow-up after the user's response.\nPHASE 2 - MEANINGFUL MOMENT: Explore one significant experience.\nPHASE 3 - GRATITUDE: Ask for 2–3 things they're thankful for.\nPHASE 4 - TOMORROW'S INTENTION: Ask for one thing they'd like to carry forward.\nPHASE 5 - CLOSING: Offer a warm, personalised goodbye.\nVOICE RULES: Keep all responses under 35 words. One question per turn. No markdown.\n```\n\nThe \"under 35 words\" constraint was the most important prompt engineering decision. Voice synthesis doesn't render markdown, and long replies feel like lectures, not conversations.\n\n```\nStarting a Session\nsession = agent.create_session(\nchannel=f\"reflection-{int(time.time())}\",\nagent_uid=\"999\",\nremote_uids=[\"*\"],\nidle_timeout=90,\nexpires_in=expires_in_hours(1),\n)\nagent_id = session.start()\n```\n\n`session.start()` is a single blocking call that provisions the agent, connects it to the RTC channel, and returns an `agent_id`. The channel name is how the browser's Agora Web SDK joins the same audio room.\n\nAfter the conversation ends, I call `session.get_history()` to retrieve the transcript and send it to GPT-4o-mini with a structured summary prompt:\n\n```\nhistory = session.get_history()\n# Format turns into a readable transcript, then:\nresponse = openai_client.chat.completions.create(\nmodel=\"gpt-4o-mini\",\nmessages=[\n{\"role\": \"system\", \"content\": SUMMARY_PROMPT},\n{\"role\": \"user\", \"content\": f\"Transcript:\\n\\n{transcript}\"},\n],\nmax_tokens=200,\n)\nreturn response.choices[0].message.content\n```\n\nThe summary appears on-screen after the session ends - a written record of what you reflected on. This is the feature that makes this more than a demo.\n\nThe browser uses the Agora Web SDK (loaded from CDN - no npm, no webpack) to join the same RTC channel:\n\n```\n// Create client and join\nrtcClient = AgoraRTC.createClient({ mode: \"rtc\", codec: \"vp8\" });\nawait rtcClient.join(app_id, channel, token || null, uid);\n// Publish microphone\nlocalMicTrack = await AgoraRTC.createMicrophoneAudioTrack({\nencoderConfig: \"speech_standard\",\n});\nawait rtcClient.publish(localMicTrack);\n// Subscribe to agent's audio and play it\nrtcClient.on(\"user-published\", async (user, mediaType) => {\nif (mediaType !== \"audio\") return;\nawait rtcClient.subscribe(user, \"audio\");\nuser.audioTrack.play();\n});\n```\n\nThree async calls: `join`, `publish`, `subscribe`. That's the entire WebRTC layer.\n\nOnce the pipeline runs, the question that matters is: does it feel like a real conversation?\n\n**Turn-taking**\n\nThe Deepgram nova-3 model detects end-of-speech accurately. Compass waits for me to finish, then responds within about 1.2–1.8 seconds (STT transcription + LLM generation + TTS synthesis). Agora's published benchmark is <650ms end-to-end - I measured closer to 1.2–1.5s for GPT-4o-mini with MiniMax TTS, which still feels conversational rather than laggy.\n\n**Interruption test**\n\nThis is where I was genuinely surprised. I spoke over Compass mid-sentence. She stopped. Immediately, cleanly, no audio bleed. I tried this five times with different timing - sometimes right as she started a word, sometimes deep into a sentence. Every time the cut was instant.\n\nThis is handled entirely by Agora's RTC infrastructure and the VAD layer. I wrote zero interruption-handling code. It just worked.\n\n**Conversation quality**\n\nThe reflection flow felt natural across few turns. The agent stayed on-topic, transitioned between phases at the right moments, and the \"under 35 words\" constraint kept responses appropriately brief for voice. The one area that needed tuning: Compass occasionally asked two questions in one turn early in testing (prompt refinement fixed this with a firm \"Ask only ONE question per turn\" rule).\n\n*Built with Agora Agents SDK (Python) + OpenAI GPT-4o-mini + Deepgram STT + MiniMax TTS*\n\n**What's genuinely great**\n\nThe builder pattern is the right abstraction. Instead of wiring five services together yourself, you declare your pipeline and the SDK handles transport, buffering, retries, and token refresh. \n\nInterruption handling working out of the box is not a small thing - in a DIY build, that's easily a full day of work.\n\n`session.think()` (not demonstrated in this project's MVP but available) is a powerful primitive: you can inject mid-session instructions into the LLM without the agent speaking them aloud. For a more sophisticated reflection agent, this could be used to nudge the agent toward a specific phase based on time elapsed.\n\n**What could be better**\n\n**The gap between quickstart and production. **The CLI quickstart (`agora init`) scaffolds a working app fast, but the gap from that template to understanding *why* each piece exists is steep. Better intermediate documentation (not just API reference, not just quickstart) would help.\n\n**`get_history()` response format. **The method exists and works, but the response shape isn't clearly documented. I had to handle three possible formats defensively. This is the kind of thing that adds 30 minutes to an otherwise 5-minute task.\n\n**Error specificity.** When I accidentally misconfigured my App Certificate, the error was a generic 401. A message like \"App Certificate mismatch - check AGORA_APP_CERTIFICATE in your environment\" would have saved 10 minutes of debugging.\n\nThese are fixable problems, not fundamental ones. The core pipeline is rock-solid.\n\nI started this build expecting to spend most of my time on infrastructure. Instead, I spent most of it on the part that matters - the conversation design, the system prompt, the reflection flow. That's the correct trade-off, and the SDK made it possible.\n\nThe Agora Agents SDK doesn't replace the REST API - it's built on top of it, and REST stays fully supported. What it does is remove the RTC plumbing so you can focus on what your agent should say and feel like, not on how audio bytes travel between browser and model.\n\nFor a real-time voice product where interruption, latency, and audio quality matter, the infrastructure Agora provides is serious. For a developer who wants to build that product in an afternoon rather than a week, this SDK is the fastest path I've found.\n\n```\npip install agora-agents fastapi uvicorn[standard] python-dotenv openai\ncp .env.example .env\n# Fill in your API keys\nuvicorn main:app - reload\n# Open http://localhost:8000\n```\n\n**GitHub (Python SDK):** [https://github.com/AgoraIO/agora-agents-python](https://github.com/AgoraIO/agora-agents-python)\n\n*#VoiceAI #AIagents #Agora #ConversationalAI #ConvoAI #OpenAI #Agora #TTS #STT*", "url": "https://wpnews.pro/news/i-built-a-voice-based-daily-reflection-companion-under-15-minutes-using-agora", "canonical_source": "https://dev.to/dear-arah/i-built-a-voice-based-daily-reflection-companion-under-15-minutes-using-agora-agents-sdk-4eki", "published_at": "2026-09-10 10:17:33+00:00", "updated_at": "2026-09-10 10:28:18.120945+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "natural-language-processing", "ai-products"], "entities": ["Agora", "Agora Agents SDK", "Deepgram", "OpenAI", "MiniMax", "GPT-4o-mini", "Compass", "FastAPI"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-voice-based-daily-reflection-companion-under-15-minutes-using-agora", "markdown": "https://wpnews.pro/news/i-built-a-voice-based-daily-reflection-companion-under-15-minutes-using-agora.md", "text": "https://wpnews.pro/news/i-built-a-voice-based-daily-reflection-companion-under-15-minutes-using-agora.txt", "jsonld": "https://wpnews.pro/news/i-built-a-voice-based-daily-reflection-companion-under-15-minutes-using-agora.jsonld"}}