# Building a Real-Time AI Voice Agent That Handles 70+ Languages > Source: > Published: 2026-07-25 01:50:44+00:00 Most small businesses still handle phone calls the old way: a human answers, or a menu tree routes the caller to a department. If it's after hours, the call goes to voicemail. If the customer speaks a language the team doesn't cover, the call drops. If the issue needs two departments, the customer repeats themselves twice. We wanted to build an AI agent that could take over those calls — not as a chatbot on a website, but as a real-time voice agent on the phone. And we wanted it to work in 70+ languages without building a separate model for each one. The result is [RingBot](https://ringbot.ai), an Agent-as-a-Service platform. This post is a short technical walkthrough of what we learned building it. A traditional voice bot works like this: That works, but the latency is noticeable. Every step adds 500ms–1s, and the conversation feels robotic. Real-time voice models like GPT Realtime, Gemini Live, and Grok Voice use an end-to-end audio pipeline. The model listens to raw audio and generates audio back directly, without going through text in the middle. That cuts latency dramatically and makes turn-taking feel natural. We built RingBot to be model-agnostic. Each real-time model has trade-offs: Instead of betting on one model, we route calls to the best model for the job. A caller can even switch models mid-conversation if that improves the experience. Our production stack looks like this: The agent orchestrator is the interesting part. It doesn't just call a model. It also manages intent classification, routing rules, tool calls (e.g., check order status, book an appointment), and escalation thresholds. The biggest surprise was that multilingual voice is now mostly a solved problem — for the model layer. The harder part is the surrounding system: Voice conversations move fast. If a caller says, "Actually, I want to change my appointment, not cancel it," the agent has to update its plan immediately. If it then transfers to a human, the receiving agent needs to know: We solve this by maintaining a structured conversation state that travels with the call. The human agent sees a short summary plus the full transcript, instead of asking the customer to repeat the story. Under the hood, this is a fairly technical system. But the end user — a small business owner — shouldn't have to think about telephony, embeddings, or model selection. So we wrapped everything in a config flow: connect your business info, upload your docs, pick a phone number, and deploy. That's the Agent-as-a-Service idea. The user hires an agent, not a toolkit. If you're building with real-time voice APIs, I'd love to hear what you're running into. And if you want to see a production example, you can check out [RingBot](https://ringbot.ai).