{"slug": "barge-in-for-eight-ai-voices-what-it-took-to-let-people-interrupt-a-live-ai-call", "title": "Barge-in for eight AI voices: what it took to let people interrupt a live AI group call", "summary": "A developer building AI Group Call, a mobile app that places two to eight AI voices on a single live voice call, described the engineering work behind reliable barge-in for multi-agent conversations. The approach runs a custom energy-based VAD on raw phone PCM instead of relying on lagging transcription events, layers echo protection after Android devices fed agent audio back into the mic, and uses a hedged, latency-benchmarked \"director\" model to classify whether an interruption was a real point or just a \"yeah.", "body_md": "I build [AI Group Call](https://aigroupcall.app/?utm_source=devto&utm_medium=article&utm_campaign=barge-in), a mobile app where you put two to eight AI voices on one live voice call and talk an idea through with them. A host, a skeptic, a couple of specialists, or models you pick (Claude, GPT, Gemini, Grok) as seats at the table.\n\nThe hardest part was not getting eight voices to talk. It was letting a human interrupt them.\n\nIn a one-on-one voice assistant, barge-in is mostly solved for you. In a room of eight, every mistake is louder: an agent that hears its own voice will answer itself, a missed \"wait, stop\" means three more people talk over you, and a cough can derail the whole conversation. Here is what ended up working.\n\nEach call has:\n\nA server-side \"conductor\" owns the floor. Only one voice speaks at a time, and the conductor decides when that changes.\n\nMy first version used transcription events to detect that you had started speaking. The problem: transcription deltas lag speech by seconds, and the transcriber I use has no server-side voice activity detection. Interruptions felt dead. You would say \"hang on\" and the agent would finish its paragraph.\n\nThe fix was to run my own energy-based VAD on the raw PCM coming from the phone:\n\nHolding first and confirming second is the key. Pausing an agent for half a second by mistake costs almost nothing. Talking over a human costs a lot.\n\nThe first time I tested on Android, the agents started answering each other in a loop. I was playing their audio on the media stream, so the hardware echo canceller had no reference signal and cancelled nothing. The mic picked up the agents and fed them straight back in.\n\nRunning the whole call in communication mode, with speakerphone forced, fixed most of it (plus a small software gain, because the voice stream is quieter). But \"most\" is not enough when eight voices are involved, so echo protection is now layered, and each layer is tested on its own:\n\nEvery barge decision logs a single line with its RMS, how long it was voiced, and who held the floor. When something regresses, it shows up in the logs instead of in a one-star review.\n\nOnce interruptions worked, a new problem appeared. People say \"yeah\" and \"mm-hmm\" while listening. With instant barge-in, every \"yeah\" killed the speaking agent, and the room would then respond to \"yeah\" as if it were a point.\n\nSo an interruption now gets a verdict. The speaking agent still stops instantly, but the transcript that arrives a second later decides what it was:\n\nBefore this, \"please stop\" earned you another round of opinions. That is funny once.\n\nThe director runs on every turn, so its latency is dead air. I benchmarked candidates on real transcripts: several models got every pick right, but one did it at about 0.5 s median while another took nearly 2 s and blew the time budget on about a quarter of turns. Accuracy was a tie; speed decided it.\n\nEven the fast one has bad days, so the director call is **hedged**, not retried: after 1.2 seconds of silence, a fallback model starts in parallel and the first valid answer wins.\n\nOn a call, you can cut in at any moment, the room goes quiet, and whoever is best placed answers you. The agents build on each other instead of taking turns reading out separate answers, which is the whole point: it is closer to [an LLM council you can talk to](https://aigroupcall.app/llm-council-voice/?utm_source=devto&utm_medium=article&utm_campaign=barge-in) than to asking three chatbots the same question.\n\nIf you are building anything multi-voice, my short list would be:\n\nIf you want to hear it, AI Group Call is on iOS and Android with three free minutes: [aigroupcall.app](https://aigroupcall.app/?utm_source=devto&utm_medium=article&utm_campaign=barge-in). Happy to answer questions about any of this in the comments.", "url": "https://wpnews.pro/news/barge-in-for-eight-ai-voices-what-it-took-to-let-people-interrupt-a-live-ai-call", "canonical_source": "https://dev.to/tashthemaker/barge-in-for-eight-ai-voices-what-it-took-to-let-people-interrupt-a-live-ai-group-call-4ll", "published_at": "2026-09-27 16:34:37+00:00", "updated_at": "2026-09-27 17:01:02.481572+00:00", "lang": "en", "topics": ["ai-agents", "ai-products", "natural-language-processing", "ai-tools"], "entities": ["AI Group Call", "Claude", "GPT", "Gemini", "Grok", "Android", "iOS"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/barge-in-for-eight-ai-voices-what-it-took-to-let-people-interrupt-a-live-ai-call", "markdown": "https://wpnews.pro/news/barge-in-for-eight-ai-voices-what-it-took-to-let-people-interrupt-a-live-ai-call.md", "text": "https://wpnews.pro/news/barge-in-for-eight-ai-voices-what-it-took-to-let-people-interrupt-a-live-ai-call.txt", "jsonld": "https://wpnews.pro/news/barge-in-for-eight-ai-voices-what-it-took-to-let-people-interrupt-a-live-ai-call.jsonld"}}