{"slug": "a-frontend-backend-architecture-for-tool-calls-in-full-duplex-speech-models", "title": "A frontend-backend architecture for tool calls in full-duplex speech models", "summary": "A frontend-backend architecture that lets full-duplex speech-to-speech models delegate tool calls to a text-based backend LLM achieved 92-97% tool-call recall and 81.2% accuracy in rejecting irrelevant calls in a single-turn evaluation, according to an arXiv paper (2609.19334v1). The system has a duplex speech-to-text frontend emit a delegation token and forward streaming ASR transcripts to the backend, whose tool-call results are injected back into the frontend via a lightweight prefill-and-repeat mechanism and synthesized with streaming TTS. Equipped with a larger backend such as Qwen3-235B-A22B, the system matched open and closed source models on Full-Duplex-Bench-V3 and significantly outperformed GPT-realtime-mini and Qwen3-Omni-30B-A3B-Instruct on EVA-Bench.", "body_md": "arXiv:2609.19334v1 Announce Type: new \nAbstract: Full-duplex speech-to-speech (S2S) models provide natural, low-latency conversational interaction and would benefit from the ability to use external tools and complete voice-agent tasks. We propose a frontend-backend architecture where a duplex speech-to-text frontend learns to emit a delegation token and forwards streaming ASR transcripts to a text-based backend LLM for tool calls. Tool-call results from the backend are injected back into the frontend through a lightweight prefill-and-repeat mechanism and then synthesized using streaming TTS to the user. Our approach largely preserves regular duplex turn-taking, interruption handling, and low-latency interaction as it requires minimal modifications to the frontend model. In a single-turn tool-call evaluation, our system achieves 92-97% tool-call recall, competitive tool-call prediction performance, and 81.2% accuracy in rejecting irrelevant calls. When equipped with a larger backend (e.g., Qwen3-235B-A22B), our system achieves competitive results on Full-Duplex-Bench-V3 compared to open and closed source models, and significantly outperforms GPT-realtime-mini and Qwen3-Omni-30B-A3B-Instruct on EVA-Bench. These results demonstrate that backend delegation is an effective and modular approach for combining natural duplex speech interaction with strong agentic tool-call capabilities.", "url": "https://wpnews.pro/news/a-frontend-backend-architecture-for-tool-calls-in-full-duplex-speech-models", "canonical_source": "https://arxiv.org/abs/2609.19334", "published_at": "2026-09-18 04:00:00+00:00", "updated_at": "2026-09-18 04:26:18.875151+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "natural-language-processing", "ai-research"], "entities": ["Qwen3-235B-A22B", "Full-Duplex-Bench-V3", "GPT-realtime-mini", "Qwen3-Omni-30B-A3B-Instruct", "EVA-Bench", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/a-frontend-backend-architecture-for-tool-calls-in-full-duplex-speech-models", "markdown": "https://wpnews.pro/news/a-frontend-backend-architecture-for-tool-calls-in-full-duplex-speech-models.md", "text": "https://wpnews.pro/news/a-frontend-backend-architecture-for-tool-calls-in-full-duplex-speech-models.txt", "jsonld": "https://wpnews.pro/news/a-frontend-backend-architecture-for-tool-calls-in-full-duplex-speech-models.jsonld"}}