A step-by-step guide to wiring Qwen Audio Agent's voice interface to Claude Code via ACP, so you can talk while it codes in the background.
What is Qwen Audio Agent and how does it work with Claude Code? #
Qwen Audio Agent is an open source application that connects a spoken conversation to agents capable of doing real work on your computer. It doesn’t code by itself. Instead, it acts as a voice front end that routes coding requests to a separate back end, in this case Claude Code, through an adapter for the agent client protocol (ACP). A local gateway sits between the two, sending your spoken instructions to Claude and bringing the results back into the conversation so you can keep talking while the actual coding task runs in the background.
TL;DR #
- Qwen Audio Agent pairs a voice front end with Claude Code as the coding back end, connected through a local gateway and an ACP adapter.
- The two systems serve different jobs: the voice model handles conversation and quick questions, while Claude Code does the actual file edits and project work.
- Interrupting the spoken reply and canceling a coding task are separate actions , and confusing the two can leave you thinking work stopped when it didn’t.
- Setup requires a supported Node version, a DashScope API key for the voice connection, and a working Claude Code installation checked independently before adding voice on top.
- The workspace folder has to be set explicitly in the configuration file, because opening a terminal in a project folder does not automatically make it the agent’s workspace.
- Voice is useful for explaining requirements, asking follow-up questions, and reporting bugs out loud, but exact file paths, code, and error messages are still best handled as text.
- Running this setup involves two separate sources of usage cost : the DashScope voice connection and your existing Claude Code account or API usage.
Other agents ship a demo. Remy ships an app. #
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
How do you install and configure the voice front end? #
The command line installation starts with a supported Node.js version (the demonstrated setup used the Node 24 series with npm 10 or newer). You can confirm your versions with node --version and npm --version. A desktop installer exists too, bundling its own runtime, but the command line path paired with the browser client is what this setup focuses on.
Installation itself uses a global npm install of the qn-audio-agent package at a specific version. After installing, running the configuration command opens an environment file, typically stored under a hidden config directory in your home folder. Inside that file you set:
- The DashScope API key , created in the provider’s console, used strictly for the voice connection.
- The real-time provider , set to DashScope.
- The voice model , referenced in the config as the Qwen audio real-time model used for conversation.
The coding back end is configured separately. Claude is not bundled into this voice model choice; it’s wired in afterward as its own component.
How do you connect Claude Code as the back end? #
Once the voice side is configured, an install command checks for the Claude components and installs whatever pieces are missing, including the ACP adapter. If Claude Code is already installed on the machine, the setup can reuse that existing installation rather than duplicating it.
Before trusting the voice layer with anything real, open Claude Code directly in a terminal and confirm sign-in or API access works on its own. Sending a small test request through plain Claude Code first rules out authentication or quota problems before you start troubleshooting through a voice interface, where it’s harder to tell whether a failure is a voice issue or a Claude issue.
After confirming Claude works independently, a setup command tied to the Claude back end checks that the integration pieces are present. This step confirms the plumbing, not your account’s usage quota. That distinction matters: a successful setup check doesn’t guarantee Claude will actually accept and run a task when you ask it to through voice.
In the configuration file, the agent protocol field needs to be set to Claude explicitly. Leaving the back end model field empty lets Claude use its own existing model configuration rather than overriding it. A permission mode field controls how the back end handles permission prompts. Setting it to native keeps Claude’s normal permission behavior intact, meaning you’ll still get prompted for decisions during a session rather than having everything silently approved.
Why does the workspace folder matter so much? #
One detail that’s easy to get wrong: opening a terminal inside a project folder does not automatically make that folder the agent’s workspace. Qwen Audio Agent has its own default workspace setting, and a Claude-specific workspace field can override the general one. The practical fix is to create a dedicated project folder, set the workspace field in the config to that folder’s full absolute path, and verify the actual configured path before sending any real task.
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
This is worth double-checking early. A voice agent confidently building an issue tracker in the wrong directory is a frustrating way to lose a demo, and it’s a purely configuration-level mistake rather than anything related to the AI itself.
How do you run and test the setup? #
With configuration done, starting the gateway process and keeping its terminal open is the first step. A second terminal runs the web UI command, which opens a local browser interface (by default on a specific local port). From there you allow microphone access, confirm the gateway and voice connection both show as connected, and ideally use headphones for initial testing so the microphone doesn’t pick up the assistant’s own voice output.
A reasonable first test is a short, low-stakes request: ask it to confirm it can hear you and respond briefly. Check the transcript against what you actually said, since any transcription error will carry forward into whatever gets sent to Claude later. A second check worth doing before any real task: ask the agent to inspect the configured project folder, report its absolute path, and list files without changing anything. Comparing that reported path against your intended folder confirms the workspace setting took effect.
What does a real voice coding session actually look like? #
A practical test case is building something small and easy to visually verify, like a basic issue tracker: a list view, a form for adding items, a completed state, filtering, and data persisted in browser local storage. Keeping the project small matters because it makes it much easier to confirm that a spoken correction actually reached the right part of the codebase.
A few behaviors are worth understanding before relying on this for real work:
Conversation continues while coding runs. You can ask an unrelated or clarifying question (for example, what local storage does and whether refreshing the page should wipe stored data) while Claude is still working on the build. The voice model answers from its own understanding; it isn’t reporting on Claude’s internal progress.
Interrupting speech doesn’t cancel a task. If you interrupt the spoken explanation to ask for something shorter, that stops the audio response. It does not cancel or whatever Claude is already doing in the back end. These are two separate systems reacting to two separate kinds of input.
Follow-up instructions get queued, not instantly applied. Asking for a requirement change (switching a theme, renaming a button) while a build is in progress may become a subsequent task rather than an instant edit mid-turn. The acknowledgement you hear is not proof the file has changed yet. Checking the actual task state and the eventual diff is the only reliable confirmation.
Canceling a task is its own explicit action, distinct from muting the microphone or closing the conversation window, both of which let background work continue uninterrupted. A read-only task (like requesting an accessibility review without edits) is a safer way to test cancellation, since there’s no risk of leaving partial edits behind if the cancellation doesn’t land before the task finishes.
Is this setup worth it for everyday coding work? #
It depends on what you want voice for. Describing a requirement aloud, asking why a particular design choice matters, or verbally reporting a bug you’re looking at are all reasonable uses of a voice interface layered over a coding agent. But exact file paths, code snippets, and error messages are still better handled as text: pasting an error or pointing to a file is faster and more reliable than narrating it character by character.
Reviewing results also still requires opening the actual app and inspecting the diff. A spoken “it’s done” is not verification. The core discipline doesn’t change just because the interface got a voice: check the behavior, check the data persistence, check that a fix didn’t break something that worked before.
There are also two separate sources of usage cost to track: the voice provider (DashScope in this setup) bills for the conversational front end, and Claude Code uses whatever account or API billing you already have configured for coding work. Extra follow-up tasks add to the Claude side independently of the voice side. The project being open source doesn’t make either of those hosted model calls free, so checking current pricing for both before running extended sessions is worth doing.
Frequently Asked Questions #
What is ACP in the context of Claude Code?
ACP stands for agent client protocol, the integration layer that lets an external front end like Qwen Audio Agent send tasks to Claude Code and receive results back. An adapter handles this connection so you don’t need to build the integration yourself.
Does interrupting the voice response stop Claude’s coding task?
No. Interrupting playback only stops the spoken audio. A task Claude has already accepted continues running in the background until it finishes or is explicitly canceled through the task controls.
Can I use a local voice model instead of DashScope?
Local voice front end options exist as alternatives, but they come with their own model and hardware requirements. Using a local voice front end also doesn’t make the Claude back end run locally; Claude Code still relies on its own configured account or API connection.
Do I need a separate Claude Code account to use this?
Yes. The voice layer doesn’t replace or bundle Claude Code access. You need Claude Code working and signed in independently before layering the voice front end on top, and your usual Claude usage or billing applies separately from the voice provider’s costs.
What happens if I close the voice conversation while Claude is working?
Closing the conversation or muting the microphone doesn’t cancel an in-progress task. Background coding work can keep running. To actually stop a task, you need to use the dedicated cancel control rather than relying on ending the conversation.