{"slug": "reachy-mini-goes-fully-local", "title": "Reachy Mini goes fully local", "summary": "Pollen Robotics and Hugging Face have released a fully local speech-to-speech pipeline for the Reachy Mini robot, eliminating the need for cloud servers or API keys. The open-source stack runs entirely on the user's machine using a cascade of Silero VAD, Parakeet-TDT STT, Gemma 4 via llama.cpp, and Qwen3-TTS, ensuring audio data never leaves the local network. This update enables private, cost-free voice conversations with the robot while allowing users to swap any component of the pipeline as new models become available.", "body_md": "# Reachy Mini goes fully local\n\n[Update on GitHub](https://github.com/huggingface/blog/blob/main/local-reachy-mini-conversation.md)\n\n[conversation app](https://github.com/pollen-robotics/reachy_mini_conversation_app)and start talking to it. Until now, you had to send your audio to a server. But not anymore. Today we'll walk you through running the whole stack locally.\n\nThis stack is powered by [ speech-to-speech](https://github.com/huggingface/speech-to-speech), our cascaded VAD → STT → LLM → TTS pipeline that exposes a Realtime API-compatible\n\n`/v1/realtime`\n\nWebSocket. Once you launch the backend, point the robot at it from the UI.Cascades are the most flexible option in the open-source landscape today, and with the right pieces they're also the fastest. We'll recommend the components we like best, but the whole point of a cascade is that you can swap them. New models drop every week.\n\nTL;DR\n\n- Deploy a local speech backend for your Reachy Mini.\n- We use our\n`speech-to-speech`\n\nlibrary, a cascade approach.- Recommended:\nllama.cppwithGemma 4,Silero VAD,Parakeet-TDT STT,Qwen3-TTS.\n\n## Quick start\n\nThis blog walks you through running conversations with Reachy Mini fully locally. No cloud, no API keys, no data leaving your machine. Here's a video showing this live:\n\n### Locally serving the LLM\n\nTo serve the LLM, we'll use Hugging Face's `llama.cpp`\n\n. If you need to install it, the simplest way is `brew install llama.cpp`\n\nor `winget install llama.cpp`\n\n, for more help, [check the docs](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md).\nFirst, we'll run:\n\n```\nllama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full\n```\n\nAnd done! The first time it will download the model, subsequent launches are fast.\n\n## What do those flags do?\n\n`-hf ggml-org/gemma-4-E4B-it-GGUF`\n\n— pulls the model straight from the Hub. First run downloads it, subsequent runs use the cache.`-np 2`\n\n— two parallel slots. Lets the server handle a second request (e.g. a quick interruption) without blocking on the first.`-c 65536`\n\n— 64k context window, shared across slots. Plenty of headroom for long conversations.`-fa on`\n\n— flash attention. Faster and lower memory, basically free on modern hardware.`--swa-full`\n\n— keeps the full sliding-window attention cache instead of recomputing it. Trades a bit of RAM for noticeably faster prompt processing on Gemma.\n\n### Setting up speech-to-speech\n\nWe'll begin by simply installing the library\n\n```\nuv pip install speech-to-speech\n```\n\nThen, while we are serving the LLM in another terminal, we can simply run:\n\n```\nspeech-to-speech --responses_api_base_url \"http://127.0.0.1:8080\" --responses_api_api_key \"\" --mode local\n```\n\nAnd you can start talking to the model through your terminal! The first time it will need to download Parakeet and Qwen3TTS, but subsequent launches are fast.\n\nHere's a video showing the local conversation mode:\n\nNow, after you've tried it in `--mode local`\n\n, you can run again the command without that option to serve speech-to-speech to the robot.\n\n### Connecting Reachy Mini to speech-to-speech\n\nOnce you have llama.cpp and speech-to-speech running, you can start the robot with the desktop app and launch the conversation app. In the UI from the conversation app, you need to choose the local mode by clicking on \"edit connection\" in the HF backend. Here's a video showing how to do it:\n\nAnd you're done. You can start talking to your robot. Every stage of the pipeline is a trade-off: there are faster TTS models with lower quality, slower STT models with higher quality. We optimized for multilingual, you might want to optimize for a single language. The rest of the blog covers how to customize.\n\n## Going deeper\n\n### Why run your own Speech-to-Speech server?\n\nHosted realtime backends are convenient, but running your own engine unlocks three things:\n\n**Privacy.** Audio never leaves your network, the entire pipeline runs on hardware you control.**No API costs.** No per-minute or per-token fees.**Full control over the pipeline.** Swap any piece: VAD, STT, LLM, TTS. Whenever something better lands on the Hub 🤗.\n\nThe `speech-to-speech`\n\nrepo gives you all of that in a single CLI. It boots a WebSocket server at `/v1/realtime`\n\nthat speaks the same protocol Reachy Mini already knows how to talk to.\n\n### Our opinionated defaults: VAD, STT, TTS\n\nA cascaded voice pipeline has four stages: VAD, STT, LLM, and TTS. For three of them, we pick solid defaults so you can focus on the LLM:\n\n| Stage | Choice | Why |\n|---|---|---|\n| VAD | Silero VAD v5 |\nTiny, accurate, runs on CPU. The de-facto default in the open-source voice-agent world. |\n| STT | Parakeet-TDT |\nStreaming-friendly, very fast, great quality on English. |\n| TTS | Qwen3-TTS |\nExpressive, low-latency, multilingual, supports custom voices. |\n\nWe are opinionated about these choices, feel free to swap them out for your own if you have a preference.\n\n### Choosing your LLM\n\nThe LLM is the layer with the most impact on latency and overall performance of the system. We support two options: **run a model locally** (llama.cpp, MLX, Transformers, vLLM), or **use a server with a Responses API** (OpenAI, Gemini, HF Inference Endpoints, llama.cpp, vLLM, etc).\n\n#### The Responses API: decouple the brain from the voice loop\n\nThe main bottleneck in the system is LLM inference latency. To address that, we support external inference engines exposed through the Responses API protocol.\n\nThe `speech-to-speech`\n\nengine therefore supports a second mode where the LLM lives in a separate process as long as it speaks the Responses API protocol. You launch your model server in one terminal, you launch the voice loop in another terminal, and the two talk over HTTP.\n\n##### Option 1: llama.cpp in one terminal, speech-to-speech in the other\n\n**Terminal 1: llama.cpp server:**\n\n```\nllama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full\n```\n\n**Terminal 2: speech-to-speech client:**\n\n```\nspeech-to-speech \\\n  --mode realtime \\\n  --stt parakeet-tdt \\\n  --tts qwen3 \\\n  --llm_backend responses-api \\\n  --model_name \"unsloth/Qwen3-4B-Instruct-2507-GGUF\" \\\n  --responses_api_base_url \"http://127.0.0.1:8080/v1\"\n```\n\n##### Option 2: vLLM in one terminal, speech-to-speech in the other\n\nRequires vLLM ≥ 0.21.0.Full support for the Responses API protocol, including tool-call streaming used by the speech-to-speech backend, landed in vLLM 0.21.0. Older versions will boot but trip up as soon as the assistant tries to call a tool.\n\nWhen serving a model through vLLM for this pipeline, three flags are effectively required:\n\n`--enable-auto-tool-choice`\n\n`--tool-call-parser <tool_parser_name>`\n\n— picks the per-family parser that turns the model's raw output into structured tool calls (e.g.`qwen3_coder`\n\nfor Qwen3 instruct models,`llama3_json`\n\nfor Llama 3,`hermes`\n\nfor Hermes-style models, ...).`--default-chat-template-kwargs '{\"enable_thinking\":false}'`\n\n: disables the`<think>`\n\nreasoning channel for models that support it. For harder agentic tasks you can flip this to`true`\n\nand let the model reason, but for a natural-feeling conversation we strongly recommend keeping it off: every thinking token is latency the user hears as silence before the robot starts speaking.\n\n**Terminal 1: vLLM inference server ( Qwen/Qwen3-4B-Instruct-2507):**\n\n```\nvllm serve Qwen/Qwen3-4B-Instruct-2507 \\\n  --port 8000 \\\n  --host 127.0.0.1 \\\n  --max-model-len 32768 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder \\\n  --default-chat-template-kwargs '{\"enable_thinking\":false}' \\\n  --speculative-config '{\"method\":\"qwen3_next_mtp\",\"num_speculative_tokens\":1}'\n```\n\nThe\n\n`--speculative-config`\n\nline enables Multi-Token Prediction (MTP). It isoptional, but it has a great impact on end-to-end latency. Leave it on whenever the model supports it.\n\n**Terminal 2: speech-to-speech client:**\n\n```\nspeech-to-speech \\\n  --mode realtime \\\n  --stt parakeet-tdt \\\n  --tts qwen3 \\\n  --llm_backend responses-api \\\n  --model_name \"Qwen/Qwen3-4B-Instruct-2507\" \\\n  --responses_api_base_url \"http://127.0.0.1:8000/v1\"\n```\n\n##### Option 3: Hugging Face Inference Endpoints\n\nSame protocol, but the model runs on a managed GPU on Hugging Face. Deploy any chat model as an Inference Endpoint, then point the voice loop at the endpoint URL:\n\n```\nspeech-to-speech \\\n  --mode realtime \\\n  --stt parakeet-tdt \\\n  --tts qwen3 \\\n  --llm_backend responses-api \\\n  --model_name \"Qwen/Qwen3-4B-Instruct-2507\" \\\n  --responses_api_base_url \"https://<your-endpoint>.endpoints.huggingface.cloud/v1\" \\\n  --responses_api_api_key \"$HF_TOKEN\"\n```\n\n##### Option 4: Hugging Face Inference Providers\n\nIf you don't want to manage your own endpoint, use an [Inference Provider](https://huggingface.co/docs/inference-providers). Hugging Face routes your request to a third-party backend (e.g. Together, Fireworks, Replicate) with a single URL:\n\n```\nspeech-to-speech \\\n  --mode realtime \\\n  --stt parakeet-tdt \\\n  --tts qwen3 \\\n  --llm_backend responses-api \\\n  --model_name \"Qwen/Qwen3.6-35B-A3B:deepinfra\" \\\n  --responses_api_base_url \"https://router.huggingface.co/v1\" \\\n  --responses_api_api_key \"$HF_TOKEN\"\n```\n\n##### Option 5: OpenAI (or any OpenAI-compatible provider)\n\nWhen you want to test against a frontier model with zero infra, point the same flag at OpenAI:\n\n```\nspeech-to-speech \\\n  --mode realtime \\\n  --stt parakeet-tdt \\\n  --tts qwen3 \\\n  --llm_backend responses-api \\\n  --model_name \"gpt-5.4\" \\\n  --responses_api_api_key \"$OPENAI_API_KEY\"\n```\n\nThe `--responses_api_*`\n\nflags work the same for any provider that implements the protocol (OpenRouter, Together, Fireworks, …). Swap the base URL and the API key, keep the rest of the pipeline identical.\n\n#### Running the LLM in-process\n\n##### Option 1: Local LLM on MLX (Apple Silicon)\n\nIf you are on a Mac, MLX is the lowest-friction way to run a real model with sane latency. We recommend **Qwen3-4B-Instruct-2507**, which is small enough to feel instant on M-series chips and capable enough to hold a conversation.\n\n```\nspeech-to-speech \\\n  --llm_backend mlx-lm \\\n  --model_name \"mlx-community/Qwen3-4B-Instruct-2507-bf16\"\n```\n\nThe server listens on `ws://127.0.0.1:8765/v1/realtime`\n\nby default. Leave it running, connect the conversation app to the local backend, and you are talking to your robot.\n\n##### Option 2: Local LLM on Transformers (CUDA / CPU / MPS)\n\nSame idea, but using vanilla `transformers`\n\n. Use this if you are on a CUDA box, on Linux, or if you want to swap models freely without re-converting weights for MLX.\n\n```\nspeech-to-speech \\\n  --llm_backend transformers \\\n  --model_name \"Qwen/Qwen3-4B-Instruct-2507\"\n```\n\nTip.`Qwen3-4B-Instruct-2507`\n\nis another good option for LLM because it gives a good speed/quality balance on a single consumer GPU. You can point`--model_name`\n\nat any HF model the backend supports — for example a larger Gemma, Qwen, or a Mistral.\n\n### Running the engine on your laptop, the app on the robot\n\nIf you are running the voice engine on your laptop and the conversation app on a Reachy Mini Wireless, the only thing that changes is the URL. Make sure the engine binds to a LAN address (not just `127.0.0.1`\n\n) and use the laptop's IP from the robot when you select the IP in the UI.\n\nIf you don't know your IP, here's how to find it:\n\n## macOS\n\n```\nipconfig getifaddr en0    # wifi\nipconfig getifaddr en1    # ethernet (sometimes en0, varies)\n```\n\n## Linux\n\n```\nhostname -I\n```\n\n## Windows\n\n```\nipconfig\n```\n\nLook for \"IPv4 Address\" under your active adapter.\n\nYou want the `192.168.x.x`\n\nor `10.x.x.x`\n\none. If you see `169.254.x.x`\n\n, you're not actually on the network.\n\n## Wrap up\n\nYou now have a fully local voice loop:\n\n- A robot listening with\n**Silero**, - transcribing with\n**Parakeet-TDT**, - thinking with whichever LLM you picked, whether that's local MLX, local Transformers, a vLLM or llama.cpp server next door, or a hosted Responses API endpoint,\n- and answering with\n**Qwen3-TTS**.\n\nStar [ huggingface/speech-to-speech](https://github.com/huggingface/speech-to-speech) and\n\n[, and come tell us in the discussions which open-source cascade you ended up running on your robot.](https://github.com/pollen-robotics/reachy_mini_conversation_app)\n\n`pollen-robotics/reachy_mini_conversation_app`", "url": "https://wpnews.pro/news/reachy-mini-goes-fully-local", "canonical_source": "https://huggingface.co/blog/local-reachy-mini-conversation", "published_at": "2026-05-27 00:00:00+00:00", "updated_at": "2026-05-27 15:39:59.530481+00:00", "lang": "en", "topics": ["robotics", "artificial-intelligence", "large-language-models", "ai-tools", "ai-infrastructure"], "entities": ["Reachy Mini", "Hugging Face", "Pollen Robotics", "Gemma 4", "Silero VAD", "Parakeet-TDT STT", "Qwen3-TTS", "llama.cpp"], "alternates": {"html": "https://wpnews.pro/news/reachy-mini-goes-fully-local", "markdown": "https://wpnews.pro/news/reachy-mini-goes-fully-local.md", "text": "https://wpnews.pro/news/reachy-mini-goes-fully-local.txt", "jsonld": "https://wpnews.pro/news/reachy-mini-goes-fully-local.jsonld"}}