Rebuilding my local voice assistant pipeline Platypush developer Fabio Manganiello replaced the assistant.openwakeword plugin with a new assistant.microwakeword plugin built on microWakeWord via the pymicro-wakeword Python bindings, after months of false positives and false negatives from openWakeWord's v0.1 models from 2023. The new plugin ships the okay_nabu, hey_jarvis, hey_mycroft and alexa models in-package, keeps the same API (models, detection_threshold, frame_duration, patience_frames, input_device, input_volume, list_models and HotwordDetectedEvent), and defaults patience_frames to 0 because the models already smooth output over a sliding window. Switching requires only pointing the configuration at assistant.microwakeword, and custom models can be added by dropping a microWakeWord JSON manifest and .tflite file into the plugin's models directory. Rebuilding my voice assistant pipeline https://blog.platypush.tech/article/Rewriting-my-voice-assistant-pipeline https://blog.platypush.tech/article/Rewriting-my-voice-assistant-pipeline Swapping openwakeword for microwakeword and making voice detection smarter I wrote an article https://blog.platypush.tech/article/Local-voice-assistant a couple of months ago describing how I rebuilt my old voice assistant pipeline in Platypush https://platypush.tech . And I wrote earlier articles describing my journey through self-hosted voice assistants - in 2024 https://blog.platypush.tech/article/The-state-of-voice-assistant-integrations-in-2024 with Picovoice https://picovoice.ai , in 2020 https://blog.platypush.tech/article/Build-custom-voice-assistants diving into the legacy assistant.google https://docs.platypush.tech/platypush/plugins/assistant.google.html integration, the now defunct assistant.alexa integration, the now defunct Snowboy integration, the now defunct DeepSpeech integration and Picovoice and the original article in 2019 https://blog.platypush.tech/article/Build-your-customizable-voice-assistant-with-Platypush describing the Google Assistant integration. For a couple of months I used the voice pipeline described in my latest article as my main driver, but after getting used to the robustness of commercial voice assistants I was unhappy with the high rates of false positives/negatives of my self-hosted pipeline, as well as its lack of robustness in detecting start/end of speech, especially in noisy environments and using cheap microphones. After a bit of experimentation, I decided to swap and fine tune a few parts, and after a couple of days of trying the results are satisfying enough for me to write a short article about it. The resulting pipeline is: Wake word detection: a new assistant.microwakeword plugin Until recently I used assistant.openwakeword https://docs.platypush.tech/platypush/plugins/assistant.openwakeword.html , which wraps openWakeWord https://github.com/dscripka/openWakeWord . It works, but the project has been quiet for a while and all of its pre-trained models are v0.1 files from 2023, mostly trained on synthetic speech. In practice I was getting both false positives the hotword triggering on background noise and false negatives the hotword not detected unless I was close to the microphone . I added a new plugin, assistant.microwakeword https://docs.platypush.tech/platypush/plugins/assistant.microwakeword.html , on top of microWakeWord https://github.com/kahrendt/microWakeWord through the pymicro-wakeword https://pypi.org/project/pymicro-wakeword/ Python bindings. microWakeWord is the engine used by ESPHome and Home Assistant Voice, its models are actively maintained, and the built-in models okay nabu , hey jarvis , hey mycroft , alexa ship with the package — no download required. The current official models are provided under the ESPHome micro-wake-word-models repository https://github.com/esphome/micro-wake-word-models . Custom models are supported too: drop a microWakeWord JSON manifest with its .tflite model in the plugin's models directory and reference it by name. The plugin exposes the same API as assistant.openwakeword same models , detection threshold , frame duration , patience frames , input device , input volume parameters, same list models action, same HotwordDetectedEvent , so switching is a matter of just changing the configuration to point to the new plugin: assistant.microwakeword: models: - alexa A few design notes: - Without an explicit detection threshold , each model uses the probability cutoff from its own manifest 0.9 for alexa , which is the value the model was validated against. - Models already smooth their output over a sliding window, so unlike openwakeword, patience frames defaults to 0 here. - The audio thread never touches the TFLite models directly: it queues frames and a reset marker to the main thread, because MicroWakeWord.reset reloads the interpreter and isn't safe to call concurrently. - Like assistant.openwakeword , the plugin releases the audio device when another assistant plugin starts a conversation, and re-acquires it when the conversation ends. Speech-to-text: assistant.openai , with a much better ear assistant.openai records audio when a conversation starts and transcribes it through Whisper. The tricky part has always been end-of-speech detection : knowing when the user has finished talking. The previous implementation used a per-frame boolean from WebRTC VAD, and any single frame classified as speech would reset the silence timer. With even a bit of background noise, phrases would only end when the maximum conversation duration was hit. The audio preprocessing layer AudioPreprocessor , shared by the STT integrations has been reworked: - Silero VAD https://github.com/snakers4/silero-vad as the primary backend through onnxruntime https://onnxruntime.ai , no PyTorch https://pytorch.org required , with WebRTC VAD and a plain RMS-based detector as fallbacks. To get an idea of Silero VAD's performance compared against other solutions for speech activity detection, take a look at the insightful article One voice detector to rule them all https://thegradient.pub/one-voice-detector-to-rule-them-all/ . The Silero model is downloaded on first use into the Platypush workdir and checksum-verified: php assistant.openai: auto = silero - webrtc - energy, depending on what's installed vad backend: auto - Hysteresis + sliding window : a frame becomes speech above the VAD threshold and stays speech until its score drops by a margin, and the per-frame decisions are smoothed by majority vote over a 0.5s window. A single noise burst can no longer keep a conversation alive. - Adaptive noise floor : the background noise level is tracked over time, and frames that aren't louder than noise floor factor default 2.0, ~6 dB times the floor are never considered as speech. On a synthetic benchmark Piper-generated phrases mixed with babble and burst noise , the old settings kept conversations open for 6–10 seconds after the speech had ended; the new pipeline ends them 1.0–1.6 seconds after the last word, which is the configured conversation end timeout https://docs.platypush.tech/platypush/plugins/assistant.openai.html platypush.plugins.assistant.openai.AssistantOpenaiPlugin.start conversation plus the smoothing lag. Since the changes on Platypush have been performed directly in the AudioPreprocessor https://git.platypush.tech/platypush/platypush/src/branch/master/platypush/plugins/assistant/ audio.py also used by other native speech-to-text engines, other speech-to-text plugins also benefit from them - e.g. if you use assistant.vosk https://docs.platypush.tech/platypush/plugins/assistant.vosk.html for full on-device detection. Responses and context: the openai plugin The transcribed phrase is processed by the openai plugin, which handles both the API key and the conversation context. By default the assistant renders the response through the configured tts plugin ; if the response ends with a question mark, the conversation continues with a follow-up. Custom hooks can intercept phrases before the default response kicks in — for example, to handle home automation commands locally without an LLM round-trip: python from platypush import run, when from platypush.message.event.assistant import SpeechRecognizedEvent @when SpeechRecognizedEvent, phrase='play the ? music' def play music : run 'music.mpd.play' And the hotword event is wired to the conversation start with a similar hook: python from platypush import run, when from platypush.message.event.assistant import HotwordDetectedEvent @when HotwordDetectedEvent def on hotword : run 'assistant.openai.start conversation' Text-to-speech: tts.openai Responses are rendered by tts.openai https://docs.platypush.tech/platypush/plugins/tts.openai.html , with volume and voice configurable. In my tests however tts.piper https://docs.platypush.tech/platypush/plugins/tts.piper.html also yielded good results, and it runs entirely locally. In my tests on a RaspberryPi 5 however it added ~3-4s of latency to process the response to audio compared to tts.openai . Since the TTS output goes to the speakers and the wake word detector listens on the microphone, the assistant plugins pause hotword detection while a conversation is active — otherwise the assistant would hear itself. The full configuration openai: api key: YOUR API KEY Or, for a custom setup using a model provider compatible with the OpenAI API, e.g. a local Ollama instance: base url: http://localhost:8000/v1 model: gpt-5 Optional context that can be passed to initialize the assistant context: role: system content: You are a useful assistant. assistant.microwakeword: models: - alexa assistant.openai: tts plugin: tts.openai conversation start sound: ~/sounds/wake word detected.wav input volume: 100 assistant.vosk: model: /path/to/vosk/model conversation start sound: ~/sounds/wake word detected.wav input volume: 100 tts.openai: output volume: 80.0 Or, for a fully local setup that uses tts.piper: tts.piper: output volume: 80.0 What changed on the code side - New assistant.microwakeword https://docs.platypush.tech/platypush/plugins/assistant.microwakeword.html plugin : microWakeWord hotword detection through pymicro-wakeword , same API as assistant.openwakeword , built-in models bundled with the package, custom models via JSON manifests. - assistant.openwakeword https://docs.platypush.tech/platypush/plugins/assistant.openwakeword.html fixes : frames are now 80ms 1280 samples, the size openWakeWord recommends instead of 500ms — longer chunks only return the highest score over their sub-frames and can miss hotwords — plus a patience frames parameter consecutive frames above threshold and an explicit detection threshold parameter default 0.5 . - Shared AudioPreprocessor https://git.platypush.tech/platypush/platypush/src/branch/master/platypush/plugins/assistant/ audio.py rework : Silero/WebRTC/energy VAD backends with vad backend: auto selection, threshold hysteresis, sliding-window majority vote, and an adaptive noise floor. Used by both assistant.openai https://docs.platypush.tech/platypush/plugins/assistant.openai.html and assistant.vosk https://docs.platypush.tech/platypush/plugins/assistant.vosk.html . - New Silero VAD wrapper https://git.platypush.tech/platypush/platypush/src/branch/master/platypush/plugins/assistant/ vad.py : ONNX inference through onnxruntime https://onnxruntime.ai , recurrent state carried across frames, checksum-verified model download into the workdir. Caveats The microphone on my RPi doesn't support 16 kHz natively, so audio is captured at 44.1 kHz and software-resampled — it works, but if detection feels sluggish software resampling is usually the first suspect. The VAD and threshold defaults were tuned on synthetic audio plus a couple of my audio recordings. Different rooms, microphones and acoustics may yield different results. You may want to adjust detection threshold on the wake word plugin and noise floor factor on the STT plugins if you don't feel satisfied with your mileage. Reactions How to interact with this page Webmentions To interact via Webmentions https://indieweb.org/Webmention , send an activity that references this URL https://blog.platypush.tech/article/Rewriting-my-voice-assistant-pipeline from a platform that supports Webmentions, such as Lemmy https://joinlemmy.org/ , WordPress with Webmention plugins https://wordpress.org/plugins/webmention/ , or any IndieWeb-compatible site https://indieweb.org/Webmention Publishing Software . ActivityPub - Follow @blog@platypush.tech https://blog.platypush.tech/@blog on your ActivityPub platform e.g. Mastodon, Misskey, Pleroma, Lemmy . - Mention @blog@platypush.tech https://blog.platypush.tech/@blog in a post to feature on the Guestbook https://blog.platypush.tech/guestbook . - Search for this URL https://blog.platypush.tech/article/Rewriting-my-voice-assistant-pipeline on your instance to find and interact with the post. - Like, boost, quote, or reply to the post to feature your activity here. It's 2026. You deserve a good local-first or local-only voice assistant on your RaspberryPi https://manganiello.tech/tags/raspberrypi blog.platypush.tech/article/Rewriting-my-voice-assistant-pipeline https://blog.platypush.tech/article/Rewriting-my-voice-assistant-pipeline