# Build a Multi-Keyword Offline Voice Controller with ONNX and TFLite > Source: > Published: 2026-10-26 09:40:12+00:00 Many voice interfaces do not need to transcribe arbitrary speech. A lamp, robot, appliance, kiosk, or media controller may only need a small vocabulary such as: ``` turn on the light turn off the light next previous stop ``` For this kind of product, a multi-keyword model can be a simpler fit than a full cloud ASR pipeline. The model answers a constrained question: which known command, if any, is present in the current audio window? This article shows the engineering shape of that system, including model metadata, local inference, command callbacks, and the tests that matter before connecting voice input to a real action. Keyword spotting (KWS) detects a known phrase or a small set of known phrases. Automatic speech recognition (ASR) produces a transcript for open-ended speech. | Product requirement | Suitable first tool | |---|---| | Start an interaction with one phrase | Single-keyword KWS | | Recognize 2–10 fixed commands | Multi-keyword KWS | | Understand arbitrary sentences | ASR plus intent parsing | | Verify who is speaking | Speaker verification | A typical two-stage voice controller looks like this: ``` always-on wake word ↓ short command window ↓ multi-keyword model ↓ command label ↓ safe application action ``` The wake word and the command vocabulary are separate concerns. Keeping the command list fixed makes the application behavior easier to test and keeps the system local. A multi-keyword model exposes a fixed output order. For example: ``` { "model_type": "multi_keyword", "keywords": ["turn on the light", "turn off the light", "next", "stop"], "model_file": "commands.onnx", "mel_time": 98, "n_mels": 32, "cons_frames": 2 } ``` The `keywords` array is part of the interface. Its order must stay aligned with the model output: ``` output[0] → turn on the light output[1] → turn off the light output[2] → next output[3] → stop ``` The application should map a recognized label to an action instead of embedding business logic inside the model. The exact API differs between Python, Android, Web, and embedded runtimes, but the application boundary can remain simple: ``` python def on_keyword(word, probability, info): if word == "turn on the light": lights.set_power(True) elif word == "turn off the light": lights.set_power(False) elif word == "next": player.next() elif word == "stop": player.stop() ``` In a real product, add a cooldown and make repeated events safe. Voice detection should not directly bypass authorization or confirmation for destructive actions such as deleting data, unlocking a door, or purchasing something. The model file is only one part of a working detector: ``` microphone ↓ mono PCM audio ↓ sample-rate conversion and framing ↓ log-Mel feature extraction ↓ ONNX or TFLite inference ↓ threshold and consecutive-frame logic ↓ command callback ``` The runtime preprocessing must match the training and export configuration. A model can appear correct in one demo and lose recall if the target application changes the sample rate, frame shape, Mel settings, or input normalization. The open-source [`onnx-wakeword`](https://github.com/voicute/onnx-wakeword) repository contains the local inference examples. The same general model concept can be delivered as ONNX for desktop, Web, Android, and Python integrations, or as TFLite/TFLite Micro for constrained embedded targets. Use the runtime that matches the target platform: | Target | Typical format or runtime | Important boundary | |---|---|---| | Web | ONNX Runtime Web | Browser microphone permission and compatibility still matter | | Android | ONNX Runtime in the open demo | The current demo is not an Android TFLite SDK | | Python / desktop | ONNX Runtime | Audio device and operating-system setup must be tested | | ESP32-S3 | INT8 TFLite Micro | The public hardware example is tied to its tested audio board | “The model is portable” does not mean that every audio device or board behaves identically. The final microphone, enclosure, operating system, and power budget need their own test. Testing only one successful utterance per command is not enough. For each command, collect: Measure at least: ``` recall = detected intended utterances / total intended utterances false accepts per hour = unwanted activations / hours of negative audio ``` Also check confusion between commands. A model can have good recall for `next` while incorrectly confusing `next` with `stop` in the same environment. If the baseline model triggers on a recurring sound, keep the false-trigger clips as hard negatives for a later training round: ``` base model ↓ scan representative negative audio ↓ collect false-trigger clips ↓ retrain with hard negatives ↓ repeat held-out evaluation ``` Do not reuse every clip as training data. Keep a separate held-out set so that the second evaluation still measures generalization. The practical workflow for a fixed-command product is: Voicute supports custom keywords in Chinese, English, Spanish, Japanese, German, French, and Hindi. The online test is a screening step; it is not a production guarantee for every microphone or enclosure. For a product with a small, known command vocabulary, one multi-keyword model can be cleaner than one detector per command or a full ASR service. The model recognizes the vocabulary; the application owns the action policy; the final device test decides whether the result is ready to ship. The open inference examples are available in [`voicute/onnx-wakeword`](https://github.com/voicute/onnx-wakeword). You can generate and test your own fixed command vocabulary at [Voicute](https://www.voicute.com/). Disclosure: I am the creator of Voicute. The model-generation service is commercial, while the local inference examples are open source. Results depend on the language, speakers, microphone, environment, and runtime configuration.