{"slug": "build-a-multi-keyword-offline-voice-controller-with-onnx-and-tflite", "title": "Build a Multi-Keyword Offline Voice Controller with ONNX and TFLite", "summary": "A developer has published a guide and open-source examples in the onnx-wakeword repository for building a multi-keyword offline voice controller that runs small fixed-vocabulary command models via ONNX Runtime or TFLite/TFLite Micro instead of a cloud ASR pipeline. The approach pairs an always-on wake word with a short command window, maps a fixed keyword array to model output indices, and routes recognized labels to application callbacks with cooldowns and safeguards against destructive actions. The writeup stresses that runtime preprocessing—sample rate, framing, log-Mel settings and normalization—must match training and export configuration or recall degrades.", "body_md": "Many voice interfaces do not need to transcribe arbitrary speech. A lamp, robot, appliance, kiosk, or media controller may only need a small vocabulary such as:\n\n```\nturn on the light\nturn off the light\nnext\nprevious\nstop\n```\n\nFor this kind of product, a multi-keyword model can be a simpler fit than a full cloud ASR pipeline. The model answers a constrained question: which known command, if any, is present in the current audio window?\n\nThis article shows the engineering shape of that system, including model metadata, local inference, command callbacks, and the tests that matter before connecting voice input to a real action.\n\nKeyword spotting (KWS) detects a known phrase or a small set of known phrases. Automatic speech recognition (ASR) produces a transcript for open-ended speech.\n\n| Product requirement | Suitable first tool | \n|---|---|\n| Start an interaction with one phrase | Single-keyword KWS | \n| Recognize 2–10 fixed commands | Multi-keyword KWS | \n| Understand arbitrary sentences | ASR plus intent parsing | \n| Verify who is speaking | Speaker verification | \n\nA typical two-stage voice controller looks like this:\n\n```\nalways-on wake word\n        ↓\nshort command window\n        ↓\nmulti-keyword model\n        ↓\ncommand label\n        ↓\nsafe application action\n```\n\nThe wake word and the command vocabulary are separate concerns. Keeping the command list fixed makes the application behavior easier to test and keeps the system local.\n\nA multi-keyword model exposes a fixed output order. For example:\n\n```\n{\n  \"model_type\": \"multi_keyword\",\n  \"keywords\": [\"turn on the light\", \"turn off the light\", \"next\", \"stop\"],\n  \"model_file\": \"commands.onnx\",\n  \"mel_time\": 98,\n  \"n_mels\": 32,\n  \"cons_frames\": 2\n}\n```\n\nThe `keywords` array is part of the interface. Its order must stay aligned with the model output:\n\n```\noutput[0] → turn on the light\noutput[1] → turn off the light\noutput[2] → next\noutput[3] → stop\n```\n\nThe application should map a recognized label to an action instead of embedding business logic inside the model.\n\nThe exact API differs between Python, Android, Web, and embedded runtimes, but the application boundary can remain simple:\n\n``` python\ndef on_keyword(word, probability, info):\n    if word == \"turn on the light\":\n        lights.set_power(True)\n    elif word == \"turn off the light\":\n        lights.set_power(False)\n    elif word == \"next\":\n        player.next()\n    elif word == \"stop\":\n        player.stop()\n```\n\nIn a real product, add a cooldown and make repeated events safe. Voice detection should not directly bypass authorization or confirmation for destructive actions such as deleting data, unlocking a door, or purchasing something.\n\nThe model file is only one part of a working detector:\n\n```\nmicrophone\n   ↓\nmono PCM audio\n   ↓\nsample-rate conversion and framing\n   ↓\nlog-Mel feature extraction\n   ↓\nONNX or TFLite inference\n   ↓\nthreshold and consecutive-frame logic\n   ↓\ncommand callback\n```\n\nThe runtime preprocessing must match the training and export configuration. A model can appear correct in one demo and lose recall if the target application changes the sample rate, frame shape, Mel settings, or input normalization.\n\nThe open-source [`onnx-wakeword`](https://github.com/voicute/onnx-wakeword) repository contains the local inference examples. The same general model concept can be delivered as ONNX for desktop, Web, Android, and Python integrations, or as TFLite/TFLite Micro for constrained embedded targets.\n\nUse the runtime that matches the target platform:\n\n| Target | Typical format or runtime | Important boundary | \n|---|---|---|\n| Web | ONNX Runtime Web | Browser microphone permission and compatibility still matter | \n| Android | ONNX Runtime in the open demo | The current demo is not an Android TFLite SDK | \n| Python / desktop | ONNX Runtime | Audio device and operating-system setup must be tested | \n| ESP32-S3 | INT8 TFLite Micro | The public hardware example is tied to its tested audio board | \n\n“The model is portable” does not mean that every audio device or board behaves identically. The final microphone, enclosure, operating system, and power budget need their own test.\n\nTesting only one successful utterance per command is not enough. For each command, collect:\n\nMeasure at least:\n\n```\nrecall = detected intended utterances / total intended utterances\nfalse accepts per hour = unwanted activations / hours of negative audio\n```\n\nAlso check confusion between commands. A model can have good recall for `next` while incorrectly confusing `next` with `stop` in the same environment.\n\nIf the baseline model triggers on a recurring sound, keep the false-trigger clips as hard negatives for a later training round:\n\n```\nbase model\n   ↓\nscan representative negative audio\n   ↓\ncollect false-trigger clips\n   ↓\nretrain with hard negatives\n   ↓\nrepeat held-out evaluation\n```\n\nDo not reuse every clip as training data. Keep a separate held-out set so that the second evaluation still measures generalization.\n\nThe practical workflow for a fixed-command product is:\n\nVoicute supports custom keywords in Chinese, English, Spanish, Japanese, German, French, and Hindi. The online test is a screening step; it is not a production guarantee for every microphone or enclosure.\n\nFor a product with a small, known command vocabulary, one multi-keyword model can be cleaner than one detector per command or a full ASR service. The model recognizes the vocabulary; the application owns the action policy; the final device test decides whether the result is ready to ship.\n\nThe open inference examples are available in [`voicute/onnx-wakeword`](https://github.com/voicute/onnx-wakeword). You can generate and test your own fixed command vocabulary at [Voicute](https://www.voicute.com/).\n\nDisclosure: I am the creator of Voicute. The model-generation service is commercial, while the local inference examples are open source. Results depend on the language, speakers, microphone, environment, and runtime configuration.", "url": "https://wpnews.pro/news/build-a-multi-keyword-offline-voice-controller-with-onnx-and-tflite", "canonical_source": "https://dev.to/buguatec/build-a-multi-keyword-offline-voice-controller-with-onnx-and-tflite-5cmc", "published_at": "2026-10-26 09:40:12+00:00", "updated_at": "2026-10-26 09:43:39.198422+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "machine-learning", "ai-products"], "entities": ["onnx-wakeword", "ONNX Runtime", "ONNX Runtime Web", "TFLite", "TFLite Micro", "ESP32-S3"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/build-a-multi-keyword-offline-voice-controller-with-onnx-and-tflite", "markdown": "https://wpnews.pro/news/build-a-multi-keyword-offline-voice-controller-with-onnx-and-tflite.md", "text": "https://wpnews.pro/news/build-a-multi-keyword-offline-voice-controller-with-onnx-and-tflite.txt", "jsonld": "https://wpnews.pro/news/build-a-multi-keyword-offline-voice-controller-with-onnx-and-tflite.jsonld"}}