The Hard Part of On-Device AI Isn’t Inference

Alder, a lightweight Mac app for on-device AI, faces its hardest engineering challenge not in inference but in delivering and managing multi-gigabyte speech and language models, according to a company blog post. The app must handle interrupted downloads, verify file integrity via size and checksum, self-heal corrupted files, and ensure model updates never break a working setup, while gating heavier models by hardware capability to match each Mac's memory budget. This reliability-focused approach is what the company says 'on-device AI' must mean in production.

The app is small. The models are not. Alder itself is a lightweight Mac app. The on-device speech and language models it downloads are hundreds of megabytes to gigabytes each. That inversion changes what “shipping software” means: the binary is the easy part, and model delivery becomes a first-class product surface with its own failure modes. Downloads Resume after interruption instead of restarting Integrity Size and checksum verified before a model is trusted Activation A model goes live only after it loads and runs Self-heal Corrupted files are detected and re-fetched Real networks fail mid-download A multi-gigabyte download will be interrupted — by sleep, by a network switch, by closing the laptop lid. Alder resumes interrupted model downloads instead of starting over, and verifies the exact size and checksum of every model file before it is ever loaded. Files on disk also rot in the real world: when a stored model fails verification later, Alder detects the corruption and repairs it rather than crashing into it. When a model file is broken, users don’t blame their disk or their network. They blame the app. So the app has to own the whole delivery path. An update must never break a working setup The scariest moment in a local-AI product is replacing a model that works. A newer model that still produces output can quietly produce worse output — so “it runs” is not the bar. Alder treats model versions conservatively: a new model must download completely, verify, load, and pass a warm-up run before it takes over, and an interrupted update can never leave the app without a working engine. Every Mac has a different budget An 8 GB MacBook Air and a 64 GB Mac Studio are different machines, and the whole toolchain — speech engine, optional cleanup model, and the apps you are actually working in — shares one memory budget. Alder gates heavier models by hardware capability so a Mac is never offered a configuration it cannot run reliably, and shows requirements before a download starts, not after it fails. The result is a boring kind of reliability: models arrive verified, updates cannot strand you, and the heaviest path your Mac is offered is one it can actually sustain. That is what “on-device AI” has to mean in production — not a demo that runs once, but a system that keeps running. How we choose which models to ship at all is covered in Why Alder runs multiple speech models https://alder.dzgapp.com/blog/why-alder-runs-multiple-speech-models/ .