Mobile AI Factory: Building Specialized Language Models for Your Mobile Device

A research paper posted to arXiv (2611.13509v1) introduces Mobile AI Factory, an agentic system in which an orchestrator LLM builds a fine-tuned, quantized small language model for a specified task and target phone by choosing between hyperparameter tuning, real data mining, and synthetic example generation, with fixed rules rolling back changes that hurt accuracy. Evaluated on ten tasks and four Android phones, the on-device models beat two five-shot frontier models on five tasks by up to 0.34 and matched them on three more, and the search came within 0.03 of exhaustively training all 27 model variants on an entity extraction task while using 19x less GPU time. The authors report that the best configuration changes across phones, and that without phone measurements Claude Opus 5.5, given full hardware and model specifications, missed the app's requirements in 25% of easy and moderate cases and 60% of hard ones; code is available at github.com/evanly-gh/SLM_Factory.

arXiv:2611.13509v1 Announce Type: new Abstract: Small language models SLMs running on mobile devices can keep user data private and work without connectivity. But they perform well only when fine-tuned for the target task and sized for the target device, a process that today requires repeated expert tuning. We present {\it Mobile AI Factory}, an agentic system that builds a fine-tuned, quantized SLM for a given task and phone. Given a task specification and a target phone, an orchestrator LLM chooses whether to tune hyperparameters, mine real data, or generate synthetic examples, while fixed rules roll back changes that hurt accuracy. The factory searches from small to large models across base models and quantizations, adapts its accuracy goal as it searches, and uses real measurements of the target phone across inference engines and backends. We evaluate Mobile AI Factory on ten tasks and four Android phones. On an unseen testset, its on-device models outperform two five-shot frontier models on five tasks, by up to 0.34, and match them on three more. Its search comes within 0.03 of exhaustively training all 27 model variants on an entity extraction task, while using 19$\times$ less GPU time. Phone measurements also matter: the best configuration changes across phones. Without phone measurements, Claude Opus 5.5, given the full hardware and model specifications, misses the app's requirements in 25\% of easy and moderate cases and in 60\% of hard ones. Code is available at: https://github.com/evanly-gh/SLM Factory.