{"slug": "workload-driven-optimization-for-on-device-real-time-subtitle-translation", "title": "Workload-Driven Optimization for On-Device Real-Time Subtitle Translation", "summary": "Researchers optimized on-device English-to-Traditional-Chinese subtitle translation for Taiwan under low-latency and privacy constraints. By replacing the original 151k-token vocabulary with a 64k-token subtitle-domain tokenizer and fine-tuning, their LocalSubs model achieved a 59.2% win rate against Google Translate on a test subset and a 1.63× speedup on Apple M2 Metal.", "body_md": "arXiv:2607.09957v1 Announce Type: new\nAbstract: This report studies on-device English-to-Traditional-Chinese subtitle translation for Taiwan under short inputs, short outputs, batch-size-one inference, low latency, and privacy constraints. These conditions limit the value of optimizations designed for long-context or high-throughput language-model serving.\nStarting from LMT-60-0.6B, preliminary profiling suggests that vocabulary projection becomes a more important decode-time cost after GGUF quantization reduces the relative cost of Transformer blocks. We replace the original 151k-token vocabulary with a 64k-token subtitle-domain tokenizer, migrate the embedding space, and adapt the model through embedding calibration followed by full supervised fine-tuning.\nOn a fixed 500-example subset of the OpenSubtitles2024 test set, the LocalSubs achieves a 59.2% tie-excluded win rate against Google Translate under GPT-4o pairwise judging. Performance is strongest on short cues and declines as cue length increases. Preliminary Apple M2 Metal measurements on a 64k-vocabulary model show a 1.63$\\times$ speedup over a 151k-vocabulary profiling baseline. The raw benchmark configuration is incomplete, so the latency result is treated as preliminary.", "url": "https://wpnews.pro/news/workload-driven-optimization-for-on-device-real-time-subtitle-translation", "canonical_source": "https://arxiv.org/abs/2607.09957", "published_at": "2026-07-14 04:00:00+00:00", "updated_at": "2026-07-14 04:33:42.950847+00:00", "lang": "en", "topics": ["machine-learning", "natural-language-processing", "ai-products"], "entities": ["LocalSubs", "Google Translate", "OpenSubtitles2024", "GPT-4o", "Apple M2 Metal", "LMT-60-0.6B"], "alternates": {"html": "https://wpnews.pro/news/workload-driven-optimization-for-on-device-real-time-subtitle-translation", "markdown": "https://wpnews.pro/news/workload-driven-optimization-for-on-device-real-time-subtitle-translation.md", "text": "https://wpnews.pro/news/workload-driven-optimization-for-on-device-real-time-subtitle-translation.txt", "jsonld": "https://wpnews.pro/news/workload-driven-optimization-for-on-device-real-time-subtitle-translation.jsonld"}}