{"slug": "wiring-android-s-workmanager-to-a-quantized-on-device-llm-for-background", "title": "Wiring Android's WorkManager to a Quantized On-Device LLM for Background Summarization", "summary": "A developer has published a pattern for wiring Android's WorkManager to a quantized on-device LLM, using llama.cpp via JNI, to run chunked document summarization in the background without OOM kills or Doze-mode deferrals. The approach uses parallel chunk Workers feeding a serial reduce Worker, with model tier selection driven by device memory ceilings — sub-1B INT4 or sub-1.5B INT4 for background work, and foreground service promotion for 3B models. The writeup notes that 7B models should be moved server-side rather than tuned to fit a 6 GB device.", "body_md": "\n\n```\n---\ntitle: \"Wiring WorkManager to On-Device LLMs for Background Summarization\"\npublished: true\ndescription: \"Schedule quantized LLM inference in Android WorkManager, handle Doze-mode constraints, promote foreground services, and choose the right model tier for mid-range devices.\"\ntags: android, kotlin, architecture, mobile\ncanonical_url: https://blog.mvpfactory.co/wiring-workmanager-on-device-llm-background-summarization\n---\n```\n\nLet me show you a pattern I use when on-device AI needs to run reliably in the background. We are wiring Android's WorkManager to a quantized LLM — specifically llama.cpp via JNI — to perform chunked document summarization without OOM kills, Doze-mode deferrals, or angry users staring at a frozen UI.\n\nBy the end of this tutorial you will have a chained Worker architecture that selects the right model tier, respects memory ceilings, and promotes to a foreground service only when the model demands it.\n\n`2.9+` on the classpath`CoroutineWorker`\nHere is the gotcha that will save you hours: the memory ceiling on a mid-range device determines your entire architecture. A Snapdragon 6 Gen 1 with 6 GB RAM leaves your app process roughly **1.8–2.2 GB** before the OOM killer becomes aggressive.\n\n| Model Size | INT8 RAM | INT4 RAM | Safe on 6 GB device? | \n|---|---|---|---|\n| 1B | ~1.0 GB | ~0.6 GB | Both tiers | \n| 1.5B (Phi-2 class) | ~1.5 GB | ~0.9 GB | Both with headroom | \n| 3B | ~3.0 GB | ~1.7 GB | INT4 only | \n| 7B | ~7.0 GB | ~4.0 GB | Neither — move server-side | \n\nFor background Workers **without** foreground promotion, target sub-1B INT4 or sub-1.5B INT4. If you find yourself rationalizing a 7B model on a 6 GB device, that is a signal to move inference server-side, not to keep tuning constraints.\n\nThe docs do not mention this, but `setRequiresBatteryNotLow` is non-negotiable for inference workloads — LLM inference drains battery fast enough to trigger system-level throttling mid-run.\n\n```\nval inferenceConstraints = Constraints.Builder()\n    .setRequiresBatteryNotLow(true)\n    .setRequiredNetworkType(NetworkType.NOT_REQUIRED)\n    .build()\n\nval summarizeRequest = OneTimeWorkRequestBuilder<SummarizationWorker>()\n    .setConstraints(inferenceConstraints)\n    .setExpedited(OutOfQuotaPolicy.RUN_AS_NON_EXPEDITED_WORK_REQUEST)\n    .setInputData(workDataOf(\"chunk_index\" to 0, \"total_chunks\" to 3))\n    .build()\n```\n\n`setExpedited` is critical for user-triggered summarization. Without it, Doze-mode deferral can push your work by **hours**. Expedited tasks require a `getForegroundInfo()` override — WorkManager calls it on older API levels to attach a notification.\n\nHere is the minimal setup to get chunked summarization working. Most teams try to load the entire document in one Worker and either blow the memory budget or hit the 10-minute execution window. The correct pattern is parallel chunk Workers feeding a serial reduce Worker.\n\n``` php\nval chunkWorkers = (0 until totalChunks).map { index ->\n    OneTimeWorkRequestBuilder<ChunkSummarizeWorker>()\n        .setInputData(workDataOf(\"chunk\" to index))\n        .build()\n}\n\nval reduceRequest = OneTimeWorkRequestBuilder<ReduceSummaryWorker>().build()\n\nWorkManager.getInstance(context)\n    .beginWith(chunkWorkers)   // parallel fan-out\n    .then(reduceRequest)       // serial reduce\n    .enqueue()\n```\n\nEach `ChunkSummarizeWorker` loads the model, runs inference on a ~500-token window, unloads, and writes its partial summary to the output `Data` map. Model load/unload per chunk costs ~200–400 ms for INT4 1B models on a Snapdragon 6 Gen 1 — expensive, but it keeps peak RSS below the OOM threshold.\n\nIf your product requires a 3B model, you must promote the Worker to a foreground service. On `CoroutineWorker`:\n\n```\noverride suspend fun getForegroundInfo(): ForegroundInfo {\n    val notification = buildSummarizationNotification()\n    return ForegroundInfo(\n        NOTIFICATION_ID,\n        notification,\n        ServiceInfo.FOREGROUND_SERVICE_TYPE_SHORT_SERVICE\n    )\n}\n```\n\n`FOREGROUND_SERVICE_TYPE_SHORT_SERVICE` (API 34+) gives you up to **3 minutes** of guaranteed execution without a declared use-case permission — the practical sweet spot for 3B INT4 inference on a chunked document.\n\n**Silent OOM kills.** The OOM killer does not throw an exception — your Worker just disappears. Profile peak RSS on your minimum-spec device under load before committing to a model size.\n\n**Forgetting `getForegroundInfo()` with expedited tasks.** WorkManager will crash on older API levels if you mark a task expedited without implementing this override.\n\n**Loading the whole document in one Worker.** Any document over ~1,500 tokens should be chunked. One Worker, one window, bounded memory.\n\n**Treating 7B as an on-device target.** At ~4 GB INT4 footprint, 7B exceeds available RAM on most 6 GB devices even with foreground promotion. Move it server-side.\n\nThree decisions determine whether this architecture ships or collapses in production. Profile peak RSS and select your model tier first — everything else follows from that number. Use chained Workers for documents over ~1,500 tokens to stay within memory and execution time bounds. Always set `setExpedited` for user-initiated work to avoid Doze-mode deferrals measured in hours.\n\nGet those three right and background LLM inference becomes a reliable product feature rather than a source of silent failures.\n\n**Further reading:** [WorkManager guides](https://developer.android.com/topic/libraries/architecture/workmanager) · [llama.android](https://github.com/shubham0204/llama.android) · [MediaPipe LLM Inference API](https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android)", "url": "https://wpnews.pro/news/wiring-android-s-workmanager-to-a-quantized-on-device-llm-for-background", "canonical_source": "https://dev.to/software_mvp-factory/wiring-androids-workmanager-to-a-quantized-on-device-llm-for-background-summarization-cd4", "published_at": "2026-09-10 14:06:16+00:00", "updated_at": "2026-09-10 14:44:58.553321+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "developer-tools", "ai-infrastructure"], "entities": ["Android", "WorkManager", "llama.cpp", "Kotlin", "Snapdragon 6 Gen 1", "Phi-2"], "alternates": {"html": "https://wpnews.pro/news/wiring-android-s-workmanager-to-a-quantized-on-device-llm-for-background", "markdown": "https://wpnews.pro/news/wiring-android-s-workmanager-to-a-quantized-on-device-llm-for-background.md", "text": "https://wpnews.pro/news/wiring-android-s-workmanager-to-a-quantized-on-device-llm-for-background.txt", "jsonld": "https://wpnews.pro/news/wiring-android-s-workmanager-to-a-quantized-on-device-llm-for-background.jsonld"}}