Wiring Android's WorkManager to a Quantized On-Device LLM for Background Summarization

A developer has published a pattern for wiring Android's WorkManager to a quantized on-device LLM, using llama.cpp via JNI, to run chunked document summarization in the background without OOM kills or Doze-mode deferrals. The approach uses parallel chunk Workers feeding a serial reduce Worker, with model tier selection driven by device memory ceilings — sub-1B INT4 or sub-1.5B INT4 for background work, and foreground service promotion for 3B models. The writeup notes that 7B models should be moved server-side rather than tuned to fit a 6 GB device.

--- title: "Wiring WorkManager to On-Device LLMs for Background Summarization" published: true description: "Schedule quantized LLM inference in Android WorkManager, handle Doze-mode constraints, promote foreground services, and choose the right model tier for mid-range devices." tags: android, kotlin, architecture, mobile canonical url: https://blog.mvpfactory.co/wiring-workmanager-on-device-llm-background-summarization --- Let me show you a pattern I use when on-device AI needs to run reliably in the background. We are wiring Android's WorkManager to a quantized LLM — specifically llama.cpp via JNI — to perform chunked document summarization without OOM kills, Doze-mode deferrals, or angry users staring at a frozen UI. By the end of this tutorial you will have a chained Worker architecture that selects the right model tier, respects memory ceilings, and promotes to a foreground service only when the model demands it. 2.9+ on the classpath CoroutineWorker Here is the gotcha that will save you hours: the memory ceiling on a mid-range device determines your entire architecture. A Snapdragon 6 Gen 1 with 6 GB RAM leaves your app process roughly 1.8–2.2 GB before the OOM killer becomes aggressive. | Model Size | INT8 RAM | INT4 RAM | Safe on 6 GB device? | |---|---|---|---| | 1B | ~1.0 GB | ~0.6 GB | Both tiers | | 1.5B Phi-2 class | ~1.5 GB | ~0.9 GB | Both with headroom | | 3B | ~3.0 GB | ~1.7 GB | INT4 only | | 7B | ~7.0 GB | ~4.0 GB | Neither — move server-side | For background Workers without foreground promotion, target sub-1B INT4 or sub-1.5B INT4. If you find yourself rationalizing a 7B model on a 6 GB device, that is a signal to move inference server-side, not to keep tuning constraints. The docs do not mention this, but setRequiresBatteryNotLow is non-negotiable for inference workloads — LLM inference drains battery fast enough to trigger system-level throttling mid-run. val inferenceConstraints = Constraints.Builder .setRequiresBatteryNotLow true .setRequiredNetworkType NetworkType.NOT REQUIRED .build val summarizeRequest = OneTimeWorkRequestBuilder<SummarizationWorker .setConstraints inferenceConstraints .setExpedited OutOfQuotaPolicy.RUN AS NON EXPEDITED WORK REQUEST .setInputData workDataOf "chunk index" to 0, "total chunks" to 3 .build setExpedited is critical for user-triggered summarization. Without it, Doze-mode deferral can push your work by hours . Expedited tasks require a getForegroundInfo override — WorkManager calls it on older API levels to attach a notification. Here is the minimal setup to get chunked summarization working. Most teams try to load the entire document in one Worker and either blow the memory budget or hit the 10-minute execution window. The correct pattern is parallel chunk Workers feeding a serial reduce Worker. php val chunkWorkers = 0 until totalChunks .map { index - OneTimeWorkRequestBuilder<ChunkSummarizeWorker .setInputData workDataOf "chunk" to index .build } val reduceRequest = OneTimeWorkRequestBuilder<ReduceSummaryWorker .build WorkManager.getInstance context .beginWith chunkWorkers // parallel fan-out .then reduceRequest // serial reduce .enqueue Each ChunkSummarizeWorker loads the model, runs inference on a ~500-token window, unloads, and writes its partial summary to the output Data map. Model load/unload per chunk costs ~200–400 ms for INT4 1B models on a Snapdragon 6 Gen 1 — expensive, but it keeps peak RSS below the OOM threshold. If your product requires a 3B model, you must promote the Worker to a foreground service. On CoroutineWorker : override suspend fun getForegroundInfo : ForegroundInfo { val notification = buildSummarizationNotification return ForegroundInfo NOTIFICATION ID, notification, ServiceInfo.FOREGROUND SERVICE TYPE SHORT SERVICE } FOREGROUND SERVICE TYPE SHORT SERVICE API 34+ gives you up to 3 minutes of guaranteed execution without a declared use-case permission — the practical sweet spot for 3B INT4 inference on a chunked document. Silent OOM kills. The OOM killer does not throw an exception — your Worker just disappears. Profile peak RSS on your minimum-spec device under load before committing to a model size. Forgetting getForegroundInfo with expedited tasks. WorkManager will crash on older API levels if you mark a task expedited without implementing this override. Loading the whole document in one Worker. Any document over ~1,500 tokens should be chunked. One Worker, one window, bounded memory. Treating 7B as an on-device target. At ~4 GB INT4 footprint, 7B exceeds available RAM on most 6 GB devices even with foreground promotion. Move it server-side. Three decisions determine whether this architecture ships or collapses in production. Profile peak RSS and select your model tier first — everything else follows from that number. Use chained Workers for documents over ~1,500 tokens to stay within memory and execution time bounds. Always set setExpedited for user-initiated work to avoid Doze-mode deferrals measured in hours. Get those three right and background LLM inference becomes a reliable product feature rather than a source of silent failures. Further reading: WorkManager guides https://developer.android.com/topic/libraries/architecture/workmanager · llama.android https://github.com/shubham0204/llama.android · MediaPipe LLM Inference API https://ai.google.dev/edge/mediapipe/solutions/genai/llm inference/android