# Wiring Android's MediaPipe LLM Inference API to a Streaming UI

> Source: <https://dev.to/software_mvp-factory/wiring-androids-mediapipe-llm-inference-api-to-a-streaming-ui-50eh>
> Published: 2026-10-28 15:07:18+00:00



```
---
title: "Streaming LLM Tokens in Android with Jetpack Compose: Back-Pressure and the Frame Budget"
published: true
description: "Wire MediaPipe's LLM Inference API to Jetpack Compose using Kotlin Flow back-pressure, correct buffer sizing, and a 16ms recomposition budget that keeps mid-range devices jank-free at 20+ tokens/sec."
tags: kotlin, android, mobile, architecture
canonical_url: https://mvpfactory.co/blog/streaming-llm-tokens-android-compose
---
```

By the end of this tutorial you will have a production-ready pattern for streaming on-device LLM token output into a Compose `Text` component — without dropped tokens, without jank on mid-range hardware, and without the recomposition storm that trips up most first implementations.

Here is the gotcha that will save you hours: **MediaPipe does not give you a suspend function. It gives you a callback on a thread you do not own.** The naive bridge will work fine on your Pixel 9 and silently drop frames on your users' Helio G88.

MediaPipe exposes token output like this:

```
inferenceSession.addQueryChunk(prompt)
inferenceSession.generateAsync(
    { partialResult, done ->
        onToken(partialResult, done)
    }
)
```

Two constraints that matter:

Ignore these two constraints and you get frame drops that only appear under thermal throttling on mid-range hardware. That is exactly the device your users own.

The cleanest bridge is a `Channel` with a bounded buffer. Here is the minimal setup to get this working:

```
fun LlmInferenceSession.tokenFlow(prompt: String): Flow<String> =
    channelFlow {
        val buffer = Channel<String>(capacity = 64, onBufferOverflow = BufferOverflow.SUSPEND)

        addQueryChunk(prompt)
        generateAsync { partial, done ->
            runBlocking { buffer.send(partial) }
            if (done) buffer.close()
        }

        for (token in buffer) send(token)
    }.flowOn(Dispatchers.Default)
```

A capacity of 64 covers roughly 2–3 seconds of burst at 25 tok/sec before `SUSPEND` kicks in. `runBlocking` inside the callback is intentional — it blocks the producer when the consumer falls behind, without dropping tokens. The docs do not mention this, but this is the only safe primitive here.

Let me show you a pattern I use in every project. Compose does not batch state writes from `collectAsState` the same way it batches layout passes. Each emission can schedule an independent recomposition.

| Device tier | Vsync budget | Max safe recompositions/sec | Safe token batch window | 
|---|---|---|---|
| Flagship (Snapdragon 8 Gen 3) | 8.3ms (120Hz) | ~60 | 16ms | 
| Mid-range (Snapdragon 6 Gen 1) | 16ms (60Hz) | ~30 | 33ms | 
| Budget (Helio G88) | 16ms (60Hz) | ~15 | 66ms | 

At 25 tokens/sec on a budget device, one recomposition per token misses frames. The fix is chunked emission with a time window:

```
fun Flow<String>.chunkByTime(windowMs: Long = 80L): Flow<String> =
    flow {
        val accumulator = StringBuilder()
        var lastEmit = System.currentTimeMillis()

        collect { token ->
            accumulator.append(token)
            val now = System.currentTimeMillis()
            if (now - lastEmit >= windowMs) {
                emit(accumulator.toString())
                accumulator.clear()
                lastEmit = now
            }
        }
        if (accumulator.isNotEmpty()) emit(accumulator.toString())
    }
```

An 80ms window caps recompositions at ~12/sec — safely below the budget-device ceiling.

```
@Composable
fun StreamingResponse(session: LlmInferenceSession, prompt: String) {
    val text by remember(prompt) {
        session.tokenFlow(prompt)
            .chunkByTime(windowMs = 80L)
            .runningReduce { acc, chunk -> acc + chunk }
    }.collectAsState(initial = "")

    Text(text = text, modifier = Modifier.fillMaxWidth())
}
```

`runningReduce` accumulates the full response string without holding a mutable `StringBuilder` in composition scope. `remember(prompt)` restarts the flow cleanly on new prompts, avoiding stale emissions.

**Do not use `MutableStateFlow` to bridge the MediaPipe callback.** Only a `Channel` with `BufferOverflow.SUSPEND` prevents dropped tokens from a native producer thread.

**Tune to your minimum supported device, not your desk Pixel.** 80ms is a safe default. Tighten to 33ms only if your floor is mid-range 60Hz and you have profiler data to back it up.

**Before shipping, verify in Android Studio's Compose Layout Inspector:** skipped frames should be zero during active streaming; recomposition count on `Text` should be ≤15/sec on your lowest-supported device.

The full pattern is three composable pieces: a `Channel`-based callback bridge, a time-windowed chunking operator, and a stateless composable that folds chunks with `runningReduce`. Each piece solves one specific constraint — thread ownership, back-pressure, and frame budget — and they stay decoupled enough to test independently.

Profile on your lowest-supported hardware before shipping. Flagship benchmarks will lie to you.