Last week I was testing an AI chat feature on my iPhone 14 (iOS 17.5, React Native 0.74, Hermes). The flow looked fine on Wi-Fi. Then I walked into an elevator mid-stream, the radio dropped, iOS suspended the app, and when I came back the conversation had silently lost the last two turns. No error. No retry. Just a gap in the transcript that only the user would notice.
The problem with testing this class of bug is that I don't control the model backend. I can't tell a hosted LLM API to hang for 40 seconds, drop a socket mid-response, or return a truncated chunk — and those are exactly the conditions my app fails under. So I started routing the app through a relay server I can kill, and suddenly reconnect behavior became a benchmarkable property instead of a vibe.
This post is the harness, the test matrix, and how I run it. Everything below is a reproducible setup, not a finished benchmark report — where I expect a result rather than having measured one, I say so.
Point your mobile app at a thin relay in front of the real model API. The relay proxies requests but lets you inject:
If your AI task survives a relay that is deliberately hostile, it has a chance of surviving real networks.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. The reason it fits here is practical: MonkeyCode offers free model access and a free server option, which means the relay plus a model backend can run without me paying for a VPS or burning API budget during long fault-injection sessions. Any equivalent free tier works for this harness — the point is that cost stops being an excuse to skip the test, not that one provider is uniquely fast or reliable (I have not benchmarked MonkeyCode's latency and make no claim about it).
Minimal Node 20 relay. It streams from an upstream OpenAI-compatible endpoint and applies a per-request fault profile from headers, so the test runner — not the app — controls the chaos:
// relay.mjs — run: node relay.mjs
import http from 'node:http';
const UPSTREAM = process.env.UPSTREAM_URL; // your model endpoint
const KEY = process.env.UPSTREAM_KEY;
http.createServer(async (req, res) => {
if (req.method !== 'POST') { res.writeHead(404); return res.end(); }
const delay = parseInt(req.headers['x-fault-delay'] || '0', 10);
const dropAfterBytes = parseInt(req.headers['x-fault-drop'] || '0', 10);
let body = '';
for await (const c of req) body += c;
if (delay) await new Promise(r => setTimeout(r, delay));
const up = await fetch(UPSTREAM, {
method: 'POST',
headers: { 'content-type': 'application/json', authorization: `Bearer ${KEY}` },
body,
});
res.writeHead(up.status, { 'content-type': up.headers.get('content-type') ?? 'application/json' });
const reader = up.body.getReader();
let sent = 0;
while (true) {
const { done, value } = await reader.read();
if (done) break;
sent += value.length;
if (dropAfterBytes && sent > dropAfterBytes) {
res.socket.destroy(); // simulate a radio drop mid-stream
return;
}
res.write(value);
}
res.end();
}).listen(8787, () => console.log('relay on :8787'));
Point the app's base URL at http://<your-lan-ip>:8787
, and drive fault profiles from the test harness:
curl -X POST http://localhost:8787/v1/chat -d '{...}'
curl -X POST http://localhost:8787/v1/chat -H 'x-fault-delay: 8000' -d '{...}'
curl -X POST http://localhost:8787/v1/chat -H 'x-fault-drop: 2048' -d '{...}'
This is the part most teams skip. Each cell is a distinct failure mode, and "works on Wi-Fi" only covers the first row. My matrix for an AI task with an in-flight request:
| # | Lifecycle transition | Fault injected | Expected behavior |
|---|---|---|---|
| 1 | None, foreground | none | stream completes, transcript persisted |
| 2 | Background 5s, return | none | stream resumes or cleanly resumes from checkpoint |
| 3 | Background 60s (iOS suspends) | delay 8s | task marked interrupted; on return, partial output shown + explicit resume prompt |
| 4 | Network drop mid-stream | drop 2KB | partial tokens preserved, retry offered, no duplicate user turn on retry |
| 5 | Airplane mode on/off | drop + offline | task queued locally, replays exactly once on reconnect |
| 6 | App killed by swipe | any | cold start restores draft + interrupted task state from disk, not memory |
| 7 | Low Power Mode + background | delay 8s | same as #3; no silent assumption that background fetch will run |
For each cell I record: device, iOS version, RN version, network (Wi-Fi/LTE/airplane), battery state, fault profile, and one of three outcomes — recovered, restarted, or silently lost. The third outcome is the one that matters; it's the bug my users found in the elevator.
The single most common root cause I've hit across these tests: task state living only in memory (a Redux store or component state) instead of being checkpointed to disk after every streamed chunk. AppState
listeners and NetInfo alone cannot save you if there is nothing durable to restore.
If your AI feature is a single fire-and-forget request with no streaming and no retry semantics, this is overkill — a unit test with a mocked 500 is enough. And if your threat model is about the provider's uptime rather than the device's radio, inject faults at the client layer (e.g., a URLProtocol/OkHttp interceptor) instead of standing up a relay.
If you try the matrix, I'd genuinely like the comparable data: device, OS version, which transition you hit, and whether the task recovered, restarted, or silently disappeared. That third column is where the interesting bugs live.
If you want to try the relay setup without provisioning anything, MonkeyCode's free model access and free server option are one way to get both halves running — but the harness above works against any endpoint you control.