I shipped an Android app mostly by pairing with Claude Code. Writing features went fine. Testing them on a real phone did not.
When an AI coding agent tests an app on a device, the usual loop is: take a screenshot, look at it, decide where to tap, tap, take another screenshot. Every step sends an image to the model. A 20-step login-and-checkout flow can cost thousands of image tokens, fill the context window, and the agent still sometimes taps the wrong thing.
Most of that work doesn't need eyes. "Tap Sign in, wait, check that Home is on screen" is a text problem.
Android can already describe the screen as text: uiautomator dump returns every visible node with its text and bounds. So instead of the agent driving the phone step by step, it writes the steps once as JSON:
{
"name": "Login works",
"app": "com.example.app",
"steps": [
{"app": "restart"},
{"type": ["Email", "test@example.com"]}, {"type": ["Password", "secret123"]},
{"tap": "Sign in", "exact": true},
{"expect": ["Home"], "wait": 3}
]
}
and runs it with one command:
$ python scenario.py scenarios/login.json
PASS Login works (5 steps, 12 s)
One line back. If a step fails, the runner prints the failing step, what was on screen, and saves a screenshot, which is usually enough for the model to fix either the test or the app:
✗ 5 expect: expected ['Home'] (found []); screen: ['Sign in', 'Wrong password', ...]
FAIL Login works step 5/5 (9 s)
The model only opens a screenshot when a visual decision is actually needed (layout, colours).
Most of the code is not the happy path; it's the stuff that broke on real phones:
uiautomator dump fails during an animation, the old ui.xml is still on the device and gets pulled again, so the test "sees" the previous screen. The runner deletes the file on both sides before every dump.adb input text sends the whole string at once. Type character by character.wlan0 has no IP.
Add a few lines to your CLAUDE.md / AGENTS.md:
## Device testing
- Never drive the phone step by step with screenshots. Write a scenario in
scenarios/<name>.json and run: python scenario.py scenarios/<name>.json | tail -3
- To see the current screen as text: python ui.py dump
I ran ~90 scenarios this way on my own app before extracting the runner into its own repo. It's Python standard library + adb, nothing to install on the phone, MIT licensed:
https://github.com/sinangumuskabak-sys/android-scenario-runner
It isn't a replacement for Espresso or Maestro. It's a single file you can hand to an agent so it tests cheaply. Issues and PRs welcome, especially from people on OEM skins I haven't tried.