Stop letting your AI agent test Android apps with screenshots A developer released android-scenario-runner, an MIT-licensed Python tool that lets AI coding agents test Android apps by driving them through JSON scenario files instead of step-by-step screenshots. The runner uses `uiautomator dump` to read the screen as text, returning a single pass/fail line and only surfacing a screenshot when a visual decision is needed, which the developer says cut image-token costs across roughly 90 scenarios run on a real device. The repo is standard library plus adb, with no on-device installation required. I shipped an Android app mostly by pairing with Claude Code. Writing features went fine. Testing them on a real phone did not. When an AI coding agent tests an app on a device, the usual loop is: take a screenshot, look at it, decide where to tap, tap, take another screenshot. Every step sends an image to the model. A 20-step login-and-checkout flow can cost thousands of image tokens, fill the context window, and the agent still sometimes taps the wrong thing. Most of that work doesn't need eyes. "Tap Sign in , wait, check that Home is on screen" is a text problem. Android can already describe the screen as text: uiautomator dump returns every visible node with its text and bounds. So instead of the agent driving the phone step by step, it writes the steps once as JSON: { "name": "Login works", "app": "com.example.app", "steps": {"app": "restart"}, {"type": "Email", "test@example.com" }, {"type": "Password", "secret123" }, {"tap": "Sign in", "exact": true}, {"expect": "Home" , "wait": 3} } and runs it with one command: bash $ python scenario.py scenarios/login.json PASS Login works 5 steps, 12 s One line back. If a step fails, the runner prints the failing step, what was on screen, and saves a screenshot, which is usually enough for the model to fix either the test or the app: ✗ 5 expect: expected 'Home' found ; screen: 'Sign in', 'Wrong password', ... FAIL Login works step 5/5 9 s The model only opens a screenshot when a visual decision is actually needed layout, colours . Most of the code is not the happy path; it's the stuff that broke on real phones: uiautomator dump fails during an animation, the old ui.xml is still on the device and gets pulled again, so the test "sees" the previous screen. The runner deletes the file on both sides before every dump. adb input text sends the whole string at once. Type character by character. wlan0 has no IP. Add a few lines to your CLAUDE.md / AGENTS.md : Device testing - Never drive the phone step by step with screenshots. Write a scenario in scenarios/