# Test Automation with agent-device (Part 1)

> Source: <https://dev.to/krenin88/test-automation-with-agent-device-part-1-7nj>
> Published: 2026-10-24 18:08:13+00:00

agent-device is an open-source tool from Callstack that lets an AI agent drive iOS and Android apps and check what is on screen. This post covers why it was picked for testing a Flutter app, how to install it for Claude Code, the basic command loop, and what to watch out for with Flutter. It is based on version 0.21.31 (2026-10-24).

Flutter ships Android and iOS apps from one codebase, so the device-testing tool has to cover iOS too. Google Artemis, the first candidate, is a natural-language testing agent, but it supports Android only; iOS is still just a roadmap item.

The other candidates looked like this:

agent-device handles iOS and Android simulators, emulators, and real devices with one tool, and its docs state Flutter support explicitly. It ships both a CLI and an MCP server, can save a verified flow as an `.ad` script to replay later, and can export to Maestro YAML. It is MIT-licensed.

Install the CLI globally and check the environment.

```
npm i -g agent-device@latest
agent-device doctor
```

`doctor` checks connected devices, toolchains, and dev-server status. In this setup it passed with only warnings for unused platforms (HarmonyOS, Vega).

Register it with Claude Code as an MCP server. With `-s user`, the same tool is available in every project.

```
claude mcp add -s user agent-device -- agent-device mcp
```

The agent-device repository includes three agent skills: `agent-device`, `ios-simulator`, and `android-emulator`. The official instruction is `npx skills add callstack/agent-device`, but Claude Code's auto mode safety check blocked it. In that case, clone the repository and copy the three folders under `skills/` into `~/.claude/skills/`.

To undo everything, run `claude mcp remove agent-device -s user` and `npm uninstall -g agent-device`, then delete the three skill folders.

The basic loop is: prepare a device, open the app, act, verify, close. Pick the device with `--platform` and `--device`.

```
agent-device boot --platform android --device "<emulator name>" --headless
agent-device install /absolute/path/app-debug.apk --platform android --device "<emulator name>"
agent-device open <bundle-id> --platform android --device "<emulator name>" --foreground
agent-device press @e5 --settle
agent-device fill @e11 "text" --settle
agent-device wait text "<expected text>"
agent-device screenshot /absolute/path/home.png
agent-device close
```

`open` reads the screen and returns a ref such as `@e5` for each element, like buttons and text fields. With `--settle`, `press` and `fill` wait until the screen goes quiet and return only what changed, so the agent can pick its next target without rereading the whole screen.

Three things commonly go wrong here:

`wait text` or `wait <selector>`, and keep screenshots as evidence only.` install` an absolute path.
For a Flutter app, build the iOS simulator app with `flutter build ios --simulator --debug` and the Android app with `flutter build apk --debug`.

agent-device reads the screen through the accessibility tree. In Flutter, that means only widgets with text show up by default; icon buttons and containers do not. Add `Semantics(identifier:)` or `semanticLabel` to those widgets so a selector can find them. Flutter's `Key` does not appear in the accessibility tree.

Labels in the accessibility tree follow the language the device is set to, so the same app reads with different label text depending on that setting. A text-based selector changes with the device language, which makes `identifier` the more stable choice.

On iOS, AdMob test banners show up as `platform_view`, `webview`, and `link` nodes. Selectors in replay scripts should avoid those nodes.
