{"slug": "test-automation-with-agent-device-part-1", "title": "Test Automation with agent-device (Part 1)", "summary": "A developer evaluated Callstack's open-source agent-device tool, which lets an AI agent drive iOS and Android apps and inspect the screen, for testing a Flutter app after Google's Android-only Artemis fell short on iOS support. The writeup covers installing the CLI and registering it as an MCP server with Claude Code, the boot/install/open/press/fill/wait loop, and Flutter-specific pitfalls: only text-bearing widgets appear in the accessibility tree unless Semantics identifiers are added, labels vary with device language, and AdMob test banners surface as platform_view, webview and link nodes on iOS.", "body_md": "agent-device is an open-source tool from Callstack that lets an AI agent drive iOS and Android apps and check what is on screen. This post covers why it was picked for testing a Flutter app, how to install it for Claude Code, the basic command loop, and what to watch out for with Flutter. It is based on version 0.21.31 (2026-10-24).\n\nFlutter ships Android and iOS apps from one codebase, so the device-testing tool has to cover iOS too. Google Artemis, the first candidate, is a natural-language testing agent, but it supports Android only; iOS is still just a roadmap item.\n\nThe other candidates looked like this:\n\nagent-device handles iOS and Android simulators, emulators, and real devices with one tool, and its docs state Flutter support explicitly. It ships both a CLI and an MCP server, can save a verified flow as an `.ad` script to replay later, and can export to Maestro YAML. It is MIT-licensed.\n\nInstall the CLI globally and check the environment.\n\n```\nnpm i -g agent-device@latest\nagent-device doctor\n```\n\n`doctor` checks connected devices, toolchains, and dev-server status. In this setup it passed with only warnings for unused platforms (HarmonyOS, Vega).\n\nRegister it with Claude Code as an MCP server. With `-s user`, the same tool is available in every project.\n\n```\nclaude mcp add -s user agent-device -- agent-device mcp\n```\n\nThe agent-device repository includes three agent skills: `agent-device`, `ios-simulator`, and `android-emulator`. The official instruction is `npx skills add callstack/agent-device`, but Claude Code's auto mode safety check blocked it. In that case, clone the repository and copy the three folders under `skills/` into `~/.claude/skills/`.\n\nTo undo everything, run `claude mcp remove agent-device -s user` and `npm uninstall -g agent-device`, then delete the three skill folders.\n\nThe basic loop is: prepare a device, open the app, act, verify, close. Pick the device with `--platform` and `--device`.\n\n```\nagent-device boot --platform android --device \"<emulator name>\" --headless\nagent-device install /absolute/path/app-debug.apk --platform android --device \"<emulator name>\"\nagent-device open <bundle-id> --platform android --device \"<emulator name>\" --foreground\nagent-device press @e5 --settle\nagent-device fill @e11 \"text\" --settle\nagent-device wait text \"<expected text>\"\nagent-device screenshot /absolute/path/home.png\nagent-device close\n```\n\n`open` reads the screen and returns a ref such as `@e5` for each element, like buttons and text fields. With `--settle`, `press` and `fill` wait until the screen goes quiet and return only what changed, so the agent can pick its next target without rereading the whole screen.\n\nThree things commonly go wrong here:\n\n`wait text` or `wait <selector>`, and keep screenshots as evidence only.` install` an absolute path.\nFor a Flutter app, build the iOS simulator app with `flutter build ios --simulator --debug` and the Android app with `flutter build apk --debug`.\n\nagent-device reads the screen through the accessibility tree. In Flutter, that means only widgets with text show up by default; icon buttons and containers do not. Add `Semantics(identifier:)` or `semanticLabel` to those widgets so a selector can find them. Flutter's `Key` does not appear in the accessibility tree.\n\nLabels in the accessibility tree follow the language the device is set to, so the same app reads with different label text depending on that setting. A text-based selector changes with the device language, which makes `identifier` the more stable choice.\n\nOn iOS, AdMob test banners show up as `platform_view`, `webview`, and `link` nodes. Selectors in replay scripts should avoid those nodes.", "url": "https://wpnews.pro/news/test-automation-with-agent-device-part-1", "canonical_source": "https://dev.to/krenin88/test-automation-with-agent-device-part-1-7nj", "published_at": "2026-10-24 18:08:13+00:00", "updated_at": "2026-10-24 18:18:10.621243+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "agent-protocols", "ai-products"], "entities": ["Callstack", "agent-device", "Claude Code", "Flutter", "Google Artemis", "Maestro", "AdMob"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/test-automation-with-agent-device-part-1", "markdown": "https://wpnews.pro/news/test-automation-with-agent-device-part-1.md", "text": "https://wpnews.pro/news/test-automation-with-agent-device-part-1.txt", "jsonld": "https://wpnews.pro/news/test-automation-with-agent-device-part-1.jsonld"}}