Yesterday I asked my agents to prepare a place where they can run my apps, inspect the result and try the controls while I keep working on my Mac. A build can pass while a button disappears behind a sheet. I want the agent to see that too. I would also like to keep the keyboard.
The setup puts Mac apps in a macOS virtual machine and iPhone and iPad apps in dedicated Simulator devices. The agent builds on my Mac, launches the app in its test environment, reads an image and uses accessibility information to find controls. Claude Code and Codex get the same scripts and instructions.
The first checks now run with Olsen, my card game, in a Golden Gate VM. Codex has captured its card table, read the accessibility tree and opened the About window through a menu command. My next test is to hand the same setup to Claude Code. You can build a similar loop with your own agent.
A desktop for the agent #
Browser automation has a useful separation. Playwright can drive a page without borrowing the browser window I’m using to answer email. Native Mac apps need somewhere for their windows and menus to go too.
Tart runs macOS virtual machines on Apple Silicon. It can run without displaying a VM window. The app still has a desktop inside the guest, where opening a menu leaves my desktop alone.
My host Mac already has Xcode and the source code. It builds the Debug app, then copies the finished bundle to the guest. The guest only needs to run it. That keeps a second Xcode installation and its build caches out of the VM.
A shell wrapper can send an app through tar without a shared source folder:
tar -cf - -C "$BUILD_PRODUCTS" MyApp.app |
tart exec -i "$MAC_VM" tar -xf - -C /Users/admin/my-app
The destination must already exist. Tart’s exec command requires its guest agent, which the non-vanilla Cirrus Labs images include. Keeping the source tree on the host also makes the boundary easier to understand. The guest receives the app and its test helpers.
The first successful run #
Ask the agent to check for Tart, existing VMs and free disk space before installing anything. The image is a large download, and the VM needs room to grow. Let it ask before down an image or changing guest settings.
For a compatible app, the initial setup looks like this:
brew install cirruslabs/cli/tart
tart clone ghcr.io/cirruslabs/macos-golden-gate-base:latest mac-test
tart set mac-test --memory 4096 --display 1920x1200
I chose Golden Gate because some of my apps require macOS 27. Check your app’s minimum OS and the available image templates before cloning. The guest also needs the libraries your app expects.
Have the wrapper start tart run --no-graphics mac-test as a managed background job. A launchd job survives the terminal command that started it. Wait for both the guest agent and the guest’s Dock before launching the app. Record which process the wrapper owns so cleanup can stop the right one.
The first goal is small: launch one known scene, capture one window and read one accessibility tree. Get that working before adding a command for every control in the app.
Seeing and using the app #
The image answers whether the screen looks right. The accessibility tree answers which element the agent can operate.
On macOS, a small helper can read the application’s accessibility tree by process ID. A useful entry contains the role, identifier, title, value and frame. Actions can use a stable accessibilityIdentifier, with text matching as a fallback. If several controls match, the helper should report the ambiguity. The guest needs Accessibility permission for the process macOS attributes these commands to. Arrange that once with the user, inside the VM.
For screenshots, the app can render its own window content to an image. That avoids asking for permission to record the whole desktop. Capture sheets separately when needed. Native AppKit views and Mac Catalyst need different capture implementations, so this belongs in the app-specific part of the setup.
System file dialogs can live in another process. Read their accessibility information and cancel them after the check. A menu command should open the menu, choose its item and close any remaining menu in one operation.
Olsen has another wrinkle. Its card table is drawn with SpriteKit. A card visible in a screenshot does not automatically become an accessibility button. The agent needs accessible game controls or a small semantic command interface before it can play a turn reliably.
Repeatable scenes and separate data #
A scene argument lets the agent jump to the screen it needs to check. Olsen already has screenshot fixtures, including soloBasicRule, that give it a known starting position. Your app might need a seeded empty library, a long list or an error state.
Keep that fixture data separate from real data. Use a distinct agent bundle identifier and a directory such as .build/agent/<session>/. Each worktree or concurrent agent needs its own instance. Review App Groups, keychain access and explicit file paths as well as the default preferences store. A new bundle identifier alone does not redirect every storage API.
Agent mode should also disable external effects such as analytics, ads and cloud sync. A disposable VM helps protect the host, but a test app can still make network requests.
The skill tells the agent to look for clipped text, overlapping controls, empty content and poor contrast. Then it tries the changed flow and checks the result again, including dark mode and a narrow window. The final report should say what it actually saw.
iPhone and iPad #
For iOS and iPadOS, create a fresh Simulator device for the session and address it by UDID. simctl can boot and control it without opening the Simulator app. Build for Simulator, install on that device, capture its screen, then shut down and delete it when the check is over.
Apple’s Xcode MCP tools can provide the interaction layer. The Xcode 27.2 installed on my Mac exposes device sessions, screenshots, UI hierarchies, taps, swipes and typing. The agent reads the latest hierarchy to find a control, performs the action and checks the returned image. XCUITest still covers repeatable regression tests.
The official Xcode plugin packages this for Claude Code:
/plugin install xcode@claude-plugins-official
Apple also documents the MCP connection for Claude Code and Codex:
claude mcp add --scope user xcode -- xcrun mcpbridge
codex mcp add xcode -- xcrun mcpbridge
Choose one Claude connection method. On Xcode 27, the headless service can be enabled once with sudo xcrun mcp-server enable. Keep its normal agent and folder approvals. The tools available depend on the installed Xcode version, so have the agent inspect them first.
My scripts own the fresh simulator and its fixture data. The agent attaches Xcode’s interaction session to that exact UDID and verifies the returned device matches. It ends the MCP session before deleting the simulator. This avoids another dependency for everyday iPhone and iPad interaction.
For Mac apps, the host’s Xcode connection still acts on the host. Tart keeps the app’s windows and menus in the guest, where the accessibility helper operates. Keep the same scene arguments across the Mac and Simulator paths where they make sense.
Several agents can then work on different apps without resetting a device someone else is using.
Moving to another Mac #
I keep the drivers and bootstrap in one skills repo. After installing Xcode and the agent clients, I can prepare another Apple Silicon Mac from that checkout:
./install.sh --repos --app-ui
./install.sh --check --app-ui
The setup asks before installing Tart, down Golden Gate and enabling Xcode’s headless service. It preserves existing VMs and MCP connections. The app repos carry their own wrappers, target settings and Debug adapters, so the setup does not depend on files left in a temporary folder on my current Mac.
./new-repo.sh my-app app also creates the wrapper and a pending configuration. Once its Xcode target exists, the creating agent uses the skill to discover the real build settings and connect the Debug bridge with isolated data. The readiness check blocks launch until that is done. An empty repo cannot supply an app window quite yet.
The same skill in Claude and Codex #
The scripts do the work. A short SKILL.md teaches the agent when and how to use them. Claude Code discovers project skills in .claude/skills/, while Codex uses .agents/skills/. One source can be linked into both locations.
I call mine se-appen, Norwegian for “see the app”. It describes the verification loop and the commands, with longer platform details in supporting files. A short section in AGENTS.md, also read through my CLAUDE.md symlink, tells the agent where to start and to ask before running a Mac app on my desktop.
For Olsen, the wrapper exposes the loop like this:
tools/app-ui --platform mac start soloBasicRule
tools/app-ui --platform mac bilde
tools/app-ui --platform mac tre
tools/app-ui --platform mac utseende dark
tools/app-ui --platform mac str 900 700
tools/app-ui --platform mac stopp
tools/app-ui --platform mac av
The agent reads the image returned by bilde before proceeding. stopp stops its app instance. av can shut down the VM once other sessions have finished.
This prompt gives your own agent enough to start:
Prepare isolated visual verification for this Apple app.
Read the repo's instructions and existing screenshot fixtures first.
For macOS, build on the host and run in a headless Tart VM.
Check installed tools, existing VMs, disk space and OS compatibility.
Ask before down a VM or changing guest system settings.
Never launch the Mac app on my desktop without asking.
Start with one deterministic scene, one app-owned window image
and one accessibility tree. Use isolated data and bundle IDs.
Add stable identifiers where the changed flow needs them.
For iOS and iPadOS, use a new headless Simulator per session.
Use Apple's Xcode MCP device tools when available, bound to its exact UDID.
Keep Mac app interaction inside the VM.
Put the commands in scripts and the workflow in one skill that
Claude Code and Codex can both discover. Inspect each image.
Report what worked, what remains unverified and how to clean up.
Hardware and account-dependent flows still need separate checks. Treat audio input, Apple Intelligence and Apple-ID features as capabilities to verify in the chosen environment. Ask before moving a test to the host.
Stop the test app when finished. Shut down the VM once no other test session is using it. My next step is to hand Olsen to Claude Code and see whether I get to keep typing.