Agentic iOS Development with RocketSim and RocketTrace

IOS developer Antoine van der Lee documented a workflow in which coding agents use RocketSim and RocketTrace to navigate, audit, and measure an app under development, reporting that Video Compressor's Home screen accessibility score rose from 0/100 (2 errors, 29 warnings across 141 audited elements) to 99/100 with 4 warnings remaining. RocketSim's accessibility-audit command returned the findings, most of them the hit_region_small rule flagging interactive elements below 28×28 points as errors and below the recommended 44×44 points as warnings, while RocketTrace records Xcode Instruments traces and returns ranked findings. Van der Lee also reported that of 57 recent users with at least one video on their device, 35 (61%) successfully compressed a video while 22 never did.

Agentic iOS development changes a lot once your coding agent can see and measure the app it’s building. Without that, an agent writes code, hopes it works, and asks you to run the app. With the right tools, it navigates the app itself, reads the accessibility tree, and checks performance with evidence. I’ve been building Video Compressor videocompression.app https://videocompression.app/ this way, using Cursor, RocketSim https://www.rocketsim.app/ , and RocketTrace https://rockettrace.app/ . In this article, I’ll share four prompts I use in my daily workflow and what each one found in my app. An agent that only reads code can’t tell you whether the app behaves right. It can only tell you that the code looks right. You still have to build the app, tap through it, and report back what you see. Two tools close that gap for me: - RocketSim lets the agent use the app in the Simulator. Through the rocketsim command, it reads what’s on screen and taps, swipes, and types. It can also run an accessibility audit and capture screenshots with device bezels. - RocketTrace lets the agent measure the app. It records an Xcode Instruments trace and returns ranked findings instead of a raw .trace package. Both come with an Agent Skill, so you only mention them in a prompt using /rocketsim or /rockettrace . If you want to know how the Simulator side works, I explained it in The iOS Simulator inside Cursor, Claude Code, and Codex https://www.avanderlee.com/ai-development/the-ios-simulator-inside-cursor-claude-code-and-codex/ . For the performance side, read Introducing RocketTrace https://www.avanderlee.com/ai-development/introducing-rockettrace-using-xcode-instruments-with-ai-agents/ . You might have noticed something in my prompts: all four run in Plan mode. I want to review what the agent found before it touches any code. Accessibility is the kind of work that’s easy to postpone because it’s tedious. You have to visit every screen, inspect every element, and keep track of what’s wrong. That’s exactly the type of work I’m happy to hand over. RocketSim reads the accessibility tree of the screen that’s visible in the Simulator, so the agent can navigate to a screen and audit it. If you want to run these audits yourself, I recommend reading Accessibility audits in the iOS Simulator https://www.rocketsim.app/blog/accessibility-audit-ios-simulator/ . The command the agent uses looks like this: rocketsim accessibility-audit --udid <simulator-udid --screen latest --format full The audit returns a score out of 100, a list of findings, and a suggested fix for each of them. The first audit of Video Compressor’s Home screen scored 0/100 with 2 errors and 29 warnings across 141 audited elements. After a few rounds of fixes, the same screen reached 99/100 with 4 warnings left. Most findings were the same rule: hit region small . An interactive element smaller than 28×28 points is an error, and anything below the recommended 44×44 points is a warning. It’s exactly the kind of detail you never notice by looking at the screen, but a person using VoiceOver or Voice Control will. I let the agent create a plan first, because a fix like this can easily change your layout. Making a button’s tap area bigger shouldn’t move it. In the plan, I check that the proposed fix expands the tappable frame instead of shrinking or shifting the design. The key metric for Video Compressor is simple: how many people who install the app end up compressing a video? I asked the agent to look at the data first, using the Amplitude MCP, before building anything. I use MCP Beast https://mcp-beast.ai/mac-app-ios-developers to get access to all my MCPs from a single place, so the agent can reach Amplitude without me configuring it in every project. Of the 57 recent users who had at least one video on their device, 35 61% successfully compressed a video. The other 22 never did. That made the first compression the place to help people, instead of adding another feature somewhere else in the app. And yes, my app just went live, so I did not have many users yet, ha Based on that, I asked the agent to implement TipKit https://developer.apple.com/documentation/tipkit tips that guide people toward their first compression. I mentioned /swiftui-expert-skill so the agent follows modern SwiftUI practices. If you want to know more about that skill, I wrote about it in SwiftUI Agent Skill: Build better views with AI https://www.avanderlee.com/ai-development/swiftui-agent-skill-build-better-views-with-ai/ . The last part of the prompt matters most for me: I ask the agent to use RocketSim to return a screenshot of each tip it implemented. TipKit tips only show up under certain conditions, so you can’t easily check them by launching the app once. By asking the agent to navigate to each state and capture a screenshot, I get a visual overview of every tip in the pull request. App Store screenshots are a chain of small manual steps: capture the screens, add a device frame, update the design, export every locale, and upload them. This prompt chains the tools for each of those steps. RocketSim captures the screens with a device bezel, the Sketch MCP updates the design file, and /helm-asc handles the export and upload to App Store Connect using the Helm CLI https://helm-app.com/features/helm-cli/ . Each tool has one job, and the agent connects them. I started by creating a design for the English locale in Sketch, and the agent duplicated it for all 18 locales. This is where the two tools work best together. RocketSim moves through the app like a user would, while RocketTrace records the performance of that session. That gives you a realistic trace without having to tap through the app yourself. I ran this on a Release build. The recording was 51.8 seconds long and RocketTrace found 178 hitches and 19 hang and stall findings. In 85% of the hitch windows, Core Animation layout work was the cause. The agent read the results in layers, starting with a short brief, then the compact list of findings, and only then the details of one finding. That’s the reason I’m not worried about the context window here. If you want to know why that matters, read my article on how to reduce token usage in Claude Code, Codex, and Cursor https://www.avanderlee.com/ai-development/reduce-token-usage-claude-code-codex-cursor/ . After my approval, the agent fixed three things on a separate branch: - A 198 ms stall on the Home screen, caused by PhotoLibraryService access combined with copies of VideoAsset values. - PhotoKit work that was done per video on the main actor. The agent changed it to fetch availability in chunks. - Duplicate thumbnail ID writes, 319 in total, which likely reset the image state while scrolling. It also decided not to fix some things. 102 of the 178 hitches happened on an unknown screen, so the plan said to record again with OSSignposter intervals first. A plan that says “we don’t have enough evidence yet” is better than a fix based on a guess. I haven’t re-recorded on the device after these changes yet. That’s the next step, and RocketTrace can compare the new recording against the old one to show whether each finding is still there. All four prompts end with a plan or start in Plan mode, and that’s on purpose. Every tool in this workflow produces evidence: an audit score, a funnel number, a screenshot, a trace finding. A plan lets me compare the evidence with the proposed change before the agent edits any code. It also makes the agent’s mistakes cheaper. If an accessibility fix changes the layout or a performance fix targets the wrong function, I can see it in the plan and correct it. Fixing it after it’s merged takes a lot more time. Is your mobile app under attack right now? https://hubs.la/Q04x jcF0 Watch a simulated mobile attack from first signal to resolution, and see why threat monitoring is essential to the complete, end-to-end protection of your mobile app. Join Guardsquare on Oct 20. Register now https://hubs.la/Q04x jcF0 . Agentic iOS development works best when your agent can verify its own work. RocketSim lets it see and use the app, and RocketTrace tells it where the app is slow. Combine that with analytics from an MCP and a plan to review, and the agent stops guessing. If you want to try this yourself, start with the prompt that fits your app best. For most apps, that’s the accessibility audit or the performance prompt. If you want to learn more about working with agents, check out my article on agentic development challenges across multiple projects https://www.avanderlee.com/ai-development/agentic-development-multi-project-challenges/ . Feel free to contact me or tweet me on Twitter https://twitter.com/twannl if you have any additional tips or feedback. Thanks