Scriptless Mobile Tests as JSON: One File for Android and iOS

A developer has described a JSON-based mobile test format that stores actions and selectors instead of recorded coordinates, allowing a single file with "platform": "mobile" to run on both Android and iOS. Each step identifies elements by text, label, or id, supports fallback selector lists, and can be hand-written, generated by an AI agent, or diffed in a pull request. The format caps flows at 500 steps, supports reusable fragments and TMS links, and was validated against a live validator before publication.

Short answer: your app renamed "Sign in" to "Log in", and every recorded test that taps it now fails on Android, then again on iOS. A test stored as a recording can usually only be fixed by recording it again. We store a mobile test as a JSON file instead: actions and selectors, no coordinates. You can write one by hand, have an AI agent write it, diff it in a pull request, and fix it by editing one line. With "platform": "mobile" , the same file runs on an Android phone and an iPhone. This post is the reference we wanted when we built it: what the format looks like, a snippet for every action, how the two platforms share one file, and what the validator says when something is wrong. Every snippet below was checked against the live validator before publishing. A recorded test usually holds three things that break: a screen position for each tap, a snapshot of the screen it was recorded on, and a device model it silently assumes. Change the layout, the phone or the OS and the recording no longer matches. A JSON flow keeps none of those. Each step names an element by what a person would call it its text or its id , and each run looks for that element on the screen as it is right now. Nothing recorded on an Android phone is ever compared against an iPhone, because nothing is recorded at all. There are three practical consequences: tap "Add to cart" in a diff, not a binary blob. launchApp . If you can describe the test to a colleague, you can write it. An AI agent can write it too, from the same description. Here is a complete flow that signs in, finds a product and adds it to the cart, on either phone: { "name": "Sign in and add to cart", "platform": "mobile", "steps": { "action": "launchApp", "appId": { "android": "com.example.shop", "ios": "com.example.Shop" } }, { "action": "tap", "selectors": { "using": "id", "value": "email input" }, { "using": "text", "value": "Email" } }, { "action": "type", "value": "{{secret.SHOP EMAIL}}" }, { "action": "tap", "selectors": { "using": "id", "value": "password input" }, { "using": "text", "value": "Password" } }, { "action": "type", "value": "{{secret.SHOP PASSWORD}}" }, { "action": "tap", "selectors": { "using": "text", "value": "Sign in" } }, { "action": "assert", "assert": { "kind": "exists" }, "selectors": { "using": "text", "value": "Home" } , "timeoutMs": 15000 }, { "action": "scroll", "selectors": { "using": "text", "value": "Running shoes" } }, { "action": "tap", "selectors": { "using": "text", "value": "Running shoes" } }, { "action": "tap", "selectors": { "using": "text", "value": "Add to cart" } }, { "action": "assert", "assert": { "kind": "textContains", "expected": "1 item" }, "selectors": { "using": "id", "value": "cart badge" } } } Eleven steps, no code, no coordinates, and no "if Android" anywhere. The rest of this post is what each of those lines can be. | Field | Value | Notes | |---|---|---| | name | any text | A name that already exists saves a second flow. Nothing is overwritten. | | platform | "android" , "ios" or "mobile" | mobile is one flow that runs on both. | | category | "test" default or "fragment" | A fragment is a reusable piece, such as a login. | | tmsLink | optional | Links the flow to its test case in TestRail, Jira or Azure DevOps. | | steps | array | Up to 500, nested steps included. | A selector is { "using": ..., "value": ... } . You can list several. If one finds nothing, or matches more than one control, the next one is tried. The section after this one explains the order. "selectors": { "using": "id", "value": "com.example.shop:id/checkout" }, { "using": "text", "value": "Checkout" } Three kinds work on both phones: text label id Text matching ignores case, extra spaces and character width, but is otherwise exact. "sign IN" finds "Sign in". "Network and Internet" does not find "Network & internet". When the wording differs between versions, regions or OS releases, make the selector a pattern. The whole text has to match, ignoring case: { "using": "text", "value": " Allow|Allow While Using App|While using the app ", "regex": true } regex works on text and label only. Keep patterns simple: the validator refuses patterns that could hang, such as nested repeats or back-references. Four more selector kinds exist. They are valid only where the step can run on one OS, which is explained under "One file, two platforms" below. | using | Platform | Example value | |---|---|---| | accessibilityId | Android. | "Open navigation drawer" | | value | iOS. | "Back" | | class | Either, inside a platform block. | "android.widget.Switch" / "XCUIElementTypeSwitch" | | xpath | Either, inside a platform block. | "// @text='Save' " | An xpath must select by attribute, as in // @text='Save' . A position-only path such as //LinearLayout 3 /Button 1 is refused, because it describes where the element sat on the day you wrote it. One rule that saves hours: the run never guesses between two controls. A "Settings" row and the "Settings" text inside it are one control, and count once. Five "Delete" buttons are five controls. If no selector narrows them to one, the step fails and says how many matched, and you add a more specific selector. A test that taps the first of five "Delete" buttons passes until the day it deletes the wrong row. Nothing in the file says where an element is. Every step that has selectors finds its element on the live screen, in five moves: If time runs out 5 seconds, or the step's timeoutMs , the step fails and says why: the element was not found, or several controls matched and it names how many. An optional step is skipped instead. Here is what each action does once its element is known: | Action | What the run does | |---|---| | launchApp | Starts the app by its package name or bundle id. No element lookup. | | tap , doubleTap , longPress | Finds the element, then touches its centre. longPress holds for durationMs . | | type | No lookup. Types into the focused field. A {{secret.KEY}} is filled in for this run only, and the saved step keeps the reference. | | scroll | Scrolls until the element is on screen. | | wait | Pauses for the given time. Nothing else. | | assert | Reads the screen every 500 ms until the condition holds or time runs out. | | if | Checks its condition the same way, then runs then or else . platformIs needs no screen read. | | pressButton , key | Presses the hardware button or key. No lookup. | It runs on the phone, not through a grid. A phone flow does not go through the Selenium or Appium grid that automation suites use. The run drives the phone directly from the machine it is plugged into. There is no hub to route through, no node to wait for and no remote session queue, so nothing sits between the step and the screen. Each step is one read of the screen and one gesture, and the time between steps is the app's time, not the infrastructure's. { "action": "launchApp", "appId": { "android": "com.example.shop", "ios": "com.example.Shop" } } A package name and a bundle id are different strings, so a mobile flow names both. A single-platform flow can use a plain string, and Android can name a launch activity: { "action": "launchApp", "appId": "com.example.shop", "appActivity": ".MainActivity" } Make it the first step. If you don't, the validator warns you that replay starts on whatever screen the phone happens to be showing. tap { "action": "tap", "selectors": { "using": "text", "value": "Sign in" } } doubleTap { "action": "doubleTap", "selectors": { "using": "id", "value": "product image" } } longPress { "action": "longPress", "selectors": { "using": "text", "value": "Size 42" } , "durationMs": 800 } durationMs is optional, from 100 to 10,000. type : enter text into the focused field { "action": "tap", "selectors": { "using": "text", "value": "Search" } }, { "action": "type", "value": "running shoes" } type has no selector. It types into whatever field has focus, so tap the field first. This is the same order a person follows, and it means the keyboard is up before any text is sent. Passwords and other secrets never go into the file. Save them once under Variables , and refer to them by name: { "action": "tap", "selectors": { "using": "text", "value": "Password" } }, { "action": "type", "value": "{{secret.SHOP PASSWORD}}" } If the field you just tapped looks like a password field and the value is literal text, the flow is rejected rather than stored. Phone flows have no environment variables: {{secret.KEY}} is the one reference they resolve. scroll : bring an element into view { "action": "scroll", "selectors": { "using": "text", "value": "Running shoes" } } You name what you want to reach, not how far to swipe. The run scrolls until the element is on screen, so the step survives a list that grows or a taller phone. wait { "action": "wait", "value": 1500 } Milliseconds, from 1 to 120,000. Reach for it last. A fixed wait is either too short on a slow network or wasted time on a fast one. To wait for a screen, assert on something it shows and give that step a longer timeoutMs : { "action": "assert", "assert": { "kind": "exists" }, "selectors": { "using": "text", "value": "Home" } , "timeoutMs": 15000 } assert : four kinds { "action": "assert", "assert": { "kind": "exists" }, "selectors": { "using": "text", "value": "Order placed" } }, { "action": "assert", "assert": { "kind": "notExists" }, "selectors": { "using": "text", "value": "Something went wrong" } }, { "action": "assert", "assert": { "kind": "textEquals", "expected": "Payment received" }, "selectors": { "using": "id", "value": "order status" } }, { "action": "assert", "assert": { "kind": "textContains", "expected": "1 item" }, "selectors": { "using": "id", "value": "cart badge" } } textEquals and textContains need expected . exists and notExists must not have it. The text comparison is the same forgiving-but-exact one that selectors use. pressButton : hardware buttons { "action": "pressButton", "button": "home" } The buttons are home , volumeUp , volumeDown and lock , and all four work on both phones. key : Android back { "action": "key", "key": "back" } or any Android key code: { "action": "key", "keyCode": 66 } iPhones have no back key, so key is Android-only. In a mobile flow it has to sit inside a platform block, which is the next section. if : branch on what is on screen { "action": "if", "condition": { "check": "exists", "timeoutMs": 3000 }, "selectors": { "using": "text", "value": " Allow|Allow While Using App|While using the app ", "regex": true } , "then": { "action": "tap", "selectors": { "using": "text", "value": " Allow|Allow While Using App|While using the app ", "regex": true } } } That handles the permission dialog that appears on a fresh install and is gone on the second run. check takes the four assert kinds. then and else each hold a list of steps, and blocks nest up to five deep. { "action": "tap", "selectors": { "using": "text", "value": "Not now" } , "optional": true, "timeoutMs": 2000 } optional: true skips the step when its element is absent, instead of failing the run. Use it for a promotion banner that appears sometimes. timeoutMs 100 to 120,000 is how long the step keeps looking for its element. The rule is short: shared selectors by default, and a block for real differences. Most steps never mention a platform. text and id resolve on both phones, and launchApp takes both app ids. When the two apps genuinely differ, wrap that part in platformIs : { "action": "if", "condition": { "check": "platformIs", "expected": "android" }, "then": { "action": "key", "key": "back" } , "else": { "action": "tap", "selectors": { "using": "value", "value": "Back" } } } else is the other phone. Inside each branch the flow is single-platform, so the Android-only key step and the iOS-only value selector become legal. There are no per-step platform overrides. Every difference lives in a block you can see, which tells a reviewer exactly where the two apps diverge. The validator tracks which OS each step can run on. A branch that can never run is an error, such as an ios block inside an Android-only flow. So is a platform-specific selector outside a block. Here is a flow with four mistakes in it: { "name": "Broken on purpose", "platform": "mobile", "steps": { "action": "launchApp", "appId": "com.example.shop" }, { "action": "tap", "selectors": { "using": "accessibilityId", "value": "login button" } }, { "action": "tap", "selectors": { "using": "text", "value": "Password" } }, { "action": "type", "value": "hunter2" }, { "action": "key", "key": "back" } } And here is the validator's reply, verbatim: Not valid: 4 errors. Nothing was saved. - steps 0 .appId: a package name and a bundle id differ; give {"android":"com.example.app","ios":"com.example.App"} - steps 1 .selectors 0 .using: "accessibilityId" is Android-only; put this step in an "if" with condition {"check":"platformIs","expected":"<android|ios "}, or make the flow single-platform - steps 3 .value: this types into a password field; a literal value is not stored. Use {{secret.KEY}} and save the secret under Variables - steps 4 .action: "key" is Android-only; put this step in an "if" with condition {"check":"platformIs","expected":"<android|ios "}, or make the flow single-platform Every error names the step, the field and the fix. A field an action does not use is an error too, not something silently dropped, so a typo like "selector" instead of "selectors" is caught before the first run. That precision is what makes the format work for agents: the reply is specific enough to correct without a human reading it. In the app. Open Test Manager → Tests and click Import JSON . Drop the .json file up to 1 MB , choose it, or paste it into the box. The flow is checked before anything is saved. Errors appear as step N · field · message , and Import stays disabled until they are fixed. Warnings, such as a {{secret.KEY}} you have not saved yet, don't block it. Click Import and the flow becomes a new test with its own T id. Through your AI assistant. Connect Claude, Cursor or any other MCP client to RobotActions and hand it the file: Validate sign-in-and-add-to-cart.json with flow validate. If it is valid, save it unchanged with flow import. flow validate is the same dry run the dialog does, and flow import is the same save. The file goes in as-is either way. Because a flow is a plain file, everything you already do with files works here. Keep flows next to the app code, review them in pull requests, and upload the reviewed version. An agent uses the same tools, plus one: flow validate a dry run , flow import save , and flow replay start run it on a device . A prompt as plain as this one is enough: Open the shop app on an Android phone and work out the sign-in and add-to-cart flow. Write it as a mobile flow, validate it, save it, then replay it on an Android phone and an iPhone and tell me which steps failed. An agent that has the device in front of it reads the real element tree before it writes a selector, so it writes id: email input because that id exists, not because it sounds plausible. If flow validate sends back an error, the error tells the agent which field to change. The agent never has to guess what the validator wanted. This is where a file beats a recording. When a release renames "Sign in" to "Log in": <name .json : the flow, plus its recordingId and version . replaces 11 steps with 11 · v3 → v4 . Click An AI assistant does the same with flow export , an edit, and flow update with that recordingId and expectedVersion . - { "action": "tap", "selectors": { "using": "text", "value": "Sign in" } }, + { "action": "tap", "selectors": { "using": "text", "value": "Log in" } }, The test keeps its T id, its test case link and its replay history. If someone else saved the test after your export, you are told before anything is overwritten, so two people can't silently undo each other's work. Every update is versioned the last ten are kept , and Version history → Revert in the flow editor undoes one. If both wordings will exist for a while, say during a staged rollout, don't pick one. Match both: { "using": "text", "value": " Sign|Log in", "regex": true } When a step fails on replay, the failure message is specific in the same way the validator is. It says the element was not found, or that several matched, and which selector it tried. Most fixes are one selector, so most fixes are one line. You don't have to choose between recording and writing. A flow recorded on a real device exports to this JSON, so you can record the first draft by hand and then tidy the selectors in an editor. One limit is worth knowing. Steps that only make sense as screen positions, such as a freeform swipe or a tap on a bare coordinate, can't be written as selectors. They are listed as left out when you export a recorded flow, and flow update refuses to rewrite that flow from JSON rather than drop them. You keep the recording, and nothing is lost behind your back. Web flows use the same envelope with "platform": "web" and a browser-shaped set of actions navigate , tap , type with a selector, hover , key , back , dialog , page-level asserts such as urlContains . They also support environment variables {{BASE URL}} . One flow can therefore run against staging and production. { "name": "Browse skills", "platform": "web", "steps": { "action": "navigate", "url": "{{BASE URL}}" }, { "action": "tap", "selectors": { "using": "role", "value": "link=Browse the skills" } }, { "action": "assert", "assert": { "kind": "urlContains", "expected": "/skills" } } } The format is ours, but the habits behind it work in any tool: name elements instead of positions, keep secrets out of test files, make every platform difference an explicit block, and fail loudly on ambiguous matches. On RobotActions, a flow runs on real Android phones and iPhones in the cloud, singly or as a suite, and links back to its case in TestRail, Jira or Azure DevOps. Write your first flow in an editor and import it, or ask your agent to write it, then replay it on both phones. Try it free · No credit card required → https://robotactions.com Originally published at robotactions.com/blog/scriptless-mobile-tests-json-android-ios https://robotactions.com/blog/scriptless-mobile-tests-json-android-ios .