{"slug": "how-to-automate-apps-with-no-api", "title": "How to Automate Apps With No API?", "summary": "A developer created Mobilerun, an open-source framework that uses LLMs with vision to automate mobile apps that lack public APIs. The tool reads screen content via accessibility trees and vision models, plans multi-step workflows, and drives real Android or iOS devices with taps, swipes, and text input. It solves the brittleness of traditional UI automation by understanding intent rather than fixed UI structure.", "body_md": "I've hit this wall more times than I can count. I need to post to a creator account, verify an SMS code on a virtual phone, or stitch a mobile workflow into a backend pipeline. Then I go looking for the API and find one of three things: there isn't one, there is one but it's locked behind a partnership form, or there is one but it's missing the exact thing I need :(\n\nFor a long time, the answer was \"tough luck, scrape the UI.\" I think that answer is finally changing, because I think now the AI agents have gotten good enough to drive a phone the same way I would. This post is about how that actually works in practice.\n\nLet's talk about why this gap exists. Most consumer apps—banking, social, messaging, dating don't ship public APIs because their business model wants you to use the app not the web.\n\nAPIs introduce things they don't want like scraping, abuse, churn from third-party clients, and it creates regulatory headaches. Even when an API exists (eg. Instagram Graph, WhatsApp Business, the various fintech sandboxes), it's almost always a narrow slice of what the app itself can do.\n\nSo if I wanted to automate real-world consumer mobile flows, an API usually wouldn't work. I needed a different abstraction.\n\nIn the past, I used UI automation frameworks like Appium, Espresso, XCUITest, and Selenium-for-mobile. The problem was they're brittle in exactly the way that matters. I'd write a script that taps a button at specific coordinates, and when the app ships a redesign, the selector moves, and my automation dies.\n\nI spent a lot of time fixing selectors than actually shipping something. I added retries, then sleeps, then \"wait for element,\" then a custom DOM diff tool, and eventually realized I'd built a small, sad browser inside my test harness.\n\nThe deeper issue is that these tools require me to describe the UI structurally, but apps don't actually have stable structure (iykyk). They have intent. When I look at a banking app, I don't think tap the third button in the second tab, I think open Payouts.\n\nLLMs with vision finally made it cheap to do what I do naturally: look at a screen, decide what to tap, do it, look at the result, and repeat. Combine vision with the a11y tree the OS already exposes, and you have a model that can both see the pixels and read the semantic structure, which is much more robust imo. Add a planner that breaks post this clip to an app and respond to comments into the dozen taps and swipes it actually requires, and you have a real mobile agent. That's the category Mobilerun works in.\n\nMobilerun is an open-source framework that lets you point an LLM at a real Android or iOS device and tell it what to do.\n\nIt reads the screen via a11y plus a vision model, plans multi-step workflows, and drives the device with taps, swipes, and text input. There's a CLI for one-off tasks, a Python SDK for embedding it into my own code, and a hosted cloud if I don't want to babysit physical phones.\n\nA few things make Mobilerun interesting if you're a developer evaluating this space:\n\nMobilerun is LLM-agnostic which means you can choose whichever LLM best fits your needs. The interesting part imo is that it works not only on emulators but also on real devices. That matters more than it sounds. Many apps like banking, dating, healthcare, anything with high security can sniff for emulator signals, missing SIMs, or suspicious network properties and quietly degrade.\n\nRunning on a real device with a real SIM and a real residential network means the app behaves the way it would in a user's hand. Mobilerun provides exactly that: persistent, dedicated mobile devices with eSIMs and locality, so the automation runs reliably.\n\nA few workflows where this approach has beaten either an API integration or traditional UI tests for me:\n\nI don't think the no API problem is going away. If anything, it's getting worse as more value moves inside walled mobile apps. What's changed is that I no longer have to choose between a brittle XPath script and giving up.\n\nVision capable LLMs plus a properly engineered runtime (a stable a11y tree, a real device, structured output, and a planner that recovers from failure) make UI-driven automation actually production-grade for the first time.\n\nIf you want to try it, pip install mobilerun is the fastest path. The framework is open source, and the cloud is there when you outgrow your laptop.", "url": "https://wpnews.pro/news/how-to-automate-apps-with-no-api", "canonical_source": "https://dev.to/priya_negi_9ffd29931ea408/how-to-automate-apps-with-no-api-2m6d", "published_at": "2026-07-09 09:01:18+00:00", "updated_at": "2026-07-09 09:11:05.578676+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "computer-vision", "ai-agents", "developer-tools"], "entities": ["Mobilerun", "Appium", "Espresso", "XCUITest", "Selenium", "Instagram Graph", "WhatsApp Business"], "alternates": {"html": "https://wpnews.pro/news/how-to-automate-apps-with-no-api", "markdown": "https://wpnews.pro/news/how-to-automate-apps-with-no-api.md", "text": "https://wpnews.pro/news/how-to-automate-apps-with-no-api.txt", "jsonld": "https://wpnews.pro/news/how-to-automate-apps-with-no-api.jsonld"}}