{"slug": "droidpath-stream-android-ui-trees-at-60hz-over-tcp-bypassing-adb", "title": "DroidPath – Stream Android UI Trees at 60Hz over TCP (Bypassing ADB)", "summary": "A GitHub project called DroidPath, published by developer V0val0x, streams Android UI trees as a live Shadow DOM over TCP port 27189 via a 631 KB single-DEX Kotlin JAR launched with app_process, bypassing a companion APK install. DroidPath's A* symbolic router completed a four-hop \"Navigate to Night Light\" task in 1,683 ms with zero intermediate LLM roundtrips, versus roughly 12,000–25,000 ms and 4–5 vision inference calls for a traditional VLM screenshot loop, and it persists screens to ~/.droidpath/shadow_graph.json while compacting near-duplicates at Jaccard similarity of 0.90 or higher. The project also ships a standalone MCP bridge (droidpath-mcp) that exposes the router to Claude, Cursor and other MCP clients over JSON-RPC 2.0 via stdio or HTTP+SSE.", "body_md": "**Persistent A* Symbolic UI Router, Headless Kotlin Shadow DOM Daemon & Standalone MCP Bridge for Android**\n\n**English** | [Русская версия (Russian)](https://github.com/V0val0x/DroidPath/blob/main/README_RU.md)\n\n**DroidPath** eliminates slow, token-heavy VLM screenshot-think-tap loops for Android automation. Instead of asking an LLM to inspect screenshots at every step, DroidPath streams a live **Shadow DOM** directly from Android Accessibility & Window APIs, compiles UI transitions into a **persistent Spatial Graph**, and navigates multi-screen workflows deterministically with **A* pathfinding** in sub-second hops — with **zero intermediate LLM roundtrips**.\n\n``` php\nflowchart LR\n    Agent[\"AI Agent (Claude / Cursor / MCP Client)\"] <-->|\"JSON-RPC 2.0 (stdio / HTTP+SSE)\"| CLI[\"droidpath-mcp / DroidPathBridge\"]\n\n    subgraph Host[\"macOS Host (Swift 5.9+ · AndroidShadowGraphKit)\"]\n        CLI <-->|\"Path Queries & Passive Learning\"| Bridge[\"ShadowGraphBridge\"]\n        Bridge --> DOM[\"ShadowDOMState + ShadowTreeIndex\"]\n        Bridge --> Graph[\"SpatialGraphEngine\\nScreenLayoutHasher · Jaccard ≥ 0.90\\n~/.droidpath/shadow_graph.json\"]\n        Bridge --> AStar[\"AStarPathfinder + MacroExecutionEngine\"]\n        Bridge -->|\"Self-Healing Recovery Hook\"| Recovery[\"SHA-256 Deploy + app_process Restart\"]\n    end\n\n    subgraph Android[\"Android Device (Zero-APK)\"]\n        Daemon[\"android-daemon.jar (Kotlin single-DEX)\\napp_process :27189 · NDJSON Delta Stream\"]\n        Sys[\"UiAutomation / Accessibility / WindowManager\"]\n    end\n\n    Daemon <-->|\"adb forward tcp:27189\"| DOM\n    AStar -->|\"adb shell input tap/swipe\"| Sys\n    Daemon --- Sys\n```\n\n1. \n**Layer 1 — Headless Kotlin Shadow DOM Daemon (`android-daemon/`):**\n  - Compiles into a standalone **631 KB single-DEX JAR** (`android-daemon.jar` ) launched via`app_process` — zero companion APK install required.\n  - Streams full snapshots and compact NDJSON incremental diffs over TCP (`:27189` ) with event debouncing, configurable tree depth, and heartbeat/watchdog protection.\n  - Deployed idempotently via SHA-256 verification (`DroidPathManager` /`deploy.sh` ) and protected by automatic self-healing recovery if Android kills the process.\n2. Compiles into a standalone \n3. \n**Layer 2 — Live Shadow DOM & Persistent Spatial Graph (`AndroidShadowGraphKit`):**\n  - `ScreenLayoutHasher` strips volatile timestamps, clocks (`12:34` ), and dates so returning to Home or Settings hashes deterministically.\n  - `SpatialGraphEngine` constructs a directed multigraph of screens (`vertices` ) and UI gestures (`edges` ), automatically compacts near-duplicate screens via Jaccard similarity (`>= 0.90` ), and persists to`~/.droidpath/shadow_graph.json` .\n4. \n**Layer 3 — Persistent A* Symbolic Router & Standalone MCP Bridge (`DroidPathBridge` + `droidpath-mcp`):**\n  - `AStarPathfinder` +`BinaryHeapPriorityQueue` computes minimum-cost multi-hop routes to any target text,`resource-id` , package, or canonical layout hash.\n  - `MacroExecutionEngine` dispatches each tap/swipe over ADB, waits for exact or canonical Shadow DOM confirmation per hop (including post-swipe`RecyclerView` fling quiescence before resolving downstream bounds), and raises structured`PathDeviationError` diagnostics on unexpected popups or screen timeouts.\n\nPrompt: *\"Navigate to Night Light\"* from Launcher Home Screen:\n\n| Metric | Traditional VLM Screenshot Loop | **DroidPath A* Router** | \n|---|---|---|\n| **Intermediate LLM Roundtrips** | 4–5 vision inference calls | **0 calls** | \n| **Total End-to-End Latency** | ~12,000–25,000 ms | **1,683 ms** (4 hops) | \n| **State Feedback** | Full PNG screenshots ( `screencap` ) | Sub-ms Shadow DOM NDJSON frames | \n\n| Standalone Agentic HUD Execution | \n|---|\n\nPrerequisites: macOS 14+, Swift 5.9+, `adb` (`brew install android-platform-tools`), JDK 17 (only needed if rebuilding `android-daemon.jar` from Kotlin source).\n\n```\ngit clone https://github.com/V0val0x/DroidPath.git\ncd DroidPath\n\n# Build standalone single-DEX Kotlin daemon (requires JDK 17 + Android d8 / build-tools or Gradle)\n./android-daemon/scripts/build-dex.sh\n\n# Run Swift + Kotlin tests\nswift test\n./android-daemon/gradlew -p android-daemon test\n\n# Build release CLI\nswift build -c release\n```\n\nOr download prebuilt `droidpath-mcp` and `android-daemon.jar` directly from [**GitHub Releases**](https://github.com/V0val0x/DroidPath/releases).\n\n```\n{\n  \"mcpServers\": {\n    \"droidpath\": {\n      \"command\": \"/path/to/droidpath-mcp\",\n      \"args\": [\"--stdio\", \"--daemon-jar\", \"/path/to/android-daemon.jar\"]\n    }\n  }\n}\n```\n\nOr launch the loopback HTTP/SSE MCP server (`127.0.0.1:8766`):\n\n```\ndroidpath-mcp --http --port 8766 --daemon-jar ./android-daemon/build/dist/android-daemon.jar\ndependencies: [\n    .package(url: \"https://github.com/V0val0x/DroidPath.git\", from: \"1.0.0\")\n]\n```\n\nAdd `\"AndroidShadowGraphKit\"` (core Shadow DOM, Spatial Graph, and A* engine) or `\"DroidPathBridge\"` (ADB deployer, persistence, and MCP core) to your target dependencies.\n\n| Tool | Description | \n|---|---|\n| `android_navigate_to_node` | Runs Layer 3 A* routing to reach a target text, `resource-id` , package, or canonical layout hash (`0x...` ) across screens | \n| `android_get_ui_tree` | Reads live interactive UI elements from the Kotlin Shadow DOM stream (with `uiautomator` XML fallback) | \n| `android_graph_stats` | Inspects persisted graph vertex/edge counts, active canonical hash, daemon state, and storage path | \n| `android_list_devices` | Lists connected USB and Wireless ADB Android devices | \n| `android_tap_element` | Taps by `(x, y)` or semantic`text` /`resource_id` /`content_desc` and records the graph edge | \n| `android_swipe` | Swipes `(x1, y1) -> (x2, y2)` and attributes the gesture into the Spatial Graph | \n| `android_type_text` | Types text into the active Android field | \n| `android_keyevent` | Sends Android keyevents ( `HOME` ,`BACK` ,`ENTER` ,`APP_SWITCH` ,`POWER` ) | \n\nPart of the native macOS ↔ Android automation suite alongside [**AndOnMac**](https://github.com/V0val0x/AndOnMac):\n\n- [**AndOnMac**](https://github.com/V0val0x/AndOnMac) — Native 120Hz ProMotion Metal Android mirroring & control app with built-in DroidPath Agentic HUD\n- [**AndroidMCPKit**](https://github.com/V0val0x/AndroidMCPKit) — Visual-to-MCP`{{parameter}}` macro studio & 3-tier Apple Vision OCR self-healing\n- [**MetalScrcpyKit**](https://github.com/V0val0x/MetalScrcpyKit) — Zero-copy VideoToolbox H.264/HEVC decoder &`CAMetalLayer` renderer\n- [**ADBConnectKit**](https://github.com/V0val0x/ADBConnectKit) — Zero-dependency Swift ADB SDK (USB, QR Wi-Fi pairing, Direct Hotspot bypass)\n- [**VisionScreenRecallKit**](https://github.com/V0val0x/VisionScreenRecallKit) — Apple Silicon Vision OCR & SQLite FTS5 screen memory\n\nMIT License © 2026 Kirill Zaitsev. See [LICENSE](https://github.com/V0val0x/DroidPath/blob/main/LICENSE).", "url": "https://wpnews.pro/news/droidpath-stream-android-ui-trees-at-60hz-over-tcp-bypassing-adb", "canonical_source": "https://github.com/V0val0x/DroidPath", "published_at": "2026-11-16 01:08:45+00:00", "updated_at": "2026-11-16 01:22:28.373623+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-tools", "developer-tools", "artificial-intelligence"], "entities": ["DroidPath", "V0val0x", "Android", "Claude", "Cursor", "Model Context Protocol", "Kotlin", "Swift"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/droidpath-stream-android-ui-trees-at-60hz-over-tcp-bypassing-adb", "markdown": "https://wpnews.pro/news/droidpath-stream-android-ui-trees-at-60hz-over-tcp-bypassing-adb.md", "text": "https://wpnews.pro/news/droidpath-stream-android-ui-trees-at-60hz-over-tcp-bypassing-adb.txt", "jsonld": "https://wpnews.pro/news/droidpath-stream-android-ui-trees-at-60hz-over-tcp-bypassing-adb.jsonld"}}