cd /news/ai-agents/droidpath-stream-android-ui-trees-at… · home › topics › ai-agents › article
[ARTICLE · art-170198] src=github.com ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

DroidPath – Stream Android UI Trees at 60Hz over TCP (Bypassing ADB)

A GitHub project called DroidPath, published by developer V0val0x, streams Android UI trees as a live Shadow DOM over TCP port 27189 via a 631 KB single-DEX Kotlin JAR launched with app_process, bypassing a companion APK install. DroidPath's A* symbolic router completed a four-hop "Navigate to Night Light" task in 1,683 ms with zero intermediate LLM roundtrips, versus roughly 12,000–25,000 ms and 4–5 vision inference calls for a traditional VLM screenshot loop, and it persists screens to ~/.droidpath/shadow_graph.json while compacting near-duplicates at Jaccard similarity of 0.90 or higher. The project also ships a standalone MCP bridge (droidpath-mcp) that exposes the router to Claude, Cursor and other MCP clients over JSON-RPC 2.0 via stdio or HTTP+SSE.

read4 min views1 publishedNov 16, 2026
DroidPath – Stream Android UI Trees at 60Hz over TCP (Bypassing ADB)
Image: Michielbdejong (auto-discovered)

Persistent A Symbolic UI Router, Headless Kotlin Shadow DOM Daemon & Standalone MCP Bridge for Android*

English | Русская версия (Russian)

DroidPath eliminates slow, token-heavy VLM screenshot-think-tap loops for Android automation. Instead of asking an LLM to inspect screenshots at every step, DroidPath streams a live Shadow DOM directly from Android Accessibility & Window APIs, compiles UI transitions into a persistent Spatial Graph, and navigates multi-screen workflows deterministically with A pathfinding* in sub-second hops — with zero intermediate LLM roundtrips.

flowchart LR
    Agent["AI Agent (Claude / Cursor / MCP Client)"] <-->|"JSON-RPC 2.0 (stdio / HTTP+SSE)"| CLI["droidpath-mcp / DroidPathBridge"]

    subgraph Host["macOS Host (Swift 5.9+ · AndroidShadowGraphKit)"]
        CLI <-->|"Path Queries & Passive Learning"| Bridge["ShadowGraphBridge"]
        Bridge --> DOM["ShadowDOMState + ShadowTreeIndex"]
        Bridge --> Graph["SpatialGraphEngine\nScreenLayoutHasher · Jaccard ≥ 0.90\n~/.droidpath/shadow_graph.json"]
        Bridge --> AStar["AStarPathfinder + MacroExecutionEngine"]
        Bridge -->|"Self-Healing Recovery Hook"| Recovery["SHA-256 Deploy + app_process Restart"]
    end

    subgraph Android["Android Device (Zero-APK)"]
        Daemon["android-daemon.jar (Kotlin single-DEX)\napp_process :27189 · NDJSON Delta Stream"]
        Sys["UiAutomation / Accessibility / WindowManager"]
    end

    Daemon <-->|"adb forward tcp:27189"| DOM
    AStar -->|"adb shell input tap/swipe"| Sys
    Daemon --- Sys

Layer 1 — Headless Kotlin Shadow DOM Daemon (android-daemon/):

  • Compiles into a standalone 631 KB single-DEX JAR (android-daemon.jar ) launched viaapp_process — zero companion APK install required.
  • Streams full snapshots and compact NDJSON incremental diffs over TCP (:27189 ) with event debouncing, configurable tree depth, and heartbeat/watchdog protection.
  • Deployed idempotently via SHA-256 verification (DroidPathManager /deploy.sh ) and protected by automatic self-healing recovery if Android kills the process.
  1. Compiles into a standalone

Layer 2 — Live Shadow DOM & Persistent Spatial Graph (AndroidShadowGraphKit):

  • ScreenLayoutHasher strips volatile timestamps, clocks (12:34 ), and dates so returning to Home or Settings hashes deterministically.
  • SpatialGraphEngine constructs a directed multigraph of screens (vertices ) and UI gestures (edges ), automatically compacts near-duplicate screens via Jaccard similarity (>= 0.90 ), and persists to~/.droidpath/shadow_graph.json .

Layer 3 — Persistent A Symbolic Router & Standalone MCP Bridge (DroidPathBridge + droidpath-mcp):*

  • AStarPathfinder +BinaryHeapPriorityQueue computes minimum-cost multi-hop routes to any target text,resource-id , package, or canonical layout hash.
  • MacroExecutionEngine dispatches each tap/swipe over ADB, waits for exact or canonical Shadow DOM confirmation per hop (including post-swipeRecyclerView fling quiescence before resolving downstream bounds), and raises structuredPathDeviationError diagnostics on unexpected popups or screen timeouts.

Prompt: "Navigate to Night Light" from Launcher Home Screen:

Metric Traditional VLM Screenshot Loop DroidPath A Router*
Intermediate LLM Roundtrips 4–5 vision inference calls 0 calls
Total End-to-End Latency ~12,000–25,000 ms 1,683 ms (4 hops)
State Feedback Full PNG screenshots ( screencap ) Sub-ms Shadow DOM NDJSON frames
Standalone Agentic HUD Execution

Prerequisites: macOS 14+, Swift 5.9+, adb (brew install android-platform-tools), JDK 17 (only needed if rebuilding android-daemon.jar from Kotlin source).

git clone https://github.com/V0val0x/DroidPath.git
cd DroidPath

./android-daemon/scripts/build-dex.sh

swift test
./android-daemon/gradlew -p android-daemon test

swift build -c release

Or download prebuilt droidpath-mcp and android-daemon.jar directly from GitHub Releases.

{
  "mcpServers": {
    "droidpath": {
      "command": "/path/to/droidpath-mcp",
      "args": ["--stdio", "--daemon-jar", "/path/to/android-daemon.jar"]
    }
  }
}

Or launch the loopback HTTP/SSE MCP server (127.0.0.1:8766):

droidpath-mcp --http --port 8766 --daemon-jar ./android-daemon/build/dist/android-daemon.jar
dependencies: [
    .package(url: "https://github.com/V0val0x/DroidPath.git", from: "1.0.0")
]

Add "AndroidShadowGraphKit" (core Shadow DOM, Spatial Graph, and A* engine) or "DroidPathBridge" (ADB deployer, persistence, and MCP core) to your target dependencies.

Tool Description
android_navigate_to_node Runs Layer 3 A* routing to reach a target text, resource-id , package, or canonical layout hash (0x... ) across screens
android_get_ui_tree Reads live interactive UI elements from the Kotlin Shadow DOM stream (with uiautomator XML fallback)
android_graph_stats Inspects persisted graph vertex/edge counts, active canonical hash, daemon state, and storage path
android_list_devices Lists connected USB and Wireless ADB Android devices
android_tap_element Taps by (x, y) or semantictext /resource_id /content_desc and records the graph edge
android_swipe Swipes (x1, y1) -> (x2, y2) and attributes the gesture into the Spatial Graph
android_type_text Types text into the active Android field
android_keyevent Sends Android keyevents ( HOME ,BACK ,ENTER ,APP_SWITCH ,POWER )

Part of the native macOS ↔ Android automation suite alongside AndOnMac:

  • AndOnMac — Native 120Hz ProMotion Metal Android mirroring & control app with built-in DroidPath Agentic HUD
  • AndroidMCPKit — Visual-to-MCP{{parameter}} macro studio & 3-tier Apple Vision OCR self-healing
  • MetalScrcpyKit — Zero-copy VideoToolbox H.264/HEVC decoder &CAMetalLayer renderer
  • ADBConnectKit — Zero-dependency Swift ADB SDK (USB, QR Wi-Fi pairing, Direct Hotspot bypass)
  • VisionScreenRecallKit — Apple Silicon Vision OCR & SQLite FTS5 screen memory

MIT License © 2026 Kirill Zaitsev. See LICENSE.

── more in #ai-agents 4 stories · sorted by recency
── more on @droidpath 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/droidpath-stream-and…] indexed:0 read:4min 2026-11-16 · —