{"slug": "ai-can-now-control-your-desktop", "title": "AI Can now control your desktop", "summary": "Clawdcursor, a new open-source desktop control tool, allows AI models to directly interact with computer interfaces through a single MCP entry point, supporting macOS, Windows, and Linux. The tool operates locally with no telemetry, using accessibility trees before resorting to vision, and provides six compact tools with a single safety chokepoint for destructive actions. It integrates with editors like Claude Code and Cursor, enabling AI agents to click by name, type by label, and read screen content without requiring screenshots or vision LLMs.", "body_md": "Any model. Any app. One MCP entry. Local-only. 6 compact tools, single safety chokepoint, no telemetry. Cheap paths first — accessibility before pixels, vision only as a last resort.\n\nPlain English in, actions out.\n\nOne MCP entry, desktop control appears as native tools.\n\nSame tools, three entry shapes. Pick once during install.\n\nAI lives in your editor (Claude Code, Cursor, Windsurf, Zed). Editor spawns clawdcursor on demand over stdio. No daemon, no port.\n\n```\n{\n  \"mcpServers\": {\n    \"clawdcursor\": {\n      \"command\": \"clawdcursor\",\n      \"args\": [\"mcp\", \"--compact\"]\n    }\n  }\n}\n```\n\nclawdcursor brings its own LLM brain (configured via `doctor`\n\n). For unattended runs, scheduled tasks, multi-process orchestration.\n\n`clawdcursor doctor`\n\n· pick a provider`clawdcursor agent`\n\n`127.0.0.1:3847/mcp`\n\nYour agent already has a brain — you just want HTTP tools. Same daemon, no built-in agent loop.\n\n`clawdcursor agent --no-llm`\n\n`:3847/mcp`\n\nA11y tree before pixels. Vision only when needed.\n\nRead the a11y tree, act on element names. No screenshot, no vision LLM.\n\nOCR when the tree is sparse, screenshot when you need pixels, vision only for canvas UIs.\n\nEvery tool call gates through one safety layer. Destructive actions need confirmation.\n\nThe recommended surface — `computer`\n\n, `accessibility`\n\n, `window`\n\n, `system`\n\n, `browser`\n\n, `task`\n\n. ~12× smaller catalog than the granular Tools surface.\n\nWindows, macOS, Linux behind a single interface. Linux covers X11 and Wayland.\n\nTCC-safe. `clawdcursor grant`\n\nhandles Accessibility + Screen Recording.\n\nNative UIA + Windows.Media.Ocr. x64 and ARM64.\n\nX11 and Wayland. AT-SPI for a11y, Tesseract for OCR.\n\nClick by name, type by label, read screen. A11y first, OCR as fallback.\n\nPlatform-aware key combos — Cmd on macOS, Ctrl elsewhere. No LLM cost.\n\nCollapse N deterministic tool calls into a single guarded, safety-gated batch. N calls → 1.\n\n`{ \"action\": \"…\" }`\n\n. The 94 granular tools (one schema per verb) are listed below for compatibility and debugging — use them when your runtime requires every primitive as a top-level MCP tool. (94 total.)\n**Compact form (recommended):** `computer({ \"action\": \"key\", \"combo\": \"mod+s\" })`\n\n— ~1,500 tokens of catalog.\n**Granular form (compat / debug):** `key_press({ \"key\": \"mod+s\" })`\n\n— 94 individual tools, one schema per verb.\nBoth produce identical effects through the same `safety.evaluate()`\n\nchokepoint.\nPass `--granular`\n\n(instead of `--compact`\n\n) to expose the granular surface over MCP.\nSee [schema.snapshot.json](https://github.com/AmrDab/clawdcursor/blob/main/schema.snapshot.json) for every parameter.\n\n```\n# Install & setup\nclawdcursor consent          # one-time desktop-control authorization\nclawdcursor grant            # macOS Accessibility + Screen Recording prompts\nclawdcursor doctor           # verify permissions, configure AI provider\nclawdcursor status           # readiness check (consent, permissions, AI config)\n\n# Run\nclawdcursor mcp              # stdio MCP server for editor hosts\nclawdcursor mcp --compact    # same, with 6 compound tools (recommended)\nclawdcursor agent            # HTTP MCP daemon at :3847/mcp, optional built-in LLM\nclawdcursor agent --no-llm   # tool surface only — your agent brings its own brain\nclawdcursor stop             # stop every running mode\nclawdcursor uninstall        # remove all clawdcursor config and data\n$\n```\n\nWorks on macOS, Linux, and Windows. The one-liner installs Node if needed, builds, and links the global shim; `npm`\n\nis the leanest. After install run `clawdcursor consent --accept`\n\nthen `clawdcursor doctor`\n\n. Pin a version with `VERSION=v1.0.0`\n\n.\n\nOpen source. Any model. Localhost only. No telemetry.", "url": "https://wpnews.pro/news/ai-can-now-control-your-desktop", "canonical_source": "https://clawdcursor.com", "published_at": "2026-06-06 23:14:25+00:00", "updated_at": "2026-06-06 23:46:51.403146+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-products", "artificial-intelligence", "ai-infrastructure"], "entities": ["Claude Code", "Cursor", "Windsurf", "Zed", "clawdcursor", "MCP", "TCC"], "alternates": {"html": "https://wpnews.pro/news/ai-can-now-control-your-desktop", "markdown": "https://wpnews.pro/news/ai-can-now-control-your-desktop.md", "text": "https://wpnews.pro/news/ai-can-now-control-your-desktop.txt", "jsonld": "https://wpnews.pro/news/ai-can-now-control-your-desktop.jsonld"}}