Cua: Open-Source Computer-Use Infrastructure & Drivers for AI Agents

An open-source project called Cua has released infrastructure for building computer-use AI agents, including cross-OS desktop drivers for macOS, Windows, and Linux that deliver background input events without hijacking the user's mouse or focus. The platform bundles sandboxed cloud desktop fleets, a lightweight CLI for managing local macOS and Linux VMs on Apple Silicon via Virtualization.Framework, and CUA-S1, a family of compact models that make fast UI element decisions from accessibility trees. Cua positions itself as an automation layer that agents such as Claude Code, Cursor, and Codex can plug into through CLI, MCP, or typed SDKs.

The landscape of AI-assisted software development is undergoing a fundamental evolution. For the past two years, AI coding assistants have operated almost entirely inside terminal buffers and text editors. However, practical software engineering tasks frequently require interacting with graphical user interfaces: verifying layout behavior in browsers, inspecting spreadsheet data, configuring desktop virtualization tools, or triaging issues across native desktop applications. This evolution is known as Computer-Use 2.0 —a paradigm where an AI agent dynamically navigates between executing shell scripts, calling REST APIs, and operating graphical applications within a single task lifecycle. Until now, developers building computer-use agents faced severe hurdles: OS-level GUI drivers were brittle, headless environments lacked display servers, and agents frequently hijacked the developer's physical mouse cursor. Cua trycua/cua is an open-source infrastructure platform designed from the ground up to solve these challenges. Here is a comprehensive breakdown of Cua's architecture, its cross-OS desktop drivers, sandboxed cloud fleets, and how it connects to modern AI agents. Cua does not attempt to be another monolithic AI agent. Instead, it acts as the operating system and automation layer that any AI agent or model can plug into. ┌─────────────────────────────────────────────────────────────┐ │ AI Agents & Models Claude Code, Cursor, Codex, CUA-S1 │ └──────────────────────────────┬──────────────────────────────┘ │ CLI / MCP / Typed SDKs ▼ ┌─────────────────────────────────────────────────────────────┐ │ Cua Driver │ │ Background event delivery for macOS, Windows, Linux │ └──────────────────────────────┬──────────────────────────────┘ │ ┌────────────────────┼────────────────────┐ ▼ ▼ ▼ Cua Fleets Lume VMs Local Desktop Isolated Cloud Apple Silicon VM Uninterrupted BG Cua is organized into five tightly integrated modules: Traditional OS-level automation tools like PyAutoGUI simulate physical hardware events. The moment an agent begins interacting with an app, your mouse jumps across the display, windows pop into the foreground, and you cannot type without interrupting the agent. Cua Driver introduces native OS-level integration across macOS, Windows, and Linux that supports background delivery . Where supported by the platform, the agent can click buttons, type values, and inspect native windows without moving your mouse pointer or stealing focus from your active code editor. For untrusted agent execution or scalable evaluation runs, running actions on your local physical workstation is undesirable. Cua Fleets allows developers to provision isolated, sandboxed Linux cloud desktops on-demand. Using the Sandbox SDK, an agent can: General-purpose LLMs like Claude 3.5 Sonnet or GPT-4o are incredible at high-level reasoning, but using them token-by-token to decide which form input to click is slow, expensive, and wasteful. Cua introduces CUA-S1 , a family of compact, specialized models designed for fast, bounded interface decisions. The initial research model CUA-S1-FORMS scores UI element decisions directly from structured accessibility trees and document elements, providing sub-second decision speed at a fraction of the compute cost. Running local macOS VMs has historically required cumbersome third-party software. Lume is a lightweight CLI tool that manages local macOS and Linux virtual machines on Apple Silicon using Apple's native Virtualization.Framework . Developers can spin up clean macOS Tahoe VMs directly from Apple restore images and connect via SSH in seconds. To build reliable computer-use agents, engineers need reproducible benchmarks. Cua Bench provides a framework to define desktop tasks, run reference solutions, verify task completion with evaluators, and export step-by-step trajectories to train future models. Cua Driver can be installed via a one-line installer across all major operating systems: /bin/bash -c "$ curl -fsSL https://cua.ai/driver/install.sh " irm https://cua.ai/driver/install.ps1 | iex If you are already using Claude Code , Codex , Cursor , or Antigravity , you can instruct your agent to install and integrate Cua automatically: Set up browser-skill and cua-driver on this machine by following https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/README.md Once installed, agents can interact with native desktop apps using clean CLI commands or through Model Context Protocol MCP server endpoints. You can run automated computer-use tasks without spinning up heavy cloud infrastructure using Python 3.12+ and uv : Install Cua Bench with browser support uv tool install 'cua-bench browser ' uv tool run --from 'cua-bench browser ' playwright install chromium Run and verify a reference task cua-bench run --task example-task The evaluator verifies agent interaction, checks UI state transitions, and outputs a normalized reward score along with action logs. As agents progress from simple code-generation scripts to comprehensive engineering partners, they require the full breadth of computing capabilities available to humans. By providing cross-platform background drivers, sandboxed cloud fleets, and specialized decision models under an open-source MIT license, Cua provides the foundational infrastructure layer needed to make Computer-Use 2.0 a reliable reality.