{"slug": "screen-relay-stream-and-control-any-desktop-at-120fps-from-a-browser-or-a-cli", "title": "Screen-relay – stream and control any desktop at 120fps from a browser or a CLI", "summary": "Screen-relay, a new desktop streaming and control tool distributed as a Node package, streams a Linux, macOS or Windows desktop to a browser as H.264 at up to 120fps and lets AI agents click, type and take screenshots one command at a time through a CLI, Node SDK, and HTTP/WebSocket API. The tool requires Node 22 or newer, starts a virtual X11/XFCE or headless Wayland/sway session on Linux, attaches to the logged-in display on macOS and Windows 10+ x64, and listens on 127.0.0.1:8080 with no token by default unless started with --public or on another address. Only one controller sends input at a time, and a successful input command confirms the engine applied the input, not that the app reacted, so a screenshot is needed to verify results.", "body_md": "Desktop streaming and control for humans and AI agents: watch a desktop in a browser, and click, type, and take screenshots from the shell.\n\nscreen-relay starts a desktop on a Linux VM, sandbox, or container, or attaches to the display of a Mac or the signed-in desktop of a Windows PC, and streams it to a browser as H.264. An agent controls the same desktop one command at a time: it takes a screenshot, clicks or types, and takes another screenshot to check the result. The `screen-relay` CLI, a Node SDK, and an HTTP and WebSocket API all reach the same session.\n\n```\nnpm install -g screen-relay\nscreen-relay install            # system packages and the engine build (first time only)\n```\n\n- **Requirements:** Node 22 or newer. On Linux: Debian, Ubuntu, Fedora, Arch Linux, or Alpine. On macOS: Homebrew and the Xcode Command Line Tools. On Windows: Windows 10 or later on x64, with a user signed in at the PC's screen.\n- **Scripts and agents:**`screen-relay install --yes` skips the confirmation. At a terminal,`install` asks before it installs packages.\n- **Wayland:**`screen-relay install --backend wayland` . The default backend on Linux is X11.\n- **Client only:** the npm install alone is enough to control a session on another machine.\n\nRequirements, backends, and Windows are covered in [Getting started](https://screen-relay.dev/docs/getting-started).\n\n```\nscreen-relay up -d                    # start a display, a desktop, and the engine\nscreen-relay run -- xfce4-terminal    # open an app on the desktop\nscreen-relay screenshot screen.png    # see the screen\nscreen-relay click 0.5 0.5            # coordinates are fractions of the screen\nscreen-relay type \"echo hello\"\nscreen-relay key Enter\nscreen-relay down\n```\n\n- `up` prints a viewer link. Open it in a browser with H.264 WebCodecs: current Chrome, Edge, or Safari.\n- The engine listens on `127.0.0.1:8080` . From another machine, forward the port with`ssh -N -L 8080:127.0.0.1:8080 user@your-vm` , or see[Remote access](https://screen-relay.dev/docs/remote-access) .\n- On a machine that is not set up yet, `up` runs the same setup as`install` first.\n- A successful input command means the engine applied the input, not that the app reacted. Take a screenshot to check.\n\nThe full walkthrough is in [Getting started](https://screen-relay.dev/docs/getting-started).\n\n`screen-relay up` starts a session: a virtual display, a desktop, and the engine. On Linux the display is X11 with XFCE or headless Wayland with sway. On a Mac the engine attaches to the display you are logged in to. On Windows it attaches to the desktop of the signed-in user. The engine captures the screen, encodes it as H.264, and serves the viewer, the stream, and the input API on one port. It lowers the bitrate on a slow link. See [How streaming works](https://screen-relay.dev/docs/how-streaming-works).\n\nA session listens on loopback with no token by default. Starting it with `--public` or on another address creates a token that every remote request must send. Only one controller sends input at a time: a viewer that took control, an SDK control session, or the CLI. A viewer opens view-only, so it can watch while the CLI or SDK sends input. Screenshots still work while another controller holds input.\n\nThe CLI ships skills that match its version:\n\n```\nscreen-relay skills get core           # the control loop, coordinates, type versus key, and errors\nscreen-relay skills get core --full    # adds every reference: CLI, config, SDK, macOS, Wayland, troubleshooting\nscreen-relay skills list               # specialized skills, such as vercel-sandbox\n```\n\nOr add this to your `AGENTS.md`:\n\n```\nUse `screen-relay` to see and control a desktop. Before the first use, run\n`screen-relay skills get core` and follow it.\n```\n\nEvery command prints JSON with `--json`. An error carries a `reason` and, when there is one, the next command to run.\n\nRun `screen-relay --help` for the full list, or `screen-relay <command> --help` for one command.\n\n| Group | Commands | Docs | \n|---|---|---|\n| Session | `up` ,`down` ,`status` ,`logs` ,`run` | [CLI reference](https://screen-relay.dev/docs/cli) | \n| Control | `screenshot` ,`click` ,`move` ,`scroll` ,`key` ,`type` ,`input` | [Client commands](https://screen-relay.dev/docs/cli#client-commands) | \n| Setup | `install` ,`doctor` | [Getting started](https://screen-relay.dev/docs/getting-started) ,[Troubleshooting](https://screen-relay.dev/docs/troubleshooting) | \n| Agents | `skills` ,`help` | [Agent skills](https://screen-relay.dev/docs/agent-skills) | \n\nFlags, environment variables, and `config.yaml` are documented in [Configuration](https://screen-relay.dev/docs/configuration).\n\nThe engine runs on Linux (x64 and ARM64, X11 and Wayland), on macOS 12.3 or later (Apple silicon and Intel), and on Windows 10 or later (x64). `screen-relay install` builds it on your machine. See [Linux](https://screen-relay.dev/docs/linux), [macOS](https://screen-relay.dev/docs/macos), and [Windows](https://screen-relay.dev/docs/windows).\n\nBuilding from source requires Node.js 22 or later, a C++17 compiler, `make`, and the libraries that `screen-relay install` installs:\n\n```\nnpm ci\nnpm run build                   # the CLI, the SDK, and the viewer\nmake                            # the engine, for the default backend\nnpm test\n```\n\nRun the engine tests with `make test`, which also needs the FFmpeg development libraries.\n\n[CONTRIBUTING.md](https://github.com/vercel-labs/screen-relay/blob/main/CONTRIBUTING.md) covers the source layout, the engine, and the tests. [AGENTS.md](https://github.com/vercel-labs/screen-relay/blob/main/AGENTS.md) covers the writing rules and the release process.\n\n- [screen-relay.dev](https://screen-relay.dev) : introduction, guides, and reference\n- [CLI reference](https://screen-relay.dev/docs/cli) and[Configuration](https://screen-relay.dev/docs/configuration)\n- [Node SDK](https://screen-relay.dev/docs/sdk) and[HTTP API](https://screen-relay.dev/docs/protocol)\n- [Remote access](https://screen-relay.dev/docs/remote-access) : tokens, TLS, and tunnels\n- [Docker](https://screen-relay.dev/docs/docker) and[Vercel Sandbox](https://screen-relay.dev/docs/vercel-sandbox)\n- [Changelog](https://github.com/vercel-labs/screen-relay/blob/main/CHANGELOG.md)\n\nApache-2.0. The engine links x264, which is licensed under GPL-2.0-or-later, from your system when `screen-relay install` builds it. See [THIRD-PARTY-NOTICES.md](https://github.com/vercel-labs/screen-relay/blob/main/THIRD-PARTY-NOTICES.md).", "url": "https://wpnews.pro/news/screen-relay-stream-and-control-any-desktop-at-120fps-from-a-browser-or-a-cli", "canonical_source": "https://github.com/vercel-labs/screen-relay", "published_at": "2026-11-19 08:00:02+00:00", "updated_at": "2026-11-19 08:15:00.621545+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-tools"], "entities": ["screen-relay", "Node.js", "X11", "Wayland", "XFCE", "sway", "Windows 10", "macOS"], "also_reported_by": [], "takeaway": "Screen-relay exposes one desktop session to a CLI, Node SDK and HTTP/WebSocket API, but a successful input command only confirms the engine applied the input, so agents must take a screenshot after each click or keystroke to verify the app reacted.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/screen-relay-stream-and-control-any-desktop-at-120fps-from-a-browser-or-a-cli", "markdown": "https://wpnews.pro/news/screen-relay-stream-and-control-any-desktop-at-120fps-from-a-browser-or-a-cli.md", "text": "https://wpnews.pro/news/screen-relay-stream-and-control-any-desktop-at-120fps-from-a-browser-or-a-cli.txt", "jsonld": "https://wpnews.pro/news/screen-relay-stream-and-control-any-desktop-at-120fps-from-a-browser-or-a-cli.jsonld"}}