cd /news/ai-agents/screen-relay-stream-and-control-any-… · home › topics › ai-agents › article
[ARTICLE · art-172418] src=github.com ↗ pub= topic=ai-agents verified=true sentiment=· neutral

Screen-relay – stream and control any desktop at 120fps from a browser or a CLI

Screen-relay, a new desktop streaming and control tool distributed as a Node package, streams a Linux, macOS or Windows desktop to a browser as H.264 at up to 120fps and lets AI agents click, type and take screenshots one command at a time through a CLI, Node SDK, and HTTP/WebSocket API. The tool requires Node 22 or newer, starts a virtual X11/XFCE or headless Wayland/sway session on Linux, attaches to the logged-in display on macOS and Windows 10+ x64, and listens on 127.0.0.1:8080 with no token by default unless started with --public or on another address. Only one controller sends input at a time, and a successful input command confirms the engine applied the input, not that the app reacted, so a screenshot is needed to verify results.

key takeaway

Screen-relay exposes one desktop session to a CLI, Node SDK and HTTP/WebSocket API, but a successful input command only confirms the engine applied the input, so agents must take a screenshot after each click or keystroke to verify the app reacted.

read4 min views6 publishedNov 19, 2026
Screen-relay – stream and control any desktop at 120fps from a browser or a CLI
Image: Michielbdejong (auto-discovered)

Desktop streaming and control for humans and AI agents: watch a desktop in a browser, and click, type, and take screenshots from the shell.

screen-relay starts a desktop on a Linux VM, sandbox, or container, or attaches to the display of a Mac or the signed-in desktop of a Windows PC, and streams it to a browser as H.264. An agent controls the same desktop one command at a time: it takes a screenshot, clicks or types, and takes another screenshot to check the result. The screen-relay CLI, a Node SDK, and an HTTP and WebSocket API all reach the same session.

npm install -g screen-relay
screen-relay install            # system packages and the engine build (first time only)
  • Requirements: Node 22 or newer. On Linux: Debian, Ubuntu, Fedora, Arch Linux, or Alpine. On macOS: Homebrew and the Xcode Command Line Tools. On Windows: Windows 10 or later on x64, with a user signed in at the PC's screen.
  • Scripts and agents:screen-relay install --yes skips the confirmation. At a terminal,install asks before it installs packages.
  • Wayland:screen-relay install --backend wayland . The default backend on Linux is X11.
  • Client only: the npm install alone is enough to control a session on another machine.

Requirements, backends, and Windows are covered in Getting started.

screen-relay up -d                    # start a display, a desktop, and the engine
screen-relay run -- xfce4-terminal    # open an app on the desktop
screen-relay screenshot screen.png    # see the screen
screen-relay click 0.5 0.5            # coordinates are fractions of the screen
screen-relay type "echo hello"
screen-relay key Enter
screen-relay down
  • up prints a viewer link. Open it in a browser with H.264 WebCodecs: current Chrome, Edge, or Safari.
  • The engine listens on 127.0.0.1:8080 . From another machine, forward the port withssh -N -L 8080:127.0.0.1:8080 user@your-vm , or seeRemote access .
  • On a machine that is not set up yet, up runs the same setup asinstall first.
  • A successful input command means the engine applied the input, not that the app reacted. Take a screenshot to check.

The full walkthrough is in Getting started.

screen-relay up starts a session: a virtual display, a desktop, and the engine. On Linux the display is X11 with XFCE or headless Wayland with sway. On a Mac the engine attaches to the display you are logged in to. On Windows it attaches to the desktop of the signed-in user. The engine captures the screen, encodes it as H.264, and serves the viewer, the stream, and the input API on one port. It lowers the bitrate on a slow link. See How streaming works.

A session listens on loopback with no token by default. Starting it with --public or on another address creates a token that every remote request must send. Only one controller sends input at a time: a viewer that took control, an SDK control session, or the CLI. A viewer opens view-only, so it can watch while the CLI or SDK sends input. Screenshots still work while another controller holds input.

The CLI ships skills that match its version:

screen-relay skills get core           # the control loop, coordinates, type versus key, and errors
screen-relay skills get core --full    # adds every reference: CLI, config, SDK, macOS, Wayland, troubleshooting
screen-relay skills list               # specialized skills, such as vercel-sandbox

Or add this to your AGENTS.md:

Use `screen-relay` to see and control a desktop. Before the first use, run
`screen-relay skills get core` and follow it.

Every command prints JSON with --json. An error carries a reason and, when there is one, the next command to run.

Run screen-relay --help for the full list, or screen-relay <command> --help for one command.

Group Commands Docs
Session up ,down ,status ,logs ,run CLI reference
Control screenshot ,click ,move ,scroll ,key ,type ,input Client commands
Setup install ,doctor Getting started ,Troubleshooting
Agents skills ,help Agent skills

Flags, environment variables, and config.yaml are documented in Configuration.

The engine runs on Linux (x64 and ARM64, X11 and Wayland), on macOS 12.3 or later (Apple silicon and Intel), and on Windows 10 or later (x64). screen-relay install builds it on your machine. See Linux, macOS, and Windows.

Building from source requires Node.js 22 or later, a C++17 compiler, make, and the libraries that screen-relay install installs:

npm ci
npm run build                   # the CLI, the SDK, and the viewer
make                            # the engine, for the default backend
npm test

Run the engine tests with make test, which also needs the FFmpeg development libraries.

CONTRIBUTING.md covers the source layout, the engine, and the tests. AGENTS.md covers the writing rules and the release process.

Apache-2.0. The engine links x264, which is licensed under GPL-2.0-or-later, from your system when screen-relay install builds it. See THIRD-PARTY-NOTICES.md.

── more in #ai-agents 4 stories · sorted by recency
── more on @screen-relay 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/screen-relay-stream-…] indexed:0 read:4min 2026-11-19 · —