# I made an SSH server that lets you remote into a PC directly from your terminal > Source: > Published: 2026-08-20 19:42:11+00:00 ``` _____ _____ __ ______ ____________ __ __ / ___// ___// / / / __ \/ ____/ ___// //_/ \__ \ \__ \/ /_/ / / / / __/ \__ \/ ,< ___/ /___/ / __ / /_/ / /___ ___/ / /| | /____//____/_/ /_/_____/_____//____/_/ |_| YOUR DESKTOP // ONE SSH SESSION // ZERO EXTRA PORTS ``` AI coding agents must read [AGENTS.md](/rylena/sshdesk/blob/main/AGENTS.md) before modifying this repository. SSHDESK is a full interactive remote desktop delivered entirely through an SSH session and displayed directly inside your terminal. Connect with the SSH client you already have: ``` # SSHDESK desktop (default) ssh desktop@example.com # Normal login shell ssh -t desktop@example.com shell # Explicitly select SSHDESK ssh -t desktop@example.com desktop ``` OpenSSH authenticates the user and launches SSHDESK as a forced command. The active graphical desktop then appears inside that same terminal. Keyboard, mouse, resize events, changed pixels, and session cleanup all travel through the one SSH PTY. There is no browser, custom SSH client, VNC/RDP listener, second password database, web server, or additional network port. Kitty, Ghostty, and WezTerm receive sharp real-pixel tiles. Every ordinary ANSI terminal receives the lower-resolution color-cell renderer, so OpenSSH, PuTTY, mobile clients, and embedded SSH terminals remain usable. Warning Anyone who can authenticate to an SSHDESK account can see and control the active graphical session. Treat it like physical console access. Keep a second administrative login available while configuring a forced command. - full desktop viewing with changed-tile/cell updates and static-frame suppression - keyboard, Ctrl/Alt/Shift, arrows, navigation keys, and F1–F12 - mouse movement, left/right/middle click, drag, and wheel scrolling - dynamic terminal resize with aspect-ratio-preserving viewport recalculation - persistent top bar and terminal title showing the connected device name - sharp palette-compressed PNG tiles through Kitty graphics, including tmux passthrough - true-color, 256-color, 16-color, Unicode, and ASCII fallbacks - latest-frame scheduling that drops stale work instead of accumulating latency - 60 FPS sharp / 30 FPS ANSI active targets with adaptive idle presentation - live FPS, latency, capture, diff, bandwidth, and update instrumentation - agent-safe screenshot and computer-use commands carried through OpenSSH - optional tmux side-by-side layout for an agent shell and visual desktop - terminal restoration and held-input release after disconnects or crashes - X11, common Wayland desktop, macOS, and Windows backend abstractions The bootstrap downloads the same SSHDESK release and selects the native installer automatically. On Linux or macOS, run this in a terminal: ``` curl -fsSL https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.sh | sh ``` On Windows, run this in PowerShell. It requests Administrator permission when needed: ``` & ([scriptblock]::Create((irm 'https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.ps1'))) ``` Both one-line entry points detect the OS, install missing Python/OpenSSH prerequisites, install SSHDESK, validate graphical access and the forced-command configuration, and start the platform's OpenSSH service. On Wayland, the Linux installer detects GNOME, KDE Plasma, or wlroots. GNOME uses one persistent Mutter/PipeWire stream with compositor-native input; KDE and wlroots install a capture command and checksum-verified `ydotoold` helper. They support common Linux distributions, macOS, and Windows 10/11. The installer asks whether to install and start Tailscale only after SSHDESK and OpenSSH setup succeeds. Tailscale carries normal OpenSSH over the private tailnet; it does not replace OpenSSH or add a second SSH authentication mode. Important Cross-platform installation does not remove OS security boundaries. macOS still asks for Screen Recording and Accessibility access. Windows OpenSSH normally runs in Session 0, so Windows forced-command desktop capture remains experimental even though the one-line installer itself is supported. Any OS can be the SSH client; Linux remains the recommended SSHDESK host. Note A one-line installer executes downloaded code with administrator permission during setup. Review [scripts/install.sh](/rylena/sshdesk/blob/main/scripts/install.sh) or [scripts/install.ps1](/rylena/sshdesk/blob/main/scripts/install.ps1) first if that is not appropriate for the machine. On Linux/macOS, use `--user USER` when automatic user detection is wrong. For unattended installs, download the script and use `--tailscale` or `--no-tailscale` : ``` curl -fsSLo /tmp/sshdesk-install.sh \ https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.sh sh /tmp/sshdesk-install.sh --user alice --no-tailscale ``` Windows PowerShell accepts `-Tailscale` or `-NoTailscale` on the downloaded script block: ``` & ([scriptblock]::Create((irm 'https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.ps1'))) -NoTailscale ``` If an older installation closes with a Wayland capture error or behaves like a slow screenshot slideshow, log into that computer's graphical desktop, open its local terminal, and rerun the one-line command above. It upgrades GNOME to the persistent PipeWire backend, installs the correct compositor dependencies, checks a real frame, and preserves the existing SSHDESK login. Then retry the ordinary SSH command from the client. SSHDESK's installer is distribution-independent. It needs Python 3.10+, a working Python `venv` , OpenSSH server, and the capture/input tools for the active display stack: | Linux session | Capture | Input | |---|---|---| | X11, any desktop | FFmpeg/XCB, MIT-SHM, or Pillow/XCB | XTest | | wlroots (Sway, Hyprland, etc.) | `grim` | `ydotool` + `ydotoold` | | GNOME Wayland | persistent Mutter + PipeWire/GStreamer | Mutter RemoteDesktop API | | KDE Plasma Wayland | `spectacle` | `ydotool` + `ydotoold` | The one-line installer handles these dependencies automatically. For a manual installation, GNOME needs PyGObject, GStreamer base introspection, and the GStreamer PipeWire plugin. Other Wayland desktops need their listed capture command and ydotool 1.0.4 or newer. FFmpeg and NumPy/OpenCV are X11 acceleration paths. Non-GNOME Wayland input requires `ydotoold` access to `/dev/uinput` ; do not run the whole SSHDESK server as root. From the repository on the server: ``` sudo ./scripts/install-server.sh \ "$USER" "$DISPLAY" "${XAUTHORITY:-$HOME/.Xauthority}" ./scripts/configure-sshd.sh "$USER" | sudo tee "/etc/ssh/sshd_config.d/90-sshdesk-$USER.conf" sudo sshd -t sudo systemctl reload ssh # some distributions call this service sshd ``` Use the active display value (`:0` , `:1` , and so on). On Wayland, preserve the logged-in graphical user's session variables when running the installer: ``` sudo --preserve-env=WAYLAND_DISPLAY,XDG_RUNTIME_DIR,XDG_SESSION_TYPE,\ XDG_CURRENT_DESKTOP,DBUS_SESSION_BUS_ADDRESS,YDOTOOL_SOCKET \ ./scripts/install-server.sh "$USER" "${DISPLAY:-}" "${XAUTHORITY:-}" ``` This records the compositor, runtime, D-Bus, and optional ydotool settings. Check the resulting root-owned `/etc/sshdesk/USER.conf` before enabling the forced command. Verify backend access first: ``` /usr/local/bin/sshdesk-server --check ``` Then connect from another terminal: ``` ssh user@server ``` A PTY is required; `ssh -T` cannot display an interactive desktop. Press `Ctrl+] Ctrl+]` to leave. To preserve a desktop owner's normal SSH shell, use a dedicated login and run only the tightly scoped server/agent entry points as the graphical user: ``` sudo useradd --create-home --shell /bin/bash sshdesk sudo ./scripts/install-server.sh \ sshdesk :0 /home/alice/.Xauthority alice ./scripts/configure-sshd.sh sshdesk | sudo tee /etc/ssh/sshd_config.d/90-sshdesk.conf sudo sshd -t && sudo systemctl reload ssh ``` The generated sudoers rule does not grant root. OpenSSH remains the only authentication system. Pass `shell` as the remote command argument after the SSH destination: ``` ssh -t user@server shell ``` Plain `ssh user@server` continues to open the desktop. The explicit equivalent is `ssh -t user@server desktop` . OpenSSH does not accept `--shell` as a local option; `shell` must appear after `user@server` so it is sent to the forced command dispatcher. The shell runs as the authenticated SSH account, never as a different `RUN_AS` desktop owner. Existing forwarding restrictions remain in effect. Anyone who can authenticate to this account can request the shell selector and receives the same command access as an ordinary shell login. An SSH client alias can make the shell connection look like a normal host: ``` Host server-shell HostName server User user RequestTTY force RemoteCommand shell ``` Then run `ssh server-shell` for the shell and `ssh user@server` for SSHDESK. The forced-command account accepts a small fixed `sshdesk-agent` command set in addition to the interactive desktop. It never evaluates a received shell string. Any AI agent that can run CLI commands and use SSH can connect; SSHDESK does not require a particular agent framework or model. Normal shell access and scripted actions at known coordinates do not require vision. To navigate an unfamiliar graphical desktop dynamically, the agent needs vision or a separate PNG analysis/OCR tool because observations contain screenshots rather than a semantic accessibility tree. The remote host must have SSHDESK configured, and the agent must have valid SSH credentials and network access. Examples: ``` ssh user@server sshdesk-agent info ssh user@server sshdesk-agent screenshot --max-width 1280 > desktop.png ssh user@server sshdesk-agent move 900 500 ssh user@server sshdesk-agent click 900 500 --button left ssh user@server sshdesk-agent scroll -3 900 500 ssh user@server sshdesk-agent type hello ssh user@server sshdesk-agent key enter ``` For reliable quoting and machine-readable responses, install SSHDESK locally and use `sshdesk-remote` . It sends bounded newline-delimited JSON to the fixed remote command: ``` sshdesk-remote user@server info sshdesk-remote user@server screenshot --output desktop.png sshdesk-remote user@server click 900 500 sshdesk-remote user@server type 'text with spaces' ``` Long-running agents can avoid process setup for every action: ``` sshdesk-remote user@server session {"id":1,"action":"observe","max_width":1280} {"id":2,"action":"click","x":900,"y":500,"button":"left"} {"id":3,"action":"type","text":"hello"} {"id":4,"action":"quit"} ``` To place a local agent shell beside the remote visual desktop, install `tmux` and run: ``` sshdesk-split user@server ``` The right pane is the normal SSHDESK connection; the left pane is available to your agent or shell and can call `sshdesk-remote` . These optional automation commands are also ordinary authenticated SSH sessions. Standard OpenSSH `ControlMaster` configuration can multiplex them over an existing connection; SSHDESK never opens another service or port. - type normally to send keyboard input - use the terminal mouse for movement, clicks, drag, and scrolling `Ctrl+S` toggles statistics (most terminals cannot distinguish`Ctrl+Shift+S` )`Ctrl+] Ctrl+]` always exits locally and is never injected- terminal resizing triggers a new viewport and full redraw without disconnecting The installer writes safe defaults to `/etc/sshdesk/USER.conf` : ``` SSHDESK_RENDER=auto SSHDESK_COLOR=auto SSHDESK_MOUSE=auto SSHDESK_UNICODE=auto SSHDESK_X11_CAPTURE=auto SSHDESK_MAX_FPS=auto SSHDESK_SCALE=auto ``` `SSHDESK_RENDER=kitty` requires sharp graphics; `ansi` forces the universal fallback. `SSHDESK_X11_CAPTURE=auto` tries continuously drained FFmpeg/XCB, then MIT-SHM, then Pillow/XCB. `SSHDESK_MAX_FPS` accepts 1–120. `SSHDESK_SCALE=auto` dynamically reduces detail when the client terminal falls behind. Fixed values from 0.25–1.0, such as 0.75, send fewer pixels all the time for smoother sessions on slower clients or networks. Linux is the primary, fully integrated OpenSSH host. Native Pillow capture plus Quartz input on macOS and SendInput on Windows are available for development and manually launched sessions. The repository-local commands below are useful for development; most users should use the one-line installers above: ``` ./scripts/install-macos.sh powershell -ExecutionPolicy Bypass -File scripts/install-windows.ps1 ``` macOS requires Screen Recording and Accessibility permission for the installed Python process. Windows hosting must execute inside the logged-in interactive desktop; the normal Windows OpenSSH service may be isolated in Session 0, so forced-command hosting there is experimental. Linux/macOS/Windows terminals are all supported as clients because the visual protocol remains standard terminal output over SSH. See [platform support](/rylena/sshdesk/blob/main/docs/platforms.md) for exact backend behavior. ``` python3 -m venv .venv --system-site-packages . .venv/bin/activate python -m pip install -e '.[fast,dev]' sshdesk-server --capture synthetic --no-input python -m unittest discover -s tests -v ruff check src tests ``` Benchmark exact rendered terminal bytes: ``` sshdesk-bench --duration 60 --columns 100 --rows 30 --color 256 ``` Python keeps platform integration and iteration straightforward today. Capture, rendering, input, session management, and terminal output are separate modules, so performance-critical pieces can move to Rust later without changing the OpenSSH user experience. [Architecture and data flow](/rylena/sshdesk/blob/main/docs/architecture.md)[Platform support](/rylena/sshdesk/blob/main/docs/platforms.md)[Client and terminal compatibility](/rylena/sshdesk/blob/main/docs/compatibility.md)[Security and permissions](/rylena/sshdesk/blob/main/docs/security.md)[Benchmark methodology](/rylena/sshdesk/blob/main/docs/benchmark.md)[Changelog](/rylena/sshdesk/blob/main/CHANGELOG.md) MIT The sharp renderer builds on the idea demonstrated by [Desktui](https://github.com/mishushakov/desktui): terminal image pixels and changed tiles can preserve far more desktop detail than character art.