{"slug": "i-made-an-ssh-server-that-lets-you-remote-into-a-pc-directly-from-your-terminal", "title": "I made an SSH server that lets you remote into a PC directly from your terminal", "summary": "Developer rylena released SSHDESK, an open-source tool that delivers a full interactive remote desktop through a standard SSH session, displayed inside the terminal without extra ports, browsers, or custom clients. The tool supports keyboard, mouse, resize events, and pixel updates over the SSH PTY, with sharp rendering for Kitty, Ghostty, and WezTerm and fallback for ANSI terminals. Installers for Linux, macOS, and Windows are available via one-line commands.", "body_md": "\n\n```\n       _____ _____ __  ______  ____________ __ __\n      / ___// ___// / / / __ \\/ ____/ ___// //_/\n      \\__ \\ \\__ \\/ /_/ / / / / __/  \\__ \\/ ,<\n     ___/ /___/ / __  / /_/ / /___ ___/ / /| |\n    /____//____/_/ /_/_____/_____//____/_/ |_|\n\n        YOUR DESKTOP  //  ONE SSH SESSION  //  ZERO EXTRA PORTS\n```\n\nAI coding agents must read [AGENTS.md](/rylena/sshdesk/blob/main/AGENTS.md) before modifying this\nrepository.\n\nSSHDESK is a full interactive remote desktop delivered entirely through an SSH session and displayed directly inside your terminal.\n\nConnect with the SSH client you already have:\n\n```\n# SSHDESK desktop (default)\nssh desktop@example.com\n\n# Normal login shell\nssh -t desktop@example.com shell\n\n# Explicitly select SSHDESK\nssh -t desktop@example.com desktop\n```\n\nOpenSSH authenticates the user and launches SSHDESK as a forced command. The active graphical desktop then appears inside that same terminal. Keyboard, mouse, resize events, changed pixels, and session cleanup all travel through the one SSH PTY. There is no browser, custom SSH client, VNC/RDP listener, second password database, web server, or additional network port.\n\nKitty, Ghostty, and WezTerm receive sharp real-pixel tiles. Every ordinary ANSI terminal receives the lower-resolution color-cell renderer, so OpenSSH, PuTTY, mobile clients, and embedded SSH terminals remain usable.\n\nWarning\n\nAnyone who can authenticate to an SSHDESK account can see and control the active graphical session. Treat it like physical console access. Keep a second administrative login available while configuring a forced command.\n\n- full desktop viewing with changed-tile/cell updates and static-frame suppression\n- keyboard, Ctrl/Alt/Shift, arrows, navigation keys, and F1–F12\n- mouse movement, left/right/middle click, drag, and wheel scrolling\n- dynamic terminal resize with aspect-ratio-preserving viewport recalculation\n- persistent top bar and terminal title showing the connected device name\n- sharp palette-compressed PNG tiles through Kitty graphics, including tmux passthrough\n- true-color, 256-color, 16-color, Unicode, and ASCII fallbacks\n- latest-frame scheduling that drops stale work instead of accumulating latency\n- 60 FPS sharp / 30 FPS ANSI active targets with adaptive idle presentation\n- live FPS, latency, capture, diff, bandwidth, and update instrumentation\n- agent-safe screenshot and computer-use commands carried through OpenSSH\n- optional tmux side-by-side layout for an agent shell and visual desktop\n- terminal restoration and held-input release after disconnects or crashes\n- X11, common Wayland desktop, macOS, and Windows backend abstractions\n\nThe bootstrap downloads the same SSHDESK release and selects the native installer automatically. On Linux or macOS, run this in a terminal:\n\n```\ncurl -fsSL https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.sh | sh\n```\n\nOn Windows, run this in PowerShell. It requests Administrator permission when needed:\n\n```\n& ([scriptblock]::Create((irm 'https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.ps1')))\n```\n\nBoth one-line entry points detect the OS, install missing Python/OpenSSH\nprerequisites, install SSHDESK, validate graphical access and the forced-command\nconfiguration, and start the platform's OpenSSH service. On Wayland, the Linux\ninstaller detects GNOME, KDE Plasma, or wlroots. GNOME uses one persistent\nMutter/PipeWire stream with compositor-native input; KDE and wlroots install a\ncapture command and checksum-verified `ydotoold`\n\nhelper. They support common Linux\ndistributions, macOS, and Windows 10/11. The installer asks whether to install\nand start Tailscale only after SSHDESK and OpenSSH setup succeeds.\nTailscale carries normal OpenSSH over the private tailnet; it does not replace\nOpenSSH or add a second SSH authentication mode.\n\nImportant\n\nCross-platform installation does not remove OS security boundaries. macOS still asks for Screen Recording and Accessibility access. Windows OpenSSH normally runs in Session 0, so Windows forced-command desktop capture remains experimental even though the one-line installer itself is supported. Any OS can be the SSH client; Linux remains the recommended SSHDESK host.\n\nNote\n\nA one-line installer executes downloaded code with administrator permission\nduring setup. Review [scripts/install.sh](/rylena/sshdesk/blob/main/scripts/install.sh) or\n[scripts/install.ps1](/rylena/sshdesk/blob/main/scripts/install.ps1) first if that is not appropriate\nfor the machine. On Linux/macOS, use `--user USER`\n\nwhen automatic user\ndetection is wrong.\n\nFor unattended installs, download the script and use `--tailscale`\n\nor\n`--no-tailscale`\n\n:\n\n```\ncurl -fsSLo /tmp/sshdesk-install.sh \\\n  https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.sh\nsh /tmp/sshdesk-install.sh --user alice --no-tailscale\n```\n\nWindows PowerShell accepts `-Tailscale`\n\nor `-NoTailscale`\n\non the downloaded\nscript block:\n\n```\n& ([scriptblock]::Create((irm 'https://raw.githubusercontent.com/rylena/sshdesk/main/scripts/install.ps1'))) -NoTailscale\n```\n\nIf an older installation closes with a Wayland capture error or behaves like a slow screenshot slideshow, log into that computer's graphical desktop, open its local terminal, and rerun the one-line command above. It upgrades GNOME to the persistent PipeWire backend, installs the correct compositor dependencies, checks a real frame, and preserves the existing SSHDESK login. Then retry the ordinary SSH command from the client.\n\nSSHDESK's installer is distribution-independent. It needs Python 3.10+, a\nworking Python `venv`\n\n, OpenSSH server, and the capture/input tools for the active\ndisplay stack:\n\n| Linux session | Capture | Input |\n|---|---|---|\n| X11, any desktop | FFmpeg/XCB, MIT-SHM, or Pillow/XCB | XTest |\n| wlroots (Sway, Hyprland, etc.) | `grim` |\n`ydotool` + `ydotoold` |\n| GNOME Wayland | persistent Mutter + PipeWire/GStreamer | Mutter RemoteDesktop API |\n| KDE Plasma Wayland | `spectacle` |\n`ydotool` + `ydotoold` |\n\nThe one-line installer handles these dependencies automatically. For a manual\ninstallation, GNOME needs PyGObject, GStreamer base introspection, and the\nGStreamer PipeWire plugin. Other Wayland desktops need their listed capture\ncommand and ydotool 1.0.4 or newer. FFmpeg and NumPy/OpenCV are X11 acceleration\npaths. Non-GNOME Wayland input requires `ydotoold`\n\naccess to `/dev/uinput`\n\n; do\nnot run the whole SSHDESK server as root.\n\nFrom the repository on the server:\n\n```\nsudo ./scripts/install-server.sh \\\n  \"$USER\" \"$DISPLAY\" \"${XAUTHORITY:-$HOME/.Xauthority}\"\n\n./scripts/configure-sshd.sh \"$USER\" |\n  sudo tee \"/etc/ssh/sshd_config.d/90-sshdesk-$USER.conf\"\nsudo sshd -t\nsudo systemctl reload ssh  # some distributions call this service sshd\n```\n\nUse the active display value (`:0`\n\n, `:1`\n\n, and so on). On Wayland, preserve the\nlogged-in graphical user's session variables when running the installer:\n\n```\nsudo --preserve-env=WAYLAND_DISPLAY,XDG_RUNTIME_DIR,XDG_SESSION_TYPE,\\\nXDG_CURRENT_DESKTOP,DBUS_SESSION_BUS_ADDRESS,YDOTOOL_SOCKET \\\n  ./scripts/install-server.sh \"$USER\" \"${DISPLAY:-}\" \"${XAUTHORITY:-}\"\n```\n\nThis records the compositor, runtime, D-Bus, and optional ydotool settings. Check the\nresulting root-owned `/etc/sshdesk/USER.conf`\n\nbefore enabling the forced command.\n\nVerify backend access first:\n\n```\n/usr/local/bin/sshdesk-server --check\n```\n\nThen connect from another terminal:\n\n```\nssh user@server\n```\n\nA PTY is required; `ssh -T`\n\ncannot display an interactive desktop. Press\n`Ctrl+] Ctrl+]`\n\nto leave.\n\nTo preserve a desktop owner's normal SSH shell, use a dedicated login and run only the tightly scoped server/agent entry points as the graphical user:\n\n```\nsudo useradd --create-home --shell /bin/bash sshdesk\nsudo ./scripts/install-server.sh \\\n  sshdesk :0 /home/alice/.Xauthority alice\n./scripts/configure-sshd.sh sshdesk |\n  sudo tee /etc/ssh/sshd_config.d/90-sshdesk.conf\nsudo sshd -t && sudo systemctl reload ssh\n```\n\nThe generated sudoers rule does not grant root. OpenSSH remains the only authentication system.\n\nPass `shell`\n\nas the remote command argument after the SSH destination:\n\n```\nssh -t user@server shell\n```\n\nPlain `ssh user@server`\n\ncontinues to open the desktop. The explicit equivalent\nis `ssh -t user@server desktop`\n\n. OpenSSH does not accept `--shell`\n\nas a local\noption; `shell`\n\nmust appear after `user@server`\n\nso it is sent to the forced\ncommand dispatcher.\n\nThe shell runs as the authenticated SSH account, never as a different `RUN_AS`\n\ndesktop owner. Existing forwarding restrictions remain in effect. Anyone who\ncan authenticate to this account can request the shell selector and receives\nthe same command access as an ordinary shell login.\n\nAn SSH client alias can make the shell connection look like a normal host:\n\n```\nHost server-shell\n    HostName server\n    User user\n    RequestTTY force\n    RemoteCommand shell\n```\n\nThen run `ssh server-shell`\n\nfor the shell and `ssh user@server`\n\nfor SSHDESK.\n\nThe forced-command account accepts a small fixed `sshdesk-agent`\n\ncommand set in\naddition to the interactive desktop. It never evaluates a received shell\nstring. Any AI agent that can run CLI commands and use SSH can connect; SSHDESK\ndoes not require a particular agent framework or model. Normal shell access and\nscripted actions at known coordinates do not require vision. To navigate an\nunfamiliar graphical desktop dynamically, the agent needs vision or a separate\nPNG analysis/OCR tool because observations contain screenshots rather than a\nsemantic accessibility tree. The remote host must have SSHDESK configured, and\nthe agent must have valid SSH credentials and network access. Examples:\n\n```\nssh user@server sshdesk-agent info\nssh user@server sshdesk-agent screenshot --max-width 1280 > desktop.png\nssh user@server sshdesk-agent move 900 500\nssh user@server sshdesk-agent click 900 500 --button left\nssh user@server sshdesk-agent scroll -3 900 500\nssh user@server sshdesk-agent type hello\nssh user@server sshdesk-agent key enter\n```\n\nFor reliable quoting and machine-readable responses, install SSHDESK locally\nand use `sshdesk-remote`\n\n. It sends bounded newline-delimited JSON to the fixed\nremote command:\n\n```\nsshdesk-remote user@server info\nsshdesk-remote user@server screenshot --output desktop.png\nsshdesk-remote user@server click 900 500\nsshdesk-remote user@server type 'text with spaces'\n```\n\nLong-running agents can avoid process setup for every action:\n\n```\nsshdesk-remote user@server session\n{\"id\":1,\"action\":\"observe\",\"max_width\":1280}\n{\"id\":2,\"action\":\"click\",\"x\":900,\"y\":500,\"button\":\"left\"}\n{\"id\":3,\"action\":\"type\",\"text\":\"hello\"}\n{\"id\":4,\"action\":\"quit\"}\n```\n\nTo place a local agent shell beside the remote visual desktop, install `tmux`\n\nand run:\n\n```\nsshdesk-split user@server\n```\n\nThe right pane is the normal SSHDESK connection; the left pane is available to\nyour agent or shell and can call `sshdesk-remote`\n\n. These optional automation\ncommands are also ordinary authenticated SSH sessions. Standard OpenSSH\n`ControlMaster`\n\nconfiguration can multiplex them over an existing connection;\nSSHDESK never opens another service or port.\n\n- type normally to send keyboard input\n- use the terminal mouse for movement, clicks, drag, and scrolling\n`Ctrl+S`\n\ntoggles statistics (most terminals cannot distinguish`Ctrl+Shift+S`\n\n)`Ctrl+] Ctrl+]`\n\nalways exits locally and is never injected- terminal resizing triggers a new viewport and full redraw without disconnecting\n\nThe installer writes safe defaults to `/etc/sshdesk/USER.conf`\n\n:\n\n```\nSSHDESK_RENDER=auto\nSSHDESK_COLOR=auto\nSSHDESK_MOUSE=auto\nSSHDESK_UNICODE=auto\nSSHDESK_X11_CAPTURE=auto\nSSHDESK_MAX_FPS=auto\nSSHDESK_SCALE=auto\n```\n\n`SSHDESK_RENDER=kitty`\n\nrequires sharp graphics; `ansi`\n\nforces the universal\nfallback. `SSHDESK_X11_CAPTURE=auto`\n\ntries continuously drained FFmpeg/XCB,\nthen MIT-SHM, then Pillow/XCB. `SSHDESK_MAX_FPS`\n\naccepts 1–120.\n`SSHDESK_SCALE=auto`\n\ndynamically reduces detail when the client terminal falls\nbehind. Fixed values from 0.25–1.0, such as 0.75, send fewer pixels all the time\nfor smoother sessions on slower clients or networks.\n\nLinux is the primary, fully integrated OpenSSH host. Native Pillow capture plus Quartz input on macOS and SendInput on Windows are available for development and manually launched sessions. The repository-local commands below are useful for development; most users should use the one-line installers above:\n\n```\n./scripts/install-macos.sh\npowershell -ExecutionPolicy Bypass -File scripts/install-windows.ps1\n```\n\nmacOS requires Screen Recording and Accessibility permission for the installed Python process. Windows hosting must execute inside the logged-in interactive desktop; the normal Windows OpenSSH service may be isolated in Session 0, so forced-command hosting there is experimental. Linux/macOS/Windows terminals are all supported as clients because the visual protocol remains standard terminal output over SSH.\n\nSee [platform support](/rylena/sshdesk/blob/main/docs/platforms.md) for exact backend behavior.\n\n```\npython3 -m venv .venv --system-site-packages\n. .venv/bin/activate\npython -m pip install -e '.[fast,dev]'\n\nsshdesk-server --capture synthetic --no-input\npython -m unittest discover -s tests -v\nruff check src tests\n```\n\nBenchmark exact rendered terminal bytes:\n\n```\nsshdesk-bench --duration 60 --columns 100 --rows 30 --color 256\n```\n\nPython keeps platform integration and iteration straightforward today. Capture, rendering, input, session management, and terminal output are separate modules, so performance-critical pieces can move to Rust later without changing the OpenSSH user experience.\n\n[Architecture and data flow](/rylena/sshdesk/blob/main/docs/architecture.md)[Platform support](/rylena/sshdesk/blob/main/docs/platforms.md)[Client and terminal compatibility](/rylena/sshdesk/blob/main/docs/compatibility.md)[Security and permissions](/rylena/sshdesk/blob/main/docs/security.md)[Benchmark methodology](/rylena/sshdesk/blob/main/docs/benchmark.md)[Changelog](/rylena/sshdesk/blob/main/CHANGELOG.md)\n\nMIT\n\nThe sharp renderer builds on the idea demonstrated by\n[Desktui](https://github.com/mishushakov/desktui): terminal image pixels and\nchanged tiles can preserve far more desktop detail than character art.", "url": "https://wpnews.pro/news/i-made-an-ssh-server-that-lets-you-remote-into-a-pc-directly-from-your-terminal", "canonical_source": "https://github.com/rylena/sshdesk", "published_at": "2026-08-20 19:42:11+00:00", "updated_at": "2026-08-20 20:16:00.751225+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools"], "entities": ["SSHDESK", "rylena", "OpenSSH", "Kitty", "Ghostty", "WezTerm", "Tailscale", "PuTTY"], "alternates": {"html": "https://wpnews.pro/news/i-made-an-ssh-server-that-lets-you-remote-into-a-pc-directly-from-your-terminal", "markdown": "https://wpnews.pro/news/i-made-an-ssh-server-that-lets-you-remote-into-a-pc-directly-from-your-terminal.md", "text": "https://wpnews.pro/news/i-made-an-ssh-server-that-lets-you-remote-into-a-pc-directly-from-your-terminal.txt", "jsonld": "https://wpnews.pro/news/i-made-an-ssh-server-that-lets-you-remote-into-a-pc-directly-from-your-terminal.jsonld"}}