# Give Your AI Agent Eyes and Hands on a Remote Desktop with rdp-cli

> Source: <https://dev.to/weerocks/give-your-ai-agent-eyes-and-hands-on-a-remote-desktop-with-rdp-cli-182e>
> Published: 2026-10-22 07:35:57+00:00

AI agents are good at generating text and calling APIs. But many real-world tasks still require interacting with graphical applications: editing a video, checking a spreadsheet, using legacy Windows software, or operating an Android phone.

`rdp-cli` provides a simple command-line workflow for letting an AI agent control RDP and VNC desktops—and verify every action through screenshots.

On Linux x86_64, download the executable and make it runnable:

```
chmod u+x ./rdp-cli-linux-x86_64
./rdp-cli-linux-x86_64 --help
```

No additional runtime or administrator access is required. For Windows and macOS, see the [quick start guide](https://rdp-cli.com/en/docs/quick-start/).

Make sure RDP is enabled on the target computer. Then connect with your username and password:

```
IFS= read -r -s -p 'Password: ' rdp_password
printf '\n'
printf '%s' "$rdp_password" | \
  ./rdp-cli-linux-x86_64 connect \
  --host 192.0.2.20 \
  --user USER \
  --password-stdin

unset rdp_password
```

The command returns a session identifier. Save it as `SESSION` for the next steps. The password is passed through standard input rather than being written directly into the command line.

Before taking an action, capture the current screen:

```
./rdp-cli-linux-x86_64 screenshot \
  --session SESSION \
  --out ./before.png
```

An AI agent can inspect `before.png`, locate a button or input field, and decide what to do next. This observe → act → observe pattern helps the agent verify the current state after every action.

After locating an input field at coordinates `X, Y`:

```
./rdp-cli-linux-x86_64 click \
  --session SESSION \
  --desktop \
  --x X \
  --y Y \
  --observation OBSERVATION

./rdp-cli-linux-x86_64 screenshot \
  --session SESSION \
  --out ./focused.png

./rdp-cli-linux-x86_64 type \
  --session SESSION \
  --text "Remote task completed." \
  --observation NEW_OBSERVATION

./rdp-cli-linux-x86_64 screenshot \
  --session SESSION \
  --out ./after.png
```

The agent can inspect the final screenshot instead of assuming the command succeeded.

A person can open the same session in a native window or browser:

```
./rdp-cli-linux-x86_64 watch \
  --session SESSION \
  --viewer window
```

The viewer starts in read-only mode and supports pausing agent input, taking control, and handing the session back.

With rdp-cli, an AI agent can work with existing desktop applications and remote environments without requiring a new API integration for every tool.

The basic loop is simple:

```
Connect → Screenshot → Act → Screenshot → Verify
```

Try the complete setup at [rdp-cli.com](https://rdp-cli.com/).

Always use authorized computers and trusted networks. Review the connection and certificate settings before using rdp-cli in production.