The Setup and the Hurdle #
The architecture is straightforward:
Host A (Linux): Runs Ollama + an LLM agent.Host B (Windows): Tailscale SSH enabled, no local AI.
I wanted to send a command like "Organize the files in my Downloads folder by type and year" to Host A, and have Host A execute the corresponding PowerShell commands on Host B.
The main friction point I hit was the environment mismatch. The agent lives in a Linux shell but needs to manipulate a Windows filesystem. If I just give the agent a generic execute_shell
tool, it tends to hallucinate ls
or mkdir
commands that fail on Windows, or it gets confused about pathing (using /
instead of \
).
My Diagnosis and Solution #
After some trial and error, I realized that giving an LLM a raw SSH terminal is a recipe for errors. The agent needs a "bridge" that translates intent into valid PowerShell.
To make this work, I moved away from a generic shell and instead defined specific tools for the agent. Instead of "Run this command," I gave it a set of high-level functions that the Python wrapper on Host A handles.
For example, I implemented a remote_move_file
tool. When the agent calls it, the backend does this:
ssh windows-node "powershell -Command Move-Item -Path 'C:\source' -Destination 'C:\dest'"
Key Takeaways for this Architecture #
If anyone else is trying to build a remote-execution AI workflow, here is what I found:
Agent Location: Yes, the LLM and agent can stay 100% on the remote machine. The target machine only needs an SSH server (Tailscale SSH makes this trivial).Tooling Strategy: Do not use a generic shell tool. Create a "wrapper" layer of tools (e.g.,list_files
,move_file
,read_log
). This prevents the LLM from guessing the OS syntax and ensures the commands are formatted for PowerShell before they ever hit the wire.Environment Awareness: You have to explicitly tell the agent in the system prompt: "You are controlling a Windows machine via SSH. All filesystem operations must use PowerShell syntax."
This approach kept my Windows machine lean while still giving me the power of a local LLM for file management. It's a much cleaner deployment than trying to cram a runtime onto every device in the house.
Next SpaceX just bought Cursor and I wonder if this kills the →
these AI tool field notes, with plenty of directly applicable cases.