3 traps I fell into while letting a phone control a desktop agent

A developer building DeepSeek Phone Harness, a mobile remote control for a desktop agent, encountered three major pitfalls during development. The relay initially missed question frames from the agent's `ask_user_question` tool, causing tasks to hang indefinitely. Additionally, the tool result payload had a nested structure that was not parsed correctly, and caching issues prevented UI updates from appearing on the phone. The developer fixed these issues and open-sourced the project under the MIT license.

On the surface it's an HTTP layer and a web page. Underneath, every step fights your assumptions. Real pitfalls, so you can avoid them. Before building DeepSeek Phone Harness, I assumed "mobile remote control for a desktop agent" was just moving a web page to a phone. After finishing it, I admit: the idea took 10 minutes; the bug-fixing took a week. Here are 3 real traps — each one made tasks "gracefully deadlock" in production. This is the nastiest one. Our relay listened to DSH's approval channel approval/requested — all good. Until one day a test task never finished . Not an error — just hung, pending climbing. Two days of digging later: DSH agents can call the ask user question tool to ask the user something — and that goes through a different channel question/requested , not the approval channel. My relay only listened for approvals, so question frames were silently dropped. Result: the agent waits on the computer for a human answer, the phone has no idea, the task hangs forever. It only continues once the user answers. Fix: ingest question frames separately, forward them as "your answer needed" cards, and use the exact same answer protocol as the official Web GUI. Lesson: in an agent's realtime event stream, every frame type exists for a reason. Miss one, and tasks die in ways you'd never guess. The phone shows "tool cards": tool name + file path + result. Name is easy, path is easy extract from arguments . Result? I assumed tool/result content was {type:'text', text:'...'} . Naive. The real shape: content: { "type": "tool-result", "content": { "type": "text", "text": "<path ...<content ..." } } Two levels of nesting , with toolCallId on the inner block. My first version only read one level: cards had names and paths, but results were always empty. Fix: recursive text collection + pairing calls to results via source.callId . Lesson: no protocol doc beats capturing one real payload. Changed the UI, opened the phone — still the old one. Refresh, restart browser, switch browser — same. Not a network issue. It's caching : the old page was cached, and Cache-Control wasn't set. Fix: serve HTML with no-store, no-cache, must-revalidate . Every open is now the latest. Lesson: for mobile web apps, cache strategy matters as much as feature code. Each one reveals the real complexity of "remote-controlling an agent from a phone": After fixing all of it, tool cards look like this on the phone: 📄 read ● done C:/Users/Joey/Documents/.../config.json ▾ tap to expand: full arguments + full result Every step the agent takes — which file, what command, the outcome — is visible. Permission requests and questions become cards. Tasks never hang because "nobody answered." Open source MIT . Stars / PRs / issues welcome: What's the weirdest bug you've hit while working with agents? Share it below — I bet it beats mine.