My AI Coding Agent Needed the Mouse. I Needed My Computer.

AigoPilot founder built a workflow that moves a project's code and AI coding sessions to a second machine so agents can drive that computer's desktop for application testing while the developer keeps using their own. The problem arose because agents testing desktop applications AigoPilot and Cilevi Screen Recorder needed mouse and keyboard control, and with several projects running, one agent's window could steal focus and send another agent's click to the wrong place — at one point constraining the founder's work for much of a day. The setup was extended to a friend's unused laptop connected over the internet, and the founder had the AI extend the transfer workflow to prepare destination machines with required tools, though installation of every development tool is not automated and unfamiliar dependencies may still need manual or AI-assisted setup.

There were days when I could barely use my own computer because an AI agent was busy testing an application I was building. It would launch the app, move the mouse, click through the interface, and type. I had to leave the controls alone. If I clicked somewhere else or brought another window forward, I could interrupt the test. Meanwhile, I had other projects to work on. This was an awkward consequence of getting more development work done with AI. I could have several coding sessions making progress, but when one of them needed my desktop, I sometimes had to wait for access to my own machine. Two of my projects are desktop applications: AigoPilot and Cilevi Screen Recorder. After an agent changed an interface, I wanted it to open the application and check what it had made. Was the control visible? Did the interaction work? Had a layout change broken something elsewhere? Reading the source and completing a build could answer some questions. They could not tell me everything I wanted to know about the running application. Much of my web development work could continue in the background. The desktop testing I was doing often needed the application in front, with the agent interacting with it much as I would. That distinction depends on how a test is run. Browser automation can also interfere with someone’s desktop, and some desktop checks can run without taking over their input. My problem was specifically the tests that needed control of the same interactive environment I was trying to use. With one application, I could sometimes step away and let the agent finish. With several projects, that became harder. One agent could be interacting with an application while another brought its own window forward. The first agent’s next click might then land in the wrong place. Adding another coding session did not give it another desktop. Neither did opening another terminal. I was already dealing with the overhead of finding the right project and session. Now, even after finding the work I wanted to continue, I sometimes could not use the computer to do it. At one point, this constrained my work for much of a day. It was enough to make me look for a different arrangement. I had spare machines available. Instead of having every project run on my main workstation, I wanted to move a project’s code and sessions to another computer and let development and testing continue there. The application could open on that machine. Its agent could use that desktop. I could keep using mine. I built this workflow into AigoPilot https://aigopilot.com , the studio I develop and use. I am its founder. The reason for building it was practical: I needed the testing work to happen somewhere that would leave my main computer available. A friend later mentioned an unused laptop at home. I connected that machine as well, so some work could run there over the internet. That made the setup useful beyond the spare computers in my own room. A home computer and a work computer could also help each other, provided the machine, connection, and project requirements were suitable. The important part was where the application actually ran. Viewing my main computer remotely would still leave its desktop occupied. Running the project on another machine gave the test its own place to work. One of the projects I moved was a game I was building with Unity. The destination laptop did not have Unity and some of the other prerequisites available on my main computer. Transferring the project files had not made it ready to continue the work. I asked the AI to help extend the transfer workflow to handle that setup as well. In the Unity case I worked through, this included preparing the destination with the tools the project needed. I have not automated installation of every development tool. For an unfamiliar dependency, the destination may still need additional setup, either with AI assistance or manually. Any required access or software licensing also needs to be handled for that machine. That experience changed how I thought about a project transfer. Along with the code, I needed to know which environment could build and run it, and what the next session was supposed to do. For someone trying a similar arrangement, I would start with one project and one spare machine. Before sending several agents over, establish that the destination can complete a small, useful task. My checklist would be: Access should be arranged for the destination rather than assumed to exist. A handoff can record where approved credentials are managed without copying secret values into its notes. Here is an example of the kind of first task I would give a destination machine. It is a suggested brief, not a transcript from one of my runs: Open the project and report missing prerequisites before changing anything. Once the environment is ready, build the application, launch it, and open its settings panel. Check that the new control appears and responds to a click. Return the code revision, build result, a screenshot, and any errors. If you cannot complete the interactive check, say where you stopped. That last sentence matters. A report that says the build passed should not leave me assuming the agent also opened and tested the interface. Moving a task to another computer addresses the input conflict. Other parts of the work can still collide. Two agents might edit the same files, use the same test account, or write their results to the same location. I would give parallel tasks a clear code state and separate output locations. Where appropriate, separate working copies or worktrees can help keep changes attributable to the task that made them. I would also ask the report to identify the machine and revision it tested. A screenshot is more useful when I can connect it to the build I actually asked the agent to check. The spare machine needs to suit the workload, too. Opening a basic settings panel is a different job from testing a demanding Unity scene or investigating recording behavior on particular hardware. An available laptop is a useful starting point; whether it can do the work still needs to be checked. I have not attached a timing study to this article, and I cannot give a measured percentage improvement. The change I noticed was that I could continue working while an agent interacted with an application on another computer. That mattered when I had several projects in progress. I could review another session, respond to a problem, or do my own work without worrying about interrupting the test with a click. I wrote about the wider workflow in My Claude Code and Codex Workflow: The Problems I Had to Solve Myself https://medium.com/@ohadi.anoush/my-claude-code-and-codex-workflow-the-problems-i-had-to-solve-myself-8cab221d8a51 . This was one of its most frustrating parts, because it affected the computer I needed for everything else. When I began using AI to build more software, I expected the difficult questions to be about code. I spent considerably more time than I expected figuring out who got to use the mouse. My AI Coding Agent Needed the Mouse. I Needed My Computer. https://pub.towardsai.net/my-ai-coding-agent-needed-the-mouse-i-needed-my-computer-b4fefe10dd4d was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.