{"slug": "read-the-logs-before-you-rewrite-the-prompt", "title": "Read the Logs Before You Rewrite the Prompt", "summary": "A developer running a multi-agent orchestrator on GPT-6.1 Sol cut its per-hour cost from $0.67 to $0.34 and shrank its instructions from about 2,100 words back to roughly 1,750 by moving deterministic decisions out of the prompt and into a resident shell script called the studio. In a replay of fifteen real past board states, the orchestrator wrote into other agents' items in ten cases before the change and none after, though a later status-snapshot bug in the script stalled the board for eight and a half hours.", "body_md": "For a while I tried to fix my orchestrator by writing to it.\n\nIt runs on GPT-6.1 Sol, and it had habits. It wrote notes into items addressed to other agents. It handed work to the expensive executor when the cheap one would have done. Every time I found one, the prompt got another paragraph explaining why not to. Sol's habits are strong enough that a polite instruction does nothing, so the paragraphs were firm, repeated themselves, and gave reasons. In a week its instructions grew from about 1,500 words to 2,100, and the last fix alone added about 400.\n\nWhat changed my approach was counting. I asked for the runs of the past day: which roles were launched, how often, and what each launch did. The orchestrator had been launched more than twenty times just to receive a finished task and file the next one. None of those launches needed a judgment. A goal's plan was already a list in order, each item with the role that should do it. The orchestrator read the list and filed the next entry, and sometimes filed it for the wrong role.\n\nAnything that can be decided deterministically is more stable in a script than in any model. That sentence is obvious, and I had still been writing prompts around it.\n\nThe team works on a board of markdown files, and a resident shell script I call the studio launches whichever agent an item is addressed to. Filing a goal's next task went into that script first. The plan already names who does each item, so when the board has nothing in progress, the studio files the next one for that role and marks it done when the task finishes. The orchestrator now sees a goal item only when I should see its result, when it is the last one, or when something went wrong. The lead executor, which plans, stopped being the default destination for anything vague.\n\nThe notes in other agents' items took a script and a sentence. Before each orchestrator pass the studio snapshots every task and review addressed to someone else, and after the pass it puts back any body the pass changed and tells the next pass what it undid. The prompt kept one boundary: into an item addressed to another agent, you write nothing, not even a note. That replaced the 400 words. To check it, I rebuilt fifteen real past states of the board in a scratch copy and ran the orchestrator over each. Before the change it wrote into another agent's item in ten of the fifteen. After, in none.\n\nTwo smaller jobs went the same way. Agent failures used to reach me as the last line of a log; now the studio reads the error code each CLI already records in its session files and retries only an overloaded server. Scratch files the agents never cleaned up are deleted once nothing on the board names them and nobody has touched them for three days.\n\nThe instructions came back down to about 1,750 words: the 400 were gone, and the rest of that week's growth came from other changes. Per hour the team ran, the orchestrator's cost fell from $0.67 to $0.34. I had been thinking of moving it to Claude Opus to cure its habits. With less and less of its old work left to it, I dropped the idea. Two nights later the team [ran a goal from start to finish without a correction from me](https://dev.to/shinpr/i-took-the-wheel-back-from-my-ai-team-then-i-gave-it-back-17ho).\n\nThe morning after that night, the board stopped moving at 8:07. I noticed eight and a half hours later.\n\nThe orchestrator had sometimes made the wrong call. The script made no call at all. The studio launches the orchestrator when an item reaches a status it should handle, and to avoid launching it twice for the same thing, it remembered which item was in which status. It took that snapshot only between passes. During one pass the orchestrator sent a finished task back to the lead executor, the lead executor finished it again, and all of that happened before the next snapshot. The studio saw the same task in the same status it had already handled and launched nothing. A sprint review sat waiting on that task all day.\n\nThe log showed this in a few minutes, and the fix was one field: the snapshot now includes the file's modification time, so a task whose file has changed counts as new even when its status reads the same.\n\nThe same stall exposed a second problem, and that one came from my own change two days before. Since the orchestrator could no longer write into the task, it had sent the task back by flipping its status, with the reason written in its own review item. To the lead executor the task looked identical to one whose process had died. Three launches in a row it rechecked the numbers it had already checked and never read why the task had come back. Work that falls short now comes back as a new task, with the shortfall written as its second line.\n\nThe habits I had tried to fix in the prompt showed up as behavior I had to notice and then count. The bugs in the scripts showed up as a line in a log. The same logs had shown me, two days earlier, which of the orchestrator's work had never needed a judgment in the first place.", "url": "https://wpnews.pro/news/read-the-logs-before-you-rewrite-the-prompt", "canonical_source": "https://dev.to/shinpr/read-the-logs-before-you-rewrite-the-prompt-4gf1", "published_at": "2026-10-22 12:07:19+00:00", "updated_at": "2026-10-22 12:12:09.673909+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "ai-infrastructure"], "entities": ["GPT-6.1 Sol", "Claude Opus"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/read-the-logs-before-you-rewrite-the-prompt", "markdown": "https://wpnews.pro/news/read-the-logs-before-you-rewrite-the-prompt.md", "text": "https://wpnews.pro/news/read-the-logs-before-you-rewrite-the-prompt.txt", "jsonld": "https://wpnews.pro/news/read-the-logs-before-you-rewrite-the-prompt.jsonld"}}