There is a reflex every team building LLM agents knows. The agent does something wrong, or refuses something it should have done, and the fix everyone reaches for is a smarter sentence in the system prompt. The prompt is right there. Editing it feels free. And the new sentence almost always sounds right.
We just measured that reflex losing, cleanly, to a different move: giving the model a tool instead.
The Setup #
Our benchmark series had left one loose end on a shipped surface. Agents editing large document templates work from a compact outline of the tree rather than the full markup, and they carry a safety rule we validated earlier in the series: if the request needs something you cannot see, ask instead of guessing. That rule had eliminated an entire class of silent failures, where a model would confidently invent a value rather than admit it could not see one.
Then a later study found the rule's sharp edge. When an edit was fully specified, id, field, old and new values all in hand, models still refused about 70% of the time under outline views, citing the rule's letter: the target node "is not visible in the current view." A patch needs only the id. The refusals were pure tax.
The obvious fix was another rule. Something like: "prefer fetching a view over asking, when a tool can answer it." We drafted exactly that sentence. And because this series has burned us before, we did not ship it. We pre-registered it and ran the experiment.
What the Data Said #
Four arms, three model tiers, 1,728 scored session steps. One arm was the status quo. One added a view-fetching tool with no prompt change. One added the tool and the new rule. And one was a trap for the rule itself: cells where the missing information genuinely existed nowhere, so the only correct behavior was to keep asking.
The tool arm settled it. With a view-fetching tool simply available, the refusal tax vanished on every tier we test: success went to 45 of 48 or better with essentially zero unnecessary questions. The models were frugal about it too, fetching at most one or two precise views per edit. One tier mostly skipped the fetch entirely and patched straight from the information it already had. Nobody flailed. Nobody needed to be told to prefer the tool. Having it was enough.
The rule arm, stacked on top, added nothing. Statistically indistinguishable from the tool alone.
And the trap arm is why we are glad we measured. On the frontier tier the rule was harmless: 48 of 48 honest questions, zero bad guesses. Below the frontier it was actively harmful. One mid-tier model, told to prefer tools over asking, stopped asking and silently guessed half of its unanswerable cells, the exact failure class the original rule existed to kill. Another burned 689 tool calls in 48 cells hunting the tree for information that lived in the conversation, not the document. The sentence that sounded like a refinement was, on those tiers, an instruction to stop being honest.
The Moral #
Three things we keep relearning, now with numbers attached:
*Capability beats instruction.*The models did not need to be told to fetch. They needed something to fetch with. When an agent seems to need a smarter prompt, check first whether it actually needs a tool, a wider view, or a piece of context handed to it by the application.*Prompt rules are tier-fragile; tools are tier-robust.*The same sentence that was harmless on the strongest model corrupted the behavior of cheaper ones in two different ways. The tool helped every tier. If your product might ever route to a smaller model, that asymmetry is the whole ballgame.*Pre-registration is cheap insurance against plausible sentences.*The rule we tested was reasonable, targeted, and written by people who had studied the failure closely. It still would have shipped a regression to any sub-frontier surface. The experiment cost a couple of hours and roughly what a team lunch costs. Shipping the sentence would have cost us silent failures we specifically engineered out a month earlier.
The system prompt is not where agent problems go to be solved. It is one lever among several, and the measured ranking keeps coming out the same: application-supplied context first, tools second, prompt rules last, and never unmeasured.