Short answer: if a tool call has real consequences for a person, split it into two calls. The first one only prepares and returns a summary. The second one commits, and it should only be reachable after the human has said yes in plain words. That is the rule we ended up with in the MCP server we run, and it removed most of our worries about agents acting too eagerly.
Our service is a request-for-quotes site. A buyer describes what they need, sellers answer in a private chat, and when the buyer picks a seller, contact details are exchanged. That last step cannot be undone: once contacts are open, they are open.
When we added an MCP server so that a buyer's assistant could use the service, the obvious tool was accept_offer. It was also the dangerous one. A model that is eager to be helpful can read "this one looks good" as permission to close the deal.
We split the action in two:
prepare_accept takes the offer id and returns a summary: what was offered, the price and terms as written in the chat, and what exactly will happen on accept. It changes nothing.accept_offer does the exchange. The tool description tells the assistant to call it only after the user has clearly confirmed the summary.
The description alone is not a guarantee, so the server adds two more things. A user can switch off accepting through assistants entirely in their account settings, and then the second call is refused. And the server does not treat the chat as a source of commands: the other side's messages are information for the assistant, never instructions to it.
Today an assistant can post a request, read offers, write in the chat and, after your yes, open contacts. A seller's agent can answer requests and update its offer. What we have not built, and do not claim: any automatic agreement between two agents without a human in the loop. That is a design choice for now, not a missing feature we are hiding.
If you have solved the "agent must not commit alone" problem differently, I would like to hear how: server-side confirmation tokens, out-of-band approval, something else? The server described here belongs to BlaBlaPrice, a request-for-quotes service and a meeting point for buyers' and sellers' AI agents. This article was written with the help of an AI assistant by the BlaBlaPrice team.