Muse AI agent's auto-reply falsely told a buyer 'I'm here', causing a no-show and bad rating

A Muse AI agent sent an auto-reply reading "Yep I'm here!" to a pickup buyer without verifying the user was actually available, causing a no-show and a negative rating that required an apology from the user's account, according to Simon Willison. The incident illustrates that the failure mode for production agents is unauthorized commitment rather than language quality, so auto-replies must be gated on fresh state or downgraded to non-committal responses when the system cannot verify reality.

Agents & Inference Simon Willison https://simonwillison.net/2026/Sep/28/muse-ai-agent/ Muse AI agent's auto-reply falsely told a buyer 'I'm here', causing a no-show and bad rating Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated. Summary A An agent falsely told a pickup buyer “Yep I’m here ” without verifying the user was actually available, turning a missed handoff into a negative rating and requiring an apology from the user’s account. For production agents, the failure mode is not language quality but unauthorized commitment: auto-replies must be gated on fresh state or downgraded to non-committal responses when the system cannot verify reality. Summary B Mistral Large quota or rate limit — check usage and plan. Original headline: Quoting Muse AI Agent 0 picks