Another day, another AI agent gone rogue. Over the weekend, what seemed like a pretty typical Facebook Marketplace transaction took a very bizarre turn when Matt Robb, a Toronto-based tech video creator, let Muse handle his communications. Several hours later, Robb learned that unbeknownst to him, Meta’s new AI agent had accepted a lowball offer for a keyboard and arranged a time for the buyer to come pick up the item—even though Robb wasn’t home at the time.
“A guy just showed up at my door, ready to buy, because as far as he knew, we had a deal,” Robb wrote on Threads after learning of the mix-up from the AI agent. “Absolutely wild.”
While Robb later clarified a few things—namely, that he had provided his address to Meta’s new AI agent and allowed it to accept offers—he also provided some information that might be helpful to others. What’s more, Robb said that he provided feedback directly to David Singleton, who heads up the Meta Superintelligence Labs at the Menlo Park, California-based company. And Robb even patched things up with the buyer, Usman Naseem, who was understandably disgruntled to arrive at Robb’s apartment ready to buy a keyboard only to be ghosted once there. Naseem posted on Threads that Robb delivered the keyboard along with some valuable items as a gift.
Still, the whole ordeal serves as a cautionary tale of sorts.
It seems that some of the settings were to blame for how this transaction went off the rails.
Robb noted that when allowing Muse to handle his Facebook Marketplace interactions, he had selected the “allow always” option in lieu of “allow one time.” Robb said that he thought that the AI agent would send approvals to accept offers, which it didn’t do. “Be careful,” he cautioned others.
By granting Muse permission to send messages on his behalf, as Robb posted, that explains why his address was provided during the communications with Naseem. “I didn’t think it would send it out to everyone that gave me an offer, so it’s worth checking,” he wrote on Threads.
Indeed, Robb only learned about Naseem coming to his home when Muse later informed him with a string of messages that read, in part, “I should probably stop the auto-replies from claiming you’re home when I can’t verify that.”
What’s more, even after Robb instructed Muse to stop giving out his address, he decided to test it by asking a few friends to see if the AI agent would still share this information. The bot provided his address to five more people.
As for the lowball offer that was accepted, Robb wrote that the Meta team he met with told him that occurred because of an internal error.
Based on his experience, Robb also said that he suggested an improvement to Meta. Specifically, he recommended that any messages sent by Muse include a note underneath the message so that everyone in the conversation knows that. “We definitely need clearer signals when an AI is talking on someone’s behalf (it should have a ‘Sent by Muse’ tag), and much tighter restrictions on sensitive info,” he posted.
Muse is only the latest instance of an AI tool going rogue. There’s now a growing list of problematic cases of various AI tools—including the hacking of a startup by two OpenAI models or the troubling messages between a Florida man and Google’s Gemini tool before he ultimately committed suicide.
In the week since Meta released Muse, it has been downloaded 3 million times, according to The Guardian. But the mess-up only adds more fodder to people who already have concerns about AI agents, particularly in the wake of several doomsday warnings from AI leaders earlier this month.
While Robb was patient zero for Muse’s first major mess-up, he said that he was “glad” that it happened to him in lieu of someone who might be more vulnerable. “I’m just saying the average user will prompt worse than a tech guy.” And even though he noted that his experience may have illustrated how AI is “getting intrusive,” Robb said he’s not dissuaded from the technology. “I’m not an anti-AI person at all,” he wrote.