Welcome to Kernel Panic!, a weekly newsletter by Lily Hay Newman and Matt Burgess from inside the new world of privacy and digital security. To receive this newsletter in your inbox each week, sign up here.
Meta’s new personal assistant, Muse, has become a viral hit, with millions down the AI agent, connecting it to bank accounts, messages, or health data, and allowing it to complete tasks for them. But as Muse takes off among consumers, data from inside the app is providing insight into how it organizes and presents information to users.
In recent days, multiple researchers have extracted Muse’s internal files and dumped the agent’s operating instructions, providing a glimpse of how the system was built and behaves. Meta has maintained that it intended for these files to be accessible in the interest of transparency, and they do provide insight into how Muse responds to prompts and questions about, for example, highly politicized or sensitive topics.
Independent AI safety and security researcher Karan Joshi was able to extract an extensive array of Muse instructions and system prompts by using the regular chat interface to essentially ask Muse to copy and share its own software files. Joshi then shared his findings with WIRED.
One of Muse’s instructions appears to be the ability for it to create “a page for every person in the user’s life.” This hourly process involves compiling data on family, partners, friends, colleagues, “collaborators,” and people you “follow,” the instructions say.
The general idea is that Muse can use its “memory” (aka structured text files) to collect information about your relationships and the important people in your life. It can then make suggestions, like giving pointers on how to improve particular relationships or, say, where to take a coffee-loving friend for breakfast. Of course, it’s not unusual for AI chatbots to track social information in general—people have been asking ChatGPT for relationship advice for years now. But it’s interesting to see how Muse is set up given Meta’s history and vast access to social network data.
“What it seemed like to me—from all these prompts, system skills data, and things that they’re feeding into Muse—is that they want to understand your relationships that you have with real people,” says Joshi. “They’re trying to know you like a friend, which is honestly pretty creepy.”
Muse’s documentation says that a page may start “sparse” and then be filled out over time, potentially including sections like Facts, History, The relationship, In common, Open threads, and Strengthening. Meta’s instructions say Muse should only use the “evidence” it has available, and that invented details are worse than an empty page.
“Where they live, what they do, the threads that recur (the apartment move, the shared savings goal),” the instructions say. They add, too, that the model could record the “dates that matter,” such as birthdays or anniversaries. A person’s history could include backstory like “the trip in March, the argument that got resolved, the milestone last week.”
The instructions also focus on recording relationship details such as “how close they are, what it is built on, how they act with each other, and what it seems to need right now.” The Strengthening section suggests ways a relationship could be improved, including, “A reason to call, a date worth remembering, something they said to circle back on, a way to be there for them that matters.”
“We are giving AI systems much more information about us than we are getting information from them,” says Carissa Véliz, an associate professor at Oxford’s Institute for Ethics in AI. “It’s not only what we explicitly tell them, but what they can infer from us—correctly or incorrectly, both concerning for different reasons—and what they can piece together from other sources of data.”
Muse is architected so each individual user has their own dedicated virtual machine, which stores user data and context. This VM is inaccessible to other agents and users can wipe memories or disconnect external services at any time. Meta says that Muse is also designed to seek human confirmation before completing actions like sending an email or making a purchase, and that it has an audit log where users can see all of the agent’s activity and future plans.
“For any agent to be useful and actually help you achieve your goals, it needs to have context about you and those you interact with,” Meta spokesperson Daniel Roberts told WIRED in a statement. “Muse gathers that based on public information and from what you’ve chosen to share, which is how it remembers the person who sent you an invoice is in fact the plumber who you previously hired to complete work in your bathroom or which flowers your spouse said they liked best.”
It’s become common for AI assistants to incorporate some sort of memory feature designed to allow greater personalization in responses and activity. Miranda Bogen, the director of the Center for Democracy and Technology’s AI Governance Lab, says Muse appears to have more emphasis on relationships and personal contacts than rivals’ systems. In general, these tools tend to offer some transparency and editing capabilities, and Muse does as well. If anything, though, Bogen notes that agents and assistants encourage people to proactively share more data with them versus focusing on culling data.
“These [AI assistant] tools are actively soliciting users to plug their whole lives in—their emails, calendars, financial institutions, everything in order to be helpful assistance,” Bogen says. “That’s dramatically more information than people might have otherwise given to some of these companies. The breadth of access to information that these tools have will lead to a ballooning of what they know about users.
Spotted something we should include next week? Let us know at [email protected]. And stay safe out there.