# How Meta Built Muse: Inside the Personal AI Agent's Eval-Driven Design > Source: > Published: 2026-10-20 00:00:00+00:00 # How Meta Built Muse: Inside the Personal AI Agent's Eval-Driven Design Meta's Muse personal agent hit millions of downloads fast. Here's how the team built it with 100+ behavior evals, separate from Meta's open models. ## What is Meta’s Muse app and why does it matter? Muse is Meta’s personal AI agent app, launched in the US on September 8th. Within a couple of weeks it had been downloaded more than 2.5 million times and briefly topped the Apple App Store charts. It handles tasks like reading email, filling in forms, booking things, making payments, and negotiating on a user’s behalf, with each user getting their own isolated virtual machine running the agent alongside their data. It matters because it’s currently the personal agent app with the most visible momentum in a category that, as of late 2025, every major AI lab is racing to own. ## TL;DR - **Muse is not powered by Meta’s open model.** The app runs on a customized, closed version of Spark, not the publicly released Spark Open weights, and Meta hasn’t detailed exactly which fine-tunes or versions are in production. - **The app was built through eval-driven development.** Meta’s team, led by Alexander Wang and Nat Friedman, wrote down more than 100 specific behaviors a personal agent needed to handle, turned each into its own evaluation, and tracked pass/fail rates like a spreadsheet of green lights and red lights. - **Nothing shipped until every row turned green.** Early checkpoints cleared roughly 80% of the evals and were still rejected; the first version to pass every single row became the model that launched inside Muse. - **The process took about seven months** with a team under 200 people working across both the underlying model and the product, starting from early prototyping with open-source agent frameworks in February. - **Meta’s separate open releases (Spark Open and Beam) are not the same project as Muse.** Spark Open is an 816 billion parameter mixture-of-experts model Meta calls its most capable US open-weights release, but it scores below several Chinese open models on independent benchmarks and sits well below Meta’s own closed frontier models. - **Pricing and tiers reflect a real commercial product.** Muse offers a free tier plus $20 and $100 monthly subscriptions, positioning it as a consumer product rather than a developer API. ## Plans first. *Then code.* Remy writes the spec, manages the build, and ships the app. ## How was Muse actually built? The origin story, as described by Alexander Wang on David Senra’s podcast, starts in February with Wang and Nat Friedman spending weeks living inside open-source “open-claw” style agents, alongside a broader team doing the same. Their shared takeaway: the underlying idea of a personal agent was genuinely compelling, but existing implementations didn’t work reliably enough for daily use. From there the team moved into parallel prototyping of two things at once: the agent system (the scaffolding, tools, and product experience) and the model itself (the thing doing the reasoning and tool calls). That dual-track approach is worth noting because it’s easy to assume a personal agent is just a wrapper around an existing LLM. Meta’s approach treated the model as something that needed its own dedicated training process to hit the reliability bar the product required. ## What is eval-driven development, and why did Meta use it? Eval-driven development means defining success criteria as testable checkpoints before you build, then refusing to ship until the system passes them. Meta’s team put this into practice very literally: they listed over 100 specific behaviors and capabilities an ideal personal agent needed to demonstrate, reading an email thread correctly, filling out a web form, completing a negotiation, and so on, and gave each one its own row in a spreadsheet with a corresponding evaluation. Each row had two thresholds: a “launch blocking” minimum and a “good” target. The team trained models, ran them against the full eval set, and tracked the percentage of rows passing. Early on, checkpoints were landing around 80% green and 20% red. Rather than treat that as good enough to ship, Wang’s team kept iterating, likely using reinforcement learning environments in post-training to specifically target the failing behaviors. The first checkpoint to go fully green across every row became the specifically trained version of Spark that now powers the Muse app in production. This is a meaningfully different discipline than benchmark chasing. Public benchmarks measure general capability. Meta’s internal evals measured whether the exact behaviors a personal agent needs to perform in the real world, reliably and repeatedly, actually worked. That distinction explains why Muse can be a strong consumer product even though the model class it’s built on doesn’t top any public leaderboard. ## Why isn’t Muse running on Meta’s open model? This is one of the more easily missed details in Meta’s recent run of releases. In the same general window, Meta shipped Spark Open (an open-weights model), plus two closed frontier models internally referred to as Wonder and Wonderful, and separately has been running Muse on its own customized version of Spark that isn’t any of those public releases. - ✕a coding agent - ✕no-code - ✕vibe coding - ✕a faster Cursor The one that tells the coding agents what to build. Spark Open is a mixture-of-experts model with 816 billion total parameters and 42 billion active per token, a 1 million token context window, and native multimodal input (text, images, video, and audio in, text out). Meta calls it the most capable US open-weights model to date, a carefully worded claim that implicitly concedes ground to Chinese open models. On the Artificial Analysis Intelligence Index, Spark Open scores 39, behind several open Chinese models in that same period, while Meta’s own closed models (Wonder and Wonderful) score 52 and 56 respectively, and Meta’s mid-tier closed model, Spark 1.3, scores 48. That gap matters for anyone hoping the open release would double as the backbone for Meta’s flagship consumer product. It doesn’t. Muse runs on a customized, closed variant of Spark that the public can’t directly inspect or test, and Meta hasn’t published which specific checkpoint or fine-tune is in production. ## Is Meta’s open model release actually competitive? On paper, Spark Open holds up reasonably well against similarly sized open models, particularly on agentic and tool-use benchmarks. Meta has also released a second open model, Beam, a smaller 501 billion parameter (23 billion active) model under a fully Apache 2.0 license, compared to Spark Open’s more restrictive Muse license, which requires attribution and a separate commercial agreement for very high-revenue deployments. Spark Open generally outperforms Beam on benchmarks like tool use and multi-step reasoning tasks, though Beam reportedly edges it out on at least one efficiency-oriented benchmark. Running Spark Open at full precision is impractical for almost anyone outside a well-resourced lab. Quantized checkpoints bring the storage footprint down substantially, and the model is available through multiple hosted inference providers at roughly the same price point. For most developers, a hosted API is the realistic way to actually use it, rather than self-hosting. ## Is the Muse personal agent worth paying attention to? For builders, Muse is worth watching less as a product to copy and more as a case study in process. The headline number, 2.5 million downloads in a couple of weeks and a brief run at the top of the App Store, shows real consumer pull for personal agents that can act on a user’s behalf rather than just answer questions. But the more durable lesson is methodological: a 7-month build cycle anchored by more than 100 concrete, measurable behavior checks, with explicit launch-blocking thresholds, produced something reliable enough to ship as a mainstream consumer app. That approach is replicable at a smaller scale. A team building its own agent doesn’t need 100 evals on day one. Starting with 20 well-defined behaviors, building a pass/fail test for each, and refusing to ship until they all clear is a far more disciplined bar than “it worked in my demo.” ## Frequently Asked Questions ### What model powers Meta’s Muse app? Muse runs on a customized, closed version of Meta’s Spark model family, not the publicly released Spark Open weights. Meta hasn’t disclosed the exact version or fine-tune in production. ### How long did it take Meta to build Muse? Based on Alexander Wang’s account, the process ran roughly seven months, starting with prototyping in February and involving a team of under 200 people working across both the model and the product. ### What is Spark Open and is it the same as Muse? Spark Open is Meta’s openly released 816 billion parameter mixture-of-experts model, which Meta calls its most capable US open-weights model. It is a separate release from Muse and is not the model that powers the Muse app. ### How much does Muse cost to use? ## Other agents ship a demo. Remy ships an app. Real backend. Real database. Real auth. Real plumbing. Remy has it all. Muse offers a free tier along with paid subscription tiers at $20 and $100 per month, structured as a consumer product rather than a developer API. ### What does “eval-driven development” mean for AI agents? It means defining the specific behaviors an agent must reliably perform, building a measurable test (eval) for each one, and only shipping once the system passes all of them, rather than relying on general benchmarks or informal testing.