# Don't Trust Anyone's Muse Wonder Benchmark. Race It Against Opus 5.5 on Your Repo > Source: > Published: 2026-10-12 20:42:56+00:00 Meta's chart says Muse Wonder beats Claude Opus 5.5 at long coding jobs: 78.8 to 74.2 on DeepSWE, 53.1 to 46.3 on SWE-Marathon ([model card](https://research.meta.ai/static/muse-wonder-model-card)). [Artificial Analysis](https://artificialanalysis.ai/models/muse-wonder) gives Wonder 52 on its Intelligence Index. Opus 5.5 [scores 58](https://artificialanalysis.ai/articles/claude-opus-5-5). Both cost $4 per million input tokens and $20 per million output (Wonder's price per [TechCrunch](https://techcrunch.com/2026/10/12/meta-releases-its-first-frontier-model-muse-wonder/), Opus's per the same Artificial Analysis article). Meta's chart and the independent index can both hold, because they measure different things, and neither ran on your code. Meta launched Muse Wonder on October 12, 2026 ([Meta newsroom](https://about.fb.com/news/2026/10/introducing-muse-wonder-models/)). In the [Hacker News thread](https://news.ycombinator.com/item?id=50059115), one commenter called it "much worse than Opus 5.5" after an image-to-HTML test. That is one prompt. Nothing below is a Muse Wonder result. What follows is the setup to get your own: the same ticket handed to two agents in two git worktrees, and a script that scores both. Wonder wins five of Meta's 14 rows against Opus 5.5, mostly long-horizon coding, legal agent work and Meta's internal memory test. It loses nine, including ProgramBench, where Opus leads 70.5 to 50.0. The row-by-row view is in [the scorecard breakdown](https://dev.to/gabrielanhaia/metas-muse-wonder-costs-exactly-what-opus-55-costs-it-loses-9-of-metas-own-14-benchmarks-399a). Meta's own footnotes tell you how far to trust any single row. For competitor runs it did in-house, Meta says they "use common settings where practical but remain best-effort: prompts, tools, and runtime may not be optimally tuned for proprietary models." And from the limitations section of the [model card](https://research.meta.ai/static/muse-wonder-model-card): "Performance can vary with quantization, serving configuration, prompts, tools, and the agent harness." No leaderboard can hold prompts, tools and harness fixed for your setup, but a bake-off on your own repo does exactly that. *One ticket, two agents building the same thing, one scale to weigh them.* Choose a real ticket with a test that fails today (or write one first), a fix that touches a few files, and an outcome you can judge. Save the prompt to a file outside the repo so every agent gets the same words, and name the test command in it. "Make `npm test -- slug` pass without changing the tests" gives the agents and you the same finish line. Next, create a key for the Meta Model API and export it under the name Meta's [coding-agents docs](https://dev.meta.ai/docs/coding-agents) use. Put it in your shell profile (or load it from your secrets manager) so any new terminal has it: ``` export MODEL_API_KEY="your-key-here" ``` The API model ID is `muse-wonder-1.0`. The API does not serve Muse Wonder Full, the stronger sibling: it is for Muse Code subscribers only ([TechCrunch](https://techcrunch.com/2026/10/12/meta-releases-its-first-frontier-model-muse-wonder/)). I build [Hermes IDE](https://hermes-ide.com), a free desktop console for coding agents, so the setup below uses it ([download it](https://hermes-ide.com/download) to follow along). The plain-git version follows session B. Press ⌘N (Ctrl+Shift+N on Windows and Linux), pick Claude Code in the agent chip, pick Opus 5.5 in the model chip, paste the prompt and press Enter. Claude Code uses its normal sign-in. The task starts in its own git worktree on a new `hermes/` branch, cut from the branch you are on. Meta documents three environment variables that point Claude Code at its API: `ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN` and `ANTHROPIC_MODEL`. The docs also tell you to set the Opus, Sonnet and Haiku aliases and the subagent model to the same ID, "so no path falls back to a Claude model". Skip those and some requests ask for a Claude model ID instead of Muse Wonder, and session B stops being a clean Muse run. Seven variables make a long command, so put them in a wrapper. Save this as `claude-muse` somewhere on your `PATH` and `chmod +x` it: ``` bash #!/usr/bin/env bash # claude-muse: Claude Code against Meta's Model API export ANTHROPIC_BASE_URL="https://api.meta.ai" export ANTHROPIC_AUTH_TOKEN="$MODEL_API_KEY" m="muse-wonder-1.0" export ANTHROPIC_MODEL="$m" export ANTHROPIC_DEFAULT_OPUS_MODEL="$m" export ANTHROPIC_DEFAULT_SONNET_MODEL="$m" export ANTHROPIC_DEFAULT_HAIKU_MODEL="$m" export CLAUDE_CODE_SUBAGENT_MODEL="$m" exec claude "$@" ``` The base URL has no `/v1`, and the key goes in `AUTH_TOKEN`, not `API_KEY`. Both details come from Meta's docs. Their worked example uses `muse-spark-1.3`; `muse-wonder-1.0` is the Wonder ID. Now press ⌘N again and pick **Custom agent** ("Any command you type"). Name it `Claude Code · Muse Wonder` and set the command to `claude-muse`. The hint under the field says it plainly: "Hermes starts this command in the session's terminal, as if you typed it." Because it's a real shell, an env-prefixed one-liner works too. When the CLI is up, paste the same prompt. Sessions A and B now run the same harness on the same ticket. What differs is the model and the API serving it. Meta's footnote on its SWE-Marathon row calls that row "not a controlled common-harness comparison". Your A-versus-B run is a common-harness one. *Real Hermes window, test build with stand-in agents. The same task in two sessions side by side, each on its own worktree branch. In your bake-off, the Custom agent takes the second slot.* The isolation part is one command per agent. From your repo: ``` git worktree add -b bakeoff/opus ../bakeoff-opus git worktree add -b bakeoff/muse ../bakeoff-muse ``` Then run `claude` in `../bakeoff-opus` (pick Opus 5.5 with `/model`) and `claude-muse` in `../bakeoff-muse`, each in its own terminal. You lose a single place to watch both, and the review step later. Hermes worktrees are plain git branches too, so nothing here locks you in. `muse` CLI Session B tests the model inside Anthropic's harness. If you want Meta's harness too, install Muse Code. Meta's installer creates a command named `muse` in `~/.local/bin`. Read the script first if you prefer, then run: ``` curl -fsSL https://dev.meta.ai/install.sh | bash ``` Add one more Custom agent with the command `muse`. Muse Code is a subscription: the [Power Usage plan](https://dev.meta.ai/products/muse-code) is $50 a month and includes Wonder Full. Meta's product page lists macOS and Windows. One early report is negative: in the same Hacker News thread, a commenter said the Muse Code sandbox "never worked across Mac or Linux". Check it on your own machine before you pay for a plan. When both agents say they're done, score them the same way. This script takes the base branch, a test command, and any number of worktree paths. For each one it prints the branch, whether the tests pass, how many files changed and how many lines went in and out against the point where the branch left the base. Save it as `compare-worktrees.sh` next to your repo and `chmod +x` it: ``` bash #!/usr/bin/env bash # Score agent worktrees against the branch they started from. # usage: compare-worktrees.sh BASE "TEST COMMAND" WORKTREE... set -u base=$1 test_cmd=$2 shift 2 for wt in "$@"; do branch=$(git -C "$wt" rev-parse --abbrev-ref HEAD) fork=$(git -C "$wt" merge-base "$base" HEAD) # count untracked files before the tests can add any new=$(git -C "$wt" ls-files -o --exclude-standard | grep -c .) (cd "$wt" && bash -c "$test_cmd") >/dev/null 2>&1 code=$? if [ "$code" -eq 0 ]; then result=PASS; else result=FAIL; fi # working tree vs fork point: committed and uncommitted edits stats=$(git -C "$wt" diff --numstat "$fork") files=$(printf '%s' "$stats" | grep -c .) added=$(printf '%s\n' "$stats" | awk '{s+=$1} END {print s+0}') removed=$(printf '%s\n' "$stats" | awk '{s+=$2} END {print s+0}') echo "== $wt" echo " branch: $branch" echo " tests: $result (exit $code)" echo " files: $files changed, $new untracked" echo " lines: +$added -$removed vs $base" done ``` It diffs the working tree against the fork point, so it counts edits an agent made but never committed. Brand-new files git doesn't track yet show up in the "untracked" count instead of the line totals. That count runs before the tests, so files the test run writes don't inflate it. Anything the agent's own test runs left behind still counts unless your `.gitignore` covers it. The output below is not from Muse Wonder or Opus 5.5. It comes from a throwaway toy repo (git 2.52, Python 3.14, `__pycache__/` in `.gitignore`) with two hand-written fixes for a broken `slugify()` standing in for agent output. Fix A rewrote the function with a regex. Fix B stripped punctuation, still produced double hyphens on extra spaces, and left a TODO in the test file: ``` bash $ git worktree add -b bakeoff/agent-a ../agent-a Preparing worktree (new branch 'bakeoff/agent-a') HEAD is now at 759ecee Initial commit $ git worktree add -b bakeoff/agent-b ../agent-b Preparing worktree (new branch 'bakeoff/agent-b') HEAD is now at 759ecee Initial commit $ ../compare-worktrees.sh main "python3 -m unittest -q" \ > ../agent-a ../agent-b == ../agent-a branch: bakeoff/agent-a tests: PASS (exit 0) files: 1 changed, 0 untracked lines: +5 -1 vs main == ../agent-b branch: bakeoff/agent-b tests: FAIL (exit 1) files: 2 changed, 0 untracked lines: +6 -1 vs main ``` In Hermes the worktrees are plain git too, so `git worktree list` in your repo prints the paths to pass in. The script covers the first three rows. The rest you note while you watch: | | A: Claude Code, Opus 5.5 | B: Claude Code, Muse Wonder | C: `muse` (optional) | |---|---|---|---| | Tests pass (exit code) | | | | | Lines +/- vs base | | | | | Files touched | | | | | Wall time to "done" | | | | | Cost | | | | | Review comments before you'd merge | | | | **Wall time.** [Artificial Analysis](https://artificialanalysis.ai/models/muse-wonder) measured Wonder's time to first token at 19.27 seconds at launch, against a 5.52-second median for comparable models. An agent makes many calls per task, so expect that to show. **Cost.** Meta's docs warn that agents pointed at a custom provider often show $0 cost, because they lack pricing metadata. Don't read cost from session B's terminal. Take the run's input, cached and output token counts and do the math: input millions × $4, cached input millions × $0.20 (the cached rate [Handy AI](https://handyai.substack.com/p/model-drop-muse-wonder) reports), output millions × $20. The per-token prices match Opus 5.5, but per-task cost can still differ. Artificial Analysis counted 130M output tokens for Wonder to run its index, against an 85M median. For session A, Hermes shows an estimated cost per session. Press ⌘G (Ctrl+Shift+G on Windows and Linux) to open the Review Desk and switch to **By turn**. You read each agent's work turn by turn instead of as one final diff. Risk flags mark lockfiles, CI workflows, secrets, new binaries and edits an agent makes to its own config. *From hermes-ide.com: a recreation of the 2.0 app with demo data. The Review Desk by turn, with a line comment for Claude Code and a drafted commit message.* A line comment goes back to the agent that wrote the line. Count those comments: that is the last row of your scorecard. Hermes knows less about a Custom agent than a built-in one, so if session B's turns look thin, read it **By file**, or fall back to the script's numbers and a plain `git diff` against your base branch. Land the winner from the Land sheet: commit, open a pull request or merge locally, with undo afterwards. Then close the other session. By hand, remove the losing worktree with `git worktree remove ../bakeoff-muse` (or `../bakeoff-opus`). One ticket is an anecdote. Run three to five tickets of different shapes before you call it, because a model that wins a refactor can lose a bug hunt. Hermes has no status signals for Custom agents. It doesn't know the command's approval model, so the attention inbox can't tell when session B or C is waiting on you. Watch their terminals. Session A keeps full status. You're also testing Muse Wonder, not Wonder Full. The API doesn't serve Full, so only session C, on a plan that includes it, reaches it. And session B is Anthropic's harness on Meta's model: fair for comparing models, maybe not how Meta tuned Wonder to work, which is why session C exists. Hermes IDE is not affiliated with or endorsed by Meta, Anthropic, OpenAI or Nous Research. Pick the ticket you'd hand an agent this week, run it through both sessions, and land the one you'd rather review again. I build Hermes IDE. It is free, source-available, and runs on macOS, Windows and Linux. It ships often: 2.2.10 came out on October 12, If you're designing the agents themselves, the *AI Agents Pocket Guide* covers the patterns behind them.