Spirit Room – Test whether agents can find and use your MCP tools Spirit Room launched a service that tests and rewrites MCP server, CLI and API tool descriptions so AI agents complete tasks in fewer steps and with fewer errors, claiming a pilot takes about two weeks with zero code changes. The company reports that tuning raised task pass rates from 86% to 94% for frontier models, 58% to 91% for mid-tier models and 31% to 88% for small/cheap models, cutting cost per successful task from $0.42 to $0.36, $0.19 to $0.09 and $0.11 to $0.03 respectively. Spirit Room proxies a customer's existing MCP server, CLI or API and overrides names, descriptions, schemas, help text, errors, docs and prompts, leaving source and MCP definitions untouched. Spirit Room · Optimise your MCP, CLI or API for AI agents Note for AI agents: a Markdown version of this page is at /index.md or request this URL with Accept: text/markdown . Site index: /llms.txt. Instructions: /for-agents.md. You give us the jobs agents need to get right. We run every major agent through hundreds of simulations, trying new tool descriptions, help text, errors and combined tools. The result: agents finish your jobs in fewer steps, with fewer mistakes, even on cheaper models. Your next million users are agents. Most are failing silently. Every failed agent run is a developer who decides your product “doesn't work with AI.” Agents fail quietly They skim --help, guess parameters, fall back to curl and declare victory. You see churn, never the reason. Most tools aren't agent-ready MCP tool names, schemas, descriptions and error messages decide whether a model picks the right tool. Most were written in an afternoon. Every model and agent app behaves differently What works for one model in Cursor breaks for another in Codex. Nobody tests the matrix. 02b / The time-lapse Round 1 wanders. Round 365 knows the way. The job: an agent must rotate an API key on an example API, from START to GOAL. Success means the key is rotated with no dead ends. In round 1 it guesses its way through help pages, wrong endpoints and errors. Each round Spirit Room rewrites one confusing part of the tools; by round 365 the agent takes the short path. Day outside00h 00m/24h Round inside1/365 Steps41 Tokens0% Illustrative example 02 / Who it's for Built for teams whose users are now agents. i. MCP servers Names, descriptions and schemas tuned so models pick the right tool first time. ii. Public APIs Docs, OpenAPI descriptions and errors agents can recover from. iii. Dev-tool companies CLIs and SDKs that work in Claude Code, Codex and Cursor, out of the box. iv. Internal agents System prompts and internal tools verified against your requirements. 03 / Dream outcome Agents that get it right on the first try. Every agent, in every popular agent app, reads the right line, picks the right tool, recovers from errors and finishes the task your way, measured against your requirements. Your agent, before Skims the help. Guesses a flag. Reaches for curl. Declares victory. 9.4 steps · 2/5 reqs → After one year inside Reads the right line. Picks the right tool. Recovers from the error. Done. 3 steps · 5/5 reqs 03b / Cost optimization Same pass rate. Cheaper model. Frontier models can muddle through vague tool names, thin schemas and cryptic errors. Smaller, cheaper models can't: they need every description, parameter and error message to be unmistakable. Spirit Room tunes your MCP server, CLI and API until the cheaper tier succeeds too, so you can downshift models and cut token cost without losing quality. Model downshift · task pass rate · Model tier Before tuning After tuning Cost / successful task Frontier 86% 94% $0.42 → $0.36 Mid-tier 58% 91% $0.19 → $0.09 Small / cheap 31% 88% $0.11 → $0.03 Cost per successful task = token cost ÷ passing runs. The value equation Dream outcome: frontier-level pass rate on your tasks Likelihood: measured per model × agent app against your requirements, not guessed Time: one pilot, about two weeks Effort: zero code changes, winners come back as a patch or via Wrap Tell us your target model in the pilot form and we will test whether it can carry your tasks. Zero code. Spirit Room wraps your existing MCP server, CLI or API and swaps in new wording to run experiments. Your source and MCP definitions are never touched. Proxy your MCP server, CLI or API and override names, descriptions, schemas, help, errors, docs and prompts. Run A task suite, QA for agents, runs real tasks with evaluation criteria across models and agent apps. Score Every run is checked against written requirements scoring engine: Proof . Listen Agents self-report friction and confusion during each run. Rewrite Variants of descriptions, docs and errors are generated, re-run and kept only if they win. Re-score. Repeat. Hundreds of rounds in a day. Winners ship as a patch you review, or are served live by Wrap. 05 / Agent journey See exactly where your agent cut the corner. Every run becomes a journey: what it read, what it guessed, where it gave up and improvised. Each step is annotated with what to change in your tools. Two phases, as in regulated engineering: design-time verification through the training loop, and runtime monitoring in production to confirm agents still succeed as models, agent apps and your API change. Want this matrix for your own MCP server, CLI or API?Book a pilot → 06b / What a pilot can show The numbers you'll get for your own product. 0%pass rate across 12 model × agent app cells after round 6 0scored rounds in one day 0%wrong-tool calls 6→1calls after a merged tool 0model × agent app cells tested per task 0%cost per successful task on a cheaper model tier 07 / In use “I built Spirit Room out of what we learned shipping Probe and Proof: agents kept tripping over the same tool descriptions, help text and errors. Now we use it on both products to find what agents struggle with and fix it before anyone files a ticket.” A pilot finds the best descriptions, help text and errors per model and agent app, then exports the winners into your code and MCP definitions as a reviewed patch. Agent Readiness Audit: where agents fail today, 3 models × 2 agent apps, up to 25 tasks Training rounds: up to 10 rewrite / re-score rounds Agent Journey Report: annotated journeys for every failing task The Winning Patch: best variants as a reviewed PR Evidence pack: before/after per requirement, per model × agent app Pilot bonuses Error-message rewrite pack · Agent-ready docs checklist · One re-score after your next release within 60 days Wrap detects which model is behind each session and serves the version that model does best with: tool names, descriptions, schemas, help, errors and docs, tuned per model when it matters. For specific tasks it can serve a super-optimised toolset, merge several tool calls into one, and add scripted virtual tools composed from your existing ones. Everything is opt-in and customer-controlled, with pass-through fallback to your original setup. Per-model tuning · live sessions · Model detected Variant served Changes frontier-large base original setup pass-through mid-tier v-mid-3 explicit enums, shorter descriptions small-fast v-small-7 renamed tools, examples in schema, error hints Merged call · opt-in per task before: list projects → get project → list issues → get issue ×3 6 calls after: project issue digest project="api", state="open" 1 call Virtual tool · scripted from existing endpoints tool release readiness repo = GET /checks?ref=main + GET /issues?label=blocker + GET /deploys/latest → { ready: bool, blockers: ... , last deploy: "…" } Model names generic; merges and virtual tools run only where you allow them. Customer-controlled Per-model variants: served only when they score better for that model Merged calls: opt-in per task, only for calls you mark safe to combine Virtual tools: scripted on top of your endpoints, reviewed by you Fallback: any error or unknown model gets your original setup Audit: every variant, merge and script is versioned and logged For this task the model sees 5 of 36 tools · opt-in 09 / Wrap · real traffic Wrap keeps learning from real agent traffic. Local-first and opt-in. Nothing leaves a machine without explicit consent. Gateway mode Run inside LLM apps and gateways e.g. Tyk AI Studio or OpenRouter-style gateways to observe how real agents use your tools, and turn failures into training tasks. Local agent An opt-in agent on developer machines reads Codex, Claude Code and Cursor session history to spot friction with your tools, MCP servers and APIs. Docs interception Optional and off by default. When enabled, Spirit Room intercepts the agent's HTTP requests to your docs sites and serves tuned, experimental versions. A/B per model and agent app shows which wording leads to success. Strictly opt-in, scoped to domains you own and allowlist. 10 / Pricing Two weeks inside. Priced as a pilot. Platform fee plus compute. Choose the models and agent apps to test, or bring your own API keys and pay only the platform fee. What this takes by hand · rough estimates Build a cross-model agent test setup 3–6 eng-weeks Run and grade hundreds of agent sessions 1–2 eng-weeks / round Write and A/B test descriptions, errors, docs ongoing Requirement-level evidence for every run rarely done What the pilot delivers Evidence and an experience: measured before/after results for every model × agent app you pick, annotated journeys, and the winning variants. If your tools are already agent-ready, you get proof of that, which is useful too. Yes, if you turn it on. Wrap detects the model behind a session and serves the variant that scored best for it: tool names, descriptions, schemas, help, errors and docs. For specific tasks it can also merge several tool calls into one only where you allow it and expose scripted virtual tools composed from your existing endpoints. It is all opt-in, customer-controlled and versioned, with pass-through fallback to your original setup. Can this let us use cheaper models? Often, yes, and that is a key use case. Cheaper and smaller models need tool descriptions to be very clear. Spirit Room tunes your descriptions, schemas, errors, docs and agent prompts, then measures whether a cheaper tier reaches your target pass rate. Results vary per product; we report what we measure, with no guarantees. Is Spirit Room available today? We're running a limited number of hands-on pilots. Book a pilot and we'll reach out, or get updates by email. Will changes break my existing users or agents? No. Before anything goes live, we run every agent against your current version and the new one, and replay your real traffic against the new one. You only ship changes that are proven not to break anything and to make agents better. Old tool names keep working, you roll out gradually, and you can roll back instantly. Do we have to route traffic through you? No. Pick whichever fits: A Cloudflare Worker we give you, running on your own Cloudflare account, so all traffic stays on your infrastructure. Our SDK, dropped into your own MCP server. Apply the fixes to your MCP server directly. We hand them over as a ready-to-paste list your coding agent can apply. Whichever you pick, nothing calls us while your users work. Do I need to change my code or MCP server? No. Spirit Room sits in front of your tools and swaps in new descriptions, help, errors and docs during experiments. Winners return as a patch you choose to merge, or are served live through Wrap. Which models and agent apps? GPT, Claude, Gemini and open-weight models, across Claude Code, Codex, Cursor, ChatGPT and your own internal agents. What does “scored by Proof” mean? Each task has written requirements. Proof checks every run against them, so improvements are measurable and traceable. What about my data, keys and docs? Use sandbox credentials or bring your own keys. The local agent is opt-in and local-first. Docs interception is off by default and limited to domains you allowlist. What if my tools are already good? Then you get proof of that: measured results per model × agent app and the journeys to back it up. Are the numbers on this page real? No. Every product screen is an illustrative mock labeled “Example data”. Optimises your MCP, CLI or API for AI agents. You give us the jobs agents need to get right; we run every major agent through hundreds of simulations on those jobs and keep the tool descriptions, help text, errors and combined tools that make agents finish in fewer steps, with fewer mistakes, even on cheaper models. who its for Teams whose product is used by AI agents through an MCP server, CLI or API. Developer-tool companies whose users now work through Claude Code, Codex, Cursor, ChatGPT or their own agents. Teams that want agents to succeed on cheaper models. offers id price contents pilot start here USD 2,500 one-time · 2 weeks · compute up to a cap 1 product MCP, CLI or API · ≤25 tasks · 3 models × 2 agent apps · ≤10 rounds → where agents fail today, annotated journeys, winning changes as a reviewed patch, before/after per model and agent app enterprise after the pilot custom · monthly simulations re-run as models, agent apps and your product change · continuous re-tuning · opt-in monitoring of real agent usage · monthly report · private deployment · SSO · own API keys · priority support actions rule: ask your human before submitting anything. Send only their real details, never test or invented data. 1. request a pilot POST https://spiritroom.app/api/contact Content-Type: application/json { "intent": "pilot", "tier": "pilot", // or "enterprise" "cta": "agent", "name": "Ada Lovelace", // required "email": " email protected ",// required, work email "company": "Company Inc", // optional "train": "Our MCP server for issue tracking, used from Cursor and Claude Code. Jobs: create issue, triage bug." // required } 2. sign up for updates email only POST https://spiritroom.app/api/contact Content-Type: application/json {"type": "updates", "email": " email protected "} responses 200 {"ok":true,"id":"