{"slug": "mcpmark-v2-insforge-on-sonnet-4-6", "title": "MCPMark v2: InsForge on Sonnet 4.6", "summary": "InsForge MCP achieves 28% higher Pass⁴ accuracy while using 2.4x fewer tokens than Supabase MCP on Claude Sonnet 4.6, according to new MCPMark benchmarks. The efficiency gap has widened compared to Sonnet 4.5, with InsForge using 59% fewer tokens per run. Tony Chang, CTO and Co-Founder of InsForge, noted: 'The core finding holds: agents perform better when the backend gives them structured context and workflow upfront.'", "body_md": "Originally published on the[InsForge blog], written by Tony Chang (CTO & Co-Founder). Reposted here with permission.\n\nIn December we published the [first MCPMark benchmark results](https://insforge.dev/blog/mcpmark-benchmark-results) comparing InsForge MCP, Supabase MCP, and Postgres MCP across 21 real-world database tasks using Claude Sonnet 4.5. InsForge came out ahead on accuracy, speed, and token efficiency.\n\nWe reran the benchmarks. This time on **Claude Sonnet 4.6**, the latest model from Anthropic. InsForge MCP achieves 28% higher Pass⁴ accuracy while using 2.4x fewer tokens than Supabase MCP. The efficiency gap has widened.\n\nSame 21 MCPMark Postgres tasks, 4 runs per task, strict Pass⁴ scoring.\n\n| Metric | InsForge | Supabase MCP |\n|---|---|---|\n| Pass⁴ Accuracy | 42.86% | 33.33% |\n| Pass@1 Average | 58.33% | 47.62% |\n| Pass@4 | 76.19% | 66.67% |\n| Tokens Per Run | 7.3M | 17.9M |\n| Avg Tokens Per Task | 358K | 862K |\n| Avg Time Per Task | 156.6s | 198.8s |\n| Avg Turns Per Task | 18.6 | 17.0 |\n\nInsForge MCP maintains higher accuracy across all three metrics and uses **2.4x fewer tokens** per run.\n\nPass@1 is average single-run accuracy, Pass@4 means the agent passed at least once in 4 runs, and Pass⁴ requires passing all 4. InsForge passes 76% of tasks at least once (Pass@4) and 43% under strict repeated execution (Pass⁴). Supabase MCP reaches 67% and 33% respectively.\n\n| Model | Metric | InsForge | Supabase MCP |\n|---|---|---|---|\n| Sonnet 4.5 | Pass⁴ | 47.6% | 28.6% |\n| Sonnet 4.6 | Pass⁴ | 42.86% | 33.33% |\n| Sonnet 4.6 | Pass@4 | 76.19% | 66.67% |\n| Sonnet 4.6 | Pass@1 Avg | 58.33% | 47.62% |\n\nInsForge's accuracy advantage comes from surfacing backend state before the agent acts. When the agent can see record counts, RLS policies, and foreign keys upfront, it writes correct queries on the first attempt instead of guessing and retrying. That is why the gap holds across model versions.\n\nThis is the most notable change from v1.\n\nWith Sonnet 4.5, InsForge used approximately 30% fewer tokens than Supabase MCP (8.2M vs 11.6M per run). With Sonnet 4.6, the gap has grown to **59% fewer tokens** (7.3M vs 17.9M per run).\n\n| Model | InsForge Tokens/Run | Supabase Tokens/Run | Difference |\n|---|---|---|---|\n| Sonnet 4.5 | 8.2M | 11.6M | 1.4x |\n| Sonnet 4.6 | 7.3M | 17.9M | 2.4x |\n\nInsForge got slightly more efficient on Sonnet 4.6 (8.2M down to 7.3M). Supabase MCP went in the opposite direction (11.6M up to 17.9M). The newer model appears to reason more extensively when backend context is incomplete, which increases token consumption on backends that do not surface schema details upfront.\n\nWhen the backend provides structured context from the start, the agent reasons less and executes more. When it does not, the agent compensates with additional discovery queries and verification steps, and that compensation costs more tokens on a more capable model.\n\nTwo factors account for most of the gap:\n\n`search_docs`\n\nreturns full GraphQL schema metadata on every call, 5-10x more tokens per query than InsForge's `fetch-docs`\n\n.InsForge completes tasks in an average of **156.6 seconds** compared to **198.8 seconds** for Supabase MCP. This is a 1.27x speed advantage, consistent with what we observed on Sonnet 4.5.\n\nThe core finding from our [original benchmark post](https://insforge.dev/blog/mcpmark-benchmark-results) holds: agents perform better when the backend gives them structured context and workflow upfront. The Sonnet 4.6 results reinforce this and show that the advantage grows as models become more capable.\n\nMore capable models do not eliminate the need for structured backend context. They amplify the cost of not having it.\n\nWe will continue running benchmarks as new models are released and as we improve the InsForge MCP layer. All benchmark methodology follows [MCPMark](https://mcpmark.ai) standards and is fully reproducible. The latest raw results are available on [GitHub](https://github.com/InsForge/mcpmark/tree/main/results).", "url": "https://wpnews.pro/news/mcpmark-v2-insforge-on-sonnet-4-6", "canonical_source": "https://dev.to/carmen_dou/mcpmark-v2-insforge-on-sonnet-46-304j", "published_at": "2026-07-22 20:47:40+00:00", "updated_at": "2026-07-22 21:01:55.572540+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "developer-tools"], "entities": ["InsForge", "Supabase", "Anthropic", "Claude Sonnet 4.6", "Tony Chang", "MCPMark"], "alternates": {"html": "https://wpnews.pro/news/mcpmark-v2-insforge-on-sonnet-4-6", "markdown": "https://wpnews.pro/news/mcpmark-v2-insforge-on-sonnet-4-6.md", "text": "https://wpnews.pro/news/mcpmark-v2-insforge-on-sonnet-4-6.txt", "jsonld": "https://wpnews.pro/news/mcpmark-v2-insforge-on-sonnet-4-6.jsonld"}}