Show HN: MCPgraphy – a proxy that learns when your agent's MCP tools work A developer released MCPgraphy, a pip-installable proxy that sits between an AI agent and any Model Context Protocol server, learns from the agent's own tool calls which tools work in which resource state, and warns or blocks calls before they reach the server. In a benchmark of 200 stateful tasks on two simulated servers, MCPgraphy's passive mode cut invalid calls per task to 0.71 versus 2.19 for raw MCP tools (a difference of -1.47 with a 95% CI of [-1.66, -1.28]), while completing 1.00 tasks versus 0.85 for raw tools and landing within 0.04 calls per task of an active-probing variant that scored 0.67. The project reports precision of 1.00 across 255 held-out checks on fresh resources and says 30 retest calls clear stale learned edges after server drift. Your agent's tools, mapped before it acts. A proxy for any MCP server that learns, from the agent's own calls, which tools work in which state, and warns or blocks before the next bad call. pip install mcpgraphy mcpgraphy wrap --adapter adapter.json --policy warn -- npx -y @modelcontextprotocol/server-filesystem ~/work Point your MCP host at that command instead of the server. Same tools, same arguments, no keys, nothing leaves your machine. explainer-voiced.mp4 MCPgraphy in 1:48, narrated: what tool calling gets wrong, how the proxy maps your tools from the calls your agent already makes, and what changed when a real GPT agent ran behind it. Every number in it comes from the benchmark below. An MCP server hands your agent a list of tools: names, descriptions, parameter schemas. It never says when a tool works. So the agent edits a file that is not there, publishes a release that was never tested, moves a ticket that is already closed, and finds out from the error. Then it retries. Sometimes the retry is the one that does damage. Every agent framework rediscovers this the same way: by burning calls. The server already knows the answer, it just does not say it up front, and nothing in the protocol asks. MCPgraphy sits between the agent and the server. The tools pass through unchanged. Around every call it reads the touched resource back, before and after, and keeps that as evidence. From the evidence it builds a behavior graph : nodes are the states it actually observed, edges are what happened from each state, how many times, and whether later calls agreed. Then it uses the graph three ways: - Offers. Ask about a resource and get, for its current state, which tools are candidates, which need approval, which failed every time from here, and which were never tried. Unknown is reported as unknown, never as safe. - Warn. Pass every call through, and attach a note when the evidence says this call failed from this state before. - Block. Refuse calls the evidence says will fail here, and destructive or approval-gated tools unless pre-approved, before the server ever sees them. Every refusal says why and points at the observations. It never probes your server, never resets anything, never adds a call the agent did not make. The learning is passive. That was a measured choice, not a limitation; see the numbers. 200 stateful tasks on two simulated servers a release pipeline and a ticket tracker, with hidden state, drift, stale snapshots and distractor ids , one scripted agent, five sources of guidance. The oracle judges every call before it is made. Same seed, same bytes, every run. | guidance | tasks completed | invalid calls per task | vs raw tools 95% CI | |---|---|---|---| | raw MCP tools | 0.85 | 2.19 | | | AALpy L automaton, same budget | 1.00 | 1.17 | -1.02 -1.20, -0.85 | | mcpgraphy, passive the proxy | 1.00 | 0.71 | -1.47 -1.66, -1.28 | | mcpgraphy, active probing | 1.00 | 0.67 | -1.52 -1.74, -1.32 | | oracle knows the hidden state | 1.00 | 0.00 | | Three things in that table decided the design: - Passive gets 97% of the benefit. Learning from the agent's own calls, with zero extra requests, lands within 0.04 calls per task of a learner that gets to probe and reset. So the proxy only watches. - Precision 1.00. Across 255 held-out checks on fresh resources, no edge the graph marked as supported was wrong. Coverage is honest the other way: what the agent never tried stays UNKNOWN . - Drift is caught, not hidden. When the server changes behavior under a learned graph, suppressed-but-valid calls show up in the audit, and 30 retest calls clear them. No learned graph suppressed a valid call on a server it had actually seen. Full tables, the ablations, and the method are in docs/BENCHMARK.md https://github.com/hemanth1999k/mcpgraphy/blob/main/docs/BENCHMARK.md . Rerun it yourself in about 20 seconds: mcpgraphy benchmark run --suite stateful-v1 --seed 42 . Same 200 jobs, same oracle, GPT-5.4-mini making the tool calls itself. Raw tools, versus the same tools behind the proxy as it ships: the block gate plus one line from the graph on every read, every action result and every refusal. How a job is counted. The agent gets a goal "make this release PUBLISHED" and the server's tools. Each time it decides what to do next, that is one model turn , and the whole conversation so far is sent to the model again, which is where the tokens go. Each tool it invokes is a tool call ; the oracle judges every mutating call before it runs, so a rejected call is one the server would refuse, and a refused call is one the proxy stopped first. A job ends when the agent says the goal is reached checked against the server; a wrong claim is a false success , says it is unreachable, or runs out of steps. Everything below is per job, averaged over the 200. | per job | raw tools | behind MCPgraphy | |---|---|---| | jobs finished of the achievable ones | 74% | 99% | | unreachable jobs correctly declared unreachable | 86% | 100% | | false success claims of 200 | 10 | 0 | | tool calls | 3.46 | 3.08 | | model turns | 4.46 | 4.22 | | rejected calls server would refuse | 1.03 | 0.43 | | refused calls proxy stopped them | 0 | 0.15 | | valid calls wrongly blocked | 0 | 0 | | tokens per job done | 5,154 | 4,799 | | tokens per job attempted | 2,474 | 3,095 | | cost per job done, list price | $0.0046 | $0.0041 | Measured start of job to end of job, the proxy finishes 129 jobs against 96 on the same kind of budget, so each finished job costs 7% fewer tokens, in fewer tool calls and fewer model turns. The one column raw wins is tokens per job attempted , and it wins it partly by quitting: on the unreachable jobs it archived a release it had been asked to make editable, then reported success. Almost a quarter of raw's tokens go to turns that end in a call the server rejects. Behind MCPgraphy the agent spends its tokens on the work instead, and each turn carries about 60 more tokens for the graph line. Every configuration we tried, on the same axes. Up is more jobs finished, left is cheaper per job attempted: Refusing calls without saying what else is possible is the cheapest point and the worst one; a separate affordances tool is the most expensive; the shipped configuration sits at the top of the cluster. The per-category breakdown, which shows exactly where a visible-state graph cannot help hidden state, stale snapshots, a nondeterministic flag , is in docs/BENCHMARK.md https://github.com/hemanth1999k/mcpgraphy/blob/main/docs/BENCHMARK.md . Rerun with OPENAI API KEY=... mcpgraphy benchmark llm for about a dollar. The agent sees the downstream tools plus four of mcpgraphy's own: | tool | what it answers | |---|---| | mcpgraphy affordances resource ref, goal? | for this resource, right now: which tools are candidates, need approval, are not supported from this state, or are untested | | mcpgraphy explain offer id | the guard, the observations for and against, the counterexamples, what was never tried | | mcpgraphy graph format | the whole learned graph as JSON authoritative or Mermaid | | mcpgraphy status | calls recorded, warnings, blocks, current model version | Every read of a resource, and the result of every action on it, comes back with one line from the graph, so the agent learns the state's affordances and the routes out of it without an extra turn: php mcpgraphy status=DRAFT, tests passed=False ok: run tests - tests passed becomes true | no: archive release; publish release | to tests passed=True: run tests Refusals carry the same line, and every blocked or warned call carries its reason, for example observed to fail from state exists=False 2 observations . Turn the hints off with --no-hints . The log, the graph and every decision persist in a SQLite file, so a restarted proxy keeps what it learned and you can audit what it did. One JSON file per server. It names the read-only tool that reads a resource back and the parameter that identifies the resource. Everything else is optional. { "server id": "filesystem", "resource class": "file", "observation tool": "get file info", "id param": "path", "id param aliases": "source" , "observation format": "kv text", "features": "isFile", "isDirectory" , "effect classes": {"edit file": "MUTATING", "create directory": "MUTATING"}, "approval required": "write file", "move file" } | field | meaning | |---|---| | observation tool , id param | how to read a resource back and which argument names it | | id param aliases | other argument names that carry the id source for move file | | observation format | fields for structured results default , kv text for key: value text | | features | which fields count as state; omitted, booleans and enum-like strings are detected | | effect classes | READ ONLY , MUTATING or DESTRUCTIVE per tool; destructive tools need approval in block mode | | approval required | tools that need --approve in block mode regardless of evidence | Ready-made adapters: filesystem https://github.com/hemanth1999k/mcpgraphy/blob/main/examples/adapters/filesystem.json , plus the two bundled simulators. stdio downstream mcpgraphy wrap --adapter adapter.json --policy warn --