cd /news/ai-agents/show-hn-mcp-crashtest-multi-step-sta… · home › topics › ai-agents › article
[ARTICLE · art-173132] src=github.com ↗ pub= topic=ai-agents verified=true sentiment=· neutral

Show HN: MCP-CrashTest – Multi-step stateful contract fuzzer for MCP servers

A developer released MCP-CrashTest v0.2.1, an open-source stateful contract fuzzer that runs guided multi-step random walks against Model Context Protocol servers inside ephemeral OS sandboxes and outputs deduplicated minimal JSON-RPC reproducers. Testing on 2026-11-20 in both `--adapter none` zero-shot mode and `--adapter auto` found that `mcp-server-sqlite` returns `isError: false` on SQL failures such as UNIQUE constraint violations, and that `mcp-server-git` passes human CLI formatting like "* main" from `git_branch` into `git_checkout`, `git_diff`, `git_show` and `git_merge`, which then fail. The tool requires Python >= 3.10 and targets agents that chain 5–20 tool calls, where single-step unit tests miss cross-tool contract breaks and stdio deadlocks.

key takeaway

MCP-CrashTest v0.2.1 fuzzes 5–20 step MCP tool chains with state-aware random walks and POSIX inode-preserving rollback, and its first runs flagged `mcp-server-sqlite` returning `isError: false` on SQL errors and `mcp-server-git` leaking "* main" CLI formatting into downstream tools.

read7 min views8 publishedNov 20, 2026
Show HN: MCP-CrashTest – Multi-step stateful contract fuzzer for MCP servers
Image: Michielbdejong (auto-discovered)

Unit tests check single tool calls. Real agents fail on multi-step contracts.

State-aware random walks with POSIX inode-preserving rollback for Model Context Protocol (MCP) servers.

Tools like MCP Inspector and standard unit tests validate single-step static calls. In production, AI agents execute 5–20 step tool chains where outputs from discovery/list tools feed downstream mutations, entities collide, and stdio pipes hang.

Inspired by reversing the state-aware random walk paradigm from ToolSpace (arXiv:2611.05036)—instead of keeping successful trajectories for model training and discarding rollbacks—MCP-CrashTest mines a dependency graph from tools/list, runs guided multi-step random walks inside ephemeral OS sandboxes, rolls back on failure to explore neighboring branches, prunes noisy intermediate steps, and outputs deduplicated minimal JSON-RPC reproducers.

Masked Errors (isError: false on SQL/Runtime Failures) — Protocol Violation (CRITICAL) Inmcp-server-sqlite (verified on2026-11-20 withMCP-CrashTest v0.2.1 inboth--adapter none pure zero-shot mode and--adapter auto ), SQL errors returnisError: false , causing agents to treat error strings as valid query results (context poisoning):

{
  "method": "tools/call",
  "params": {
    "name": "write_query",
    "arguments": {
      "query": "INSERT INTO users (id, name, email) VALUES (1, 'CollisionBob', 'bob_new@example.com');"
    }
  },
  "response": {
    "text": "Database error: UNIQUE constraint failed: users.id",
    "isError": false
  }
}

CLI Output Leaks Across Tools — Cross-Tool Contract Break (HIGH) Inmcp-server-git ,git_branch outputs human CLI formatting ("* main" ). Passing that branch name directly togit_checkout ,git_diff ,git_show , orgit_merge fails:

[
  { "tool": "git_branch", "response": { "text": "* main", "isError": false } },
  { "tool": "git_merge", "arguments": { "source_branch": "* main" }, "response": { "text": "Error executing tool git_merge: Ref '* main' did not resolve to an object", "isError": true } }
]

Silent State Corruption & Stdio Deadlocks — Protocol Violations (CRITICAL) Read-after-write verification catches tools that reportisError: false without persisting state, as well as hungstdio buffers or severed child-process pipes (with automatic subprocess recycling). 4. Agent Contract Smells — UX & Consistency Pitfalls (MEDIUM / LOW / INFO) Explicitly separated from protocol bugs: undocumented glob-onlysearch_files matching, relative vs. parent-joined path contracts, no-upsert skip behavior (create_entities ), and intra-serverisError inconsistencies.

Requires Python >= 3.10.

uvx mcp-crashtest --target sqlite --walks 15 --depth 6

git clone https://github.com/SnowyLumiere/mcp-crashtest.git
cd mcp-crashtest
pip install -e ".[dev]"

Use -- to pass the server command cleanly without option collisions, and --adapter none to run pure zero-shot generic stateful fuzzing with zero domain-specific assumptions:

mcp-crashtest \
  --target my-server \
  --adapter none \
  --sandbox directory \
  --seed 42 \
  --walks 20 \
  --depth 6 \
  --fail-on-crash \
  --output reports/my_server_report.md \
  -- python3 -m my_mcp_server --root {sandbox}

For stateless/API-only servers, use --sandbox none (context-memory rollback and branch exploration still run):

mcp-crashtest \
  --target my-api-server \
  --adapter none \
  --sandbox none \
  --walks 20 \
  -- node dist/index.js --env test

Requires npx (filesystem / memory) or uvx + git (sqlite / git):

mcp-crashtest --target sqlite --walks 15 --depth 6 --output reports/sqlite_crash_report.md
mcp-crashtest --target git --walks 15 --depth 6 --output reports/git_crash_report.md
mcp-crashtest --target filesystem --walks 15 --depth 6 --output reports/filesystem_crash_report.md
mcp-crashtest --target memory --walks 15 --depth 6 --output reports/memory_crash_report.md

MCP-CrashTest separates the generic zero-shot stateful engine from optional domain contract adapters (mcp_crashtest/adapters/):

Capability Generic Mode ( --adapter none ) Built-in Domain Adapters ( --adapter auto )
Schema Graph Mining Infers CREATE /READ /MUTATE /DELETE +DATA /STATE /RETRY edges fromtools/list ; handles Pydantic v2anyOf /oneOf /enum andproperties: null Adds domain-specific edge bindings for filesystem ,sqlite ,memory ,git
Context Flow & Fuzzing Extracts IDs/keys/URIs/paths from args & JSON responses; injects boundary strings, traversal paths, extreme ints Adds domain-specific probes (e.g. PK collisions, glob-vs-substring queries, raw * main branch passing)
Post-Condition Probes Probe A ( RETRY_TRAP ), Generic Probe B (Read-After-Write onid /key /uri ), Probe C (SCHEMA_BREAK ), Probe D (STDIO_DEADLOCK ) Adds calibrated CONTRACT_SMELL checks (e.g.search_files glob pitfall,describe_table``[] ,create_entities no-upsert skip)
Report Provenance Every finding is tagged with source: generic Every adapter finding is explicitly tagged source: adapter:<name>
--sandbox CLI Placeholders Injected Env Var Seeded Initial Data
directory(default) {sandbox} ,{work_dir} MCP_SANDBOX_DIR sample.txt ,data_dir/config.json ,data_dir/nested_note.txt
git {sandbox} ,{work_dir} MCP_SANDBOX_DIR Initialized main branch + initial commit (commit.gpgsign=false )
sqlite {sandbox} ,{db_path} MCP_SANDBOX_DB_PATH test_target.db with seededusers table (2 rows)
memory {sandbox} ,{memory_path} MEMORY_FILE_PATH Isolated temp path memory_graph.json
none — — No local filesystem sandbox (context rollback still active)

Verified on 2026-11-20 (--walks 15 --depth 6 --rollbacks 3 --seed 42, Python 3.12, mcp SDK v2.3.0 client with full v1.x/v2.x compatibility; pruned reports in reports/):

Server Mode Protocol Violations Contract Smells Headline Minimal Reproducers Report
mcp-server-sqlite --adapter none(pure zero-shot) 3 ( CRITICAL ) 0 write_query ,create_table ,read_query return"Error: ..." withisError: false (1-step each,source: generic ) —
mcp-server-sqlite --adapter auto 3 ( CRITICAL ) 1 (MEDIUM ) SQL errors return isError: false (×3, 1-step);describe_table returns[] on missing table (1-step) sqlite
mcp-server-git --adapter auto 0 4 ( HIGH``SCHEMA_BREAK ) git_branch leaks'* main' , breakinggit_checkout ,git_diff ,git_show ,git_merge (2-step each) git
@modelcontextprotocol/server-filesystem --adapter auto 0 4 (MEDIUM ) list_directory bare filenames vsread_file /read_text_file /get_file_info (2-step);search_files glob-only miss (1-step) filesystem
@modelcontextprotocol/server-memory --adapter auto 0 3 (LOW /INFO ) create_entities skips existing entities withisError: false (2-step);delete_* idempotent0 differs fromadd_* (1-step) memory

Notes:

  • The sqlite preset launches uvx --with "mcp<2" mcp-server-sqlite because upstream mcp-server-sqlite currently uses the mcp 1.x @server.list_resources() API and fails to boot under mcp 2.x. MCP-CrashTest itself supports both mcp SDK v1.x (inputSchema/ isError) and v2.x ( input_schema/ is_error). reports/MCP_CRASHTEST_COMPREHENSIVE_DIAGNOSIS_20261106.md is preserved as the historical v0.2.0 archive.
  1. Strict isError Semantics : Never returnisError: false on SQL errors, constraint violations, or failed mutations.
  2. Cross-Tool I/O Symmetry : Discovery/list tool outputs must be directly consumable by downstream tools without CLI decorations (* main ) or stripped parent paths.
  3. Explicit Matching & Upsert Contracts : Document glob vs. substring matching and provide explicit upsert/conflict errors instead of ambiguousisError: false skips.
  4. Actionable Errors & Pipe Backpressure : Return offending parameter names and suggested corrective tools; paginate large responses to preventstdio deadlocks.
Flag Default Description
--target "custom" Target label or built-in preset ( filesystem ,sqlite ,memory ,git )
--adapter "auto" Domain adapter mode: auto ,none (pure generic), or explicit adapter name(s)
--cmd /-- <cmd...> None Server executable (prefer POSIX -- passthrough to avoid flag clashes)
--sandbox "directory" directory |sqlite |git |memory |none
--walks 15 Number of random walk chains
--depth 6 Max step depth per walk
--rollbacks 3 Max rollback retries per walk
--fuzz-ratio 0.25 Boundary fuzzing probability ( 0.0 –1.0 )
--timeout 8.0 Per-step & tools/list timeout in seconds
--init-timeout 20.0 Session initialize() timeout in seconds
--seed None Local deterministic RNG seed (does not mutate global random state)
--lang "en" Report language: en |zh
--fail-on-crash False Exit with code 1 if findings matching--fail-on are detected (for CI gates)
--fail-on "any" CI failure scope: any (all findings) |protocol (PROTOCOL_VIOLATION only)
--output "reports/mcp_crash_report.md" Output Markdown report path
--version — Print version and exit
  • Transport : Currently supports**stdio** transport only.Streamable HTTP / SSE support is on the roadmap.
  • SQLite Sandbox : In-place binary file rollback assumes per-call SQLite connections (such asmcp-server-sqlite ). Long-lived connection pools with in-memory page caches require session restarts between walks.
  • Process Memory : For servers keeping unpersisted in-process state, sandbox rollback restores disk state only.
import asyncio
from mcp_crashtest.cli import run_crash_test

async def main():
    crashes = await run_crash_test(
        cmd="python3",
        args=["my_server.py", "--root", "{sandbox}"],
        target_name="my-server",
        adapter_mode="none",
        sandbox_type="directory",
        seed=42,
        lang="en",
        output_report_path="reports/my_server.md",
    )
    print(f"Hits: {len(crashes)} | Unique: {len({c.fingerprint for c in crashes})}")

asyncio.run(main())

Run the test suite (zero third-party dependencies required for unittest):

PYTHONPATH=. python3 -m unittest discover -s tests -v
PYTHONPATH=. pytest tests/ -v

Released under the MIT License.

── more in #ai-agents 4 stories · sorted by recency
── more on @mcp-crashtest 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/show-hn-mcp-crashtes…] indexed:0 read:7min 2026-11-20 · —