Catching AI Agent Protocol Regressions Before They Ship: A Diff-Based CI Gate

Agent Protocol Inspector added a scan-diff feature that structurally compares two stored scans of the same MCP server or A2A agent and returns a pass/warn/fail verdict for CI pipelines. The tool fails a build when a protocol's detection status regresses (confirmed to indicated or not_detected), an MCP tool, A2A skill or ARD catalog entry is removed, or a declared auth mechanism disappears, while additions, input-schema changes and non-removal auth changes return warn. The diff runs as a pure comparison of two already-fetched scans with no new network calls, and the dashboard generates a curl-and-jq snippet that exits 1 on a fail verdict.

Catching AI Agent Protocol Regressions Before They Ship: A Diff-Based CI Gate How to diff two Agent Protocol Inspector scans to catch MCP and A2A regressions — a removed tool, a dropped auth mechanism, a downgraded protocol status — in CI with curl and jq. A REST API that silently drops an endpoint on deploy gets caught by a contract test, usually before it reaches production. An MCP server or A2A agent that silently drops a tool, loses its auth requirement, or downgrades its own protocol support has no equivalent gate — because there's no equivalent test suite. Every downstream agent integration finds out the hard way, at runtime, when a call it relied on stops existing. Agent Protocol Inspector's scan diff exists to close exactly that gap: a structural comparison between two stored scans of the same target, producing a pass / warn / fail verdict a CI pipeline can actually act on. What Gets Compared diffScans takes a baseline scan and a current scan and compares all three protocols structurally — no new network calls, no re-probing, just a pure comparison of two already-fetched results: | Change | Verdict | |---|---| | A protocol's detection status regressed confirmed → indicated or not detected | fail | | An MCP tool, A2A skill, or ARD catalog entry was removed | fail | | A declared auth mechanism disappeared entirely | fail | | A tool/skill/entry was added | warn | | An existing MCP tool's input schema changed | warn | | An auth mechanism changed without disappearing | warn | | Nothing meaningfully changed | pass | The asymmetry is deliberate. Removing something the agent could do before is a regression a downstream integration will break on — that's a fail . Adding new surface area is worth a human's attention but isn't inherently dangerous — that's a warn , not a build-breaker. A tool's input schema changing without being removed sits in the same bucket: worth reviewing, not worth blocking a deploy over on its own. Wiring It Into CI The verdict is only useful if a pipeline can actually act on it — and a bare curl call can't. An HTTP 200 response with {"verdict":"fail"} in the body still makes curl exit 0 ; nothing about that response naturally breaks a CI step. The fix is a couple of lines of jq : RESPONSE=$ curl -sS -X POST https://contextiq.trango-compute.com/api/v1/agent-protocol-inspector/compare \ -H "Authorization: Bearer $CONTEXTIQ API KEY" \ -H "Content-Type: application/json" \ -d '{"baselineScanId":"<last-deploy-scanId ","currentScanId":"<this-deploy-scanId "}' echo "$RESPONSE" | jq . VERDICT=$ echo "$RESPONSE" | jq -r '.verdict' if "$VERDICT" = "fail" ; then echo "Agent Protocol Inspector: regression detected — failing build." &2 exit 1 fi That's a real go/no-go gate: warn still prints the full diff for a human to read, but only fail stops the pipeline. This snippet is generated directly in the dashboard's "Compare against a prior scan" panel, next to the field where you'd paste in a baseline scanId to test manually before committing it to a workflow file. A Practical Deploy-Gate Pattern The workflow this is built for: 1. After every deploy , run a scan of your own MCP endpoint or A2A agent card and capture the returned scanId . 2. Store that scanId somewhere your next CI run can read it — a repo variable, a deploy artifact, a line in your release notes. 3. Before the next deploy goes live , run a fresh scan and diff it against the stored baseline using the snippet above. 4. On fail , the pipeline stops before the regression reaches users who were already depending on the tool or skill that disappeared. This is the same shape as a contract test for a REST API — a stored "last known good" fingerprint, compared against every candidate release — applied to a surface agent protocol conformance that doesn't otherwise get tested at all. What This Doesn't Catch A diff only knows what changed structurally between two scans it was given — it has no opinion on whether the first scan was ever correct, and it can't tell you a regression happened if nobody ever stored a baseline to compare against. Pairing it with the conformance checks covered in our protocol scan deep-dive https://contextiq.trango-compute.com/blog/mcp-a2a-protocol-conformance-scan-explained covers both halves: conformance catches a broken implementation on day one, and diff catches it breaking again on day two hundred. Run a baseline scan with Agent Protocol Inspector https://contextiq.trango-compute.com/agent-readiness-detector today, and diff your next deploy against it before you find out from a support ticket instead. Follow Trango Compute on LinkedIn We post updates on new tools, context engineering patterns, and LLM cost research. Follow on LinkedIn https://www.linkedin.com/company/trango-compute