# Catching AI Agent Protocol Regressions Before They Ship: A Diff-Based CI Gate

> Source: <https://contextiq.trango-compute.com/blog/ai-agent-protocol-regression-detection-ci-diff>
> Published: 2026-09-18 00:00:00+00:00

# Catching AI Agent Protocol Regressions Before They Ship: A Diff-Based CI Gate

How to diff two Agent Protocol Inspector scans to catch MCP and A2A regressions — a removed tool, a dropped auth mechanism, a downgraded protocol status — in CI with curl and jq.

A REST API that silently drops an endpoint on deploy gets caught by a contract test, usually before it reaches production. An MCP server or A2A agent that silently drops a tool, loses its auth requirement, or downgrades its own protocol support has no equivalent gate — because there's no equivalent test suite. Every downstream agent integration finds out the hard way, at runtime, when a call it relied on stops existing.

Agent Protocol Inspector's **scan diff** exists to close exactly that gap: a structural comparison between two stored scans of the same target, producing a **pass / warn / fail** verdict a CI pipeline can actually act on.

## What Gets Compared

`diffScans` takes a baseline scan and a current scan and compares all three protocols structurally — no new network calls, no re-probing, just a pure comparison of two already-fetched results:

| Change | Verdict | 
|---|---|
| A protocol's detection status regressed ( `confirmed` →`indicated` or`not_detected` ) | **fail** | 
| An MCP tool, A2A skill, or ARD catalog entry was removed | **fail** | 
| A declared auth mechanism disappeared entirely | **fail** | 
| A tool/skill/entry was added | warn | 
| An existing MCP tool's input schema changed | warn | 
| An auth mechanism changed (without disappearing) | warn | 
| Nothing meaningfully changed | pass | 

The asymmetry is deliberate. Removing something the agent could do before is a regression a downstream integration will break on — that's a **fail**. Adding new surface area is worth a human's attention but isn't inherently dangerous — that's a **warn**, not a build-breaker. A tool's input schema changing without being removed sits in the same bucket: worth reviewing, not worth blocking a deploy over on its own.

## Wiring It Into CI

The verdict is only useful if a pipeline can actually act on it — and a bare `curl` call can't. An HTTP `200` response with `{"verdict":"fail"}` in the body still makes `curl` exit `0`; nothing about that response naturally breaks a CI step. The fix is a couple of lines of `jq`:

```
RESPONSE=$(curl -sS -X POST https://contextiq.trango-compute.com/api/v1/agent-protocol-inspector/compare \
  -H "Authorization: Bearer $CONTEXTIQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"baselineScanId":"<last-deploy-scanId>","currentScanId":"<this-deploy-scanId>"}')

echo "$RESPONSE" | jq .

VERDICT=$(echo "$RESPONSE" | jq -r '.verdict')
if [ "$VERDICT" = "fail" ]; then
  echo "Agent Protocol Inspector: regression detected — failing build." >&2
  exit 1
fi
```

That's a real go/no-go gate: `warn` still prints the full diff for a human to read, but only `fail` stops the pipeline. This snippet is generated directly in the dashboard's "Compare against a prior scan" panel, next to the field where you'd paste in a baseline `scanId` to test manually before committing it to a workflow file.

## A Practical Deploy-Gate Pattern

The workflow this is built for:

1. **After every deploy** , run a scan of your own MCP endpoint or A2A agent card and capture the returned`scanId` .
2. **Store that `scanId`** somewhere your next CI run can read it — a repo variable, a deploy artifact, a line in your release notes.
3. **Before the next deploy goes live** , run a fresh scan and diff it against the stored baseline using the snippet above.
4. **On `fail`** , the pipeline stops before the regression reaches users who were already depending on the tool or skill that disappeared.

This is the same shape as a contract test for a REST API — a stored "last known good" fingerprint, compared against every candidate release — applied to a surface (agent protocol conformance) that doesn't otherwise get tested at all.

## What This Doesn't Catch

A diff only knows what changed structurally between two scans it was given — it has no opinion on whether the *first* scan was ever correct, and it can't tell you a regression happened if nobody ever stored a baseline to compare against. Pairing it with the [conformance checks covered in our protocol scan deep-dive](https://contextiq.trango-compute.com/blog/mcp-a2a-protocol-conformance-scan-explained) covers both halves: conformance catches a broken implementation on day one, and diff catches it breaking again on day two hundred.

Run a baseline scan with [Agent Protocol Inspector](https://contextiq.trango-compute.com/agent-readiness-detector) today, and diff your next deploy against it before you find out from a support ticket instead.

Follow Trango Compute on LinkedIn

We post updates on new tools, context engineering patterns, and LLM cost research.

[Follow on LinkedIn](https://www.linkedin.com/company/trango-compute)