I Let GitHub Copilot CLI Read a Failed AI-Agent Trace—Here’s What It Found

A developer demonstrated that GitHub Copilot CLI can diagnose AI-agent failures from structured execution traces, using the AgentInspect MCP server to analyze a controlled TypeScript fixture. The experiment revealed a subtle bug where a formatter used the wrong optional field, causing a generic 'not enough project context' response despite successful retrieval.

I use GitHub Copilot CLI for practical development work: understanding unfamiliar code, reviewing changes, generating tests, and debugging from the terminal. After writing my beginner’s guide to GitHub Copilot CLI https://dev.to/raajaryan/github-copilot-cli-for-beginners-2026-guide-real-use-cases-setup-prompting-workflow-tips-33gd , I wanted to test a harder question: Can a coding assistant diagnose an AI-agent failure from structured execution evidence instead of a wall of logs? This matters to me as a full-stack developer working with Next.js, TypeScript, MERN applications, and AI features across the broader TheCampusCoders ecosystem. A coding assistant can read source code, but an agent failure often depends on the path taken at runtime: which tool ran, what happened between retrieval and generation, and whether a technically successful run produced the expected behavior. For this experiment, I connected AgentInspect https://github.com/rajudandigam/agent-inspect to GitHub Copilot CLI through AgentInspect’s read-only MCP server. I used a controlled, keyless TypeScript fixture so that the same failure could be reproduced without an API key or a nondeterministic model response. Disclosure:I tested AgentInspect independently for the workflow described here. The maintainer reviewed the commands for technical accuracy; the conclusions are my own. @agent-inspect/mcp-server was inPreviewwhen I ran this experiment. The fixture represents a small project-help agent—the kind of assistant that could answer setup questions about a Next.js or MERN repository. Its path was simple: The retriever returned this shape: type RetrievedChunk = { id: string; text?: string; content?: string; }; But the formatter used the wrong optional field: function formatContext chunks: RetrievedChunk { return chunks .map chunk = chunk.content ?? "" // Bug: the data is in text .join "\n\n" ; } TypeScript did not reject this because content was permitted by the loose integration type. Retrieval succeeded, the formatter did not throw, and the answer step still ran. The visible symptom was only a generic “not enough project context” response. This was exactly the kind of bug I wanted to test. There was no dramatic stack trace pointing to one bad line. I tested with Node.js 20 or newer and pinned both AgentInspect packages to the published 6.17.4 baseline: npm install agent-inspect@6.17.4 npm install --save-dev @agent-inspect/mcp-server@6.17.4 I wrapped the workflow with inspectRun and named the important boundaries with step : js import { inspectRun, observeOutcome, step, } from "agent-inspect"; await inspectRun "project-help-agent-broken", async = { const chunks = await step.tool "retrieve project docs", retrieveProjectDocs, ; const context = await step "format context", = formatContext chunks , { type: "logic", metadata: { returnedField: "text", mapperField: "content", retrievedChunkCount: chunks.length, }, }, ; await step "validate context", async = { await observeOutcome "context available", { expectation: "At least one retrieved chunk contributes usable context", status: context.trim .length 0 ? "passed" : "failed", method: "custom", actual: { retrievedChunkCount: chunks.length, usableCharacterCount: context.trim .length, }, } ; }, { type: "logic", metadata: { retrievedChunkCount: chunks.length, usableCharacterCount: context.trim .length, }, }, ; return step.llm "deterministic-demo-model", = generateAnswer context , ; }, { traceDir: ".agent-inspect", silent: true, metadata: { scenario: "field-mapping-regression" }, }, ; Two details are important here. First, step.llm labels a boundary; it does not call a provider by itself. My fixture used a deterministic function so the experiment focused on the debugging loop rather than model variance. Second, I recorded counts and field names—not raw project documents, prompts, or answers. AgentInspect uses metadata-only capture by default, but metadata is still data, so I kept it bounded and non-sensitive. I ran the fixture, listed the latest trace, and inspected its report: npx agent-inspect list --dir .agent-inspect npx agent-inspect report <run-id --dir .agent-inspect The interesting result was this: Status: success Steps: 4 1 LLM, 1 tool, 2 logic Observed outcomes Total: 1 passed 0, failed 1, unknown 0, skipped 0 context available: failed Expectation: At least one retrieved chunk contributes usable context The JavaScript function completed. The tool completed. The model-shaped step completed. Yet the behavioral outcome failed. I turned that observation into a deterministic check: npx agent-inspect check <run-id \ --dir .agent-inspect \ --fail-on-observation failed Check status: fail Summary: 1 failed, 0 warning s , 0 error s outcome.status: Observed outcome count 1 matched failed That exit code is useful in CI, but I still needed to diagnose the cause. This is where I brought in Copilot CLI. GitHub Copilot CLI supports local stdio MCP servers. I added a project-level .mcp.json file: { "mcpServers": { "agent-inspect": { "type": "local", "command": "npx", "args": "-y", "@agent-inspect/mcp-server@6.17.4", "--dir", ".agent-inspect" , "tools": "list recent runs", "get trace facts", "get execution tree", "get first causal failure", "get failed observations", "compare runs" } } } Copilot CLI also supports user-level configuration at ~/.copilot/mcp-config.json , but I preferred project-level scope for this test. On first launch, Copilot asked me to trust the folder before loading its MCP configuration. I verified the server and its tools from Copilot CLI: /mcp show agent-inspect The MCP server exposes 12 flagship read-only tools, plus compatibility tools. I enabled only the six needed for this experiment to keep the available tool set focused. I did not ask Copilot to “fix my agent.” That is too open-ended and makes it easy to mix evidence with guesses. Instead, I used this prompt: Use the AgentInspect MCP tools to inspect the latest project-help-agent-broken run. Separate your response into: 1. persisted facts, 2. the first causal failure, 3. your likely diagnosis, 4. the source code I should inspect. Do not edit any files until I approve the proposed change. Copilot’s useful answer was not a magical explanation. It was a short chain grounded in the trace: | What Copilot reported | Evidence available through AgentInspect | |---|---| | Retrieval completed | retrieve project docs was a finished tool | | Five chunks crossed the retrieval boundary | retrievedChunkCount: 5 | | The formatter expected a different field | returnedField: "text" , mapperField: "content" | | The prompt boundary had no usable context | usableCharacterCount: 0 | | The failed behavior was explicitly observed | context available had status failed | | The answer step still ran | One finished LLM-labelled step appeared after validation | AgentInspect’s get first causal failure result identified the failed observed outcome and its parent step. It did not claim that adjacent events were causally linked just because their timestamps were close. Copilot then inspected the formatter source and proposed a one-line change: js - .map chunk = chunk.content ?? "" + .map chunk = chunk.text ?? "" That diagnosis made sense, but I still reviewed the retriever’s runtime shape and TypeScript type before accepting it. A coding assistant’s explanation is a hypothesis; the persisted trace facts and the source contract are the evidence. After changing the mapping, I ran the same fixture again and checked the new trace: npx agent-inspect check <fixed-run-id \ --dir .agent-inspect \ --fail-on-observation failed Check status: pass Summary: 0 failed, 0 warning s , 0 error s The important result was not simply that the answer looked better. The same deterministic expectation that failed before now passed, while the intended execution path remained retrieval → formatting → validation → generation. This gave me a more disciplined loop: Failed behavior → local trace → bounded MCP facts → Copilot hypothesis → human review → code change → rerun → deterministic check The boundary is worth understanding. The AgentInspect MCP server did: It did not : Copilot CLI may have its own file-editing and shell tools. Those capabilities belong to Copilot, not to AgentInspect’s read-only MCP server. I deliberately required approval before any edit. AgentInspect’s core capture and MCP process are local and do not upload data to AgentInspect by default. That does not mean the combined Copilot workflow is entirely offline. When I connect an MCP server, its returned facts become available to the connected client. Copilot may send that context to its model service according to my GitHub plan, settings, organization policies, and applicable terms. My practical rules are: Local trace storage is valuable, but it does not remove the need for a data-handling decision when another tool reads those traces. The strongest part of the experiment was search-space reduction . Instead of asking Copilot to read hundreds of interleaved log lines, I gave it a small evidence surface with named steps, bounded metadata, and an explicit failed outcome. It was especially useful because the program did not crash. Traditional error-first debugging would have started in the wrong place. There were also clear limitations: I did not find an autonomous debugging system—and I do not think that should be the goal. What I found was a useful division of responsibility: For TypeScript developers already using Copilot CLI, this is a more reliable pattern than pasting a giant log and asking, “What went wrong?” The coding assistant still reasons probabilistically, but it starts from better evidence. If you want to reproduce the same loop, start with a synthetic failure, keep capture metadata-only, and make Copilot separate facts from inferences. That separation was the most valuable part of the experiment.