{"slug": "i-built-agent-inspect-to-debug-typescript-ai-agent-trajectories", "title": "I built agent-inspect to debug TypeScript AI agent trajectories", "summary": "Developer Raju Dandigam has released agent-inspect, an open-source local evidence debugger and trajectory-test toolkit for TypeScript AI agents. The tool converts a single local trace into a readable execution tree, a deterministic regression gate, and a shareable Evidence v2 bundle, addressing the challenge of debugging complex agent flows where flat logs and output-only tests fail to reveal wrong paths. AgentInspect supports manual instrumentation and adapters for AI SDK, OpenAI Agents JS, LangChain, and LangGraph, with no account or default upload required.", "body_md": "Your AI agent failed.\n\nAgain.\n\nThe final answer is wrong, but the logs look fine:\n\n```\ntool call started\nmodel call started\ntool call completed\nmodel call completed\nfallback used\nerror: timeout\n```\n\nWhich tool caused the timeout? Did the model answer before retrieval finished? Was the fallback expected? Did the agent call the same tool twice?\n\nThis is where `console.log`\n\nstops feeling like debugging and starts feeling like archaeology.\n\nI kept hitting this problem while building TypeScript AI agents. Once the flow moved beyond a single model call, the debugging loop became a system:\n\n```\nplan → retrieve → rank → generate → validate → maybe retry → maybe hand off\n```\n\nFlat logs lost the structure. Output only tests could miss a bad path that happened to produce a plausible answer. Model graded evals helped with semantic quality, but they were a poor fit for every deterministic CI rule. And raw traces were too risky to paste casually into issues or pull requests.\n\nSo I built [agent-inspect](https://github.com/rajudandigam/agent-inspect).\n\nAgentInspect is a local evidence debugger and trajectory-test toolkit for TypeScript AI agents.\n\nIt turns one local trace into three things: a readable execution tree, a deterministic regression gate, and a derived Evidence v2 bundle that you can review before sharing.\n\nNo account. No collector. No default upload. Metadata only by default.\n\n```\none local JSONL trace\n├─ Debug   → view · report · explain\n├─ Prevent → check · contract · CI\n└─ Share   → redact · bundle · verify\n```\n\nA support agent can return a plausible answer after doing almost everything wrong.\n\nThe healthy path might be:\n\n```\nplan-request\n└─ retrieve_policy\n   └─ rank-results\n      └─ generate_answer\n         └─ policyShown: passed\n```\n\nThe regression might be:\n\n```\ngenerate_answer          <- answered before retrieval\nretrieve_policy\nretrieve_policy          <- duplicate call\nsearch_docs              <- wrong tool, failed\npolicyShown: failed\n```\n\nAn output only test may pass. A flat log may contain every event. Neither makes the wrong path obvious.\n\nThe final answer is only one fact about the run. Tool choice, ordering, repetition, completion, duration, token usage, and observed outcomes are facts too. Together, those facts form the agent's **trajectory**.\n\nThat trajectory should be inspectable. It should also be testable.\n\nYou can start with manual instrumentation:\n\n``` js\nimport { inspectRun, observeOutcome, step } from \"agent-inspect\";\n\nconst answer = await inspectRun(\n  \"support-agent\",\n  async () => {\n    const policy = await step(\n      \"retrieve_policy\",\n      () => retrievePolicy(),\n      {\n        type: \"tool\",\n        metadata: { toolName: \"retrieve_policy\" },\n      },\n    );\n\n    const result = await step(\n      \"generate_answer\",\n      () => draftAnswer(policy),\n      {\n        type: \"llm\",\n        metadata: { model: \"your-model\" },\n      },\n    );\n\n    await observeOutcome(\"policyShown\", {\n      expectation: \"The answer cites a retrieved policy\",\n      status: \"passed\",\n      method: \"custom\",\n    });\n\n    return result;\n  },\n  { traceDir: \".agent-inspect\" },\n);\n```\n\nThe wrapper records those boundaries as local JSONL while preserving the application's return value and errors. Raw prompts and model outputs are not required for the core workflow.\n\nIf your application already emits structured logs or uses AI SDK, OpenAI Agents JS, LangChain, or LangGraph you can use an adapter or reader instead of wrapping every step manually.\n\nThe shortest path uses a generated synthetic demo:\n\n```\nnpm install agent-inspect\nnpx agent-inspect init --yes\nnode examples/agent-inspect-demo.mjs\nnpx agent-inspect list --dir .agent-inspect\n```\n\n`init`\n\nwrites a small config and demo into your project. The demo does not call a model or upload a trace.\n\nCopy the run ID printed by `list`\n\n, then use the same local artifact for the three jobs below.\n\n```\nnpx agent-inspect view <run-id> --dir .agent-inspect --summary\nnpx agent-inspect report <run-id> --dir .agent-inspect\nnpx agent-inspect explain <run-id> --dir .agent-inspect\n```\n\nThe tree restores the structure that flat logs lose: nested steps, tool and model calls, durations, safe metadata, errors, and observed outcomes. `explain`\n\nsummarizes local trace facts deterministically; its default path makes no provider call.\n\nThe useful question changes from:\n\n```\nDid the run fail?\n```\n\nto:\n\n```\nWhere did the passing and failing trajectories first diverge?\n```\n\nThat distinction matters when the visible answer looks fine but the agent skipped a required retrieval, safety, or validation step.\n\nSome agent quality questions are subjective. Helpfulness, tone, and open ended answer quality can benefit from model graded evaluation.\n\nBut many regressions are structural:\n\n`retrieve_policy`\n\ncalled?`search_docs`\n\ntool appear?Those checks do not need another model. They can be deterministic:\n\n```\nnpx agent-inspect check <run-id> --dir .agent-inspect \\\n  --preset trajectory \\\n  --required-tool retrieve_policy \\\n  --forbidden-tool search_docs \\\n  --fail-on-observation failed\n```\n\nThe preset and explicit shorthand rules are additive. A healthy run exits `0`\n\n; a trajectory-rule failure exits `1`\n\n.\n\nFor the committed regression fixture, the result is concrete:\n\n```\nCheck status: fail\nSummary: 2 failed, 0 warning(s), 0 error(s)\n\n- outcome.status: Observed outcome count 1 matched [failed].\n- tool.usage: Forbidden tool search_docs appeared.\n```\n\nSame trace. Same rules. Same verdict. No model judge and no provider call in the check path.\n\nThat makes it suitable for a normal CI step. If your test fixture writes a trace to a stable path:\n\n```\n- name: Run deterministic agent fixture\n  run: node run-agent-fixture.mjs\n\n- name: Check agent trajectory\n  run: |\n    npx agent-inspect check .agent-inspect/ci-run.jsonl \\\n      --preset trajectory \\\n      --required-tool retrieve_policy \\\n      --fail-on-observation failed \\\n      --evidence-on fail\n```\n\n`--evidence-on fail`\n\nwrites local Evidence for triage when the check fails. It does not upload the artifact.\n\nWhen CLI flags outgrow one command, the Beta TraceContract API expresses the same expectations in TypeScript:\n\n``` js\nimport { openTraceFile } from \"agent-inspect/readers\";\nimport {\n  defineTraceContract,\n  evaluateTraceContractRead,\n} from \"agent-inspect/checks\";\n\nconst read = await openTraceFile(\"./.agent-inspect/demo-regression.jsonl\");\n\nconst contract = defineTraceContract({\n  run: { requireCompleted: true },\n  tools: {\n    required: [\"retrieve_policy\"],\n    forbidden: [\"search_docs\"],\n  },\n  observations: { failOn: [\"failed\"] },\n});\n\nconst result = evaluateTraceContractRead(read, contract);\nif (result.status !== \"pass\") process.exitCode = 1;\n```\n\nThe principle is simple: use deterministic trace facts for structural CI rules, and reserve model grading for semantic quality.\n\nA failing trace is often the best debugging artifact. It can also contain prompts, tool arguments, retrieved documents, customer identifiers, error messages, or secrets.\n\nThe collaboration strategy should not be “paste the raw trace into Slack.”\n\nAgentInspect keeps the source trace read-only and creates a derived bundle:\n\n```\nnpx agent-inspect verify-safe <run-id> --dir .agent-inspect\nnpx agent-inspect bundle <run-id> --dir .agent-inspect \\\n  --profile share \\\n  --out ./evidence\nnpx agent-inspect bundle verify ./evidence\n```\n\nThe bundle can include:\n\n```\nevidence.html          self-contained offline review surface\nevidence.json          versioned manifest and SHA-256 file hashes\ntrace.jsonl            redacted derived trace\ncheck-results.json     deterministic findings\nredaction-report.json  detector summary without secret values\nsummary.md             human-readable overview\n```\n\n`bundle verify`\n\nchecks the manifest, listed files, hashes, assessment, and provenance offline.\n\nThis is an integrity check. It is not a signature or a compliance certificate.\n\nThe wording matters: the artifact is **share-checked**, not “certified safe.” `verify-safe`\n\nand redaction are best-effort controls. Review the generated HTML and safety results before attaching a bundle to a pull request, incident, or public issue.\n\nThe local evidence model is not tied to one agent framework.\n\n| Your stack | Capture path |\n|---|---|\n| Custom TypeScript functions or classes |\n`inspectRun` , `step` , `observe` , or `createInspector`\n|\n| Vercel AI SDK | `@agent-inspect/ai-sdk` |\n| OpenAI Agents JS | `@agent-inspect/openai-agents` |\n| LangChain or LangGraph | `@agent-inspect/langchain` |\n| Existing structured logs |\n`agent-inspect logs` readers |\n| OpenInference or OTLP JSON | local standards readers |\n| Vitest or Jest | reporters plus experimental trace matchers |\n\nThe root package is enough for custom capture, the CLI, deterministic checks, and Evidence. Optional packages add only the integration you need.\n\nThe Preview MCP path exposes configured local evidence through bounded, read-only tools:\n\n```\nnpx agent-inspect mcp configure --client cursor\n```\n\nThe command is a dry run by default, so you can review the generated configuration before enabling it.\n\nA connected coding assistant can then investigate the same TraceFacts used by the CLI: What failed first? Which required tool was missing? What changed between the passing and failing runs?\n\nThis is not replay, an auto fix engine, or a hidden upload path. It is optional read-only access to explicitly configured local evidence.\n\nAgentInspect owns the laptop to pull request evidence loop:\n\n```\ncapture locally\n→ understand the path\n→ fail CI on structural drift\n→ derive reviewable evidence\n```\n\nIt complements hosted observability and evaluation platforms. Use hosted tools when you need production dashboards, long term retention, fleet wide alerting, team wide trace search, hosted datasets, or prompt management.\n\nUse AgentInspect when you need to inspect one TypeScript agent run immediately, enforce deterministic trajectory expectations in CI, compare a passing and failing local run, or hand off a redacted, hash-verifiable artifact.\n\nThe boundary is intentional. AgentInspect is not:\n\nThe current release is **6.17.2**, requires Node.js 20 or newer, uses persisted schema `1.0`\n\n, and is MIT licensed. Legacy v0.1 and v0.2 traces remain readable.\n\n```\nnpm install agent-inspect\nnpx agent-inspect init --yes\nnode examples/agent-inspect-demo.mjs\nnpx agent-inspect list --dir .agent-inspect\n```\n\nThen inspect, check, and derive Evidence from the run:\n\n```\nnpx agent-inspect view <run-id> --dir .agent-inspect --summary\nnpx agent-inspect check <run-id> --dir .agent-inspect --preset trajectory\nnpx agent-inspect bundle <run-id> --dir .agent-inspect \\\n  --profile share \\\n  --out ./evidence\nnpx agent-inspect bundle verify ./evidence\n```\n\nOne local trace should be able to tell you what the agent did, prove that the regression stays fixed, and give a teammate evidence they can review without making upload the price of admission.\n\nAgentInspect is open source and MIT licensed. If this workflow is useful to you:\n\nMost of all, **leave a comment below**.\n\nHow do you debug agent runs today?\n\nWhich trajectory rule would you put in CI first?\n\nAnd if AgentInspect does not fit your workflow, tell me why that feedback is just as useful.", "url": "https://wpnews.pro/news/i-built-agent-inspect-to-debug-typescript-ai-agent-trajectories", "canonical_source": "https://dev.to/raju_dandigam/i-built-agent-inspect-to-debug-typescript-ai-agent-trajectories-2jg6", "published_at": "2026-08-25 17:12:09+00:00", "updated_at": "2026-08-25 17:45:35.354838+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-tools", "mlops"], "entities": ["agent-inspect", "Raju Dandigam", "AI SDK", "OpenAI Agents JS", "LangChain", "LangGraph", "Evidence v2"], "alternates": {"html": "https://wpnews.pro/news/i-built-agent-inspect-to-debug-typescript-ai-agent-trajectories", "markdown": "https://wpnews.pro/news/i-built-agent-inspect-to-debug-typescript-ai-agent-trajectories.md", "text": "https://wpnews.pro/news/i-built-agent-inspect-to-debug-typescript-ai-agent-trajectories.txt", "jsonld": "https://wpnews.pro/news/i-built-agent-inspect-to-debug-typescript-ai-agent-trajectories.jsonld"}}