Conversation Regression Testing for AI Agents: Catch Multi-Turn Failures Before Production

An agent can give a convincing final answer and still fail the user three turns earlier. It may forget an account constraint, call the wrong tool, accept a correction it should reject, or carry a stale fact into every later decision. A one-prompt test will happily pass.

That is why production agents need conversation regression testing: replaying a complete, realistic interaction after a change and checking properties that span the whole trajectory. This guide shows how to build a small, useful suite without pretending model output will be byte-for-byte deterministic.

The payoff is concrete: when you change a model, prompt, tool, retrieval source, or memory policy, CI can tell you whether a familiar customer journey still completes safely—and where it first went wrong.

Single-turn tests are still valuable. They catch malformed structured output, unsafe tool arguments, and obvious retrieval errors quickly. But a customer-facing agent is stateful. Each turn changes what the next turn sees.

Consider a support agent helping a customer change a subscription:

An agent can answer turn 5 politely while violating the constraint introduced at turn 3. A final-answer judge may call the response helpful; your billing system will call it a defect.

Conversation tests reveal four failures that prompt tests routinely hide:

The key idea is simple: test a conversation as a sequence of state transitions, not as a bag of independent answers.

Do not begin by generating thousands of synthetic chats. Start with 10–20 journeys that represent expensive, frequent, or risky work:

Good fixtures come from anonymized support transcripts, bug reports, sales-engineering handoffs, and incidents. Remove personal data, replace identifiers with stable test values, and record the business rule the journey protects. A fixture is not a transcript archive; it is an executable statement of what must remain true.

Here is a compact TypeScript shape:

type Turn = {
  user: string;
  expected?: {
    tool?: string;
    mustInclude?: string[];
    mustNotInclude?: string[];
  };
};

type ConversationFixture = {
  id: string;
  risk: "low" | "medium" | "high";
  initialState: Record<string, unknown>;
  turns: Turn[];
  invariants: string[];
};

const cancelBeforeWrite: ConversationFixture = {
  id: "cancel-before-plan-change",
  risk: "high",
  initialState: { workspaceId: "ws_test", plan: "annual" },
  turns: [
    { user: "Change us to the monthly plan.", expected: { tool: "quote_plan_change" } },
    { user: "Wait, do not make any changes yet." },
    { user: "What would the prorated amount be?", expected: { tool: "quote_plan_change" } }
  ],
  invariants: ["no plan_change write occurs", "withdrawal is acknowledged"]
};

This fixture is deliberately small. It tests a real product promise: a quote is not authorization, and a withdrawn request cannot leak into a later tool call.

You cannot debug a failed conversation with only the final text. Store a trace for each turn with the input, selected tool, sanitized arguments, tool result, state revision, prompt revision, model ID, and an immutable run ID.

Keep secrets and raw customer data out of fixtures. Store references or redacted values instead. If a tool result changes outside your control, use a test double for CI and separately run a small nightly suite against a safe staging environment.

type TraceStep = {
  turn: number;
  stateVersion: string;
  assistantText: string;
  toolCalls: Array<{ name: string; args: unknown; resultCode: string }>;
  promptVersion: string;
  model: string;
};

The distinction matters. A replay fixture supplies stable inputs. A trace explains the observed trajectory. Together they let a reviewer answer, “Did the model change, did our prompt change, or did a tool contract change?”

Exact-string assertions are brittle because legitimate wording changes. Purely subjective judging is brittle because it can miss deterministic safety failures. Use both, in layers.

Make product and security rules ordinary code. Examples:

function assertNoWriteAfterWithdrawal(trace: TraceStep[]) {
  const withdrewAt = trace.findIndex(s =>
    /do not make|cancel|withdraw/i.test(s.assistantText)
  );
  const writes = trace.slice(withdrewAt + 1)
    .flatMap(s => s.toolCalls)
    .filter(c => c.name === "change_plan");

  if (writes.length) throw new Error("plan changed after withdrawal");
}

function assertWorkspaceScope(trace: TraceStep[]) {
  for (const call of trace.flatMap(s => s.toolCalls)) {
    if (call.name === "get_invoice" && !JSON.stringify(call.args).includes("ws_test")) {
      throw new Error("tool call escaped fixture workspace");
    }
  }
}

Prefer assertions on permissions, schema validity, tool order, idempotency keys, source citations, and state transitions. They are fast, explainable, and cheap enough for every pull request.

Then use a separate model or human review for properties that cannot be reduced to a rule: did the agent acknowledge the correction, ask a necessary clarifying question, or preserve the user’s goal?

Give the judge a narrow rubric and the relevant evidence. Do not ask, “Is this good?” Ask, “After the user withdrew authorization, did the agent promise or initiate a plan change? Return pass, fail, or uncertain with the first supporting turn.” Treat uncertain as review-needed, not as a pass.

Run high-risk fixtures several times. A pass rate of 10/10 is more meaningful than a fortunate 1/1, but do not turn every CI job into an expensive Monte Carlo experiment. A practical split is:

Set a budget per suite. Regression testing should prevent surprise spend, not create it.

The most useful output is not “conversation failed.” It is “turn 2 selected an unscoped invoice lookup; turns 3–5 inherited the wrong workspace.” That transforms an opaque judge score into an engineering task.

For each test, record:

first_failure_turn; This separation prevents a common mistake: fixing the last bad answer instead of the earlier decision that made it inevitable. It also makes failures comparable across releases.

Linear transcripts cover the happy path. Real conversations fork at moments of uncertainty: a user corrects an assumption, refuses an action, or reveals a constraint. You do not need to regenerate the identical first four turns for every branch.

Snapshot the safe state at a branch point and run alternatives from there:

const branches = [
  "Yes, apply the change.",
  "No, cancel that request.",
  "Use a different workspace instead."
];

for (const nextUserMessage of branches) {
  const result = await runFromSnapshot({ snapshot: quoteState, nextUserMessage });
  await scoreConversation(result);
}

Forking improves coverage while controlling token cost. It is especially useful for consent, payments, account access, and any agent that can trigger an external write.

Run conversation regression tests whenever any behavior-bearing input changes:

Tag fixtures by risk. A low-risk content-answering journey may warn on a judge regression. A high-risk write journey should block a release when a deterministic invariant fails.

Keep a short review packet with the baseline trace, candidate trace, score changes, and first failure. This is far better than burying a 30-turn transcript in CI logs.

Every production failure should become one of three things: a fixed invariant, a replay fixture, or a reason not to add a test. The last category is legitimate when the incident was infrastructure-only and already covered elsewhere—but make that decision explicit.

Use this loop:

Over time, the suite becomes an operational memory of how your agent has actually failed. That is more valuable than a generic leaderboard score because it reflects the work your users ask it to do.

Before calling conversation regression testing complete, confirm:

The goal is not to prove an agent will never fail. It is to make familiar failures difficult to reintroduce—and to detect a new class of failure before it reaches a customer.

It is a repeatable test of a whole agent interaction, rather than one prompt and one reply. The test replays a realistic sequence of user turns and tool responses, then checks cross-turn rules such as context retention, permission boundaries, and task completion.

An evaluation harness can cover many workflow-level checks. Conversation regression testing is a focused layer within that practice: it preserves known multi-turn journeys and compares behavior after a change. Its central outputs are cross-turn invariants, replay evidence, and first-failure attribution.

Usually no. Exact prose makes tests fragile. Assert deterministic product rules first, then use narrow rubrics for meaning. Exact matches are appropriate for structured output, tool arguments, required disclaimers, and other stable contracts.

Start with 10–20. Prioritize journeys with writes, money, access, regulated information, high volume, or prior incidents. Add a fixture whenever a production failure reveals a behavior you do not want to see twice.

Yes. Mock external tools, run deterministic checks on every pull request, keep the PR suite small, and reserve repeated simulations for nightly or release-candidate runs. Fork from saved state instead of regenerating shared conversation prefixes.

Block on deterministic failures involving permissions, tenant boundaries, unintended writes, unsafe tool calls, invalid structured output, or required policy behavior. Treat subjective quality-score changes as warnings unless they cross a deliberate, reviewed threshold.