# Stop Snapshot-Testing Voice Conversations: Verify the Turn Protocol Instead

> Source: <https://dev.to/susiewang/stop-snapshot-testing-voice-conversations-verify-the-turn-protocol-instead-109j>
> Published: 2026-10-25 17:08:59+00:00

A real-time voice companion can behave correctly while its test suite reports failure.

The usual culprit is not that “the LLM knows better.” It is that the test encoded an assumption the product never promised: an exact sentence, a fixed callback order, or playback beginning within an arbitrary sleep interval.

That creates an uncomfortable engineering tension. You do not want to dismiss failures as harmless AI variability, but you also do not want brittle tests blocking valid behavior.

A better approach is to test the **conversation protocol** deterministically and evaluate the **conversation content** separately. For a Tencent RTC conversational AI application, that means verifying facts such as:

We will build that test oracle in TypeScript, reproduce the dangerous races, and then map it onto the live voice pipeline.

Consider this test:

```
expect(answer).toBe("Your meeting begins at 3 PM.");
```

The model returns:

```
The meeting starts at 3 PM.
```

The assertion failed, but that alone does not establish a product defect. The test mixed three different questions:

Only the last question involves linguistic variability. The first two should remain deterministic.

The human decision is therefore not “trust the model or trust the test.” It is: **which product contract was this test supposed to enforce?**

A production voice companion is not one AI call. Keep these responsibilities separate:

```
Microphone / RTC media
        ↓
Speech recognition
        ↓
Application turn coordinator
        ↓
LLM
        ↓
Speech synthesis
        ↓
RTC playback
```

The application coordinator owns the current turn. Neither a late model response nor a delayed playback callback gets to decide that it is still relevant.

Tencent RTC documents Conversational AI as a real-time voice interaction scenario that can connect users with multiple LLM providers. Its LLM configuration guidance also covers OpenAI-compatible models and request identifiers used for routing and observability:

The event and effect names below belong to our application. They are not Tencent RTC SDK method names.

```
mkdir voice-protocol-oracle
cd voice-protocol-oracle
npm init -y
npm install --save-dev typescript vitest tsx @types/node
npm pkg set type=module
npm pkg set scripts.test="vitest run"
mkdir src
```

Create `src/session.ts`:

```
export type Phase =
  | "idle"
  | "userSpeaking"
  | "transcribing"
  | "thinking"
  | "preparingAudio"
  | "speaking"
  | "recovery"
  | "offline";

export type State = {
  phase: Phase;
  turn: number;
  requestId?: string;
};

export type Event =
  | { type: "USER_STARTED" }
  | { type: "USER_ENDED" }
  | {
      type: "TRANSCRIPT_READY";
      turn: number;
      requestId: string;
      text: string;
    }
  | { type: "TRANSCRIPT_FAILED"; turn: number }
  | {
      type: "MODEL_DONE";
      turn: number;
      requestId: string;
      text: string;
    }
  | { type: "MODEL_TIMEOUT"; turn: number; requestId: string }
  | { type: "PLAYBACK_STARTED"; turn: number; requestId: string }
  | { type: "PLAYBACK_ENDED"; turn: number; requestId: string }
  | { type: "PLAYBACK_FAILED"; turn: number; requestId: string }
  | { type: "DISCONNECTED" };

export type Effect =
  | { type: "REQUEST_TRANSCRIPT"; turn: number }
  | {
      type: "REQUEST_MODEL";
      turn: number;
      requestId: string;
      text: string;
    }
  | {
      type: "SPEAK";
      turn: number;
      requestId: string;
      text: string;
    }
  | { type: "CANCEL_AUDIO"; turn: number; requestId: string }
  | { type: "NOTICE"; message: string }
  | { type: "LOG_STALE"; source: string; turn: number; requestId?: string };

export const initialState: State = {
  phase: "idle",
  turn: 0,
};

function isCurrent(
  state: State,
  turn: number,
  requestId?: string,
): boolean {
  return (
    state.turn === turn &&
    (requestId === undefined || state.requestId === requestId)
  );
}

function stale(
  state: State,
  source: string,
  turn: number,
  requestId?: string,
): [State, Effect[]] {
  return [
    state,
    [{ type: "LOG_STALE", source, turn, requestId }],
  ];
}

export function reduce(
  state: State,
  event: Event,
): [State, Effect[]] {
  switch (event.type) {
    case "USER_STARTED": {
      if (state.phase === "userSpeaking") return [state, []];

      const effects: Effect[] = [];

      if (
        (state.phase === "preparingAudio" || state.phase === "speaking") &&
        state.requestId
      ) {
        effects.push({
          type: "CANCEL_AUDIO",
          turn: state.turn,
          requestId: state.requestId,
        });
      }

      return [
        {
          phase: "userSpeaking",
          turn: state.turn + 1,
        },
        effects,
      ];
    }

    case "USER_ENDED": {
      if (state.phase !== "userSpeaking") return [state, []];

      return [
        { ...state, phase: "transcribing" },
        [{ type: "REQUEST_TRANSCRIPT", turn: state.turn }],
      ];
    }

    case "TRANSCRIPT_READY": {
      if (
        state.phase !== "transcribing" ||
        !isCurrent(state, event.turn)
      ) {
        return stale(state, "transcript", event.turn, event.requestId);
      }

      if (!event.text.trim()) {
        return [
          { ...state, phase: "recovery" },
          [{ type: "NOTICE", message: "I did not catch that. Please try again." }],
        ];
      }

      return [
        {
          ...state,
          phase: "thinking",
          requestId: event.requestId,
        },
        [
          {
            type: "REQUEST_MODEL",
            turn: event.turn,
            requestId: event.requestId,
            text: event.text,
          },
        ],
      ];
    }

    case "TRANSCRIPT_FAILED": {
      if (state.phase !== "transcribing" || !isCurrent(state, event.turn)) {
        return stale(state, "transcript-error", event.turn);
      }

      return [
        { ...state, phase: "recovery" },
        [{ type: "NOTICE", message: "Speech recognition is unavailable. Try again." }],
      ];
    }

    case "MODEL_DONE": {
      if (
        state.phase !== "thinking" ||
        !isCurrent(state, event.turn, event.requestId)
      ) {
        return stale(state, "model", event.turn, event.requestId);
      }

      if (!event.text.trim()) {
        return [
          { ...state, phase: "recovery" },
          [{ type: "NOTICE", message: "No answer was produced. Please retry." }],
        ];
      }

      return [
        { ...state, phase: "preparingAudio" },
        [
          {
            type: "SPEAK",
            turn: event.turn,
            requestId: event.requestId,
            text: event.text,
          },
        ],
      ];
    }

    case "MODEL_TIMEOUT": {
      if (
        state.phase !== "thinking" ||
        !isCurrent(state, event.turn, event.requestId)
      ) {
        return stale(state, "model-timeout", event.turn, event.requestId);
      }

      return [
        { ...state, phase: "recovery" },
        [{ type: "NOTICE", message: "The response took too long. Please try again." }],
      ];
    }

    case "PLAYBACK_STARTED": {
      if (
        state.phase === "preparingAudio" &&
        isCurrent(state, event.turn, event.requestId)
      ) {
        return [{ ...state, phase: "speaking" }, []];
      }

      // Playback may have crossed an interruption or cancellation boundary.
      // Stop it instead of accepting its late callback.
      return [
        state,
        [
          {
            type: "CANCEL_AUDIO",
            turn: event.turn,
            requestId: event.requestId,
          },
          {
            type: "LOG_STALE",
            source: "playback-start",
            turn: event.turn,
            requestId: event.requestId,
          },
        ],
      ];
    }

    case "PLAYBACK_ENDED": {
      if (
        state.phase !== "speaking" ||
        !isCurrent(state, event.turn, event.requestId)
      ) {
        return stale(state, "playback-end", event.turn, event.requestId);
      }

      return [{ phase: "idle", turn: state.turn }, []];
    }

    case "PLAYBACK_FAILED": {
      if (
        !["preparingAudio", "speaking"].includes(state.phase) ||
        !isCurrent(state, event.turn, event.requestId)
      ) {
        return stale(state, "playback-error", event.turn, event.requestId);
      }

      return [
        { ...state, phase: "recovery" },
        [{ type: "NOTICE", message: "Audio playback failed. A text answer can be shown." }],
      ];
    }

    case "DISCONNECTED": {
      const effects: Effect[] = [];

      if (
        (state.phase === "preparingAudio" || state.phase === "speaking") &&
        state.requestId
      ) {
        effects.push({
          type: "CANCEL_AUDIO",
          turn: state.turn,
          requestId: state.requestId,
        });
      }

      effects.push({ type: "NOTICE", message: "Connection lost. Reconnect to continue." });
      return [{ ...state, phase: "offline" }, effects];
    }
  }
}
```

There are two important design choices here.

First, `turn` changes as soon as new user speech begins. That immediately invalidates work from the previous turn.

Second, a playback callback is not automatically trusted. If audio starts after its turn was interrupted, the coordinator issues another cancellation rather than moving back into `speaking`.

Create `src/session.test.ts`:

``` js
import { describe, expect, it } from "vitest";
import {
  type Effect,
  type Event,
  initialState,
  reduce,
} from "./session.js";

function scenario() {
  let state = initialState;

  return {
    step(event: Event): Effect[] {
      const [next, effects] = reduce(state, event);
      state = next;
      return effects;
    },
    state() {
      return state;
    },
  };
}

describe("voice turn protocol", () => {
  it("rejects an LLM result from an interrupted turn", () => {
    const s = scenario();

    s.step({ type: "USER_STARTED" });
    s.step({ type: "USER_ENDED" });
    s.step({
      type: "TRANSCRIPT_READY",
      turn: 1,
      requestId: "req-1",
      text: "Tell me a story",
    });

    // The user begins a replacement turn before req-1 completes.
    s.step({ type: "USER_STARTED" });
    s.step({ type: "USER_ENDED" });
    s.step({
      type: "TRANSCRIPT_READY",
      turn: 2,
      requestId: "req-2",
      text: "Actually, stop",
    });

    const effects = s.step({
      type: "MODEL_DONE",
      turn: 1,
      requestId: "req-1",
      text: "Once upon a time...",
    });

    expect(effects.some((effect) => effect.type === "SPEAK")).toBe(false);
    expect(s.state()).toMatchObject({
      phase: "thinking",
      turn: 2,
      requestId: "req-2",
    });
  });

  it("cancels playback that starts after a barge-in", () => {
    const s = scenario();

    s.step({ type: "USER_STARTED" });
    s.step({ type: "USER_ENDED" });
    s.step({
      type: "TRANSCRIPT_READY",
      turn: 1,
      requestId: "req-1",
      text: "Say hello",
    });
    s.step({
      type: "MODEL_DONE",
      turn: 1,
      requestId: "req-1",
      text: "Hello!",
    });

    const interruption = s.step({ type: "USER_STARTED" });
    expect(interruption).toContainEqual({
      type: "CANCEL_AUDIO",
      turn: 1,
      requestId: "req-1",
    });

    const lateStart = s.step({
      type: "PLAYBACK_STARTED",
      turn: 1,
      requestId: "req-1",
    });

    expect(lateStart).toContainEqual({
      type: "CANCEL_AUDIO",
      turn: 1,
      requestId: "req-1",
    });
    expect(s.state().phase).toBe("userSpeaking");
  });

  it("does not speak a response that arrives after timeout", () => {
    const s = scenario();

    s.step({ type: "USER_STARTED" });
    s.step({ type: "USER_ENDED" });
    s.step({
      type: "TRANSCRIPT_READY",
      turn: 1,
      requestId: "req-1",
      text: "What should we do today?",
    });
    s.step({
      type: "MODEL_TIMEOUT",
      turn: 1,
      requestId: "req-1",
    });

    const lateResult = s.step({
      type: "MODEL_DONE",
      turn: 1,
      requestId: "req-1",
      text: "Let us go for a walk.",
    });

    expect(lateResult.some((effect) => effect.type === "SPEAK")).toBe(false);
    expect(s.state().phase).toBe("recovery");
  });

  it.each([
    "The meeting starts at 3 PM.",
    "Your meeting begins at 3 PM.",
  ])("accepts a protocol-valid wording: %s", (answer) => {
    const s = scenario();

    s.step({ type: "USER_STARTED" });
    s.step({ type: "USER_ENDED" });
    s.step({
      type: "TRANSCRIPT_READY",
      turn: 1,
      requestId: "req-1",
      text: "When is my meeting?",
    });

    const effects = s.step({
      type: "MODEL_DONE",
      turn: 1,
      requestId: "req-1",
      text: answer,
    });

    expect(effects).toContainEqual({
      type: "SPEAK",
      turn: 1,
      requestId: "req-1",
      text: answer,
    });
  });
});
```

Run the suite:

```
npm test
```

These tests do not claim that both meeting answers are factually correct. They establish that wording variation is not a turn-protocol failure. Factual correctness needs its own evidence and oracle.

A practical test strategy separates concerns:

| Layer | Suitable oracle | Example failure | 
|---|---|---|
| Turn protocol | Deterministic state and effect assertions | Interrupted turn begins playback | 
| Safety and policy | Rules, schemas, allowlists, and human-approved cases | Companion gives disallowed advice | 
| Open-ended quality | Rubrics, reference facts, sampled human review | Answer is confusing despite being allowed | 

An LLM-based evaluator can assist the third layer, but it should not silently become the authority for the first two. If an evaluator says a response is acceptable while the response belongs to a stale turn, the application must still reject it.

Likewise, when a test and an LLM disagree, inspect the contract before changing either one:

This turns “the model was right” from a rhetorical conclusion into a reviewable engineering decision.

Keep vendor and network operations behind ports:

```
export interface VoicePorts {
  requestTranscript(turn: number): Promise<void>;

  requestModel(input: {
    turn: number;
    requestId: string;
    text: string;
  }): Promise<void>;

  speak(input: {
    turn: number;
    requestId: string;
    text: string;
  }): Promise<void>;

  cancelAudio(input: {
    turn: number;
    requestId: string;
  }): Promise<void>;

  showNotice(message: string): void;
}
```

Your effect dispatcher translates application effects into the configured speech recognition, LLM, synthesis, and RTC operations. The corresponding callbacks are translated back into reducer events.

For the LLM path, preserve the same application request identifier through the configured integration where supported. The Tencent RTC [LLM configuration documentation](https://trtc.io/document/68338) is the implementation reference for connecting an OpenAI-compatible model or an agent platform such as Dify or Coze.

Do not let the adapter mutate session state directly. Its job is to report outcomes such as `MODEL_DONE`, `MODEL_TIMEOUT`, or `PLAYBACK_FAILED`. The reducer decides whether those outcomes are still current.

A single “response latency” number cannot tell you where a turn stalled. Record timestamps around boundaries such as:

```
type TurnTiming = {
  turn: number;
  requestId: string;
  userEndedAt?: number;
  transcriptReadyAt?: number;
  modelRequestedAt?: number;
  modelCompletedAt?: number;
  playbackRequestedAt?: number;
  playbackStartedAt?: number;
};
```

From those fields, derive separate measurements for:

Choose timeout budgets from measurements in your own supported environments. Do not hide a slow phase behind automatic retries: a retry can produce a second, even later answer after the user has moved on.

The user-visible fallback should also match the failed phase. “I didn’t catch that” is appropriate for transcription failure but misleading when the model timed out.

Unit tests prove reducer behavior. Staging drills prove that adapters preserve it.

Start one utterance, begin another, and release the first transcript afterward.

**Expected:** The old transcript is logged as stale and never reaches the LLM.

Delay model completion beyond the application deadline.

**Expected:** Recovery remains visible. The late answer is not synthesized.

Introduce a delay between requesting synthesis and receiving the playback-start callback. Interrupt during that gap.

**Expected:** The late audio is cancelled immediately. State remains with the user’s new turn.

Make speech output unavailable while retaining the model result.

**Expected:** The interface exposes a clear fallback, such as displaying the text. It must not pretend the answer was spoken.

Disconnect the RTC path while audio is active.

**Expected:** Local playback is stopped, the session becomes visibly offline, and reconnection requires an explicit application policy.

Deliver `MODEL_DONE` or `PLAYBACK_ENDED` twice.

**Expected:** The duplicate cannot trigger a second spoken answer or advance a later turn.

A social or companion experience should make its AI nature clear and provide visible controls for muting, interrupting, ending the session, and managing any retained transcript data. Moderation and privacy decisions belong to the application design; fluent output is not evidence that those decisions are safe.

For character dialogue, AI companions, voice rooms, and related use cases, Tencent RTC also describes the broader [Social Entertainment solution](https://trtc.io/solutions/social-entertainment). Treat that page as scenario context, while using the product documentation for implementation details.

Before shipping, verify that:

The useful reframing is simple: a failed test is evidence of disagreement, not automatic proof that either the implementation or the model is correct. In a live voice application, the strongest tests are the ones that defend turn ownership and human control while leaving room for valid language variation.

*Relationship disclosure: This article was produced in connection with Tencent RTC. Official Tencent RTC documentation was used as the implementation reference.*

Which assertion in your current AI test suite is really a protocol guarantee—and which one is only an accidental preference for a particular wording?