{"slug": "stop-sending-half-sentences-to-openai-in-a-tencent-rtc-voice-companion", "title": "Stop Sending Half-Sentences to OpenAI in a Tencent RTC Voice Companion", "summary": "A developer published a Python tutorial building an \"utterance commit gate\" for Tencent RTC Conversational AI voice applications, a state-machine controller that waits for a configurable quiet window before committing a transcript turn to an LLM. The controller tracks six states (LISTENING, SETTLING, THINKING, MODERATING, SPEAKING, RECOVERING), correlates downstream LLM, moderation and speech-synthesis operations by request ID, and makes recovery explicit, using only the Python standard library on Python 3.11+.", "body_md": "Voice-companion demos invite an obvious question: *Which model makes the smartest or most charming character?*\n\nThat is rarely the first production decision. A capable model still feels broken if your application sends it half a sentence, repeats a finalized transcript, or plays a response after the user has already started a new turn.\n\nThe practical tension is between responsiveness and certainty:\n\nThis tutorial builds a small Python **utterance commit gate** for a Tencent RTC Conversational AI application. It waits for a configurable quiet window, correlates every downstream operation, and makes recovery explicit. The result is not a smarter model. It is a voice pipeline that gives the model coherent turns.\n\nKeep the media and AI responsibilities separate:\n\n```\nMicrophone\n   │\n   ▼\nTencent RTC media transport\n   │\n   ▼\nSpeech recognition ── partial/final segments\n   │\n   ▼\nUtterance commit gate  ← this tutorial\n   │\n   ├── LLM request\n   ├── output moderation\n   └── speech synthesis\n            │\n            ▼\n      RTC audio playback\n```\n\nTencent RTC's Conversational AI scenario supports real-time voice interaction with multiple LLM providers. Its LLM configuration documentation also describes connecting OpenAI-compatible models and carrying request identifiers for routing and observability:\n\nThe Python types below are deliberately application-owned. They are **not Tencent RTC SDK API names**. Your adapter translates actual speech, model, moderation, synthesis, and RTC callbacks into these events.\n\nOur controller recognizes six states:\n\n| State | Meaning | Permitted next step | \n|---|---|---|\n| `LISTENING` | Collecting partial or final transcript segments | Wait or begin settling | \n| `SETTLING` | All known segments are final; quiet timer is running | Commit or accept another segment | \n| `THINKING` | One LLM request owns the turn | Moderate its result or cancel it | \n| `MODERATING` | The generated draft is being checked | Speak it or enter recovery | \n| `SPEAKING` | Approved audio is being presented | Finish or be interrupted | \n| `RECOVERING` | The current operation has an uncertain or failed outcome | Resume explicitly | \n\nThree invariants matter more than the exact state names:\n\nThis example uses only the Python standard library and Python 3.11 or newer.\n\n```\nmkdir rtc-turn-commit\ncd rtc-turn-commit\ntouch companion.py test_companion.py\n```\n\nAdd the controller to `companion.py`:\n\n``` python\nfrom __future__ import annotations\n\nfrom dataclasses import dataclass\nfrom enum import Enum, auto\nfrom typing import Optional\n\nclass Mode(Enum):\n    LISTENING = auto()\n    SETTLING = auto()\n    THINKING = auto()\n    MODERATING = auto()\n    SPEAKING = auto()\n    RECOVERING = auto()\n    STOPPED = auto()\n\n@dataclass(frozen=True)\nclass Transcript:\n    epoch: int\n    segment_id: str\n    revision: int\n    text: str\n    final: bool\n\n@dataclass\nclass StoredSegment:\n    revision: int\n    text: str\n    final: bool\n    order: int\n\n@dataclass(frozen=True)\nclass Effect:\n    kind: str\n    request_id: Optional[str] = None\n    text: Optional[str] = None\n\nclass TurnCommitter:\n    def __init__(\n        self,\n        settle_ms: int = 350,\n        transcript_timeout_ms: int = 8_000,\n    ) -> None:\n        if settle_ms < 0:\n            raise ValueError(\"settle_ms must be non-negative\")\n        if transcript_timeout_ms <= settle_ms:\n            raise ValueError(\"transcript timeout must exceed settle time\")\n\n        self.settle_ms = settle_ms\n        self.transcript_timeout_ms = transcript_timeout_ms\n        self.mode = Mode.LISTENING\n        self.epoch = 1\n\n        self._segments: dict[str, StoredSegment] = {}\n        self._consumed_segment_ids: set[str] = set()\n        self._next_order = 0\n        self._first_segment_at: Optional[int] = None\n        self._settle_deadline: Optional[int] = None\n\n        self._turn_number = 0\n        self._active_request: Optional[str] = None\n\n    @property\n    def active_request(self) -> Optional[str]:\n        return self._active_request\n\n    def on_transcript(self, event: Transcript, now_ms: int) -> list[Effect]:\n        if self.mode not in (Mode.LISTENING, Mode.SETTLING):\n            return []\n        if event.epoch != self.epoch:\n            return []\n        if event.segment_id in self._consumed_segment_ids:\n            return []\n\n        previous = self._segments.get(event.segment_id)\n        if previous and event.revision <= previous.revision:\n            return []\n\n        if previous is None:\n            order = self._next_order\n            self._next_order += 1\n        else:\n            order = previous.order\n\n        self._segments[event.segment_id] = StoredSegment(\n            revision=event.revision,\n            text=event.text.strip(),\n            final=event.final,\n            order=order,\n        )\n\n        if self._first_segment_at is None:\n            self._first_segment_at = now_ms\n\n        if self._segments and all(s.final for s in self._segments.values()):\n            self.mode = Mode.SETTLING\n            self._settle_deadline = now_ms + self.settle_ms\n        else:\n            self.mode = Mode.LISTENING\n            self._settle_deadline = None\n\n        return []\n\n    def tick(self, now_ms: int) -> list[Effect]:\n        if (\n            self._first_segment_at is not None\n            and now_ms - self._first_segment_at >= self.transcript_timeout_ms\n            and self.mode in (Mode.LISTENING, Mode.SETTLING)\n        ):\n            self._clear_transcript()\n            self.mode = Mode.RECOVERING\n            return [Effect(\"status\", text=\"transcript_timeout\")]\n\n        if (\n            self.mode != Mode.SETTLING\n            or self._settle_deadline is None\n            or now_ms < self._settle_deadline\n        ):\n            return []\n\n        ordered = sorted(self._segments.items(), key=lambda item: item[1].order)\n        text = \" \".join(segment.text for _, segment in ordered if segment.text)\n        text = \" \".join(text.split())\n\n        for segment_id, _ in ordered:\n            self._consumed_segment_ids.add(segment_id)\n        self._clear_transcript()\n\n        if not text:\n            self.mode = Mode.LISTENING\n            return []\n\n        self._turn_number += 1\n        request_id = f\"session-{self.epoch}-turn-{self._turn_number}\"\n        self._active_request = request_id\n        self.mode = Mode.THINKING\n\n        return [Effect(\"generate\", request_id=request_id, text=text)]\n\n    def on_llm_result(self, request_id: str, draft: str) -> list[Effect]:\n        if self.mode != Mode.THINKING or request_id != self._active_request:\n            return []\n\n        self.mode = Mode.MODERATING\n        return [Effect(\"moderate\", request_id=request_id, text=draft)]\n\n    def on_moderation_result(\n        self,\n        request_id: str,\n        allowed: bool,\n        approved_text: str = \"\",\n    ) -> list[Effect]:\n        if self.mode != Mode.MODERATING or request_id != self._active_request:\n            return []\n\n        if not allowed:\n            self._active_request = None\n            self.mode = Mode.RECOVERING\n            return [Effect(\"status\", text=\"reply_blocked\")]\n\n        self.mode = Mode.SPEAKING\n        return [\n            Effect(\"speak\", request_id=request_id, text=approved_text)\n        ]\n\n    def on_user_speech_started(self) -> list[Effect]:\n        effects: list[Effect] = []\n\n        if self.mode in (Mode.THINKING, Mode.MODERATING):\n            effects.append(Effect(\"cancel_generation\", self._active_request))\n        elif self.mode == Mode.SPEAKING:\n            effects.append(Effect(\"stop_speech\", self._active_request))\n\n        if self.mode in (Mode.THINKING, Mode.MODERATING, Mode.SPEAKING):\n            self._active_request = None\n            self.mode = Mode.LISTENING\n\n        return effects\n\n    def on_speech_finished(self, request_id: str) -> list[Effect]:\n        if self.mode != Mode.SPEAKING or request_id != self._active_request:\n            return []\n\n        self._active_request = None\n        self.mode = Mode.LISTENING\n        return []\n\n    def on_stage_error(self, request_id: str, stage: str) -> list[Effect]:\n        if request_id != self._active_request:\n            return []\n\n        self._active_request = None\n        self.mode = Mode.RECOVERING\n        return [Effect(\"status\", text=f\"{stage}_failed\")]\n\n    def on_disconnect(self) -> list[Effect]:\n        effects: list[Effect] = []\n\n        if self.mode in (Mode.THINKING, Mode.MODERATING):\n            effects.append(Effect(\"cancel_generation\", self._active_request))\n        elif self.mode == Mode.SPEAKING:\n            effects.append(Effect(\"stop_speech\", self._active_request))\n\n        self.epoch += 1\n        self._active_request = None\n        self._consumed_segment_ids.clear()\n        self._clear_transcript()\n        self.mode = Mode.RECOVERING\n        effects.append(Effect(\"status\", text=\"connection_lost\"))\n        return effects\n\n    def resume(self) -> None:\n        if self.mode == Mode.RECOVERING:\n            self.mode = Mode.LISTENING\n\n    def stop(self) -> None:\n        self._active_request = None\n        self._clear_transcript()\n        self.mode = Mode.STOPPED\n\n    def _clear_transcript(self) -> None:\n        self._segments.clear()\n        self._first_segment_at = None\n        self._settle_deadline = None\n```\n\nAdd deterministic tests to `test_companion.py`:\n\n``` python\nimport unittest\n\nfrom companion import Effect, Mode, Transcript, TurnCommitter\n\nclass TurnCommitterTests(unittest.TestCase):\n    def test_two_final_segments_become_one_model_request(self) -> None:\n        c = TurnCommitter(settle_ms=300)\n\n        c.on_transcript(\n            Transcript(c.epoch, \"a\", 1, \"Could you explain\", True),\n            now_ms=0,\n        )\n        self.assertEqual(c.tick(299), [])\n\n        # A second final segment restarts the settling window.\n        c.on_transcript(\n            Transcript(c.epoch, \"b\", 1, \"Python generators?\", True),\n            now_ms=250,\n        )\n        self.assertEqual(c.tick(549), [])\n\n        self.assertEqual(\n            c.tick(550),\n            [\n                Effect(\n                    \"generate\",\n                    \"session-1-turn-1\",\n                    \"Could you explain Python generators?\",\n                )\n            ],\n        )\n        self.assertEqual(c.mode, Mode.THINKING)\n\n    def test_duplicate_revision_cannot_create_a_second_turn(self) -> None:\n        c = TurnCommitter(settle_ms=100)\n        event = Transcript(c.epoch, \"segment-1\", 2, \"Hello\", True)\n\n        c.on_transcript(event, now_ms=0)\n        first = c.tick(100)\n        self.assertEqual(len(first), 1)\n\n        c.on_user_speech_started()\n        c.on_transcript(event, now_ms=200)\n        self.assertEqual(c.tick(500), [])\n\n    def test_interruption_invalidates_late_model_output(self) -> None:\n        c = TurnCommitter(settle_ms=100)\n        c.on_transcript(\n            Transcript(c.epoch, \"a\", 1, \"Tell me a story\", True),\n            now_ms=0,\n        )\n        request = c.tick(100)[0].request_id\n\n        self.assertEqual(\n            c.on_user_speech_started(),\n            [Effect(\"cancel_generation\", request)],\n        )\n\n        # Cancellation is best-effort. The old callback may still arrive.\n        self.assertEqual(c.on_llm_result(request, \"Once upon a time\"), [])\n        self.assertEqual(c.mode, Mode.LISTENING)\n\n    def test_reconnect_rejects_old_transcript_callbacks(self) -> None:\n        c = TurnCommitter(settle_ms=100)\n        old_epoch = c.epoch\n\n        c.on_disconnect()\n        c.resume()\n\n        c.on_transcript(\n            Transcript(old_epoch, \"late\", 1, \"Old audio\", True),\n            now_ms=0,\n        )\n        self.assertEqual(c.tick(500), [])\n        self.assertEqual(c.mode, Mode.LISTENING)\n\n    def test_partial_transcript_eventually_enters_recovery(self) -> None:\n        c = TurnCommitter(\n            settle_ms=100,\n            transcript_timeout_ms=1_000,\n        )\n        c.on_transcript(\n            Transcript(c.epoch, \"a\", 1, \"unfinished\", False),\n            now_ms=0,\n        )\n\n        self.assertEqual(\n            c.tick(1_000),\n            [Effect(\"status\", text=\"transcript_timeout\")],\n        )\n        self.assertEqual(c.mode, Mode.RECOVERING)\n\nif __name__ == \"__main__\":\n    unittest.main()\n```\n\nRun the suite:\n\n```\npython -m unittest -v\n```\n\nYou should see five passing tests. More importantly, the tests prove behavioral properties rather than relying on real-time sleeps:\n\nKeep provider-specific code outside the controller. A simplified dispatcher might look like this:\n\n``` python\nasync def apply_effect(effect, services, controller):\n    try:\n        if effect.kind == \"generate\":\n            draft = await services.llm.generate(\n                prompt=effect.text,\n                correlation_id=effect.request_id,\n            )\n            for next_effect in controller.on_llm_result(\n                effect.request_id,\n                draft,\n            ):\n                await apply_effect(next_effect, services, controller)\n\n        elif effect.kind == \"moderate\":\n            result = await services.moderation.check(effect.text)\n            next_effects = controller.on_moderation_result(\n                effect.request_id,\n                allowed=result.allowed,\n                approved_text=result.approved_text,\n            )\n            for next_effect in next_effects:\n                await apply_effect(next_effect, services, controller)\n\n        elif effect.kind == \"speak\":\n            await services.speech.play(\n                text=effect.text,\n                correlation_id=effect.request_id,\n            )\n\n        elif effect.kind == \"cancel_generation\":\n            await services.llm.cancel(effect.request_id)\n\n        elif effect.kind == \"stop_speech\":\n            await services.speech.stop(effect.request_id)\n\n        elif effect.kind == \"status\":\n            services.ui.show_status(effect.text)\n\n    except Exception:\n        # Classify the actual stage in production rather than using\n        # one broad exception handler.\n        for recovery_effect in controller.on_stage_error(\n            effect.request_id,\n            effect.kind,\n        ):\n            services.ui.show_status(recovery_effect.text)\n```\n\n`services.llm`, `services.moderation`, and `services.speech` are ports owned by your application. Map `correlation_id` to the request-identifier mechanism supported by your configured integration; do not assume the pseudocode argument name is an SDK field.\n\nThe live event flow is then:\n\n`on_transcript()`.` tick()` from your application timer.`on_user_speech_started()` when your turn-detection policy confirms a new user turn.`on_speech_finished()` only after playback completion is confirmed.`on_disconnect()` before attempting to rejoin.`resume()` after the media and recognition path is ready again.\nThe Tencent RTC [Social Entertainment solution](https://trtc.io/solutions/social-entertainment) includes AI virtual companions and character dialogue among its scenarios. The commit gate applies equally to a playful character, an assistant, or an AI host because it controls turns rather than personality.\n\nDo not copy `350 ms` into production and call it solved. It is an initial configuration value, not a universal latency target.\n\nChoose it with a replay set containing your application's actual conversational patterns:\n\n| Observation | Likely adjustment | Trade-off | \n|---|---|---|\n| Clause-ending pauses create separate requests | Increase the settling window | Slower response start | \n| Complete commands feel delayed | Decrease the settling window | More premature commits | \n| Recognition emits several final segments per sentence | Keep segment aggregation enabled | Requires stable ordering and IDs | \n| Very long partials never finalize | Review recognition behavior and timeout policy | Recovery may ask the user to repeat | \n\nRecord at least these timestamps per request identifier:\n\n```\nfirst_transcript_at\nlast_final_segment_at\nutterance_committed_at\nllm_started_at\nllm_finished_at\nmoderation_finished_at\nspeech_playback_started_at\nspeech_playback_finished_at\n```\n\nThis separates distinct delays. A single “AI latency” number cannot tell you whether time was spent waiting for a turn boundary, generating text, checking the output, synthesizing speech, or starting playback.\n\nThe transcript timeout enters `RECOVERING` and emits `transcript_timeout`. Show a visible and, when possible, accessible prompt such as “I didn't catch the end of that.” Do not submit the partial text as though it were confirmed.\n\nProvider cancellation may fail or arrive too late. The request-ID check is the authoritative defense: a callback without current ownership produces no speech.\n\nDo not silently bypass the stage. Move to recovery, keep the draft out of synthesis, and let the user retry. For a social companion, moderation and user-visible control belong in the runtime design, not only in the prompt.\n\nThe generated answer may be valid while its delivery outcome is unknown. Show a failure state and return to listening only through an explicit recovery action. Automatically generating a new answer could duplicate content.\n\nIncrement the epoch before rejoining. Old ASR callbacks are then structurally incapable of forming a new request. Start a fresh turn rather than pretending an interrupted utterance continued seamlessly.\n\nThis in-memory example intentionally does not resume an in-flight turn. A restarted service should create a new epoch and expose the interruption. If you later persist state, store the epoch, active request identifier, mode, and committed transcript together; restoring only the transcript can replay a turn without restoring its ownership rules.\n\nRun these drills with real audio and your configured providers:\n\nA model can demonstrably generate engaging character dialogue. That does not demonstrate that a live companion can identify a completed turn, recover from a partition, or respect an interruption.\n\nThe human decision underneath the model comparison is where to spend the next engineering cycle. Before changing models or expanding the character prompt, inspect ten imperfect voice sessions. If you find split sentences, duplicate requests, or stale replies, add a commit gate and correlation logging first. Model evaluation becomes more meaningful once every candidate receives the same coherent input turns.\n\n**Disclosure:** This article was produced in connection with Tencent RTC. Official Tencent RTC documentation was used as the implementation reference.", "url": "https://wpnews.pro/news/stop-sending-half-sentences-to-openai-in-a-tencent-rtc-voice-companion", "canonical_source": "https://dev.to/susiewang/stop-sending-half-sentences-to-openai-in-a-tencent-rtc-voice-companion-3ni6", "published_at": "2026-09-26 17:11:27+00:00", "updated_at": "2026-09-26 17:28:56.062384+00:00", "lang": "en", "topics": ["ai-agents", "natural-language-processing", "ai-tools", "developer-tools"], "entities": ["Tencent RTC", "OpenAI", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/stop-sending-half-sentences-to-openai-in-a-tencent-rtc-voice-companion", "markdown": "https://wpnews.pro/news/stop-sending-half-sentences-to-openai-in-a-tencent-rtc-voice-companion.md", "text": "https://wpnews.pro/news/stop-sending-half-sentences-to-openai-in-a-tencent-rtc-voice-companion.txt", "jsonld": "https://wpnews.pro/news/stop-sending-half-sentences-to-openai-in-a-tencent-rtc-voice-companion.jsonld"}}