# Show HN: We gave a voice agent an inner monologue so it stops forgetting

> Source: <https://waterr.ai/research/monologue>
> Published: 2026-08-04 14:10:05+00:00

# Monologue: *the agent talks to itself while it listens*

A research preview of Monologue — the reasoning harness that powers thinking inside Waterr's AI meetings. A slot-typed text-side sidecar runs in parallel with a frozen real-time voice model, so a low-latency conversation can be backed by deliberate reasoning without paying the latency cost in-band.

Human conversation is bidirectional and real-time. Between our turns, we think, and that thinking is what keeps a long conversation coherent, what surfaces real value, what lets us hold a thread across the whole hour. State-of-the-art real-time voice models don't do this. They may carry frontier-level intelligence, but the moment you put them into a long, high-stakes conversation they bottleneck: no room to reason between turns, no memory of a rule set six turns ago, no coherent thread across half an hour. Today we're publishing our first research note on the architecture that closes that gap: Monologue (the architecture our whitepaper calls ORI, the Omni-model Reasoning Infrastructure). Monologue lets a real-time voice model think in chunks *while the user is still speaking*, so its next move is deliberate and informed by everything that came before, without skyrocketing the per-minute price to a level nobody can afford in production.

The result, measured on a public benchmark: Thinking-tier conversation quality at Instant-tier latency, at roughly a third of the cost of the nearest Thinking-tier system – on the same frozen backbone. It ships as an API at a flat $0.03 per conversation-minute, planner included. The model that speaks should not have to be the model that thinks.

## The failure everyone who has talked to a voice AI has met

Thirty minutes into a screening interview, the candidate said it in minute four: *"I can't relocate before March."* In minute thirty-one, a bare real-time voice model cheerfully proposes a February start date on-site. The candidate notices. The meeting is over in every way that matters, even though it keeps going.

That moment – the forgotten fact, the contradicted commitment, the dropped instruction, the ignored correction – is the signature failure of real-time voice AI, and it is catalogued systematically by [Audio MultiChallenge (Scale AI, 2025)](https://scale.com/research/audiomc). It is not four separate bugs. It is one failure: information routing under a latency constraint. Real-time voice models – Google's `gemini-live-2.5-flash-native-audio`

, OpenAI's `gpt-realtime-2`

, Thinking Machines' `interaction-small`

– live inside a ~200–500 ms turn-taking window that forecloses the extended chain-of-thought budgets text-mode reasoning models routinely use. Even where a `thinking_config`

is exposed on paper, the current generation of production speech-to-speech SKUs typically rejects it at request time.

The model has enough capacity. It does not have the right context at the right moment. Our ablation shows this directly: an oracle planner that sees the literal upcoming user turn lifts factual recall by +24.1 points on the same frozen backbone. The bottleneck is not the model. It is what reaches the model, and when.

The engineering question is therefore not *how to make the voice model reason harder in-band* – that breaks the pacing that makes a conversation feel human, and multiplies the per-minute cost. It is where the informing deliberation lives. (Decision researchers have a name for this shape: fast expert decisions are good because the deliberate work already happened, somewhere else, in advance.) Prompt engineering pre-loads context statically and cannot adapt as the conversation moves. Long-term memory gives the fast path a bigger buffer but does no fresh reasoning over it. In-band reasoning makes the user wait.

## Our approach

### The interface: an AI that is in the meeting with you

Waterr's product surface is what we call an omni-model interface: you are present in a video call with the AI, and the AI participates through audio. Video-in captures your face, environment, and screen; audio-out returns natural speech. No lip-sync, no talking-head avatar, no synthetic reciprocity. (The name describes the product surface; the evaluation in this note covers the audio reasoning path.)

This asymmetry is deliberate. In production observation, users tired of synthetic reciprocity quickly; they wanted a productive collaborator that speaks, not a synthetic interlocutor to perform reciprocity toward. The user is a face plus a voice plus a room; the AI is a voice plus a reasoning process. That is a feature, not an omission – and it means the interface pushes far more context in per unit time than a voice-only agent, which makes the sidecar's job both more valuable (there is more to reason over) and more tractable (higher-fidelity signal to reason from). The lineage is the omni-model line from Alibaba's Qwen team – Qwen2.5-Omni (Qwen Team, 2025) – adopted at the harness level rather than inside a single model: the sidecar can call the vision, retrieval, and tool capabilities of whichever reasoning model it runs, on top of a frozen audio backbone.

Monologue is the mechanism that turns that dense multimodal input into cross-turn coherence at conversational speed. It has been the deep-think processor inside Waterr's production meeting engine since June.

### The reasoning sidecar: thinking between the turns

Our starting point is a frozen speech-to-speech backbone (`gemini-live-2.5-flash-native-audio`

) that cannot reason in-band. Several design choices make Monologue work.

Off-band reasoning, not in-band. The sidecar is a separate text-side model that observes the transcript, produces a briefing, and injects it into the voice model's session in the audio gap that already exists between turns – before the next audio streams in. Wall time is unchanged; the user hears no pause.

Slot-typed thoughts, not free-form summarisation. Our first sidecar emitted three unstructured analytical bullets. It was null on factual recall (−0.4 pts on the benchmark's memory axis) – the voice model treated free-form bullets as background chatter. The failure was not summarisation; it was the *carrier shape*. Monologue instead uses a slot-typed contract: `MEMORY`

(verbatim user-stated facts – quoted, not paraphrased), `CONSTRAINT`

(the bot's own prior commitments plus an enforcement clause), `TRAJECTORY`

(the prescriptive action for the next reply). Each slot is one full sentence; a runtime parser stitches them into a flowing background paragraph the model reads before it speaks. In the vignette above, the `MEMORY`

slot is carrying *"user said: 'I can't relocate before March'"* into the model's context at minute thirty-one. Under this contract the sidecar recovers +7 to +16 pts on factual recall (depending on baseline run) – a substantial fraction of the oracle-context gap – without any leak of the upcoming user turn.

Injection format is a first-class variable. Injecting content between turns risks the model treating it as a completed prior turn and going silent – we hit exactly this with a 23% empty-reply rate on our first slot-typed planner. Joining the slots into one paragraph and softening the wrapper to `[Background note – read silently before you speak your reply. Do not read this aloud.]`

dropped the empty rate to 3.2%. Silent-injection stability is something we measure on every backend we support.

Mid-turn injection: think while they're still speaking. The sidecar doesn't have to wait for the user to finish. A parallel STT transcript feeds a debouncer that inspects each partial as it arrives and releases the planner only when four gates all pass – the transcript has stayed stable for ≥ 0.6 s, is at least 20 characters long, has grown by 25 characters since the last firing, and 2.5 s have passed since the last firing. On a long user turn the planner can fire two or three times before the user is done, and each slot update is injected silently into the live session – so the model already has the latest reasoning loaded when it opens its reply. Currently supported on Gemini Live only (needs the parallel STT stream); all four gates are per-deployment tunable.

A zero-leak protocol. The sidecar sees turns 1..K−1 only, never the upcoming user turn. This is what makes the number reproducible in production, where the sidecar cannot see the future. The oracle condition (which does see turn K) is reserved as a diagnostic ceiling.

Backbone-agnostic by construction. The sidecar is a contract, not a model: any reasoning model can fill it (Claude Sonnet 5 does, in this preview, at ≈$0.004 per call), and any voice backbone that exposes a mid-session injection primitive can receive it. This is the strategic property of the design: every frontier-model improvement, from any vendor, makes Waterr meetings better the week it ships – we pick the best backbone and the best planner per deployment, and re-pick when the frontier moves. Vendors will ship thinking-capable voice models natively; Monologue wraps those too. A vendor optimises a model. We optimise the meeting.

### What it does in a meeting

- Remembers what was said. Verbatim facts from early turns – names, numbers, constraints – reach the model at the moment they're needed (
`MEMORY`

). - Keeps its own commitments. The model is reminded of what it already promised, with an enforcement clause (
`CONSTRAINT`

). - Follows the brief. The next reply's concrete action – the rule being applied, the edit being made – is named before the model speaks (
`TRAJECTORY`

). - Knows where it is in the meeting. Opening, middle, closing, final-minutes: a closing question is answered differently than an opening one.
- Dials thinking depth per deployment. The sidecar can call any reasoning model on the price/quality curve; the backbone stays fast.
- Adds no conversation wall time. 22.5 s bare vs 22.5 s with the sidecar per benchmark conversation – the sidecar is structurally off the latency path.
- Stays out of the transcript. Briefings are read silently; empty-reply rate in evaluation is 3.2% (5/154 conversations).

## The evidence

### The setup

We evaluate on [Audio MultiChallenge (Scale AI, 2025)](https://scale.com/research/audiomc) – multi-turn conversations balanced across four failure axes, judged per-rubric by an LLM judge, with the bare backbone re-run three independent times so every effect is measured against the benchmark's own run-to-run noise.

### Results

Monologue lifts the same frozen backbone from 38.50% to 69.8% aggregate APR at n=140, paired against three independent baseline runs. That is +1.8 pts above the text-mode Sonnet-5 reasoning ceiling measured on the same 90 paired conversations – three of the four AudioMC axes meet or exceed that ceiling, and factual recall (`INFERENCE_MEMORY`

) beats it by +17.6 pts (a slot-typed verbatim carrier over an audio-native backbone outperforms a frontier text reasoner reading the transcript). Planner p90 is 3.5 s, empty-transcript rate 0 of 139. In cross-vendor terms: that is Thinking-tier quality reached at Instant-tier latency, from an Instant-tier backbone.

| Instant · external Thinking | Thinking | Instant | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Gemini Live 2.5 + Monologue*Waterr · headline | GPT Realtime 2xHigh reasoning | Gemini 3.1 Flash LiveThinking | Gemini 2.5 Flash Native AudioThinking · preview | Gemini Live 2.5*bare backbone | Gemini 3.1 Flash LiveGoogle · Live 3.1 | Gemini 2.5 Flash Native AudioGoogle · preview | GPT Realtime 2default | TML Interaction Small276B MoE, 12B active | ||
| AudioMC · APR (%) | Aggregate140 conversations, 4 axes | 69.8* | 48.5 | 36.1 | 21.5 | 38.50* | 26.8 | 13.9 | 37.6 | 43.4 |
`INFERENCE_MEMORY` multi-turn factual recall |
54.9* | – | – | – | 22.76* | – | – | – | – | |
`SELF_COHERENCE` no self-contradiction |
77.2* | – | – | – | 29.89* | – | – | – | – | |
`INSTRUCTION_RETENTION` rule-following across turns |
76.9* | – | – | – | 45.45* | – | – | – | – | |
`VOICE_EDITING` mid-conversation corrections |
67.5* | – | – | – | 57.79* | – | – | – | – | |
| Cost & Latency | Cost / meeting-minuteaudio-only list price | ≈ $0.020 | ≈ $0.08 | ≈ $0.030 | ≈ $0.018 | ≈ $0.012 | ≈ $0.012 | ≈ $0.012 | – | – |
| Wall time / conversationend-to-end, seconds | 22.5 | – | – | – | 22.5 | – | – | – | – | |
| best per row on Scale (Scale's judge) · our system · * Waterr AI numbers measured under our own harness (Claude Sonnet 4.6 judge, Gemini Live 2.5 GA release). All other numbers from the Scale AudioMC leaderboard snapshot 2026-06-23 under Scale's undisclosed judge; both harnesses use the teacher-forced protocol standard to AudioMC. Blank per-axis cells are unmeasured on Scale's leaderboard, not zeros. Cost row: 30 s user + 30 s assistant audio at vendor list prices (re-verified 2026-07-03 against Google and OpenAI pricing pages); Thinking and xHigh cells include estimated reasoning-token surcharges that vendors do not publish. |

[Audio MultiChallenge (Scale AI, 2025)](https://scale.com/research/audiomc). Monologue-augmented Gemini Live measures 69.8% aggregate APR under our own harness at n=140. Other columns are vendors' published numbers (snapshot 2026-06-23) and are not directly comparable – different judge, and bare models vs a model-plus-planner system.

Per axis, in meeting terms (paired vs the primary baseline, with the spread across all three baseline runs):

| What breaks in meetings | Benchmark axis | Delta vs primary baseline | Across all baselines |
|---|---|---|---|
| Forgets facts you stated | `INFERENCE_MEMORY` | +16.29 | +7.0 to +16.3 – positive vs all three |
| Drops your instructions | `INSTRUCTION_RETENTION` | +5.41 | +5.4 to +16.2 – positive vs all three |
| Contradicts itself | `SELF_COHERENCE` | +20.33 | −0.3 to +20.3 – unresolved; noise ≈ effect, so we don't claim it |
| Ignores your corrections | `VOICE_EDITING` | −1.09 | −1.1 to +1.2 – null; already the backbone's strongest axis |

The two axes the slot contract directly targets with verbatim carriers move against every baseline we have. Self-coherence looks spectacular against one baseline and vanishes against another – on an axis where identical bare runs differ by 20 points, we don't claim it.

### Pricing, cost, and latency

Pricing. The Monologue API is a flat $0.03 per conversation-minute, planner included – roughly 60% below `gpt-realtime-2`

at xHigh (~$0.08/min), the only system in its quality territory. Flat-rate means no token metering, and every backbone improvement we adopt is priced in, not passed on.

Underlying cost. End-to-end unit cost, including planner tokens at production turn rates, is $0.03–0.05 per meeting-minute (raw audio is ≈$0.012/min at list prices; the planner adds ≈$0.004 per call at 3–4 calls per minute; prompt and history overhead make up the rest). That is roughly half of `gpt-realtime-2`

at xHigh reasoning (~$0.08/min at moderate token consumption).

Latency. 22.5 s (bare) vs 22.5 s (Monologue) end-to-end wall time per conversation. The sidecar runs in a gap that already exists.

## What this doesn't show yet

We hold ourselves to the evidence, so three honest edges, stated plainly (the full limitations analysis is in the research paper):

- The routing is measured at its ceiling. In this evaluation the sidecar's axis-aware routing uses the benchmark's axis labels as hints; a production planner would self-classify from conversation history. Re-running with self-classified routing is the immediate next step.
- The headline run's judge shares a vendor with the planner. A cross-vendor agreement check on earlier-iteration records was strong (κ = 0.89 on genuinely double-judged rubrics); re-judging this run's sample with a second vendor is queued and gates any harder magnitude claim.
- Point estimates are directional, not decimal-place claims. n=124 paired conversations; per-axis cells are small, which is why we publish the spread across three baselines rather than a single flattering column.

The road ahead, ordered by expected value: self-classified axis routing; a confirmation run on a fresh sample; cross-vendor re-judge; a retrieval-augmented planner targeting the residual oracle gap on factual recall; streaming and deferred planners; scenario-adherence evaluation on Waterr's actual production surface; cross-backbone deployment on OpenAI Realtime and other Live-capable backbones.

## Try it, test it, or challenge it

Monologue runs inside every Waterr AI meeting today. If you run interviews, discovery calls, or training sessions, you can experience it directly at [waterr.ai](https://waterr.ai). If you build voice-first products, the Monologue API is available at a flat $0.03 per conversation-minute – reach us at

for access, to test the sidecar on your own scenarios, or to have a reasoning backend evaluated on our harness. The raw JSONL, statistics code, and reproducibility pins are available on request.[[email protected]](/cdn-cgi/l/email-protection)

## Citation

Please cite this work as:

```
Sharma, H. and Waterr Research, "Monologue (Monologue): An Omni-model
Reasoning Infrastructure for Real-Time Voice AI",
Waterr Research: Research Notes, July 2026.
```

BibTeX:

```
@article{waterr2026ori,
  author  = {Sharma, Harshit and {Waterr Research}},
  title   = {Monologue (ORI): An Omni-model Reasoning
             Infrastructure for Real-Time Voice AI},
  journal = {Waterr Research: Research Notes},
  year    = {2026},
  month   = {July},
  note    = {https://waterr.ai/research/monologue.html}
}
```