{"slug": "a-transcript-is-a-choice", "title": "A Transcript Is a Choice", "summary": "Symphony, a steerable speech recognition model, nearly matches the purpose-built CrisperWhisper 2.0 on spoken disfluencies, scoring an F1 of 91.83% versus 92.46% on shared categories, according to a blog post by Ian Eaves. The post argues that transcription is a choice, as voice agents may need verbatim, clean, or structured transcripts, and that Symphony can adapt to each need.", "body_md": "[Blog](/blog)- A Transcript Is a Choice\n\n# A Transcript Is a Choice\n\nIan Eaves\n\nSymphony nearly matches a purpose-built verbatim transcription model on spoken disfluencies, and can turn the same recording into the representation each part of a voice agent needs.\n\nA voice agent hears:\n\n\"Um, can you move my Tuesday appointment to Thurs, Thursday at, uh, three thirty? Actually, make that four. Four works better.\"\n\nWhat should its transcript say?\n\nA verbatim transcript would preserve the fillers, the interrupted word, the abandoned time, and the eventual correction. A clean transcript would say something much simpler: the appointment should be moved to Thursday at 4:00. Agentic tasks might even need a structured interpretation explaining that 3:30 was proposed but ultimately replaced.\n\nAll three outputs can be reasonably described as \"accurate.\" They simply answer different questions.\n\n## Disfluency is not the same as transcription error\n\nPeople rarely speak in finished sentences. Spontaneous speech contains fillers such as \"um,\" repeated words, interrupted fragments, false starts, and corrections made halfway through an instruction.\n\nThese features are collectively described as **disfluencies**, but the category includes more than meaningless noise. In the example above, the correction from 3:30 to 4:00 determines what the agent actually needs to do. A system can preserve what someone said verbatim, or it can produce the clean text they apparently intended.\n\nMost speech recognition systems do not expose that decision explicitly, which can lead to inconsistent behavior across similar utterances. Making matters worse, ordinary Word Error Rate (WER) does not capture this distinction cleanly. A filler counted as an error against a clean reference might be exactly what a verbatim system was asked to preserve.\n\n## Purpose-built vs. general-purpose models\n\nSome companies, like Nyra Labs, are working to address this by purpose-building ASR models explicitly designed to handle disfluencies, and their published benchmarks show strong results. Steerable models like Symphony handle disfluencies (among other issues) as the caller needs, rather than being locked into a single purpose-built workflow. When we tested Symphony against Nyra's complete English benchmark using a simple verbatim-transcription instruction, Symphony performed almost identically to the frontier CrisperWhisper model.\n\n### Sound-excluded disfluency metric, English\n\n| Metric | Symphony | CrisperWhisper 2.0 | Difference |\n|---|---|---|---|\n| Precision | 88.93% | 88.29% | +0.64 pp |\n| Recall | 94.93% | 97.03% | -2.10 pp |\n| F1 | 91.83% | 92.46% | approximately -0.6 pp |\n\n::: callout{type=\"note\" title=\"One qualification\"} Symphony does not currently emit canonical vocal-sound labels such as [laughter] or [breath] under this configuration. We therefore recalculated the disfluency metric over the three categories both models produce: fillers, repetitions, and cut-offs. :::\n\nOn these shared categories, Symphony comes within roughly six-tenths of a percentage point of CrisperWhisper 2.0 on disfluency F1.\n\nThe precision and recall split is informative. Symphony was slightly less likely to introduce a disfluency that was not present, while CrisperWhisper recovered slightly more of the disfluencies in the reference transcript. Overall, the models were near parity despite CrisperWhisper being purpose-built for controllable verbatim speech recognition.\n\n## On the vagaries of transcription\n\nRecognizing disfluencies is useful, but a voice agent often needs more than a verbatim transcript. Take this example snippet from a hypothetical phone conversation\n\nDepending on application requirements an agent might need any of\n\n- A verbatim account of what was spoken.\n- A clean transcript of the speaker's final intended meaning.\n- A structured (JSON) interpretation of the corrections and uncertainty in the request.\n\nFor applications which narrowly require a solution like (1) traditional ASR models can be great solutions. Likewise, applications which need a pure intent driven transcript like (2) can benefit from purpose built models like CrisperWhisper. The muddy middle is where things get difficult; what if you need structured outputs, or customization over how the transcript is processed? What if you need both a verbatim and cleaned transcript in a single request?\n\nFor these sorts of tasks steerable multilingual models, like Symphony, can be a better match because the application can define it's own transcription protocol. In practice this actually ends up being much simpler than it sounds. If you need a single unadulturated transcript you can us something like this\n\n**Instructions**\n\n```\nTranscribe this audio verbatim.\nPreserve filler words, repetitions, false starts, interrupted words,\nand self-corrections exactly as spoken.\nReturn only the transcript.\n```\n\n**Transcript**\n\nUm, can you move my Tuesday appointment to Thurs, Thursday at uh three thirty, Actually, make that four. Four works better.\n\nThe output will respect the natural flow of the users real speech. Similarly, if you need disfluency handling you can use an instruction like this\n\n**Instructions**\n\n```\nProduce a clean transcript of the speaker's final intended request.\nRemove incidental filler words, false starts, restarts, and abandoned values.\nApply explicit self-corrections so that only the speaker's final resolved values\nremain. Preserve any uncertainty that the speaker does not resolve. Do not infer\nor add details that are not supported by the audio.\nOutput only the clean intended transcript.\n```\n\n**Transcript**\n\nCan you move my Tuesday appointment to Thursday at 4:00?\n\nIt can even flexibly produce JSON output matching your specific transcription requirements.\n\n**Instructions**\n\n```\nInterpret the attached voice request. Return one bare JSON object and nothing else.\nUse exactly these fields:\n- \"final_request\": the speaker's final resolved request as a string\n- \"corrections\": an array of objects with string fields \"abandoned\" and \"replacement\"\n- \"unresolved_uncertainty\": an array of strings\n- \"material_disfluencies\": an array of short strings\nIdentify explicit corrections and distinguish abandoned values from final values.\nInclude only uncertainty that remains unresolved. In \"material_disfluencies\", list\neach distinct audible filler, interrupted word or restart, repetition, and\nself-correction exactly once, in order of occurrence. Do not duplicate entries.\nUse no more than ten entries. Do not infer unsupported details. Do not include a\ntranscript, explanation, or Markdown fence.\n```\n\n**Response**\n\n```\n{\n  \"final_request\": \"Can you move my Tuesday appointment to Thursday at 4:00?\",\n  \"corrections\": [\n    {\n      \"abandoned\": \"3:30\",\n      \"replacement\": \"4:00\"\n    }\n  ],\n  \"unresolved_uncertainty\": [],\n  \"material_disfluencies\": [\n    \"um\",\n    \"uh\",\n    \"thurs\",\n    \"three thirty\",\n    \"actually\",\n    \"make that four\",\n    \"four works better\"\n  ]\n}\n```\n\nUnder a strict output-format test, the cut-off notation was the one limitation. Symphony recognized that the speaker began and restarted the word, but it normalized the boundary rather than marking it with the requested symbol. That is better understood as a formatting limitation than a failure to recover the spoken repair. Nyra's own benchmark reports several cut-off metrics because exact fragments are acoustically ambiguous, and its headline cut-off F1 gives credit when a model preserves an aligned token at the interruption even without a dedicated cut-off marker ([Nyra Labs](https://nyra-labs.com/research/nyra-verbatim-speech-benchmark)).\n\n## Not all repetition means the same thing\n\nWe also tested whether Symphony could distinguish two recordings containing superficially similar repetition:\n\n\"I really, really need this fixed today.\"\n\nand:\n\n\"I, I need this fixed today.\"\n\nIn both cases, a purely mechanical transcriber can observe that a word was repeated. The more useful question is why.\n\nFor the first recording, Symphony preserved both instances of \"really\" and classified the repetition as emphasis. Its explanation treated the repetition as an intensifier rather than as an abandoned attempt to form the sentence.\n\nFor the second recording, Symphony preserved both instances of \"I\" but classified the repetition as a speech repair. It interpreted the first \"I\" as a false start followed by an immediate restart.\n\nThe words alone do not fully explain this distinction. The same surface behavior, a repeated token, can carry different functions depending on the word, its delivery, and the surrounding language.\n\n## The transcript should fit the task\n\nOn Nyra's English benchmark, Symphony nearly matches a purpose-built verbatim transcription model on fillers, repetitions, and cut-offs. In controlled tests, it also used those signals to recover the speaker's final request, identify a corrected value, and distinguish deliberate emphasis from a speech repair.\n\nEvery transcription task has its own specific needs and requirements. Disfluency is an extremely important part of that story, but it is only one part. The broader problem is to build systems that can reliably preserve, remove, or interpret the features of the transcript required for its task.\n\nTo get started with Symphony, [create an account](https://sprag.ai/auth/signup).", "url": "https://wpnews.pro/news/a-transcript-is-a-choice", "canonical_source": "https://sprag.ai/blog/a-transcript-is-a-choice", "published_at": "2026-08-18 16:22:44+00:00", "updated_at": "2026-08-18 16:42:16.538782+00:00", "lang": "en", "topics": ["natural-language-processing", "artificial-intelligence"], "entities": ["Symphony", "CrisperWhisper 2.0", "Nyra Labs", "Ian Eaves"], "alternates": {"html": "https://wpnews.pro/news/a-transcript-is-a-choice", "markdown": "https://wpnews.pro/news/a-transcript-is-a-choice.md", "text": "https://wpnews.pro/news/a-transcript-is-a-choice.txt", "jsonld": "https://wpnews.pro/news/a-transcript-is-a-choice.jsonld"}}