{"slug": "stop-wrestling-with-asr-the-complete-guide-to-gemini-3-5-transcribe", "title": "Stop Wrestling with ASR: The Complete Guide to Gemini 3.5 Transcribe 🎙️", "summary": "Google has introduced Gemini 3.5 Transcribe, a dedicated speech-to-text model built on Gemini's audio understanding core, optimized for fast, accurate, and cost-effective transcription. The model supports native speaker diarization, word-level timestamps, and vocabulary biasing, and is accessible via the Google GenAI SDK and Google AI Studio.", "body_md": "You’ve probably used Gemini to analyze hours of video, summarize podcasts, or answer questions from recorded meetings (if you didn't you should, it's extremely useful!). But when all you need is a clean, hyper-accurate, and structured transcript from audio, spinning up a huge reasoning model with complicated prompts often feels like using a sledgehammer to crack a nut.\n\nEnter **Gemini 3.5 Transcribe** (`gemini-3.5-transcribe`\n\n).\n\nIt's Google's dedicated speech-to-text model built on Gemini's audio understanding core, optimized specifically for fast, accurate, and cost-effective transcription. Whether you want an exact court-reporter transcript with millisecond timestamps, or a reading-optimized summary that removes all your awkward *\"ums\"* and *\"uhs\"*, this model handles it natively with zero prompt gymnastics.\n\n🚀\n\nHands-on first:If you want to jump straight into running the code yourself, open the interactive[! It's ready to run so you can dirrectly experience how the model work.]Gemini Transcribe Colab notebook\n\nPrefer a visual UI with zero coding? You can also test speech recognition directly in[.]Google AI Studio\n\nHere's what you'll find in this guide:\n\nBefore looking at the code, let's get the mental model straight. You might wonder: *\"Can't I just upload an MP3 to Gemini 3.7 and say 'Transcribe this'?\"*\n\nYou can, but here is why `gemini-3.5-transcribe`\n\nis different:\n\n| Feature | General Audio Understanding (e.g. Gemini 3.7) | Dedicated Transcribe (`gemini-3.5-transcribe` ) |\n|---|---|---|\nPrimary Job |\nReasoning, Q&A, sentiment analysis, audio chat | High-throughput, precise speech-to-text |\nSpeaker Diarization |\nPrompt-dependent (can hallucinate turns) | Native segment labeling (`spk:0` , `spk:1` ) |\nTimestamps |\nApproximate timecodes via text prompt | True word-level millisecond offsets in metadata |\nVocabulary Biasing |\nSystem prompt instructions | Native acoustic biasing dictionary (up to 1,000 terms) |\nCost & Latency |\nFull multimodal LLM generation overhead | Optimized lightweight speech pipeline |\n\nPro tip:If you need toask questionsabout what happened in an audio file (\"What was the action item for Alice?\"), use a multimodal model like Gemini 3.7. If you need thetranscript itself, subtitles, or cleaned dictation notes, use Gemini Transcribe!\n\nThe Gemini 3.5 Transcribe model runs on the modern **Google GenAI SDK** (`google-genai`\n\nv2.0+) using the [Interactions API](https://ai.google.dev/gemini-api/docs/interactions-overview).\n\nFirst, install the SDK:\n\n```\npip install -U \"google-genai>=2.0.0\"\n```\n\nMake sure you have an API key from [Google AI Studio](https://aistudio.google.com/app/apikey), set it as `GEMINI_API_KEY`\n\n, and let's look at how audio gets passed to the model:\n\n``` python\nfrom google import genai\n\nclient = genai.Client()\n\n# 1. Upload your audio file via the Files API\naudio_file = client.files.upload(file=\"meeting_recap.mp3\")\n\n# 2. Request transcription using the uploaded file's URI\ninteraction = client.interactions.create(\n    model=\"gemini-3.5-transcribe\",\n    input=[{\"type\": \"audio\", \"uri\": audio_file.uri}],\n)\n\nprint(interaction.output_text)\n```\n\nWatch the demo video below to see the baseline transcription in action—handling natural speech and bilingual code-switching with ease:\n\nWhen dealing with audio and video, you never want to inline raw audio bytes as base64 in your API requests—it blows up the payload size by 33%, easily hits network timeouts, and requires re-uploading the same bytes if you want to rerun a query.\n\nThe ** Files API** solves this cleanly:\n\nAs you saw in the video above, Gemini Transcribe automatically identifies spoken languages out of the box and seamlessly handles **code-switching** (when someone mixes multiple languages in the same sentence—like switching between French and English mid-sentence, which happens to me all the time!).\n\nHowever, if you know your audio is exclusively in a specific language or regional dialect, you can pass explicit **BCP-47 language codes** in `transcription_config`\n\nto bias recognition:\n\n```\ninteraction = client.interactions.create(\n    model=\"gemini-3.5-transcribe\",\n    input=[{\"type\": \"audio\", \"uri\": spanish_audio.uri}],\n    generation_config={\n        \"transcription_config\": {\n            # Explicit language hint\n            \"language_codes\": [\"es-ES\"],\n        }\n    },\n)\n\nprint(interaction.output_text)\n```\n\nNote:Leaving`language_codes=[]`\n\n(or omitting it) enables full automatic detection across[85+ supported languages and locales]. Check out the[Audio Transcription Documentation]for the complete list of language codes.\n\nEvery developer has suffered from an ASR model mangling proper names, confusing specialized libraries with everyday dictionary words (turning *\"ScaNN\"* into *\"scan\"*, or *\"Qdrant\"* into *\"quadrant\"*), or inventing phonetically similar terms (*\"Sitsi\"* instead of *\"CitC\"*, *\"Thiago\"* instead of *\"Tiago\"*).\n\nWith `custom_vocabulary`\n\n, you can pass a list of up to **1,000 domain-specific terms** that the model will bias towards:\n\n```\ninteraction = client.interactions.create(\n    model=\"gemini-3.5-transcribe\",\n    input=[{\"type\": \"audio\", \"uri\": team_briefing.uri}],\n    generation_config={\n        \"transcription_config\": {\n            \"custom_vocabulary\": [\n                \"Guillaume Vernade\",\n                \"ScaNN\",\n                \"Qdrant\",\n                \"Cilium\",\n                \"Weaviate\",\n                \"Milvus\",\n                \"Buganizer\",\n                \"Tiago\",\n                \"CitC\",\n                \"CL\",\n                \"spaCy\",\n            ],\n        }\n    },\n)\n\nprint(interaction.output_text)\n```\n\nWatch the side-by-side comparison video below to see how the model behaves with and without custom vocabulary biasing:\n\n| Without Custom Vocabulary (Default ASR) | With `custom_vocabulary` (100% Precision) |\n|---|---|\n\"For our vector benchmarks, sync with Guillaume **Vernat* in Paris to compare Scan against Quadrant while Syllium handles the traffic.\"* |\n\"For our vector benchmarks, sync with Guillaume **Vernade* in Paris to compare ScaNN against Qdrant while Cilium handles the traffic.\"* |\n\"We also need to evaluate Weaviate against Milvus, assign the buganizer ticket to **Thiago, and test the changes in **Sitsi* before submitting the CL.\"* |\n\"We also need to evaluate Weaviate against Milvus, assign the Buganizer ticket to **Tiago, and test the changes in **CitC* before submitting the CL.\"* |\n\"Finally, run a quick smoke test with **Spacey* to validate the tokenization pipeline before deploying.\"* |\n\"Finally, run a quick smoke test with **spaCy* to validate the tokenization pipeline before deploying.\"* |\n\nNotice how default speech recognition falls back to phonetic dictionary guesses (**Vernat**, **Scan**, **Quadrant**, **Syllium**, **Thiago**, **Sitsi**, **Spacey**). By contrast, supplying `custom_vocabulary`\n\nguarantees that names of team members, niche tools, internal infrastructure, and open-source libraries are transcribed with 100% precision.\n\nPro tip:Don't just put acronyms in your custom vocabulary. Add proper names of team members, internal service codenames, GitHub repo handles, product brand names, and niche industry terminology.\n\nThis is hands down my favorite capability of Gemini 3.5 Transcribe.\n\nBy default, speech-to-text models operate in ** verbatim** mode: they write down\n\nWhen you're transcribing a speech rehearsal, interview, or voice memo, reading raw verbatim text is painful:\n\n```\n--- Verbatim output ---\n\"Uh, hello. Good evening, everyone. Um, I'd like to start by, well, first of all, thank you all for coming. Today is, um, a very special day, or rather, evening? No, afternoon? Right, evening. We are here to celebrate, uh, sorry, let me just find my notes. Ah, here. We are here to honor, no, not honor, but, um, to mark the launch of our new, sorry, my glasses are a bit foggy, the new marketing campaign. No, wait, product campaign? Product, yes. Um, where was I? Ah, yes. It has been a long journey, a very, uh, challenging, well, not challenging in a bad way, but, you know, difficult? No, rewarding. Rewarding is the word. So, um, yes, cheers to, wait, we don't have glasses yet. Thank you.\"\n```\n\nIf you switch `mode={\"type\": \"smart\"}`\n\n, the model performs intelligent reading optimization:\n\nHere is how you turn it on:\n\n```\ninteraction_smart = client.interactions.create(\n    model=\"gemini-3.5-transcribe\",\n    input=[{\"type\": \"audio\", \"uri\": audio_file.uri}],\n    generation_config={\n        \"transcription_config\": {\n            \"mode\": {\n                \"type\": \"smart\",\n            },\n        }\n    },\n)\n\nprint(interaction_smart.output_text)\n```\n\nLook at the cleaned result on that exact same rehearsal audio:\n\n```\n--- Smart transcription output ---\nGood evening everyone. First of all, thank you all for coming. Today is a very special evening. We are here to mark the launch of our new product campaign.\n\nIt has been a long journey, a very rewarding one. So, cheers to that.\n```\n\nWatch the side-by-side comparison video below to see how the raw disfluencies are stripped while listening:\n\n*(If the video doesn't load, you can listen to rehearsing.wav directly.)*\n\nImportant caveat:Because Smart transcription uses language modeling to clean up disfluencies and structure the output,it might slightly rewrite, omit, or rephrase parts of what was saidto make it sound natural and concise. If you are doing verbatim court reporting, medical transcription, or subtitle syncing where every exact syllable matters, stick with`verbatim`\n\nmode!Also note that Smart mode is\n\nincompatible with word-level timestamps and speaker diarization(which require`{\"type\": \"verbatim\"}`\n\n).\n\nNeed to know who spoke during a multi-person meeting or podcast? Enable **diarization** with `diarization_mode=\"speaker\"`\n\n:\n\n```\ninteraction = client.interactions.create(\n    model=\"gemini-3.5-transcribe\",\n    input=[{\"type\": \"audio\", \"uri\": meeting_audio.uri}],\n    generation_config={\n        \"transcription_config\": {\n            \"mode\": {\n                \"type\": \"verbatim\",\n                \"diarization_mode\": \"speaker\",\n            },\n        }\n    },\n)\n```\n\nTo extract each speaker turn cleanly, iterate through the step annotations:\n\n``` python\ndef print_diarized_transcript(interaction):\n  words = []\n  for step in getattr(interaction, \"steps\", []) or []:\n    for content in getattr(step, \"content\", []) or []:\n      for annotation in getattr(content, \"annotations\", []) or []:\n        if getattr(annotation, \"type\", None) == \"word_info\":\n          words.append(annotation)\n\n  current_speaker = None\n  current_turn = []\n\n  for w in words:\n    speaker = getattr(w, \"speaker\", \"spk:0\")\n    if speaker != current_speaker:\n      if current_turn:\n        print(f\"[{current_speaker}]: {' '.join(current_turn)}\")\n      current_speaker = speaker\n      current_turn = [w.text]\n    else:\n      current_turn.append(w.text)\n\n  if current_turn:\n    print(f\"[{current_speaker}]: {' '.join(current_turn)}\")\n\nprint_diarized_transcript(interaction)\n```\n\nOutput:\n\n```\n[spk:0]: One chocolatine, please.\n[spk:1]: Tiago, arrête. It is a pain au chocolat.\n[spk:0]: Wait, a guy from the south west told me it's chocolatine.\n[spk:1]: Do not listen to them. 90% of France and the entire universe calls it pain au chocolat. Chocolatine is a myth.\n[spk:0]: Meu Deus, you French are intense. In Brazil, people fight the exact same way over bolacha versus biscoito.\n[spk:1]: Well, here pain au chocolat is the only real word.\n[spk:0]: Fine. Two pain au chocolat, please. As long as it has chocolate, tá valendo.\n```\n\nWatch the demo video below where two colleagues debate *pain au chocolat* vs. *chocolatine*. **Notice how the waveform line dynamically changes color (Cyan for Tiago, Orange for his colleague) as each speaker takes turns:**\n\n*(Direct audio link: listen to pain_au_chocolat.wav)*\n\nWhen you need exact synchronization—for example, to jump to specific points in a video, build interactive transcripts, or align text with waveforms—you can request word-level millisecond start and end offsets.\n\nConfigure `timestamp_granularities=[\"word\"]`\n\n(and optionally combine it with `diarization_mode=\"speaker\"`\n\n):\n\n```\ninteraction = client.interactions.create(\n    model=\"gemini-3.5-transcribe\",\n    input=[{\"type\": \"audio\", \"uri\": audio_file.uri}],\n    generation_config={\n        \"transcription_config\": {\n            \"mode\": {\n                \"type\": \"verbatim\",\n                \"timestamp_granularities\": [\"word\"],\n                \"diarization_mode\": \"speaker\",\n            },\n        }\n    },\n)\n```\n\nEach recognized word comes back with its exact time offsets (and speaker turn) attached in the content annotations:\n\n```\nwords = []\nfor step in getattr(interaction, \"steps\", []) or []:\n  for content in getattr(step, \"content\", []) or []:\n    for annotation in getattr(content, \"annotations\", []) or []:\n      if getattr(annotation, \"type\", None) == \"word_info\":\n        words.append(annotation)\n\nfor w in words[:6]:\n  spk = getattr(w, \"speaker\", \"spk:0\")\n  print(f\"[{w.start_offset:>7} -> {w.end_offset:>7}] ({spk}) {w.text}\")\n```\n\nOutput:\n\n``` php\n[ 0.000s ->  0.400s] (spk:0) One\n[ 0.400s ->  1.200s] (spk:0) chocolatine,\n[ 1.200s ->  1.800s] (spk:0) please.\n[ 3.200s ->  3.700s] (spk:1) Tiago,\n[ 3.700s ->  4.200s] (spk:1) arrête.\n[ 4.200s ->  4.500s] (spk:1) It\n```\n\nHaving millisecond-level offsets for every individual word unlocks huge capabilities:\n\n`.srt`\n\n/ `.ass`\n\n)💡\n\nBehind the scenes:That's actually what I did to make the demo videos above! The word timestamps provided the exact millisecond timing to align the subtitle cards, highlight the custom terms (\"oatmilk\"), and trigger the color switch of the waveform line from Cyan to Orange when the speaker changed.If you want the complete Python function to convert these word annotations into standard\n\n`.srt`\n\nsubtitle files, you can find it directly in the[interactive Cookbook Colab notebook].\n\nHere is a quick cheat sheet to pick the right settings for your use case:\n\n| Use Case | Mode | Diarization | Timestamps | Custom Vocab |\n|---|---|---|---|---|\nMeeting Notes / Voice Memos |\n`smart` |\nNo | No | Optional |\nVideo Subtitles / Closed Captions |\n`verbatim` |\nOptional | `[\"word\"]` |\nHighly recommended |\nPodcast / Multi-speaker Interview |\n`verbatim` |\n`speaker` |\n`[\"word\"]` |\nHighly recommended |\nLegal / Compliance Audio Logs |\n`verbatim` |\n`speaker` |\n`[\"word\"]` |\nOptional |\nSearch Indexing & Embeddings |\n`smart` |\nNo | No | Optional |\n\nEverything we covered above is for **pre-recorded audio files** (unary mode via the Files API).\n\nGemini also supports **real-time live streaming transcription** over WebSockets using `gemini-3.5-transcribe-live`\n\nand the Live API. It lets you stream raw 16-bit PCM chunks (100ms each) directly from a microphone and receive instantaneous interim partial hypotheses (`interim_input_transcription`\n\n) and finalized text as speech occurs.\n\nHowever, streaming real-time WebSockets with asynchronous Python workers (`asyncio`\n\n), handling audio chunking, and managing ephemeral valet tokens for secure client apps is quite a bit more complex and deserves its own dedicated tutorial.\n\nIf you want to dive straight into live streaming code right now:\n\nGemini 3.5 Transcribe gives you the best of both worlds: strict, millisecond-accurate verbatim data when you need timestamps and diarization, and an intelligent, disfluency-stripping smart mode when you want clean text for human eyes.\n\nHave you tried using `smart`\n\nmode on your own voice recordings or meetings? Drop your thoughts and edge cases in the comments below! 🚀", "url": "https://wpnews.pro/news/stop-wrestling-with-asr-the-complete-guide-to-gemini-3-5-transcribe", "canonical_source": "https://dev.to/googleai/stop-wrestling-with-asr-the-complete-guide-to-gemini-35-transcribe-1m6i", "published_at": "2026-08-28 13:34:29+00:00", "updated_at": "2026-08-28 13:50:48.926103+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-products", "developer-tools"], "entities": ["Google", "Gemini 3.5 Transcribe", "Google GenAI SDK", "Google AI Studio"], "alternates": {"html": "https://wpnews.pro/news/stop-wrestling-with-asr-the-complete-guide-to-gemini-3-5-transcribe", "markdown": "https://wpnews.pro/news/stop-wrestling-with-asr-the-complete-guide-to-gemini-3-5-transcribe.md", "text": "https://wpnews.pro/news/stop-wrestling-with-asr-the-complete-guide-to-gemini-3-5-transcribe.txt", "jsonld": "https://wpnews.pro/news/stop-wrestling-with-asr-the-complete-guide-to-gemini-3-5-transcribe.jsonld"}}