Previously: My AI Agent Reads Handwritten Timesheets. It Gets the Receipts 100% Right and the Names 14% Right., how my Odoo bot reads scanned paperwork and flags what it can’t read.
This is a bonus post about what my agents have been working on lately, which is this newsletter itself. Every post here is now also a podcast episode, the episodes are turning into videos, and the voice reading them is turning into mine.
Step 1: One post, one episode #
The rule was simple: each post becomes one episode, with a teaser for the next one. Episodes 1–8 are live on the Substack podcast feed. Only Ep 1 went out by email. The rest are on the web, so your inbox doesn’t get every post twice. I also added the show to Spotify.
Those eight episodes are read by a stock open-source voice: Kokoro’s “Fenrir.” It’s clear and it’s free, but it isn’t me.
Step 2: Videos #
Next came video versions. Each one has:
- The show art and an animated waveform.
- Burned-in captions.
- An illustration for each section of the post.
- A key-points sidebar, so you can skim while you listen.
Step 3: The voice #
Round 1: “This voice is worse.” The agents tried Chatterbox, another open-source voice model. I listened and said exactly that. Back to Fenrir.
Round 2: A custom Kokoro voice. Instead of swapping models, the agents mixed Kokoro voices: mostly Fenrir, with a little Eric and a little Adam. They also built a pronunciation lexicon of 53 words and 4 phrases. It spells out initialisms, reads clock times properly, and finally says “La Pierre” right.
Round 3: My actual voice. I asked whether I could just read a post and send the recording. So I read post 1 into my phone. The recording was 13 minutes 32 seconds with a strong 120 Hz hum. After cleanup, about 10.6 minutes were usable.
From that, the agents built five candidates:
- A: a Chatterbox Turbo clone, from a short sample of my voice.
- B: the original Chatterbox model, cloned the same way.
- C: a Kokoro voice nudged step by step toward mine.
- D: Fenrir’s reading, converted to sound like me.
- D+: the custom Kokoro voice and lexicon, then converted to my voice.
My verdict: “A and B sound good, but D pronounces better.” The A versions sounded like me but sometimes misread words. The D versions read perfectly but sounded less like me.
A+: Record it, check it, retake it #
So the agents fixed A’s weak spot. A+ works the way a careful human narrator would:
- Read the script one sentence at a time in my cloned voice, from a 15-second sample.
- Transcribe each take with a speech-to-text model and compare it to the script.
- If the words don’t match, retake it with a new random seed, up to 4 takes.
- Hard words get a respelling list, so the model sees how to say them.
My favorite failure: the agents tried respelling “Kokoro” as “Ko-ko-ro.” The clone read it as “Coco Rowe.” That respelling got dropped.
I picked A+. Episode 2 is being re-voiced with it now, and future episodes use it too. The intros now say: “I write this show through my AI agents, and an AI version of my voice reads it to you.”
The limits were real #
- No GPU. Everything runs on an 8-core box with 15 GB of memory. Training a voice from scratch wasn’t an option, so the agents cloned and blended instead.
- Out-of-memory kills. Running two models at once crashed jobs. The fix was one model at a time, capped threads, and checkpoints so a job resumes where it stopped.
- Time. A+ takes roughly an hour per episode. D+ took about 20 minutes. I chose quality.
- Licenses. Only MIT- and Apache-licensed models were used. Chatterbox adds an inaudible watermark, so the audio can be identified as AI-generated. I’m fine with that, because I’m telling you anyway.
- Live episodes stay as they are. Substack can’t swap the audio on a published podcast post, so episodes 2–8 keep Fenrir. My voice shows up in the videos and new episodes.
What I learned #
- “Sounds like me” and “reads correctly” are two different scores. No single candidate won both until the agents added a check.
- Check every sentence. The retake loop came from my own one-line review. A transcription check turned “sometimes misreads” into something the agents can catch on their own.
- Say it’s AI. The intro tells you up front, and the watermark backs it up.
— Blake (written through my AI agents) Previously: My AI Agent Reads Handwritten Timesheets. It Gets the Receipts 100% Right and the Names 14% Right., how my Odoo bot reads scanned paperwork and flags what it can’t read.