Last weekend I got mad at voicemail. Not at any individual voicemail — at the entire concept. Someone calls, leaves a 30-second message, and then I have to remember to check it. By the time I do, the urgent ones are old news and the spam ones have wasted my attention. So I built an AI that listens for me.
This post is about the build — what I chose, what I got wrong, and the two specific decisions that turned a flaky demo into something I'm actually comfortable shipping.
The canonical code example is in the Telnyx code examples repo:
Telnyx has a public repo of around 480 code examples. The vast majority are too complex for a six-minute demo. I needed something that was small enough to teach the pattern, real enough to be useful, and visual enough to work on camera.
I picked ai-voicemail-transcription-forwarding-python
because:
I cloned the example, filled in my API key, and the AI part worked great. Then I tested the SMS delivery. It failed. Every time.
The bug: the send_sms
function was passing the caller's phone number as the from
field. The owner's number was set as to
. So the API was saying "send an SMS from this random number I don't own, to this number." Telnyx (correctly) rejects that.
This was in the upstream team-telnyx/telnyx-code-examples
repo. It's listed as "production-ready." It is not production-ready. It would never have sent a single SMS in production.
I dug through the rest of the code and found twelve issues total. The takeaway was the same: read example code skeptically, even from authoritative sources.
This one took me a while. The upstream example used client.calls.actions.record_start()
and client.calls.actions.transcription_start()
. Neither of those methods exist in the current Telnyx SDK. The actual names are start_recording()
and start_transcription()
.
Even worse: start_recording()
has a built-in transcription
parameter. You can do recording + transcription in one call instead of two. The upstream example was doing it the hard way AND using wrong method names.
Then there was the language_code
parameter on the speak()
call. The SDK uses language
, not language_code
. The speak returned 200 but then a call.speak.failed
webhook arrived with a 400 "not well-formed" error. The greeting never played.
This was the sneakiest bug. I assumed the transcription would stream in chunks via call.transcription
webhooks while the caller was speaking. That's how the upstream example was written.
In reality, the transcription arrives as a single call.recording.transcription.saved
webhook AFTER the caller hangs up, with the full text in a transcription_text
field. Different event name, different payload structure, different timing.
The fix: the app now waits for the transcription on hangup instead of processing immediately. If the transcript arrives after the session is popped, it recreates the session and processes it.
The first version used max_tokens=300
for the LLM call. It worked in my unit tests. It failed in production. The AI Inference call returned, but the JSON got truncated mid-word, like {"priority":"urgent","summary":"Sarah reports
. Why?
Because the model — moonshotai/Kimi-K2.6
— has reasoning tokens. When you ask it to think step-by-step, the reasoning comes out of the same token budget as the answer. With max_tokens=300
, my answer was getting cut off after the first ~50 tokens of actual JSON because Kimi had burned 250 tokens reasoning about whether it should return JSON.
The fix was bumping max_tokens
to 1500. Rule of thumb: when using reasoning models, budget 5-10x more tokens than the visible output needs.
The upstream example used voice="female"
which is the basic TTS tier. It sounded robotic. I upgraded to AWS.Polly.Joanna-Neural
with service_level="premium"
— a natural, human-like voice. One parameter change, dramatic quality difference.
A few things I'd change if I was starting over:
call.recording.saved
webhook, so the audio is just sitting in Telnyx's storage with no way to retrieve it from my app.The hardest part wasn't the code. It was figuring out what the AI should actually decide. "Urgent vs normal vs spam" feels obvious, but the edge cases are where the value is:
I haven't solved this. The next iteration would be a small eval set — twenty voicemails with my labels — and a prompt I tune against it.
This is the first of three apps I'm building for a YouTube series. The next one is an AI language tutor — call a phone number, practice a foreign language with an AI that adapts to your level. Same skeleton. By the third video, the audience will have seen the pattern three times and should be able to build anything on top of it.
If you want to build the voicemail yourself, the repo has the full code. The state machine itself fits in one screen — the rest is error handling, logging, and JSON parsing defenses. Should take you 30 minutes including the inevitable debugging.