Welcome back to This Last week at Romi. We've been really busy so this post hits a bit late than our scheduled Friday one!
In Voice AI, saving 200–300 milliseconds sounds tiny on paper. In practice, it’s the difference between an awkward robotic delay and a natural conversation.
This week, we took a hard look at our pipeline to eliminate latency and clean out old code that was getting in the way. Here is what we changed:
Sync to Async Tools: Originally, tool execution was synchronous. When a user created a task, the pipeline froze waiting for database writes. Making our tools asynchronous means the voice keeps flowing naturally while items are saved in the background. #
7 Dedup Layers Down to 2: To stop duplicate tasks from hitting the database, we kept piling on safeguards over time until we had 7 layers of deduplication. They slowed down the pipeline and began contradicting each other. With stronger prompt reinforcement and a quick backend cleanup, we cut that down to just 2 layers. #
Ditching "Thinking" Models: Since our agent relies heavily on fast tool calls, thinking models added unnecessary reasoning delay before speaking. We swapped to a model optimized strictly for speed and accurate tool invocation. #
Push-to-Talk Toggle: To complement these backend speedups, we added a toggle betweenNatural Flow andPush-to-Talk . Now users can think out loud or speak in noisy rooms without the model prematurely triggering unwanted tasks.
Sometimes performance wins aren't about writing complex new code—they're about deleting redundant safeguards, running tasks in the background, and picking the right model for the job.
What's the worst bit of legacy "safety code" you've had to clean out of your stack?
By Vlad - CTO@RomiADHD