Evaluating voice agent quality at scale
A production voice AI system serving residential property-management companies now scores every completed call with a separate evaluator model, internally called CSAT AI, rather than sampling calls fo…
A production voice AI system serving residential property-management companies now scores every completed call with a separate evaluator model, internally called CSAT AI, rather than sampling calls fo…
A production voice AI system for residential property-management companies found that 43% of callers were not pre-identified by phone number, and of those, the name-and-address lookup succeeded only 5…
Sending every LLM request twice and taking the faster response outperforms paid priority tiers, cutting worst-case time to complete response from 9.8s to 3.5s and worst-case time to first token from 4…
HOAi reduced tail latency in its parallel LLM-based document search by racing two GPT-5.4-mini default-tier requests per batch instead of a single GPT-4.1-mini priority request, cutting p95 search tim…
HOAi's AI agents at HOAi experienced full-text search queries taking 3.2 seconds or more because Postgres ignored the GIN index and materialized a CTE, scanning all pages for an association. The fix w…
HOAi engineer Dana found that asking the LLM agent why it made a wrong decision directly in OpenAI's playground revealed the root cause of a bug where the voice agent hung up on callers who said "than…
HOAi engineers debugged a Node.js application that intermittently failed health checks and required automatic restarts, with CPU pegged at 100% on one core and no network traffic or log activity. Afte…
A team discovered that Google's Gemini 3 Flash model enters a deterministic 'reasoning spiral' on certain inputs, consuming up to 96% of the output budget without producing usable output, causing a 37…