{"slug": "five-ways-a-voice-agent-tells-you-it-booked-a-table-when-it-didn-t", "title": "Five ways a voice agent tells you it booked a table when it didn't", "summary": "A maintainer of Oathra, an open-source runtime for AI phone-call agents, documented five ways voice agents falsely report a successful restaurant booking, including a case where a clerk's later \"fully booked\" remark failed to revoke an earlier confirmation. The project shipped a fix in PR #37 that treats callee availability words such as \"full\" or \"closed\" as revoking a strictly earlier confirmation, with tests covering both directions. The maintainer argues the model should not decide whether a call succeeded; separate code must read the other party's words.", "body_md": "I maintain [Oathra](https://github.com/FORIFOR/oathra), an open-source runtime for AI agents that make\n\nphone calls. Its one opinion is that the model does not get to decide whether the call succeeded. A\n\nseparate piece of code reads the other party's words and decides.\n\nThat sounds like belt and braces until you look at what clerks actually say. Here are five replies to\n\nthe same request. Only one of them is a booking. A model asked \"did this succeed?\" says yes to at least\n\nfour, because all five *sound* like a yes.\n\nThe request, every time:\n\n```\nAI: Could I book a table for two at 7:30 pm on September 25? The name is Tanaka.\nThem: Sure, I will pencil you in for September 25 at 7:30 pm and call you back to confirm.\n```\n\nVerdict: **incomplete**, `{ date, time, partySize }` extracted, `confirmed` missing.\n\n\"Sure\" is agreement. The date, the time and the party size are all there, and they are all correct.\n\nThe only thing missing is the thing you called about. A completion check that counts filled fields\n\npasses this. A check that requires the callee to have committed does not.\n\n```\nThem: We can hold September 25 at 7:30 pm for now, but it is not confirmed yet.\n```\n\nVerdict: **incomplete**, nothing extracted at all.\n\nThis one is interesting because the clerk is being maximally clear, and a naive extractor still walks\n\naway with `time = 19:30`. The clause carrying the time is the one being negated. If you extract values\n\nper-utterance instead of per-clause, \"not confirmed yet\" and \"7:30 pm\" end up in different variables and\n\nthe negation is lost on the way.\n\n```\nThem: The only thing left that evening is 9 pm. Would that do?\n```\n\nVerdict: **incomplete**, nothing extracted.\n\nA number was said. It was not offered as your booking; it was offered as a question. The trap here is\n\nthat the agent's *next* turn is usually \"9 pm works, thank you\" — and if your extractor took `21:00`\n\nfrom the clerk's turn and your agent then says something agreeable, you have a fully populated booking\n\nthat nobody ever agreed to. The clerk's turn has to stay a proposal until the caller accepts it and the\n\nclerk acknowledges the acceptance.\n\n```\nThem: Yes, we have you down for two at 7:30 pm on September 25.\nThem: Sorry, that day is fully booked after all.\n```\n\nVerdict: **incomplete**, `{ date, time, partySize }` extracted, `confirmed` gone.\n\nThis is the one I got wrong. Until yesterday, Oathra reported this as **completed**.\n\nMy retraction pattern required the refusal to name the booking — \"we cannot take the reservation\",\n\n\"the booking is cancelled\". A clerk who simply says the slot is gone does not phrase it that way. They\n\nsay the table is taken, the day is private-hire, they are closed that day. The booking is equally dead\n\nand my code called it a success.\n\nThe fix is narrow on purpose: an *availability* word from the callee (full, private hire, closed,\n\n\"fully booked\") revokes a confirmation spoken strictly earlier. Not any refusal — \"we can't take cards\"\n\nafter a booking is a payment remark, not a cancellation. And only *earlier*, so the ordinary \"7 pm is\n\nfull but 7:30 is free\" that happens **before** a booking is untouched.\n\nIt went out as [PR #37](https://github.com/FORIFOR/oathra/pull/37) with tests in both directions.\n\n```\nThem: You are all set for September 25 at 7:30 pm, party of two.\n```\n\nVerdict: **completed**, `{ date: 2026-09-25, time: 19:30, partySize: 2, confirmed: true }`.\n\nThis one also failed until today, in a way I find more embarrassing than #4: it returned **nothing at all**. Not \"unconfirmed\" — empty.\n\nThe engine had two separate ideas, \"the callee agreed to a value the caller proposed\" and \"the callee\n\nconfirmed the booking\", and \"you are all set\" was in the second list but not the first. So nothing the\n\ncaller had proposed was ever verified; and because a confirmation is bound to the values that were\n\nsettled when it was spoken, a confirmation with no settled values behind it is stale and gets dropped.\n\nTwo lists that should have overlapped, and the result is a blank screen for a perfectly normal sentence.\n\nI only found it because I pasted an English log into my own public checker while writing a comment on\n\nsomeone else's thread. Three of five natural English confirmations worked. The Japanese side, which I\n\nuse daily, was fine. The lesson is not about regexes; it is that the language you don't test in is the\n\nlanguage that's broken.\n\nEvery release runs 10,000 seeded adversarial dialogues — the five shapes above plus voicemail,\n\ntransfers, hold-then-reply, dialect confirmations, wrong restatements — against a hard gate:\n\n```\nFalse Completion: 0 / 10000 adversarial runs\n```\n\nA false completion is when the runtime reports \"completed\" and the callee's own ground truth says they\n\nnever committed. Zero is the only passing number. It does not prove the checker is right about\n\neverything; it proves the specific ways I know a call can lie are all covered, and it fails loudly the\n\nmoment a change reopens one.\n\nThe inverse error — reporting \"incomplete\" when the table really was booked — is allowed to happen. It\n\nis a phone call you make again. The other direction is a customer standing outside a restaurant.\n\nIf you run a voice agent, the fastest version of this is to paste one of your own transcripts into the\n\nchecker. No install, no key, nothing leaves the tab:\n\n[https://forifor.github.io/oathra/en/check.html](https://forifor.github.io/oathra/en/check.html)\n\nOne turn per line, with a speaker:\n\n```\nAI: Could I book a table for two at 7:30 pm on September 25? The name is Tanaka.\nThem: Sure, I will pencil you in and call you back to confirm.\n```\n\nIf it reads your clerk's yes as a no, that is a bug I want — the supported phrasings are a list, and\n\nlists are always short somewhere. The repo is Apache-2.0:\n\n[FORIFOR/oathra](https://github.com/FORIFOR/oathra).", "url": "https://wpnews.pro/news/five-ways-a-voice-agent-tells-you-it-booked-a-table-when-it-didn-t", "canonical_source": "https://dev.to/forifor/five-ways-a-voice-agent-tells-you-it-booked-a-table-when-it-didnt-1lpj", "published_at": "2026-09-22 03:24:48+00:00", "updated_at": "2026-09-22 03:52:59.982417+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "natural-language-processing", "ai-safety", "developer-tools"], "entities": ["Oathra", "GitHub", "PR #37"], "alternates": {"html": "https://wpnews.pro/news/five-ways-a-voice-agent-tells-you-it-booked-a-table-when-it-didn-t", "markdown": "https://wpnews.pro/news/five-ways-a-voice-agent-tells-you-it-booked-a-table-when-it-didn-t.md", "text": "https://wpnews.pro/news/five-ways-a-voice-agent-tells-you-it-booked-a-table-when-it-didn-t.txt", "jsonld": "https://wpnews.pro/news/five-ways-a-voice-agent-tells-you-it-booked-a-table-when-it-didn-t.jsonld"}}