8 of my AI agent's 30 test calls failed. Every one was my fault.

A developer building an AI intake agent for US home-service contractors found that 8 of 30 scripted test calls failed, all due to prompt and harness design flaws rather than model errors. The failures included a prompt that pointed the model at a schema file path it could not open, an emergency guardrail that told the agent to stop collecting fields without saying when to resume, and raw JSON emitted mid-conversation that a text-to-speech layer would have read aloud to a caller reporting a possible gas leak. The developer fixed the issues by injecting the real schema into the prompt, adding a safety-critical minimum of address and callback number, and flagging the JSON-in-transcript behavior that no automated check caught.

The record said there was a gas leak. It did not say where. { "urgency": "emergency", "emergency type": "gas", "callback number": null, "service address": { "street": null, "city": null, "postal code": null }, "outcome": "escalated" } My AI intake agent wrote that after a caller said their furnace was out and the basement "kind of smells like eggs." It told them to leave the house and call 911 from outside, which was the right thing to say. Then it handed an on-call dispatcher an emergency with no address and no phone number. Nobody could be sent. Nobody could call back. The model did exactly what I told it to. That was the problem. An intake agent for home service contractors in the US: HVAC, plumbing, roofing. It answers the phone or an SMS, or a web form , works out how urgent the problem is, collects what a dispatcher needs, and writes one JSON record at the end that goes into the contractor's system. Before letting it near a real caller I wrote 30 scripted test calls, ten per trade. The caller's lines are fixed. The agent replies turn by turn, and a runner scores the result on nine criteria. Four are checked by plain code: is the record valid against the schema C1 , is the urgency right C2 , are the required fields filled C3 , is the emergency type right C4 . The other five need a second model as judge, and I haven't run those yet. Everything below is the deterministic half, run against gpt-5.5 through a local gateway. I started with a single case, hvac-01 , the gas leak above. First failure: "no intake object found." Except there was one. The agent had written a perfectly reasonable JSON object. My checker looks for a schema version field to recognise the record, and it wasn't there. It wasn't there because my prompt said this: At the end of every conversation, emit a single JSON object conforming to 01-intake-agent/schema/intake.schema.json . That's a file path. The model can't open files. It never saw the schema, so it invented its own shape from the field tables in the prompt: trade fields dumped at the top level, "gas smell" where the enum says "gas" , no outcome at all. Fixing schema version alone would have failed three more ways. The fix was to stop describing the schema and start including it. The runner now reads the schema file and injects it into the prompt, the same file the validator loads: schema text = args.schema.read text encoding="utf-8-sig" variables.setdefault "intake schema", schema text the real code wraps it in a json fence system = build prompt args.prompt, args.guardrails, args.trade module , variables validator = Draft202012Validator json.loads schema text The contract the agent reads and the contract it's graded on can't drift apart anymore. Second failure: the record at the top of this post. My emergency guardrail, G4, said: If any of these appears at any point in the conversation, stop the intake immediately. Do not finish your question. Do not collect remaining fields. It never said when to start again. So when the caller said "Okay, I'm outside," safe and still on the line, the agent asked for nothing. It obeyed. G4 now has a section called the safety-critical minimum. Once the caller confirms they're out, ask exactly two things, one at a time: the address, and a number to call back. Nothing else. If they panic or hang up, write what you have and mark it abandoned . Third failure, and this one no score caught. I was reading the transcript to debug the second problem and saw this: CALLER: Okay hang on AGENT: { "schema version": "1.0", "trade": "hvac", "urgency": "emergency", ... The caller said "hang on" and the agent answered with raw JSON. It did it three times in one call. On a voice line, the text-to-speech layer reads whatever the agent returns. So somebody standing in their yard next to a gas leak would have heard a JSON object read out loud. C1 passed anyway, because the checker reads the agent's last turn first and the last turn happened to contain exactly one object. None of my nine criteria looks for this. I only found it because I was reading a transcript for a different reason. The prompt now says the record is written once, when the call is actually over, and that "hang on" and "one sec" are not the end of a call. With hvac-01 passing I ran all thirty. Eight failed: | Case | What the checker said | What was actually wrong | |---|---|---| | plm-01, plm-10, rf-02 | missing callback number, service address | the scripted caller never says them | | hvac-03, plm-02 | missing property type | the prompt forbids guessing, and never says that "my kitchen sink" means residential | | hvac-09 | expected same day, got routine | nothing in the kit says a repeat failure is urgent | | rf-10 | expected same day, got routine | roofing triage has no line for commercial tenants | | rf-01 | expected eme rgency type null, got "other" | I never defined what emergency type means | The first row is the one that stings. Three emergency test cases demanded a phone number and an address that no line in the script ever supplied. No agent could pass them, however well it asked. I'd already found four cases like that in the HVAC file while debugging hvac-01 , fixed those, and assumed plumbing and roofing were fine. I hadn't checked. That's seven of thirty cases, almost a quarter of the suite, that were impossible to pass. I would have spent days tuning the prompt against them. The two urgency failures are a different kind of embarrassing. A customer calling because the tech was there on Tuesday and it's doing the same thing again is the call a contractor most needs to see today. A leak over a shop's storeroom costs the tenant money by the hour. I knew both of these. I had never written them down, so the agent had nothing to go on and picked the mildest reading. Both are now rules in the core prompt that outrank the trade modules. This is the thing I keep coming back to. Every failure was a place where my kit said two things that couldn't both be true, and the model picked one: call meta.started at must be a string When the agent "ignores" an instruction now, the first thing I do is look for the other sentence in my own prompt that it followed instead. So far it's always been there. Two more, and both were in the runner, not the prompt. The nudge that manufactured a failure. At the end of each case the runner sends one last message, caller disconnected , so an agent that got cut off still writes its record. It sent it every time, unconditionally. rf-05 is a web form. Name, phone, address and the problem all arrive in the first message. The agent wrote a complete, correct record straight away. Then the runner told it the caller had disconnected, and it wrote a second record with every field null. The checker reads the last turn first. The empty record won. That case had passed on two earlier runs and failed on the third. For a while it looked like randomness. It was a trap in the harness that finally got sprung. Nudge only when the agent hasn't written a record yet. already, = extract json messages -1 "content" if already is None: messages.append {"role": "user", "content": " caller disconnected "} messages.append {"role": "assistant", "content": model.reply system, messages } The quota wall that deleted everything. Partway through a roofing run my API quota ran out. The runner retried three times, then died with a Python traceback, and the six cases it had already scored were gone because it only wrote the report at the very end. If you run evals on a subscription or a free tier, this will happen to you. Now the runner recognises a quota error including a 429 that the gateway wraps as a 503 , stops cleanly, writes everything it has with "complete": false , and has a --resume flag that skips cases already scored. For the emergency type problem I added a rule: it names which G4 emergency branch fired, and it's null if none did. rf-01 , water pouring through a ceiling during a storm, started passing. On the next run rf-02 failed instead. A bedroom ceiling "bulging down, like a balloon full of water" came back as an emergency with emergency type: null . The agent had checked the G4 table, found rows for gas, electrical, flooding, carbon monoxide and injury, found nothing structural, and did what I'd just told it to. The schema had structural in its enum. G4 never did. I'd fixed one contradiction and exposed the next one. G4 has a structural row now, including "don't puncture it or try to drain it," which is what you actually want a caller to hear about a ceiling full of water. So I'm not going to tell you it's 30/30. There's one thing I still don't measure. My agent says "Leave building now" and "Stay out of building," dropping the articles, like a telegram. It's correct and it passes every check I have. It also sounds like a robot to someone who is scared. None of my deterministic checks catch it, so that's what the judge run is for.