Anthropic Batch API: 112 of My 1,842 Jobs Never Came Back

A developer running a mock voice interview platform found that 112 of 1,842 scoring jobs pushed through the Anthropic Batch API over 30 days silently produced no usable result — a 6.1% failure rate. The failures stemmed from treating batch.processing_status == "ended" as proof that every request succeeded, ignoring per-request result types (71 errored, 24 expired) and 17 responses truncated by max_tokens, plus matching results by position instead of custom_id. The developer's fix is to check batch.request_counts and each result line's result.type, and to key results by an opaque custom_id.

A user emailed me a screenshot of an empty report. Score field blank. I opened the database and found the row sitting there with score = NULL , created four hours earlier, no error logged anywhere. My worker had run, my batch had finished, my code had written nothing and moved on cheerfully. That was the third one that week. So I pulled 30 days of logs and counted: I had pushed 1,842 scoring jobs through the Anthropic Batch API, and 112 of them never produced a usable result. Not 112 crashes. 112 silent holes, because "the batch ended" and "my requests succeeded" are two completely different facts and I had written code that assumed they were the same one. batch.processing status == "ended" does request counts and switch on each result line's result.type succeeded / errored / expired / canceled . stop reason: "max tokens" is a truncated JSON body. My parser swallowed it and wrote a null. It is asynchronous bulk inference at roughly half price. You hand Anthropic a list of up to 100,000 requests, each wrapped in a custom id , and poll until the batch ends. The tradeoff is the entire point: you give up latency guarantees and you get a discount. There is a 24-hour ceiling on processing. My workload fit on paper. I run a platform that does mock voice interviews and hands back a written report afterward, plus a separate scoring pass over the candidate's portfolio. The portfolio pass is nightly, nobody is watching, and it is a fat prompt. Perfect batch job. The interview report is not a batch job, and I put it in one anyway. That was the first mistake and I'll get to it. batch = client.messages.batches.create requests= { "custom id": session uuid, not the user's email "params": { "model": "claude-sonnet-5", "max tokens": 4096, "messages": {"role": "user", "content": prompt} , }, } for session uuid, prompt in pending One small thing worth doing on day one: custom id comes back to you in the results file, so make it an opaque internal ID. I used the session UUID. Do not put an email address there because it was convenient. Because I only checked one flag. Here is the real breakdown from the 30-day run: | result.type | count | |---|---| | succeeded | 1,747 | | errored | 71 | | expired | 24 | | canceled | 0 | Of the 71 errored: 52 overloaded error , 14 invalid request error transcripts that blew past my own token budget , 5 generic api error . Then, of the 1,747 that succeeded, 17 came back with stop reason: "max tokens" — valid API responses containing half a JSON object. My json.loads threw, my except block logged at DEBUG, and the row stayed null. 71 + 24 + 17 = 112. A 6.1% silent failure rate on a pipeline I thought was green. processing status: "ended" mean my requests succeeded? No. ended means Anthropic is done working on the batch, including the parts it gave up on. The per-request outcome lives in two places: batch.request counts , and the JSONL results stream. My original loop, in full embarrassing honesty: while batch.processing status = "ended": time.sleep 30 batch = client.messages.batches.retrieve batch.id results = list client.messages.batches.results batch.id for session, entry in zip pending sessions, results : <-- both bugs live here save report session.id, entry.result.message.content 0 .text Two failures in one line. entry.result.message does not exist on an errored or expired entry, so those raise an AttributeError I was catching too broadly. And zip against pending sessions assumes the results arrive in submission order. They do not. Because batch results come back in arbitrary order and I matched by position instead of by custom id . When every request succeeds, the order is often close enough to input order that nothing looks wrong. The moment one request errors out, the list shortens by one and every entry after it shifts up a slot . Candidate A gets Candidate B's report. This is the bug that actually scared me. No exception, no alert, no 500. Just quietly wrong output rendered in a nice template. I found it by reading a report that praised a Kubernetes project the person had never mentioned. Full disclosure: the product is Preterview https://preterview.com/en , which I built and run. It does voice interviews with three interviewer styles and returns a written report, and a report is the entire deliverable, so shipping one person's feedback to another person's account is about as bad as my failure modes get. Two sessions were affected before I caught it. I emailed both people, which is a conversation I recommend avoiding by writing the loop correctly the first time. The fix is four lines and I should have written them on day one: by id = {} for entry in client.messages.batches.results batch.id : by id entry.custom id = entry never trust order for session in pending sessions: entry = by id.get session.id if entry is None: mark retry session.id, "missing from results" continue match entry.result.type: case "succeeded": msg = entry.result.message if msg.stop reason == "max tokens": mark retry session.id, "truncated" else: save report session.id, msg.content 0 .text case "errored": mark retry session.id, entry.result.error.type case "expired" | "canceled": mark retry session.id, entry.result.type Every branch writes a row. There is no path through that block where a request disappears. Median 9 minutes across my 96 batches. p90 was 51 minutes. p99 was 6 hours 20 minutes. And 24 requests, spread across two unlucky batches, sat until the 24-hour ceiling and came back expired . That distribution is the whole design constraint. The median tempts you into thinking batch is a slightly slower sync API. It is not. You have to build for the tail, because the tail is where your users live. Which is why the post-interview report is back on the synchronous API, where it always belonged. Someone just finished talking for 25 minutes; they are not going to refresh for six hours. The nightly portfolio rescoring stayed on batch, and that is the workload the discount was designed for. The piece of this I'd write first next time is not the submitter. It is the reconciler: a table of custom id, batch id, status, attempts and a cron job that sweeps it. Rules I landed on after the postmortem: custom id exists in the database as pending before the batch is created. If it never comes back, it is already visible as a gap. stop reason on success. max tokens is a failure wearing a success costume. After the reconciler shipped, the next 30 days ran 2,100-ish requests with zero unresolved rows. Still had errors. Still had a handful of expiries. But every one of them now ends up retried or dead-lettered instead of vanishing into a null column. So what happens to the jobs that never come back from the Anthropic Batch API? Nothing happens to them. That is the problem. A batch reaching processing status: "ended" only means Anthropic stopped working on it, and the results JSONL can contain errored , expired , or canceled entries alongside successes, in arbitrary order, plus succeeded entries truncated at max tokens . If your consumer zips results against the input list and reads .result.message without checking .result.type , those requests are not reported as failures — they are dropped, or worse, shifted onto the wrong record. Match on custom id , handle all four result types, verify stop reason , and reconcile submitted-versus-resolved counts per batch. In my 30-day run that was the difference between 6.1% silent data loss and a clean retry queue, at half the API cost. Written by the developer behind Preterview https://preterview.com/en , an interview prep platform.