Your Worker Returned 500 and the Log Says `outcome: "ok"`

A developer running AI Change Watch on Cloudflare Workers discovered that the platform's telemetry API silently truncates query results, returning only about 10% of events when querying a 24-hour window in one request. By slicing the time range into four-hour chunks, they retrieved 956 events instead of nine, revealing multiple unrelated causes for 500 errors. The developer also warns that the 'outcome' field indicates runtime success, not HTTP status, so filtering on 'outcome' misses application errors.

I run AI Change Watch https://aichangewatch.com/?src=devto , a small independent project that It runs on Cloudflare Workers, which means that when someone tells me "your site 500'd an hour ago", the obvious tool is useless. wrangler tail is a live stream . It shows you what is happening now. It cannot show you an hour ago. There is a way to read the past, and there are four traps in it that cost me most of a day. Workers can write their invocation logs to a queryable store, and a REST endpoint reads it back. First the worker has to be opted in — this is the whole config: // wrangler.jsonc { "observability": { "enabled": true } } Then you can ask for events in a time range: curl -sX POST \ "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT/workers/observability/telemetry/query" \ -H "Authorization: Bearer $CF API TOKEN" \ -H "Content-Type: application/json" \ -d '{ "queryId": "anything", "timeframe": { "from": 1786000000000, "to": 1786003600000 }, "limit": 500, "view": "events", "parameters": { "datasets": "cloudflare-workers" , "filters": { "id": "f1", "key": "$workers.event.response.status", "type": "number", "operation": "eq", "value": 500 } } }' from and to are epoch milliseconds , not ISO strings. A read-scoped API token is enough — the one I already had for deploys worked unchanged. Each event carries more than you would guess: $workers.outcome ok | exceededCpu | canceled $workers.cpuTimeMs / $workers.wallTimeMs $workers.event.request.path / .search $workers.event.request.headers 'user-agent' $workers.event.request.cf.asOrganization the ASN owner $workers.event.response.status $metadata.error That is the tool. Now the traps. I started by filtering on the field that sounds right: { "key": "$workers.outcome", "operation": "eq", "value": "exceededCpu" } and found nothing, repeatedly, while the site was demonstrably returning 500s. Because for every one of them: $workers.outcome = "ok" The worker ran. It produced a response. It returned it. That the response was an error page is not the runtime's problem — the invocation succeeded. outcome describes the worker, not the HTTP result. So outcome is the wrong axis for application errors. Filter on $workers.event.response.status for what the user saw, and read $metadata.error for the throw. outcome is for failures the runtime itself noticed: CPU limit, cancellation. This is worth internalising because it inverts the usual relationship. In most stacks "the request failed" and "the handler failed" are the same event. At the edge they are two different fields — and the one with the friendlier name is the one that will not tell you. This is the one that actually cost me the day. I asked for 5xx across a 24-hour window, got nine events, and concluded I was chasing a single bug. The same 24 hours, walked in 4-hour slices and concatenated, returned 956 — across 92 URLs and three unrelated causes. I re-ran the comparison today, on 404s, to check it was not a one-off: 24h asked as one query → 26 events same 24h in 4h slices → 266 events The single query returned 10% of what was there. Not a rounding difference — a different conclusion. And nothing in the response says so: no truncation flag, no "results were sampled" field. You get a well-formed answer that happens to be mostly missing. So the loop, not the query: js const out = ; for let h = 24; h 0; h -= 4 { const from = Date.now - h 3600 000; const to = Date.now - h - 4 3600 000; const ev = await queryEvents { from, to, limit: 500 } ; if ev.length = 500 console.warn slice ${h}h hit the limit — narrow it ; out.push ...ev ; } The ev.length = 500 check matters as much as the slicing. A slice that returns exactly your limit is truncated, and you have to narrow that slice further. Without the warning you cannot tell "500 events happened" from "500 events fit". exists matches empty strings, and includes ignores case Two smaller ones, both of which produced confidently wrong numbers before I noticed. operation: "exists" matches a key that is present but empty. I wanted requests Cloudflare had { "key": "$workers.event.request.cf.verifiedBotCategory", "operation": "exists" } That field is present on every request and is "" on almost all of them, so the filter matched the entire dataset and I briefly believed the whole site was bot traffic. Use exists only for keys genuinely absent on what you are excluding — sec-fetch-mode is a real example, since non-browsers do not send it. operation: "includes" is case-insensitive. Filtering user agents for bot and for Bot returnedThe events view and the dashboard's invocation count disagree, and both are right. On one day my web worker showed 24,085 telemetry events against 8,772 invocations — roughly 2.7 events per invocation. So: workersInvocationsAdaptive in the GraphQL analytics API, not from counting eventsMixing them gives a number that is wrong by a factor you cannot see. // Past-tense debugging: "what 500'd between 3am and 4am". async function queryEvents { from, to, limit = 500, filters } { const r = await fetch https://api.cloudflare.com/client/v4/accounts/${ACCOUNT}/workers/observability/telemetry/query , { method: 'POST', headers: { Authorization: Bearer ${TOKEN} , 'Content-Type': 'application/json' }, body: JSON.stringify { queryId: 'q', timeframe: { from, to }, limit, view: 'events', parameters: { datasets: 'cloudflare-workers' , filters }, } } ; const d = await r.json ; return d?.result?.events?.events ?? ; } …called from the slicing loop above, with the results grouped in plain JavaScript rather than by asking the API to group them. view: "calculations" with a groupBy on a high-cardinality key returns only a few groups, quietly — the same failure mode as trap 2: a well-formed answer that is mostly missing. The retention window is limited. I have reliably queried three days back and would not build a workflow that assumes more; for anything you need to keep, pull it out and store it yourself. One more, learned the embarrassing way: cf.asOrganization is the ASN owner, not the bot. Requests wrangler tail is for watching. For asking, use the telemetry API — and remember that outcome: "ok" means the worker succeeded, not that your user did , and that a query covering a wide window will hand The tracker this came out of is at aichangewatch.com — it watches AI vendor docs for changes, and the 500s that started all this were a REST detail endpoint quietly falling through to its collection endpoint.