Backend Error Tracking Evidence for Silent Cron Jobs and AI Tutoring Workers

An engineer running a small edtech backend outlined an approach for tracking silent failures in scheduled lesson-planning jobs and AI tutoring agent loops by combining structured exception events with explicit run-completion signals tied to a shared runId. The design records queued, started and completed events per run, attributes model usage and latency at each agent step, and treats a missed completion deadline as the primary alert, with captured exceptions serving as supporting evidence about cause. The engineer argued this gives a solo operator enough context to tell whether a slow or expensive run is also broken, without maintaining an elaborate telemetry pipeline.

A small edtech backend needs one failure record that connects each scheduled lesson-planning run, background worker step, and web request to the AI agent loop's latency and cost. The least complex workable choice is exception capture plus explicit run completion signals, joined by a stable run ID. Exception capture alone cannot prove that expected work happened. A heartbeat alone cannot explain which agent step failed. TL;DR: choose the approach that preserves both negative evidence an exception occurred and positive evidence the expected run started and finished . Attribute model usage at each agent step, then roll it up by run, course, and tenant. That gives a solo operator enough context to search, resolve, and decide whether a slow or expensive run is also broken. | Approach | Thrown errors | Missing or stuck work | Agent-loop cost attribution | Operational load | |---|---|---|---|---| | Exception capture only | Strong | Weak | Weak unless enriched | Low | | Completion checks only | Weak | Strong for scheduled work | Weak | Low | | Structured events plus completion checks | Strong | Strong | Strong with step-level usage | Moderate | | Metrics and logs assembled during an incident | Variable | Variable | Possible, but slow to reconstruct | High | The recommendation is structured exception events plus completion checks. Keep the event schema small. A one-person SaaS should spend its engineering hours on the tutoring workflow, not on maintaining an elaborate telemetry pipeline. A web API can return a successful response while enqueueing work that never completes. A cron process can exit cleanly after selecting zero rows because its filter is wrong. A worker can catch an exception, acknowledge the message, and quietly discard the only useful stack trace. These are different failures, even if a user sees the same result: no generated lesson plan. Start with the expected state transition. For one planning run, define queued , started , and completed events. Record an exception event if execution fails. The absence of completed after the workload's deadline is itself a signal; no thrown exception is required. This is why a generic process-alive check is insufficient. It says the worker can answer, not that a particular lesson-planning run crossed its boundary. Google SRE's monitoring guidance separates symptoms from causes and frames latency, traffic, errors, and saturation as core signals. For this workload, a run deadline is the symptom-facing error signal, while the captured exception is evidence about cause. Keep both. Consider one concrete run. The API accepts a teacher's request, assigns runId plan 42 , and queues the planner. The first worker spends 800 ms classifying the assignment, then a second step retrieves curriculum context. A retry starts after the worker lease expires. Nothing throws at the web boundary, and a process health check stays green throughout. Without run events, the operator sees a successful request and two unrelated model-usage entries. With the shared ID, the timeline shows two attempts, no terminal completion, and usage attached to the unfinished run. The alert should come from the missed completion deadline. The exception, if one exists, should remain supporting evidence. This distinction matters for an agent loop because a silent retry can add latency and cost before anyone sees a stack trace. One ID. Two kinds of proof. An AI agent loop may call a model several times: classify the assignment, retrieve curriculum context, draft a response, and verify it. A total attached only to the HTTP request hides retries and background continuation. A total attached only to a worker hides the originating tenant and course. Use one runId across the request, queue message, worker, and scheduled reconciliation job. Give each model call a step and record its duration plus the usage values returned by the model provider. Do not estimate usage from string length when authoritative usage is available. Also keep the model identifier actually returned or selected for that call; attribution without that dimension makes later comparisons ambiguous. The useful unit is an agent step. The useful rollup is a run. This split lets an operator answer two separate questions: "Which step made this run slow?" and "Which incomplete runs still incurred model usage?" A revenue-per-hour lens changes the implementation threshold. I would accept a plain append-only event sink and a daily aggregate before building a bespoke analytics layer. Ship weekly. Outsource undifferentiated storage and alert delivery when the contract allows data export and stable identifiers. The differentiating work is the tutoring loop and the rules that decide when a generated answer is safe to show. The instrumentation boundary should be boring. It should also survive a queue hop. The following example emits structured events through a generic sink; the same envelope can go to a log appender, an event stream, or a database table. type RunEvent = { runId: string; tenantId: string; courseId: string; source: "api" | "cron" | "worker"; stage: "queued" | "started" | "model call" | "completed" | "failed"; step?: string; durationMs?: number; inputTokens?: number; outputTokens?: number; model?: string; errorType?: string; occurredAt: string; }; interface EventSink { write event: RunEvent : Promise<void ; } async function runPlanningStep sink: EventSink, context: Pick<RunEvent, "runId" | "tenantId" | "courseId" , execute: = Promise<{ model: string; inputTokens: number; outputTokens: number; } , : Promise<void { const startedAt = Date.now ; try { const usage = await execute ; await sink.write { ...context, source: "worker", stage: "model call", step: "draft lesson plan", durationMs: Date.now - startedAt, ...usage, occurredAt: new Date .toISOString , } ; } catch error { await sink.write { ...context, source: "worker", stage: "failed", step: "draft lesson plan", durationMs: Date.now - startedAt, errorType: error instanceof Error ? error.name : "UnknownError", occurredAt: new Date .toISOString , } ; throw error; } } Do not put prompts, student text, access tokens, or raw model responses into this envelope. Identifiers should support operational correlation without turning the error index into a second content store. Apply retention and access controls according to the data carried by the identifiers. There is a specific trap here: if sink.write fails inside the catch branch, the telemetry failure can replace the original error. Production code should preserve the original exception and treat sink delivery according to an explicit policy, such as bounded buffering with a separate delivery-failure counter. The right policy depends on whether losing an event is worse than delaying the job. First, test boundary coverage. Exercise four cases before adopting a setup: an API handler throws; a queued worker throws; a scheduled run never starts; and a run starts but never completes. Search for the runId in every case. Confirm that the original error type remains visible and that the missing completion becomes actionable after the declared deadline. This is more revealing than comparing feature lists. Second, test join quality. Pick 1 synthetic planning run with 2 model steps and 1 retry. Verify that step durations and provider-reported usage roll up exactly once. Retries need distinct attempt identifiers or idempotent event keys; otherwise the cost view may double count after redelivery. The same discipline applies to completion events. Deployment deserves a dull rollout. Add events in shadow mode, inspect cardinality and sensitive fields, then enable alerts for missed completion. Alert on user-visible or deadline-visible symptoms. Route the diagnostic detail into the linked run record. This keeps a single worker exception from becoming three pages while still preserving evidence for resolution. Completion checks alone are reasonable for a tiny, deterministic cron job whose failure mode is simply "did not finish by the deadline" and whose local logs are already retained and searchable. They have a low integration cost and answer the scheduling question directly. They are a poor fit once the job fans out, retries, or spends money across several agent steps. The combined design has a real limitation: more event types create more schema governance, storage, and cardinality work. If a job has one step, no queue, no retry, and no per-run model usage, choose completion checks instead. Extra correlation fields would add maintenance without improving the operational decision. That trade-off is acceptable only when the join answers a question someone will act on. Exception capture alone can also be enough for a synchronous API where errors propagate consistently and there is no deferred work. The boundary moves as soon as the endpoint accepts a request before the lesson plan exists. At that point, success means more than an HTTP response. For an AI tutoring pipeline, the combined design earns its extra schema work because it exposes costly incomplete runs. That is the decision rule: add the second signal when absence, delay, or retry can consume resources without throwing an exception at the caller. Resolve against a run timeline, not a pile of unrelated stack traces.