What Happens When an AI Agent Runs Longer Than Your HTTP Request?

A developer outlines the architectural failure that occurs when AI agents outlive the HTTP requests that trigger them, arguing that the request/response model breaks down for long-running agent workflows. The proposed fix is to treat each agent run as a first-class resource: return 202 Accepted with a run ID, execute the work asynchronously via a worker or queue, and support status polling, progress events, cancellation, and idempotency keys to prevent duplicate side effects.

A user clicks “Run agent” , your backend receives a normal HTTP request, and the agent starts doing what agents do: calling tools, reading documents, querying APIs, waiting for a human approval, retrying a flaky search, and generating a long report. Two minutes later, your load balancer returns 504 Gateway Timeout . The user sees an error. The agent, depending on where it is running, may still be alive. It may still be spending tokens. It may have already sent an email, created a ticket, or updated a record. Then the user clicks retry. Now you may have two agents doing overlapping work, with no clean way to know which one is authoritative. That is the core problem: An HTTP request is a short-lived delivery mechanism. An AI agent run is often a long-lived state machine. When the agent outlives the request, the request/response model stops being the right abstraction. You need run IDs, durable state, queues, idempotency, cancellation, progress events, and a way to resume after failure. This article is about what breaks when an AI agent runs longer than the HTTP request that started it — and how to design systems that survive that reality. 202 Accepted , and execute the work asynchronously. A typical HTTP request has a simple lifecycle: client connects client sends request server processes server responds connection closes An AI agent run has a much messier lifecycle: queued started planning waiting for model waiting for tool waiting for approval retrying streaming progress completed / failed / cancelled Those two lifecycles do not line up. HTTP assumes the server can finish quickly enough for the client to stay interested. Agent work often cannot make that promise. The agent may be waiting on: The first mistake is pretending the HTTP request is the agent. It is not. The request is only the trigger. Once you accept that, the architecture changes. Scenario: Your frontend calls POST /agent-runs and waits for the final answer. The agent takes longer than your gateway timeout. The client receives a 504 , but the backend worker continues executing. Why it matters: Now you have split-brain behavior. The user thinks the operation failed. The system may still be doing work, consuming tokens, calling APIs, and mutating state. This is especially dangerous when the agent has side effects. A timed-out read-only query is annoying. A timed-out agent that can send messages, create orders, or delete records is an incident waiting to happen. Solution: Do not run long-lived agents synchronously inside the HTTP request. Create a run record, return 202 Accepted , and execute the agent asynchronously. python import express from "express"; import crypto from "node:crypto"; const app = express ; app.use express.json ; app.post "/agent-runs", async req, res = { const idempotencyKey = req.get "Idempotency-Key" ?? crypto.randomUUID ; const run = await startAgentRunOnce { idempotencyKey, input: req.body, } ; res .status 202 .setHeader "Location", /agent-runs/${run.id} .json { runId: run.id, status: run.status, statusUrl: /agent-runs/${run.id} , eventsUrl: /agent-runs/${run.id}/events , } ; } ; The client receives a run ID immediately. The actual agent work happens in a worker, queue, or workflow engine. Why this works: The HTTP request is no longer responsible for completing the agent run. It is only responsible for starting it safely. The run becomes a first-class resource: POST /agent-runs → start GET /agent-runs/:id → status GET /agent-runs/:id/events → progress stream POST /agent-runs/:id/cancel → cancel 💡 Practical note: 202 Accepted is the honest response for work that has been received but not completed. It is not a hack. It is the correct HTTP semantics. Scenario: The user does not see a result, so they click “Run” again. Or the mobile app automatically retries after a network drop. Now two agent runs are executing for the same intent. Why it matters: Agents are not pure functions. If the agent can call tools, retrying from scratch can produce different decisions and duplicate side effects. A normal API retry problem becomes much worse with agents because the retry may: Solution: Require an idempotency key for agent creation. A simple Postgres pattern: CREATE TABLE agent runs id uuid PRIMARY KEY, idempotency key text NOT NULL UNIQUE, input jsonb NOT NULL, status text NOT NULL, created at timestamptz NOT NULL DEFAULT now ; Then insert safely: INSERT INTO agent runs id, idempotency key, input, status VALUES $1, $2, $3, 'queued' ON CONFLICT idempotency key DO NOTHING RETURNING id; If no row is returned, fetch the existing run: SELECT id, status FROM agent runs WHERE idempotency key = $1; In application code: async function startAgentRunOnce params: { idempotencyKey: string; input: unknown; } { const runId = crypto.randomUUID ; const inserted = await db.query INSERT INTO agent runs id, idempotency key, input, status VALUES $1, $2, $3, 'queued' ON CONFLICT idempotency key DO NOTHING RETURNING id, status , runId, params.idempotencyKey, JSON.stringify params.input ; if inserted.rows.length 0 { return inserted.rows 0 ; } const existing = await db.query SELECT id, status FROM agent runs WHERE idempotency key = $1 , params.idempotencyKey ; return existing.rows 0 ; } Why this works: The idempotency key becomes the client’s declaration of intent. If the same intent is submitted again, the system returns the existing run instead of creating a new one. ⚠️ Gotcha: Idempotency keys need a meaningful scope. A key like run-agent is useless. Use something tied to the user, tenant, action, and input hash, or let the client generate a UUID per logical attempt. Scenario: Your agent keeps its conversation history, current plan, and tool-call progress in process memory. Then you deploy a new version, a container scales in, or the machine restarts. The run disappears. Why it matters: Long-running agents need durable state. If the process can die before the run finishes, the state must live somewhere else. This is not just about crashes. In production, processes restart all the time: If your agent state is only in memory, you do not have a long-running agent. You have a fragile process with amnesia. Solution: Persist the agent run state in a durable store. The exact store can be Postgres, Redis, DynamoDB, or a workflow engine, but the state model should be explicit. A useful run state shape: type AgentRunStatus = | "queued" | "running" | "waiting for approval" | "completed" | "failed" | "cancelled"; interface AgentRunState { runId: string; status: AgentRunStatus; input: unknown; cursor?: string; messages: AgentMessage ; pendingToolCall?: ToolCall; artifacts: Artifact ; error?: string; updatedAt: string; } More important than the exact fields is the discipline: Why this works: The worker becomes replaceable. The run state survives independently of any one process. This also makes debugging much easier. When someone asks, “What is the agent doing right now?” you can answer from stored state instead of guessing from logs. 🚨 Production warning: If you cannot answer “What step is this agent on?” without attaching a debugger, your agent is not production-ready. Scenario: You use Server-Sent Events or WebSockets to stream agent progress. The connection drops. The user reloads the page. Now the frontend has missed events and has no reliable way to recover. Why it matters: Streaming is excellent for user experience. It is terrible as the only record of what happened. A stream is ephemeral. It tells you what is happening now, or what happened recently, but it does not by itself answer: Solution: Keep durable run state as the source of truth. Use streaming as a notification layer on top of that state. A resilient SSE endpoint should support reconnection: js app.get "/agent-runs/:id/events", async req, res = { const runId = req.params.id; res.setHeader "Content-Type", "text/event-stream" ; res.setHeader "Cache-Control", "no-cache" ; res.setHeader "Connection", "keep-alive" ; res.flushHeaders ; const lastEventId = Number req.headers "last-event-id" ?? "0" ; const missedEvents = await eventStore.eventsSince runId, lastEventId ; for const event of missedEvents { sendEvent res, event ; } const unsubscribe = eventStore.subscribe runId, event = { sendEvent res, event ; } ; const heartbeat = setInterval = { res.write ": ping\n\n" ; }, 15000 ; req.on "close", = { clearInterval heartbeat ; unsubscribe ; } ; } ; function sendEvent res: express.Response, event: { id: number; type: string; payload: unknown } { res.write id: ${event.id}\n ; res.write event: ${event.type}\n ; res.write data: ${JSON.stringify event.payload }\n\n ; } The important part is Last-Event-ID . When the client reconnects, the server can replay missed events instead of pretending the connection never dropped. Why this works: The client can lose the stream without losing the run. The UI becomes a projection of durable state, not the only place where progress exists. 💡 Practical note: Heartbeats are not optional. Many proxies and load balancers close idle connections quietly. A periodic comment line like : ping helps keep the stream alive. Scenario: Your agent calls three tools: search, create ticket, send email. The email tool succeeds, then the run crashes before the state is saved. The worker restarts and retries. Now the email may be sent again. Why it matters: Agents with tools are not just inference loops. They are distributed systems that take actions. Once an agent can affect external systems, you have the usual distributed-workflow problems: But agents make these problems harder because the sequence of actions may not be deterministic. Solution: Make tool execution idempotent wherever possible. Give each meaningful tool call a stable idempotency key derived from the run and the logical operation. async function sendInvoiceEmail run: AgentRunState, invoiceId: string { const toolCallKey = ${run.runId}:send invoice email:${invoiceId} ; const existing = await externalCallStore.find toolCallKey ; if existing { return existing.result; } const result = await emailClient.send { idempotencyKey: toolCallKey, to: run.input.customerEmail, subject: Invoice ${invoiceId} , template: "invoice", } ; await externalCallStore.record toolCallKey, result ; return result; } If the downstream API does not support idempotency keys, you still need local deduplication: js const alreadyPerformed = await sideEffectLog.exists toolCallKey ; if alreadyPerformed { return sideEffectLog.resultFor toolCallKey ; } For destructive or irreversible operations, consider requiring explicit approval or using a two-phase pattern: propose action store proposal wait for approval execute once record result Why this works: You are separating “the agent decided to do something” from “the system actually did it.” That separation gives you a place to enforce safety, retries, and auditing. 🧠 The important part: If a tool call can happen twice, the agent run is not safe to retry unless that tool call is idempotent or guarded. Scenario: The user clicks “Cancel” while the agent is running. Your API updates a database row. The agent, currently waiting on a model call or external tool, has no idea. Why it matters: Cancellation is easy as a UI concept and hard as an execution concept. If the agent does not check for cancellation, it keeps doing work. If it checks too late, it may perform side effects after the user asked it to stop. Solution: Treat cancellation as a cooperative protocol. First, expose a cancellation endpoint: js app.post "/agent-runs/:id/cancel", async req, res = { const runId = req.params.id; await agentRunStore.requestCancellation runId ; res.status 202 .json { runId, status: "cancel requested", } ; } ; Then make the worker check cancellation at safe boundaries: async function executeAgentRun runId: string, signal: AbortSignal { while signal.aborted { const run = await agentRunStore.get runId ; if run return; if run.status === "cancel requested" { await agentRunStore.markCancelled runId, { reason: "user requested", } ; return; } const nextStep = await planNextStep run ; if nextStep { await agentRunStore.markCompleted runId ; return; } await executeStep runId, nextStep, signal ; } } The critical detail is where you check. Good cancellation points: Bad cancellation strategy: Why this works: Cancellation becomes part of the run lifecycle instead of an afterthought. ⚠️ Gotcha: Cancellation does not automatically undo side effects. If the agent already sent the email, cancellation may only mean “stop doing more work.” Your system needs to know the difference. Scenario: Your agent needs approval before sending a high-risk email. The approval may come in ten seconds, ten hours, or three days. Now your agent is not merely long-running. It is suspended. Why it matters: Many agent systems are designed for “slow API calls,” not for “pause until a human responds.” These are different problems. A slow API call can be handled with timeouts and retries. A human pause requires: If you keep the agent process alive while waiting for a human, you are wasting resources. If you do not persist the pause, you lose the run. Solution: Model waiting states explicitly. await agentRunStore.update runId, { status: "waiting for approval", pendingAction: { type: "send customer email", payload: emailDraft, requestedAt: new Date .toISOString , expiresAt: new Date Date.now + 48 60 60 1000 .toISOString , }, } ; Then resume when the approval arrives: js app.post "/agent-runs/:id/approvals", async req, res = { const runId = req.params.id; const { approved, approver } = req.body; await agentRunStore.resolveApproval runId, { approved, approver, resolvedAt: new Date .toISOString , } ; await queue.enqueue "agent.resume", { runId } ; res.status 202 .json { runId, status: "resuming" } ; } ; This is where durable execution engines start to make sense. They are designed for workflows that can wait for long periods without keeping a process alive. Why this works: The agent run becomes a workflow with explicit pause and resume points. That is far more honest than pretending a human approval is just another fast function call. 💡 Practical note: If your agent can wait for humans, design expiration early. A pending action that lives forever becomes a security and compliance problem. Scenario: The agent starts with the user’s access token. Twenty minutes later, it needs to call another API. The token has expired. Why it matters: User sessions and agent lifetimes do not naturally align. A user may: Meanwhile, the agent may still be running. If you casually pass the original request token into a long-lived background worker, you create both security and reliability problems. Solution: Decide explicitly what identity the agent runs under. Common patterns: Useful when the run is brief and the token lifetime is sufficient. Risk: the token expires mid-run. Useful when the agent acts on behalf of the user for longer periods. Requirements: Useful when the agent performs system-level work. Useful when the agent needs approval for high-risk actions. Example: agent requests permission to refund payment system pauses run user re-authenticates system resumes run with fresh approval A practical authorization check before a sensitive tool call: async function authorizeToolCall run: AgentRunState, tool: ToolDefinition { const policy = await policyStore.forRun run.runId ; if policy.allows tool.name { throw new Error Tool ${tool.name} is not allowed for this run ; } if tool.requiresRecentUserApproval { const approval = await approvalStore.latest run.runId, tool.name ; if approval || approval.approvedAt < minutesAgo 5 { await agentRunStore.update run.runId, { status: "waiting for approval", pendingAction: { type: tool.name, payload: tool.input, }, } ; throw new ToolCallPaused "Recent approval required" ; } } } Why this works: You stop assuming that the original HTTP request’s auth context is valid forever. 🚨 Production warning: Do not solve token expiration by giving agents broad admin tokens “just to keep things moving.” That turns every long-running agent into a privilege-escalation risk. Not every agent needs the same architecture. The right choice depends on how long the agent can run, whether it has side effects, and whether humans are involved. | Approach | Best for | Complexity | Weakness | |---|---|---|---| | Synchronous HTTP | Very fast, read-only agent calls | Low | Breaks as soon as work is slow or unreliable | | Async run + polling | Simple background agent jobs | Medium | Polling can be inefficient; needs run store | | SSE/WebSocket progress | Interactive UX with live updates | Medium | Connection loss handling is required | | Queue workers | Scalable background execution | Medium | Retry and state discipline needed | | Durable workflow engine | Multi-step, long-lived, human-in-loop agents | High | More operational complexity | Use synchronous request/response when: POST /summarize-text Use async runs when: POST /agent-runs GET /agent-runs/:id Use SSE or WebSockets when: But keep durable state underneath. Use a durable workflow engine when: This is the category where “agent” starts looking less like a chat endpoint and more like business process automation. If I were shipping an AI agent feature that could run longer than a normal HTTP request, I would use this shape: Client ↓ API layer ↓ Run store ↓ Queue / workflow engine ↓ Agent worker ↓ Tool execution layer ↓ Event store / notifications Responsible for: Stores: The most important design decision is this: The HTTP request starts and observes the run. It does not own the run. That one mental shift prevents a large class of production pain. Before letting an agent run longer than the HTTP request that started it, I’d want these boxes checked. The deeper truth is that long-running AI agents are not just “slower APIs.” They are workflows with non-deterministic planning, external side effects, and user expectations. HTTP can trigger them. HTTP can report on them. HTTP can stream progress from them. But once the agent can outlive the request, HTTP should not be the container for the entire execution. The request is the doorway. The agent run is the process. Design them separately, and the system becomes far easier to operate. Design them as the same thing, and every timeout, retry, deploy, and disconnected browser becomes a potential corruption of the run.