{"slug": "what-happens-when-an-ai-agent-runs-longer-than-your-http-request", "title": "What Happens When an AI Agent Runs Longer Than Your HTTP Request?", "summary": "A developer outlines the architectural failure that occurs when AI agents outlive the HTTP requests that trigger them, arguing that the request/response model breaks down for long-running agent workflows. The proposed fix is to treat each agent run as a first-class resource: return 202 Accepted with a run ID, execute the work asynchronously via a worker or queue, and support status polling, progress events, cancellation, and idempotency keys to prevent duplicate side effects.", "body_md": "A user clicks **“Run agent”**, your backend receives a normal HTTP request, and the agent starts doing what agents do: calling tools, reading documents, querying APIs, waiting for a human approval, retrying a flaky search, and generating a long report.\n\nTwo minutes later, your load balancer returns `504 Gateway Timeout`.\n\nThe user sees an error.\n\nThe agent, depending on where it is running, may still be alive. It may still be spending tokens. It may have already sent an email, created a ticket, or updated a record. Then the user clicks retry. Now you may have two agents doing overlapping work, with no clean way to know which one is authoritative.\n\nThat is the core problem:\n\nAn HTTP request is a short-lived delivery mechanism.\n\nAn AI agent run is often a long-lived state machine.\n\nWhen the agent outlives the request, the request/response model stops being the right abstraction. You need run IDs, durable state, queues, idempotency, cancellation, progress events, and a way to resume after failure.\n\nThis article is about what breaks when an AI agent runs longer than the HTTP request that started it — and how to design systems that survive that reality.\n\n`202 Accepted`, and execute the work asynchronously.\nA typical HTTP request has a simple lifecycle:\n\n```\nclient connects\nclient sends request\nserver processes\nserver responds\nconnection closes\n```\n\nAn AI agent run has a much messier lifecycle:\n\n```\nqueued\nstarted\nplanning\nwaiting for model\nwaiting for tool\nwaiting for approval\nretrying\nstreaming progress\ncompleted / failed / cancelled\n```\n\nThose two lifecycles do not line up.\n\nHTTP assumes the server can finish quickly enough for the client to stay interested. Agent work often cannot make that promise. The agent may be waiting on:\n\nThe first mistake is pretending the HTTP request is the agent. It is not. The request is only the trigger.\n\nOnce you accept that, the architecture changes.\n\n**Scenario:**\n\nYour frontend calls `POST /agent-runs` and waits for the final answer. The agent takes longer than your gateway timeout. The client receives a `504`, but the backend worker continues executing.\n\n**Why it matters:**\n\nNow you have split-brain behavior. The user thinks the operation failed. The system may still be doing work, consuming tokens, calling APIs, and mutating state.\n\nThis is especially dangerous when the agent has side effects. A timed-out read-only query is annoying. A timed-out agent that can send messages, create orders, or delete records is an incident waiting to happen.\n\n**Solution:**\n\nDo not run long-lived agents synchronously inside the HTTP request. Create a run record, return `202 Accepted`, and execute the agent asynchronously.\n\n``` python\nimport express from \"express\";\nimport crypto from \"node:crypto\";\n\nconst app = express();\napp.use(express.json());\n\napp.post(\"/agent-runs\", async (req, res) => {\n  const idempotencyKey = req.get(\"Idempotency-Key\") ?? crypto.randomUUID();\n\n  const run = await startAgentRunOnce({\n    idempotencyKey,\n    input: req.body,\n  });\n\n  res\n    .status(202)\n    .setHeader(\"Location\", `/agent-runs/${run.id}`)\n    .json({\n      runId: run.id,\n      status: run.status,\n      statusUrl: `/agent-runs/${run.id}`,\n      eventsUrl: `/agent-runs/${run.id}/events`,\n    });\n});\n```\n\nThe client receives a run ID immediately. The actual agent work happens in a worker, queue, or workflow engine.\n\n**Why this works:**\n\nThe HTTP request is no longer responsible for completing the agent run. It is only responsible for starting it safely.\n\nThe run becomes a first-class resource:\n\n```\nPOST /agent-runs        → start\nGET  /agent-runs/:id    → status\nGET  /agent-runs/:id/events → progress stream\nPOST /agent-runs/:id/cancel → cancel\n```\n\n💡 Practical note:\n\n`202 Accepted` is the honest response for work that has been received but not completed. It is not a hack. It is the correct HTTP semantics.\n\n**Scenario:**\n\nThe user does not see a result, so they click **“Run”** again. Or the mobile app automatically retries after a network drop. Now two agent runs are executing for the same intent.\n\n**Why it matters:**\n\nAgents are not pure functions. If the agent can call tools, retrying from scratch can produce different decisions and duplicate side effects.\n\nA normal API retry problem becomes much worse with agents because the retry may:\n\n**Solution:**\n\nRequire an idempotency key for agent creation.\n\nA simple Postgres pattern:\n\n```\nCREATE TABLE agent_runs (\n  id uuid PRIMARY KEY,\n  idempotency_key text NOT NULL UNIQUE,\n  input jsonb NOT NULL,\n  status text NOT NULL,\n  created_at timestamptz NOT NULL DEFAULT now()\n);\n```\n\nThen insert safely:\n\n```\nINSERT INTO agent_runs (id, idempotency_key, input, status)\nVALUES ($1, $2, $3, 'queued')\nON CONFLICT (idempotency_key) DO NOTHING\nRETURNING id;\n```\n\nIf no row is returned, fetch the existing run:\n\n```\nSELECT id, status\nFROM agent_runs\nWHERE idempotency_key = $1;\n```\n\nIn application code:\n\n```\nasync function startAgentRunOnce(params: {\n  idempotencyKey: string;\n  input: unknown;\n}) {\n  const runId = crypto.randomUUID();\n\n  const inserted = await db.query(\n    `\n      INSERT INTO agent_runs (id, idempotency_key, input, status)\n      VALUES ($1, $2, $3, 'queued')\n      ON CONFLICT (idempotency_key) DO NOTHING\n      RETURNING id, status\n    `,\n    [runId, params.idempotencyKey, JSON.stringify(params.input)]\n  );\n\n  if (inserted.rows.length > 0) {\n    return inserted.rows[0];\n  }\n\n  const existing = await db.query(\n    `\n      SELECT id, status\n      FROM agent_runs\n      WHERE idempotency_key = $1\n    `,\n    [params.idempotencyKey]\n  );\n\n  return existing.rows[0];\n}\n```\n\n**Why this works:**\n\nThe idempotency key becomes the client’s declaration of intent. If the same intent is submitted again, the system returns the existing run instead of creating a new one.\n\n⚠️ Gotcha:\n\nIdempotency keys need a meaningful scope. A key like `run-agent` is useless. Use something tied to the user, tenant, action, and input hash, or let the client generate a UUID per logical attempt.\n\n**Scenario:**\n\nYour agent keeps its conversation history, current plan, and tool-call progress in process memory. Then you deploy a new version, a container scales in, or the machine restarts.\n\nThe run disappears.\n\n**Why it matters:**\n\nLong-running agents need durable state. If the process can die before the run finishes, the state must live somewhere else.\n\nThis is not just about crashes. In production, processes restart all the time:\n\nIf your agent state is only in memory, you do not have a long-running agent. You have a fragile process with amnesia.\n\n**Solution:**\n\nPersist the agent run state in a durable store. The exact store can be Postgres, Redis, DynamoDB, or a workflow engine, but the state model should be explicit.\n\nA useful run state shape:\n\n```\ntype AgentRunStatus =\n  | \"queued\"\n  | \"running\"\n  | \"waiting_for_approval\"\n  | \"completed\"\n  | \"failed\"\n  | \"cancelled\";\n\ninterface AgentRunState {\n  runId: string;\n  status: AgentRunStatus;\n  input: unknown;\n  cursor?: string;\n  messages: AgentMessage[];\n  pendingToolCall?: ToolCall;\n  artifacts: Artifact[];\n  error?: string;\n  updatedAt: string;\n}\n```\n\nMore important than the exact fields is the discipline:\n\n**Why this works:**\n\nThe worker becomes replaceable. The run state survives independently of any one process.\n\nThis also makes debugging much easier. When someone asks, “What is the agent doing right now?” you can answer from stored state instead of guessing from logs.\n\n🚨 Production warning:\n\nIf you cannot answer “What step is this agent on?” without attaching a debugger, your agent is not production-ready.\n\n**Scenario:**\n\nYou use Server-Sent Events or WebSockets to stream agent progress. The connection drops. The user reloads the page. Now the frontend has missed events and has no reliable way to recover.\n\n**Why it matters:**\n\nStreaming is excellent for user experience. It is terrible as the only record of what happened.\n\nA stream is ephemeral. It tells you what is happening now, or what happened recently, but it does not by itself answer:\n\n**Solution:**\n\nKeep durable run state as the source of truth. Use streaming as a notification layer on top of that state.\n\nA resilient SSE endpoint should support reconnection:\n\n``` js\napp.get(\"/agent-runs/:id/events\", async (req, res) => {\n  const runId = req.params.id;\n\n  res.setHeader(\"Content-Type\", \"text/event-stream\");\n  res.setHeader(\"Cache-Control\", \"no-cache\");\n  res.setHeader(\"Connection\", \"keep-alive\");\n  res.flushHeaders();\n\n  const lastEventId = Number(req.headers[\"last-event-id\"] ?? \"0\");\n\n  const missedEvents = await eventStore.eventsSince(runId, lastEventId);\n\n  for (const event of missedEvents) {\n    sendEvent(res, event);\n  }\n\n  const unsubscribe = eventStore.subscribe(runId, (event) => {\n    sendEvent(res, event);\n  });\n\n  const heartbeat = setInterval(() => {\n    res.write(\": ping\\n\\n\");\n  }, 15000);\n\n  req.on(\"close\", () => {\n    clearInterval(heartbeat);\n    unsubscribe();\n  });\n});\n\nfunction sendEvent(\n  res: express.Response,\n  event: { id: number; type: string; payload: unknown }\n) {\n  res.write(`id: ${event.id}\\n`);\n  res.write(`event: ${event.type}\\n`);\n  res.write(`data: ${JSON.stringify(event.payload)}\\n\\n`);\n}\n```\n\nThe important part is `Last-Event-ID`. When the client reconnects, the server can replay missed events instead of pretending the connection never dropped.\n\n**Why this works:**\n\nThe client can lose the stream without losing the run. The UI becomes a projection of durable state, not the only place where progress exists.\n\n💡 Practical note:\n\nHeartbeats are not optional. Many proxies and load balancers close idle connections quietly. A periodic comment line like `: ping` helps keep the stream alive.\n\n**Scenario:**\n\nYour agent calls three tools: search, create ticket, send email. The email tool succeeds, then the run crashes before the state is saved. The worker restarts and retries. Now the email may be sent again.\n\n**Why it matters:**\n\nAgents with tools are not just inference loops. They are distributed systems that take actions.\n\nOnce an agent can affect external systems, you have the usual distributed-workflow problems:\n\nBut agents make these problems harder because the sequence of actions may not be deterministic.\n\n**Solution:**\n\nMake tool execution idempotent wherever possible. Give each meaningful tool call a stable idempotency key derived from the run and the logical operation.\n\n```\nasync function sendInvoiceEmail(run: AgentRunState, invoiceId: string) {\n  const toolCallKey = `${run.runId}:send_invoice_email:${invoiceId}`;\n\n  const existing = await externalCallStore.find(toolCallKey);\n\n  if (existing) {\n    return existing.result;\n  }\n\n  const result = await emailClient.send({\n    idempotencyKey: toolCallKey,\n    to: run.input.customerEmail,\n    subject: `Invoice ${invoiceId}`,\n    template: \"invoice\",\n  });\n\n  await externalCallStore.record(toolCallKey, result);\n\n  return result;\n}\n```\n\nIf the downstream API does not support idempotency keys, you still need local deduplication:\n\n``` js\nconst alreadyPerformed = await sideEffectLog.exists(toolCallKey);\n\nif (alreadyPerformed) {\n  return sideEffectLog.resultFor(toolCallKey);\n}\n```\n\nFor destructive or irreversible operations, consider requiring explicit approval or using a two-phase pattern:\n\n```\npropose action\nstore proposal\nwait for approval\nexecute once\nrecord result\n```\n\n**Why this works:**\n\nYou are separating “the agent decided to do something” from “the system actually did it.” That separation gives you a place to enforce safety, retries, and auditing.\n\n🧠 The important part:\n\nIf a tool call can happen twice, the agent run is not safe to retry unless that tool call is idempotent or guarded.\n\n**Scenario:**\n\nThe user clicks **“Cancel”** while the agent is running. Your API updates a database row. The agent, currently waiting on a model call or external tool, has no idea.\n\n**Why it matters:**\n\nCancellation is easy as a UI concept and hard as an execution concept. If the agent does not check for cancellation, it keeps doing work. If it checks too late, it may perform side effects after the user asked it to stop.\n\n**Solution:**\n\nTreat cancellation as a cooperative protocol.\n\nFirst, expose a cancellation endpoint:\n\n``` js\napp.post(\"/agent-runs/:id/cancel\", async (req, res) => {\n  const runId = req.params.id;\n\n  await agentRunStore.requestCancellation(runId);\n\n  res.status(202).json({\n    runId,\n    status: \"cancel_requested\",\n  });\n});\n```\n\nThen make the worker check cancellation at safe boundaries:\n\n```\nasync function executeAgentRun(runId: string, signal: AbortSignal) {\n  while (!signal.aborted) {\n    const run = await agentRunStore.get(runId);\n\n    if (!run) return;\n\n    if (run.status === \"cancel_requested\") {\n      await agentRunStore.markCancelled(runId, {\n        reason: \"user_requested\",\n      });\n      return;\n    }\n\n    const nextStep = await planNextStep(run);\n\n    if (!nextStep) {\n      await agentRunStore.markCompleted(runId);\n      return;\n    }\n\n    await executeStep(runId, nextStep, signal);\n  }\n}\n```\n\nThe critical detail is where you check.\n\nGood cancellation points:\n\nBad cancellation strategy:\n\n**Why this works:**\n\nCancellation becomes part of the run lifecycle instead of an afterthought.\n\n⚠️ Gotcha:\n\nCancellation does not automatically undo side effects. If the agent already sent the email, cancellation may only mean “stop doing more work.” Your system needs to know the difference.\n\n**Scenario:**\n\nYour agent needs approval before sending a high-risk email. The approval may come in ten seconds, ten hours, or three days.\n\nNow your agent is not merely long-running. It is suspended.\n\n**Why it matters:**\n\nMany agent systems are designed for “slow API calls,” not for “pause until a human responds.” These are different problems.\n\nA slow API call can be handled with timeouts and retries. A human pause requires:\n\nIf you keep the agent process alive while waiting for a human, you are wasting resources. If you do not persist the pause, you lose the run.\n\n**Solution:**\n\nModel waiting states explicitly.\n\n```\nawait agentRunStore.update(runId, {\n  status: \"waiting_for_approval\",\n  pendingAction: {\n    type: \"send_customer_email\",\n    payload: emailDraft,\n    requestedAt: new Date().toISOString(),\n    expiresAt: new Date(Date.now() + 48 * 60 * 60 * 1000).toISOString(),\n  },\n});\n```\n\nThen resume when the approval arrives:\n\n``` js\napp.post(\"/agent-runs/:id/approvals\", async (req, res) => {\n  const runId = req.params.id;\n  const { approved, approver } = req.body;\n\n  await agentRunStore.resolveApproval(runId, {\n    approved,\n    approver,\n    resolvedAt: new Date().toISOString(),\n  });\n\n  await queue.enqueue(\"agent.resume\", { runId });\n\n  res.status(202).json({ runId, status: \"resuming\" });\n});\n```\n\nThis is where durable execution engines start to make sense. They are designed for workflows that can wait for long periods without keeping a process alive.\n\n**Why this works:**\n\nThe agent run becomes a workflow with explicit pause and resume points. That is far more honest than pretending a human approval is just another fast function call.\n\n💡 Practical note:\n\nIf your agent can wait for humans, design expiration early. A pending action that lives forever becomes a security and compliance problem.\n\n**Scenario:**\n\nThe agent starts with the user’s access token. Twenty minutes later, it needs to call another API. The token has expired.\n\n**Why it matters:**\n\nUser sessions and agent lifetimes do not naturally align.\n\nA user may:\n\nMeanwhile, the agent may still be running.\n\nIf you casually pass the original request token into a long-lived background worker, you create both security and reliability problems.\n\n**Solution:**\n\nDecide explicitly what identity the agent runs under.\n\nCommon patterns:\n\nUseful when the run is brief and the token lifetime is sufficient.\n\nRisk: the token expires mid-run.\n\nUseful when the agent acts on behalf of the user for longer periods.\n\nRequirements:\n\nUseful when the agent performs system-level work.\n\nUseful when the agent needs approval for high-risk actions.\n\nExample:\n\n```\nagent requests permission to refund payment\nsystem pauses run\nuser re-authenticates\nsystem resumes run with fresh approval\n```\n\nA practical authorization check before a sensitive tool call:\n\n```\nasync function authorizeToolCall(run: AgentRunState, tool: ToolDefinition) {\n  const policy = await policyStore.forRun(run.runId);\n\n  if (!policy.allows(tool.name)) {\n    throw new Error(`Tool ${tool.name} is not allowed for this run`);\n  }\n\n  if (tool.requiresRecentUserApproval) {\n    const approval = await approvalStore.latest(run.runId, tool.name);\n\n    if (!approval || approval.approvedAt < minutesAgo(5)) {\n      await agentRunStore.update(run.runId, {\n        status: \"waiting_for_approval\",\n        pendingAction: {\n          type: tool.name,\n          payload: tool.input,\n        },\n      });\n\n      throw new ToolCallPaused(\"Recent approval required\");\n    }\n  }\n}\n```\n\n**Why this works:**\n\nYou stop assuming that the original HTTP request’s auth context is valid forever.\n\n🚨 Production warning:\n\nDo not solve token expiration by giving agents broad admin tokens “just to keep things moving.” That turns every long-running agent into a privilege-escalation risk.\n\nNot every agent needs the same architecture. The right choice depends on how long the agent can run, whether it has side effects, and whether humans are involved.\n\n| Approach | Best for | Complexity | Weakness | \n|---|---|---|---|\n| Synchronous HTTP | Very fast, read-only agent calls | Low | Breaks as soon as work is slow or unreliable | \n| Async run + polling | Simple background agent jobs | Medium | Polling can be inefficient; needs run store | \n| SSE/WebSocket progress | Interactive UX with live updates | Medium | Connection loss handling is required | \n| Queue workers | Scalable background execution | Medium | Retry and state discipline needed | \n| Durable workflow engine | Multi-step, long-lived, human-in-loop agents | High | More operational complexity | \n\nUse synchronous request/response when:\n\n```\nPOST /summarize-text\n```\n\nUse async runs when:\n\n```\nPOST /agent-runs\nGET /agent-runs/:id\n```\n\nUse SSE or WebSockets when:\n\nBut keep durable state underneath.\n\nUse a durable workflow engine when:\n\nThis is the category where “agent” starts looking less like a chat endpoint and more like business process automation.\n\nIf I were shipping an AI agent feature that could run longer than a normal HTTP request, I would use this shape:\n\n```\nClient\n  ↓\nAPI layer\n  ↓\nRun store\n  ↓\nQueue / workflow engine\n  ↓\nAgent worker\n  ↓\nTool execution layer\n  ↓\nEvent store / notifications\n```\n\nResponsible for:\n\nStores:\n\nThe most important design decision is this:\n\nThe HTTP request starts and observes the run.\n\nIt does not own the run.\n\nThat one mental shift prevents a large class of production pain.\n\nBefore letting an agent run longer than the HTTP request that started it, I’d want these boxes checked.\n\nThe deeper truth is that long-running AI agents are not just “slower APIs.” They are workflows with non-deterministic planning, external side effects, and user expectations.\n\nHTTP can trigger them. HTTP can report on them. HTTP can stream progress from them.\n\nBut once the agent can outlive the request, HTTP should not be the container for the entire execution.\n\nThe request is the doorway.\n\nThe agent run is the process.\n\nDesign them separately, and the system becomes far easier to operate. Design them as the same thing, and every timeout, retry, deploy, and disconnected browser becomes a potential corruption of the run.", "url": "https://wpnews.pro/news/what-happens-when-an-ai-agent-runs-longer-than-your-http-request", "canonical_source": "https://dev.to/hosseinhezami/what-happens-when-an-ai-agent-runs-longer-than-your-http-request-288o", "published_at": "2026-09-10 07:08:55+00:00", "updated_at": "2026-09-10 07:22:43.509168+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "developer-tools", "mlops"], "entities": ["HTTP", "Express", "Node.js"], "alternates": {"html": "https://wpnews.pro/news/what-happens-when-an-ai-agent-runs-longer-than-your-http-request", "markdown": "https://wpnews.pro/news/what-happens-when-an-ai-agent-runs-longer-than-your-http-request.md", "text": "https://wpnews.pro/news/what-happens-when-an-ai-agent-runs-longer-than-your-http-request.txt", "jsonld": "https://wpnews.pro/news/what-happens-when-an-ai-agent-runs-longer-than-your-http-request.jsonld"}}