Your AI Agent Doesn't Belong Inside an HTTP Request

wpnews.pro

A user sends a message to your AI application.

Your server receives:

POST /api/chat

So you do what seems natural:

app.post('/api/chat', async (c) => {
  const result = await runAgent(await c.req.json());

  return c.json(result);
});

It works.

Then your Agent gets smarter. Instead of one LLM call, it starts doing this:

User
  │
  ▼
Agent
  │
  ├── LLM call
  ├── search
  ├── database query
  ├── tool call
  ├── another LLM call
  ├── file generation
  └── final LLM call

The request that used to take 5 seconds now takes 30 seconds, then 2 minutes, then 10.

The HTTP request is still there, but the HTTP connection is a poor place to represent the lifetime of an Agent.

Consider this architecture:

Browser
   │
   │ POST /api/agent
   ▼
Node.js
   │
   │ await
   ▼
Agent
   │
   ├── LLM
   ├── Tool
   ├── LLM
   ├── Tool
   └── LLM
   │
   ▼
HTTP response

The entire Agent run is coupled to one HTTP request. That raises an uncomfortable question:

What happens when the browser disconnects?

A server can detect a client disconnect through the request's AbortSignal and combine it with a server timeout:

const signal = AbortSignal.any([
  timeoutSignal,
  c.req.raw.signal,
]);

Then it passes the signal into the upstream LLM request:

await fetch(LLM_URL, { signal });

For a normal streaming chat response, this is what you want. If the user closes the tab halfway through, continuing to generate is wasteful.

An Agent is different.

Compare two operations.

User
  │
  ▼
POST /chat
  │
  ▼
LLM
  │
  ▼
stream response

The operation exists to serve the current connection. If the user leaves, cancelling the generation makes sense.

User
  │
  ▼
POST /agents
  │
  ▼
Agent
  │
  ├── search web
  ├── inspect documents
  ├── call tools
  ├── write files
  ├── run code
  └── generate final report

This is no longer a response. It is a task, and tasks have their own lifecycle.

Tying an Agent to the connection says:

"This Agent exists as long as this browser connection exists."

What the user actually asked is:

"Perform this task for me."

Those are different lifetimes. The browser connection might last 10 seconds while the task takes 10 minutes:

Browser connection
       │
       │ short-lived
       ▼
HTTP request

                    Agent task
                       │
                       │ long-lived
                       ▼
                  Task lifecycle

This is the same boundary the previous article ran into with cancellation. A client disconnect does not prove the server cancelled the work, and a server cancellation does not prove the provider stopped inference. Each of those is a separate fact.

Now take it one step further. Suppose the Agent has already searched, queried the database, and generated a file, and then the user closes the browser. If the request was:

"Research these 20 documents and generate a report."

you probably don't want:

Browser closed
     ↓
Agent killed
     ↓
report lost

The user didn't cancel the task. They closed the UI.

Instead of running the Agent inside the request, the request creates a task and returns:

POST /api/agents
{
  "taskId": "task_01J..."
}

The HTTP request is finished. The Agent continues on its own:

HTTP request
     │
     ▼
create task ──────────► task_123
     │                      │
     ▼                      ▼
HTTP response             Agent
                            │
                            ├── LLM
                            ├── Tool
                            └── LLM

The task ID is an identity for the work itself:

Before:  HTTP request = work
After:   task ID      = work

That lets you store the task and make the database, not the connection, the source of truth:

type AgentTask = {
  id: string;
  status: 'queued' | 'running' | 'completed' | 'failed' | 'cancelled';
  createdAt: Date;
  startedAt?: Date;
  completedAt?: Date;
  error?: string;
};

The browser stops owning the Agent. It owns a subscription to the Agent's progress:

POST /api/agents                  → task_123
GET  /api/agents/task_123         → current status
GET  /api/agents/task_123/events  → live progress

A client can connect, disconnect, and reconnect. The task doesn't care.

Reconnection becomes ordinary. The user starts an Agent at 12:00 and the browser crashes at 12:01. At 12:05 the user reopens the app, and the frontend asks:

GET /api/agents/task_123
{
  "id": "task_123",
  "status": "completed"
}

Nothing required the original TCP connection to survive five minutes.

The code for this article is a runnable demo: aiengineering-book/agent-task-demo. It runs the same simulated Agent bound to a request and as a task, drops the connection partway through, cancels at different points, evicts a subscriber that stops reading, and checks what survives each time.

This doesn't mean you should stop streaming. Streaming is still the right way to deliver progress. What changes is what the stream represents:

Streaming becomes a view of a task, not the task itself.

The task emits events:

task.started
tool.started
tool.completed
llm.started
llm.delta
llm.completed
task.completed

and any number of browsers can render them as they arrive:

                  Agent task
                      │
                      │ events
                      ▼
              ┌───────────────┐
              │ Event stream  │
              └───────┬───────┘
                      │
              ┌───────┴────────┐
              ▼                ▼
          Browser A        Browser B

An SSE response is a subscription to task_123. It is not where the Agent lives:

Task lifecycle
───────────────────────────────────────►

        Agent running

Browser A
───────────────X

                         Browser B
                         ────────────────►

The subscription can disappear without destroying the work. The LLM stream itself moves one level down: it feeds the task's event stream instead of feeding the HTTP response directly.

Once the task is independent of HTTP, the server no longer has to execute it either. The API accepts the request, validates input, creates the task, and hands it to a worker. The worker owns execution.

With a task, "the browser went away" and "the user wants this stopped" stop being the same event.

Browser
   X
   │
   ▼
subscription ended
Agent continues
POST /api/agents/task_123/cancel

Now the user has said explicitly: stop this task. That is a different semantic, and it lets you keep several policies side by side:

Event	What happens
User disconnects	The subscription ends. The Agent continues.
User presses Stop	The task is cancelled.
Task runs longer than its limit (for example 30 minutes)	The task is cancelled.
Server shuts down (SIGTERM)	Task state is persisted. The task resumes later.

These are four different events. Collapsing them into one AbortSignal hides the difference between them.

A cancel request also needs a precise answer. "Cancelled" can mean different things depending on how far the work had got, and a status that hides the difference is misleading:

Outcome	Meaning
Client detached	The subscription ended. Nothing was cancelled.
Queued work cancelled	The task had not started. Nothing ran.
Tool call cancelled	A running step stopped before it finished.
Side effect already committed	A step finished before the cancel arrived and cannot be undone.

Return which of these happened. In the demo, the cancel endpoint answers step_cancelled or side_effect_committed. Give the task an ID that appears in every log line across the browser, the API, the worker, and the provider call, so the sequence can be reconstructed afterwards.

Decoupling the task from the connection doesn't make the hard parts disappear. It moves them somewhere you have to design for.

queued, running, and one of completed, failed, cancelled. The task decides these states, not the HTTP connection. None of these are specific to AI. They are the standard problems of background work, and Agents run into them early because a single Agent run is already many asynchronous steps. Each deserves more space than this article can give it.

Not every operation needs a task. For plain chat:

request
  ↓
LLM
  ↓
stream
  ↓
response

binding the work to the connection is reasonable, and cancelling when the user leaves is often exactly right.

The previous article measured one provider on one day: DeepSeek, streaming chat requests, aborted at two points. Billing followed the abort point and stayed far below the cost of running each request to its token limit. For plain chat in that test, cancel-on-disconnect was cheap and effective. That is one provider, not a general statement, but it points to where the real reason to decouple Agents lies. It isn't cost. It is that an Agent run has side effects and results the user still wants after the tab is gone.

Ask one question:

If the user closes the browser, should the work still exist?

If the answer is no, use connection cancellation. If the answer is yes, create a task and return its ID.

You can sort most AI operations into three patterns:

Pattern	Shape	Examples	Disconnect
Request-bound	HTTP → LLM → HTTP response	chat completion, short transformations, simple streaming	Can cancel the work
Persisted	HTTP → create work → storage → background processing	document indexing, file processing, generation written to history	Should not cancel the work
Resumable Agent task	HTTP → task ID → worker (LLM, tools, checkpoints, state)	research agents, coding agents, multi-step workflows	The UI is just a client of the task

The rule isn't "never run an Agent inside an HTTP request". It is:

Don't bind the lifetime of durable work to the lifetime of an ephemeral connection.

The first generation of AI applications looked like this:

HTTP request → LLM → HTTP response

That is fine for chat. Agents introduce something different:

HTTP request → create task → Agent → many asynchronous operations → durable result

Once work can outlive the browser connection, the connection should no longer own it:

┌──────────────────┐
│   Connection     │
│                  │
│ short-lived      │
│ disposable       │
│ reconnectable    │
└────────┬─────────┘
         │
         │ subscribes to
         ▼
┌──────────────────┐
│      Task        │
│                  │
│ durable          │
│ cancellable      │
│ resumable        │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  Agent Runtime   │
│                  │
│ LLMs             │
│ Tools            │
│ State            │
│ Checkpoints      │
└──────────────────┘

The browser connection is ephemeral. The Agent task is the thing that matters.

If the work should survive the connection, don't make the connection own the work.

This is the third article in a series about what happens when a user closes the tab. Part 1 covers how a disconnect reaches your server and how to propagate it upstream with AbortSignal. Part 2 measures whether aborting the request actually stops the provider.

The demo code is in aiengineering-book/agent-task-demo.

The server-side patterns this article builds on, including the difference between non-persisted chat work, persisted work, and long-running Agent tasks, are covered in Chapter 3 of 《AI Engineering with TypeScript — A Comprehensive Guide to Building AI Agents》 at Leanpub. The book's companion code is at github.com/aiengineering-book/code.

source & further reading

dev.to — original article Why a multi-agent system must not average LLM-as-Judge Position Bias: Swapping A/B Flipped 87 of 300 Verdicts October at Agentic Finance Graph: signed passes, a Merkle-sum ledger, and the corrections we made in public

Your AI Agent Doesn't Belong Inside an HTTP Request

Run your AI side-project on zahid.host