Your LLM App Is Wasting Money: What Happens When Users Close the Tab?

wpnews.pro

You build an AI chat application.

A user sends:

"Explain how distributed systems work."

Your server calls an LLM API and starts streaming the answer:

LLM
 │
 ├── "Distributed"
 ├── "systems"
 ├── "are"
 ├── ...
 │
 ▼
Browser

Everything looks great.

Then the user closes the browser tab.

The response disappears.

But what about the LLM request?

Is it still running?

If your server doesn't explicitly propagate cancellation, the answer may be yes.

And that's not just a correctness problem. For an AI application, it can become a cost problem.

A user can abandon a generation after 500 tokens, while your backend continues paying for the remaining thousands of tokens.

In this article, we'll build the cancellation path for a TypeScript LLM server:

Browser
   │
   │ client disconnect
   ▼
Hono / Node.js
   │
   │ AbortSignal
   ▼
fetch()
   │
   │ cancellation
   ▼
LLM API

Along the way, we'll look at:

AbortController

and AbortSignal

workConsider the simplest possible server:

app.post('/api/chat', async (c) => {
  const response = await fetch(LLM_API_URL, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${API_KEY}`,
    },
    body: JSON.stringify({
      model: '...',
      messages: [
        {
          role: 'user',
          content: 'Explain distributed systems',
        },
      ],
      stream: true,
    }),
  });

  return new Response(response.body, {
    headers: {
      'Content-Type': 'text/event-stream',
    },
  });
});

This works.

The browser sends a request:

Browser
   │
   │ POST /api/chat
   ▼
Server
   │
   │ fetch()
   ▼
LLM API

The LLM starts generating tokens, and the server streams them back to the browser.

But now:

Browser
   │
   X
   │
   │ tab closed

The browser is gone.

What happens to the fetch()

call to the LLM?

Nothing automatically tells your application to cancel it.

The server and the upstream LLM request are separate operations.

Your server needs to explicitly connect their lifecycles.

There are actually two HTTP connections here.

       Connection #1
Browser ────────────────► Server
                             │
                             │ Connection #2
                             ▼
                         LLM API

The browser controls Connection #1.

Your server controls Connection #2.

When the browser closes the tab:

Browser                  Server                  LLM API
   │                        │                       │
   │──── HTTP request ─────►│──── HTTP request ───►│
   │                        │                       │
   X                        │                       │
   │                        │                       │
   │ connection closed      │                       │
   │                        │                       │
                            │                       │
                            │──── still connected ─►│

The LLM API doesn't magically know that the browser disappeared.

Your server has to propagate the cancellation:

Browser
   X
   │
   ▼
Server
   │
   │ abort()
   ▼
LLM API

This is where AbortController

comes in.

The Web Platform already gives us a standard cancellation mechanism:

const controller = new AbortController();

const response = await fetch(url, {
  signal: controller.signal,
});

// Later:
controller.abort();

The important part is:

signal: controller.signal

The AbortSignal

is passed into the operation.

When:

controller.abort();

is called, APIs that support the signal can terminate the operation.

Node.js supports AbortSignal

throughout its asynchronous APIs, including streams and HTTP-related operations.

This gives us a useful mental model:

AbortController
       │
       │ signal
       ▼
 ┌─────────────┐
 │ async work  │
 └─────────────┘
       │
       │ abort()
       ▼
   cancelled

The controller doesn't need to know what it's cancelling.

It simply broadcasts:

"Stop."

Any operation that received its signal can react accordingly.

Now we need to answer the first question:

How does the server know that the browser has disconnected?

With Hono, the request exposes the underlying request signal:

c.req.raw.signal

This signal is aborted when the client connection is terminated.

So we can connect it directly to the LLM request:

app.post('/api/chat', async (c) => {
  const response = await fetch(LLM_API_URL, {
    method: 'POST',
    signal: c.req.raw.signal,
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${API_KEY}`,
    },
    body: JSON.stringify({
      model: '...',
      messages: [
        {
          role: 'user',
          content: 'Explain distributed systems',
        },
      ],
      stream: true,
    }),
  });

  return new Response(response.body, {
    headers: {
      'Content-Type': 'text/event-stream',
    },
  });
});

Now the lifecycle looks like this:

Browser
   │
   │ request
   ▼
Hono
   │
   │ c.req.raw.signal
   ▼
fetch()
   │
   ▼
LLM API

If the browser disconnects:

Browser
   X
   │
   ▼
c.req.raw.signal
   │
   │ aborted
   ▼
fetch()
   │
   │ cancelled
   ▼
LLM API

This is the critical connection that many first versions of AI applications miss.

There is another failure mode.

What if the LLM API simply takes too long?

You don't want a request hanging forever.

So we have two independent cancellation conditions:

             ┌─────────────────┐
             │ Client closes   │
             │ browser         │
             └────────┬────────┘
                      │
                      ▼
                   CANCEL
                      ▲
                      │
             ┌────────┴────────┐
             │ Server timeout  │
             │ 30 seconds      │
             └─────────────────┘

We want:

Cancel if

eithercondition occurs.

Modern JavaScript gives us exactly that:

AbortSignal.any()

AbortSignal.any()

creates a signal that aborts when any of the supplied signals aborts. It is available in Node.js 20+ and later versions.

So:

const timeoutSignal = AbortSignal.timeout(30_000);

const signal = AbortSignal.any([
  timeoutSignal,
  c.req.raw.signal,
]);

Now one signal represents both conditions.

Putting it together:

app.post('/api/chat', async (c) => {
  const signal = AbortSignal.any([
    AbortSignal.timeout(30_000),
    c.req.raw.signal,
  ]);

  try {
    const response = await fetch(LLM_API_URL, {
      method: 'POST',
      signal,
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${API_KEY}`,
      },
      body: JSON.stringify({
        model: '...',
        messages: [
          {
            role: 'user',
            content: 'Explain distributed systems',
          },
        ],
        stream: true,
      }),
    });

    return new Response(response.body, {
      headers: {
        'Content-Type': 'text/event-stream',
        'Cache-Control': 'no-cache',
      },
    });
  } catch (error) {
    if (signal.aborted) {
      console.log('LLM request cancelled');
    }

    throw error;
  }
});

There are now two ways the request can terminate:

                  ┌── browser closes
                  │
                  │
AbortSignal.any ──┤
                  │
                  │
                  └── 30s timeout
                         │
                         ▼
                       abort
                         │
                         ▼
                     fetch()
                         │
                         ▼
                      LLM API

This is much better than manually maintaining separate timers and disconnect handlers.

Cancellation becomes particularly important when you're streaming LLM output.

Without streaming:

Browser ── request ──► Server ──► LLM

                         20 seconds

Browser ◄────────────── complete response

With streaming:

Browser ◄── token ── token ── token ── token ── ...

The request may stay alive for tens of seconds or even minutes.

That creates a much larger cancellation window.

The architecture is essentially a stream pipeline:

LLM API
   │
   │ chunks
   ▼
SSE parser
   │
   │ events
   ▼
TransformStream
   │
   │ events
   ▼
HTTP response
   │
   ▼
Browser

The important thing is that the stream is not a special "AI" mechanism.

It's just a stream-processing pipeline.

Node.js and the Web Streams API provide cancellation mechanisms through AbortSignal

, and stream operations can be terminated when their signal is aborted.

This is why understanding server-side streams is so useful when building AI applications.

There's another subtle problem with streaming.

Suppose the LLM sends:

data: Hello\n\n
data: world\n\n

You might imagine that your HTTP client receives exactly those chunks.

It doesn't have to.

The network might give you:

data: Hel

then:

lo\n\ndata: wor

then:

ld\n\n

A TCP chunk is not necessarily an application-level message.

So your streaming pipeline needs to buffer incomplete data:

Network chunks
      │
      ▼
┌──────────────┐
│    Buffer    │
└──────┬───────┘
       │
       ▼
Complete SSE events
       │
       ▼
Application

A TransformStream

is a natural fit:

function createLineTransform(): TransformStream<string, string> {
  let buffer = '';

  return new TransformStream({
    transform(chunk, controller) {
      buffer += chunk;

      const lines = buffer.split('\n');

      // The last fragment may be incomplete.
      buffer = lines.pop() ?? '';

      for (const line of lines) {
        if (line.trim()) {
          controller.enqueue(line);
        }
      }
    },

    flush(controller) {
      if (buffer.trim()) {
        controller.enqueue(buffer);
      }
    },
  });
}

The pattern is:

upstream
   ↓
chunks
   ↓
buffer
   ↓
complete messages
   ↓
business logic

This same pattern appears when processing large files, SSE responses, and LLM streaming responses. The key abstraction is incremental processing, not AI.

There's one more reason to treat this as a stream pipeline.

What if the producer is faster than the consumer?

LLM
 │
 │ very fast
 ▼
TransformStream
 │
 │ very fast
 ▼
Network
 │
 │ slow
 ▼
Browser

If data were allowed to accumulate indefinitely, memory usage could grow.

Streams solve this with backpressure.

Conceptually:

LLM
 │
 ▼
Transform
 │
 ▼
Network
 │
 ▼
Browser
 ▲
 │
 └──── backpressure

When the downstream consumer cannot keep up, the stream machinery can stop pushing data upstream until capacity becomes available.

This is one of the major reasons streams are preferable to accumulating the entire response in memory.

And it is the same reason the following two approaches are fundamentally different:

// Wait for everything
const result = await response.text();

versus:

// Process incrementally
for await (const chunk of stream) {
  process(chunk);
}

For AI applications, incremental processing is what makes token-by-token responses possible.

Now return to our original question.

Suppose:

User starts generation
        │
        ▼
LLM generates 5,000 tokens
        │
        │
        ├── User closes tab after 500 tokens
        │
        ▼
Server continues generating

The exact financial impact depends on the model, provider, request, caching, and billing model.

But the engineering principle is simple:

If an operation no longer has a consumer, you should explicitly decide whether the operation should continue.

For an interactive chat response, continuing is usually wasteful.

The user isn't going to read tokens that have nowhere to go.

Cancellation gives you a way to release the work:

500 tokens generated
        │
        ▼
client disconnect
        │
        ▼
AbortSignal
        │
        ▼
cancel upstream request

The benefit isn't only token cost.

You also release:

Here's the important architectural distinction.

Client disconnect does not always mean "cancel the task."

Consider four operations.

User asks question
        ↓
LLM generates response
        ↓
User closes tab

Cancel it.

The result has no value if the user has abandoned it.

User uploads PDF
        ↓
Server starts indexing
        ↓
User closes browser

Don't necessarily cancel it.

The indexing operation is part of a persistent workflow.

The user's browser is merely observing the operation.

User submits form
        ↓
Server writes database
        ↓
Browser disconnects

Usually, you want the database operation to complete.

The database write is a business operation, not a streaming response.

Consider an Agent run:

User
 │
 ▼
POST /agent/run
 │
 ▼
Agent
 │
 ├── search
 ├── browse
 ├── call tools
 ├── generate
 └── ...

This might take several minutes.

Binding the entire Agent lifecycle to an HTTP connection is usually the wrong architecture.

Instead:

POST /agent/run
        │
        ▼
      taskId
        │
        ▼
background worker
        │
        ├── tool calls
        ├── LLM calls
        └── state

The frontend can then subscribe to the task:

Browser ───────► task status
       ◄──────── SSE / WebSocket / polling

Now closing the browser doesn't necessarily destroy the Agent run.

This distinction is important:

Cancellation is a business decision, not merely a technical decision.

Instead of thinking:

"The browser disconnected, so cancel everything."

Think:

"What is the lifecycle of this operation?"

There are two fundamentally different types of work:

                  Operation
                      │
          ┌───────────┴───────────┐
          │                       │
     Connection-bound        Task-bound
          │                       │
          ▼                       ▼
    Chat streaming           Agent run
    autocomplete             file indexing
    live response             database write
          │                       │
          ▼                       ▼
    disconnect → cancel      disconnect → continue

This distinction becomes increasingly important as an AI application grows.

Cancellation is not necessarily a normal application error.

For example:

try {
  await fetch(url, { signal });
} catch (error) {
  if (signal.aborted) {
    // Expected cancellation
    return;
  }

  throw error;
}

Compare that with:

429 Rate Limit
502 Upstream Failure
500 Internal Error
AbortError

These represent different things.

A production server should distinguish:

A useful error hierarchy might look like:

AppError
├── ValidationError
├── UnauthorizedError
├── RateLimitError
├── ExternalServiceError
└── ...

Then a global error handler can convert known application failures into consistent API responses while unexpected programmer errors are logged separately.

This kind of centralized error handling is especially important on servers because an unhandled error can affect many users rather than just one browser tab.

Putting everything together:

                         Browser
                            │
                            │ POST /chat
                            ▼
                    ┌───────────────┐
                    │     Hono      │
                    │               │
                    │ Zod validate  │
                    └───────┬───────┘
                            │
                            ▼
                  ┌───────────────────┐
                  │   AbortSignal     │
                  │                   │
                  │ client disconnect │
                  │        OR         │
                  │ 30s timeout       │
                  └─────────┬─────────┘
                            │
                            ▼
                     ┌────────────┐
                     │  fetch()   │
                     └─────┬──────┘
                           │
                           │ streaming
                           ▼
                     ┌────────────┐
                     │  LLM API   │
                     └─────┬──────┘
                           │
                           │ chunks
                           ▼
                    ┌────────────────┐
                    │ TransformStream│
                    │                │
                    │ parse / buffer │
                    └───────┬────────┘
                            │
                            │ SSE
                            ▼
                         Browser

There are several independent pieces here:

Type safety

Zod → validated input → typed route

Streaming

LLM → chunks → TransformStream → SSE

Cancellation

disconnect ─┐
            ├─→ AbortSignal → fetch()
timeout ────┘

Error handling

LLM / application errors
          ↓
     error hierarchy
          ↓
    global handler
          ↓
    consistent API

None of these mechanisms is specifically an "AI framework."

They're server-side engineering primitives.

And that's exactly why they matter.

When you first build an LLM application, it's tempting to think about the model first:

Which model?
Which prompt?
Which agent framework?
Which vector database?

But once the application has real users, many of the expensive problems are much less glamorous:

What happens when the user disconnects?

What happens when the model takes 2 minutes?

What happens when the provider returns 429?

What happens when the browser can't consume the stream fast enough?

What happens when the request times out?

What happens when an Agent outlives the HTTP connection?

These are server engineering problems.

And TypeScript gives you excellent primitives for solving them:

Promises
Streams
TransformStream
AbortController
AbortSignal
async iterators
typed errors
Zod
Hono

Once you understand these primitives, an LLM streaming server stops looking like mysterious AI infrastructure.

It's just a carefully designed asynchronous pipeline.

And that is the important shift:

AI engineering is still software engineering.

The model may be probabilistic.

The server shouldn't be.

If you want to go deeper into the underlying primitives, the Node.js documentation covers AbortSignal

, stream cancellation, and Web Streams in detail.

This article is also based on the server-side TypeScript patterns covered in Chapter 3 of 《AI Engineering with TypeScript — A Comprehensive Guide to Building AI Agents》at Leanpub, particularly the sections on Streams, cancellation, and type-safe API design. The chapter explicitly treats LLM streaming as an instance of the same streaming model used elsewhere in Node.js rather than as a separate AI-specific abstraction.

If you're building AI applications with TypeScript, these are the foundations worth understanding before adding more sophisticated agent frameworks.

source & further reading

dev.to — original article "[Jul 13] Watching AI Be Confidently Wrong" The 4,000-Token Tax: Auto-Disabling Claude Code Plugins You Haven't Touched in 30 Days Building a Production AI Platform: Kubernetes, GitOps, IaC, Security, and Observability

Your LLM App Is Wasting Money: What Happens When Users Close the Tab?

Run your AI side-project on zahid.host