A user clicks “Research this topic.”
Your app accepts one request.
Behind the scenes, the agent searches the web, reads pages, calls the model, retries a failed tool, and searches again.
Your rate limiter sees one request.
Your infrastructure sees a growing chain of work.
Limiting requests at the entrance doesn’t limit everything that happens afterward.
If you’re building an AI feature, here’s how to keep that work under control.
Imagine a research task that uses:
That’s an illustrative workflow, but it shows the problem: one user action can trigger many billable operations.
Track usage for the entire task, including retries and background work.
A useful starting point is to record:
task_id
account_id
model_calls
tool_calls
input_tokens
output_tokens
elapsed_time
estimated_cost
stop_reason
Keep sensitive document content and credentials out of routine logs.
A task needs an explicit stopping point.
For example:
const limits = {
maxModelCalls: 8,
maxToolCalls: 15,
maxDurationMs: 90_000,
maxEstimatedCostUsd: 0.30,
};
These numbers are examples, not recommended defaults.
Choose limits based on the feature, its pricing, and the quality users need.
Enforce them in your application. A prompt saying “don’t spend too much” cannot replace a budget check.
Before starting another operation, check whether the task has enough remaining budget.
You may not know the final charge before a model call completes.
Reserve a conservative allowance before dispatching it, using input size, output limits, and applicable pricing. Reconcile that reservation against reported usage afterward.
Include paid tools, search services, and other billable operations where relevant.
If several workers share a budget, reserve usage atomically. Otherwise, they can all see the same remaining balance and spend it simultaneously.
A budget counter only works if concurrent work cannot bypass it.
A failed operation can become an expensive loop:
Search failed → retry → model chooses another search → retry again.
Set limits at two levels:
Also detect repeated actions that produce no progress.
If an agent keeps fetching the same page or repeating the same unsuccessful query, stop or ask the user for clarification.
Activity is not evidence of progress.
A request count treats a short question and a huge document as one request each.
Their resource requirements can differ considerably.
Set appropriate limits on:
OWASP identifies uncontrolled LLM resource consumption as a risk and recommends controls including input validation, rate limits, and user quotas.
Make limits understandable to users. “This document exceeds the supported length” is more useful than a mysterious failure after a long wait.
A user closes the tab.
Does the research stop?
If your backend job continues, the interface disappearing has changed nothing about the remaining work.
Propagate cancellation to queued jobs and workers. Check cancellation before each new model or tool call.
Attempt to cancel in-flight operations where supported, while recognizing that work already performed may still be billable.
Cancellation should stop future work as promptly as possible.
A bounded task can still become expensive when someone starts hundreds of them.
Combine:
Use provider controls as another layer where available. Verify whether a provider’s “budget” actually blocks spending or merely sends an alert.
An alert tells you something happened. A blocking control prevents additional work.
When a task reaches its limit, don’t pretend the research is complete.
Return the useful work already collected:
“I reviewed six sources and reached this task’s research limit. Here are the findings so far, along with the questions that remain.”
You can offer an explicit continuation if the product supports it.
A partial answer with clear limits is easier to trust than a confident answer hiding unfinished work.
Test these situations:
Check whether your workers actually stop—not just whether the interface displays an error.
Before adding another autonomous step, ask:
“What is the maximum work one user action can trigger?”
If you can’t answer, measure the workflow and add enforceable limits.
A useful agent needs room to work. Your application needs control over how much work it can start.
Does your AI feature limit incoming requests—or the full task behind each request?