{"slug": "your-ai-app-has-a-rate-limit-your-agent-can-still-run-up-the-bill", "title": "Your AI App Has a Rate Limit. Your Agent Can Still Run Up the Bill.", "summary": "A developer outlined an approach for bounding the cost of multi-step AI agents, arguing that request-level rate limits fail to cap the many billable model and tool calls a single user action can trigger. The proposed design tracks usage per task (model calls, tool calls, tokens, elapsed time, estimated cost), enforces explicit per-task and per-account budgets in application code, reserves spending atomically before dispatching calls, detects retry loops that make no progress, and propagates cancellation to queued and in-flight work. The writeup cites OWASP's guidance on uncontrolled LLM resource consumption and recommends returning partial results with a clear stop reason when a task hits its limit.", "body_md": "A user clicks **“Research this topic.”**\n\nYour app accepts one request.\n\nBehind the scenes, the agent searches the web, reads pages, calls the model, retries a failed tool, and searches again.\n\nYour rate limiter sees one request.\n\nYour infrastructure sees a growing chain of work.\n\n**Limiting requests at the entrance doesn’t limit everything that happens afterward.**\n\nIf you’re building an AI feature, here’s how to keep that work under control.\n\nImagine a research task that uses:\n\nThat’s an illustrative workflow, but it shows the problem: one user action can trigger many billable operations.\n\nTrack usage for the entire task, including retries and background work.\n\nA useful starting point is to record:\n\n```\ntask_id\naccount_id\nmodel_calls\ntool_calls\ninput_tokens\noutput_tokens\nelapsed_time\nestimated_cost\nstop_reason\n```\n\nKeep sensitive document content and credentials out of routine logs.\n\nA task needs an explicit stopping point.\n\nFor example:\n\n``` js\nconst limits = {\n  maxModelCalls: 8,\n  maxToolCalls: 15,\n  maxDurationMs: 90_000,\n  maxEstimatedCostUsd: 0.30,\n};\n```\n\nThese numbers are examples, not recommended defaults.\n\nChoose limits based on the feature, its pricing, and the quality users need.\n\n**Enforce them in your application.** A prompt saying “don’t spend too much” cannot replace a budget check.\n\nBefore starting another operation, check whether the task has enough remaining budget.\n\nYou may not know the final charge before a model call completes.\n\nReserve a conservative allowance before dispatching it, using input size, output limits, and applicable pricing. Reconcile that reservation against reported usage afterward.\n\nInclude paid tools, search services, and other billable operations where relevant.\n\nIf several workers share a budget, reserve usage atomically. Otherwise, they can all see the same remaining balance and spend it simultaneously.\n\nA budget counter only works if concurrent work cannot bypass it.\n\nA failed operation can become an expensive loop:\n\nSearch failed → retry → model chooses another search → retry again.\n\nSet limits at two levels:\n\nAlso detect repeated actions that produce no progress.\n\nIf an agent keeps fetching the same page or repeating the same unsuccessful query, stop or ask the user for clarification.\n\n**Activity is not evidence of progress.**\n\nA request count treats a short question and a huge document as one request each.\n\nTheir resource requirements can differ considerably.\n\nSet appropriate limits on:\n\nOWASP identifies uncontrolled LLM resource consumption as a risk and recommends controls including input validation, rate limits, and user quotas.\n\nMake limits understandable to users. “This document exceeds the supported length” is more useful than a mysterious failure after a long wait.\n\nA user closes the tab.\n\nDoes the research stop?\n\nIf your backend job continues, the interface disappearing has changed nothing about the remaining work.\n\nPropagate cancellation to queued jobs and workers. Check cancellation before each new model or tool call.\n\nAttempt to cancel in-flight operations where supported, while recognizing that work already performed may still be billable.\n\nCancellation should stop future work as promptly as possible.\n\nA bounded task can still become expensive when someone starts hundreds of them.\n\nCombine:\n\nUse provider controls as another layer where available. Verify whether a provider’s “budget” actually blocks spending or merely sends an alert.\n\n**An alert tells you something happened. A blocking control prevents additional work.**\n\nWhen a task reaches its limit, don’t pretend the research is complete.\n\nReturn the useful work already collected:\n\n“I reviewed six sources and reached this task’s research limit. Here are the findings so far, along with the questions that remain.”\n\nYou can offer an explicit continuation if the product supports it.\n\nA partial answer with clear limits is easier to trust than a confident answer hiding unfinished work.\n\nTest these situations:\n\nCheck whether your workers actually stop—not just whether the interface displays an error.\n\nBefore adding another autonomous step, ask:\n\n**“What is the maximum work one user action can trigger?”**\n\nIf you can’t answer, measure the workflow and add enforceable limits.\n\nA useful agent needs room to work. Your application needs control over how much work it can start.\n\n**Does your AI feature limit incoming requests—or the full task behind each request?**", "url": "https://wpnews.pro/news/your-ai-app-has-a-rate-limit-your-agent-can-still-run-up-the-bill", "canonical_source": "https://dev.to/robertadam987_/your-ai-app-has-a-rate-limit-your-agent-can-still-run-up-the-bill-1pa5", "published_at": "2026-10-25 04:09:57+00:00", "updated_at": "2026-10-25 04:11:00.829585+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "ai-tools", "mlops", "ai-safety"], "entities": ["OWASP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-ai-app-has-a-rate-limit-your-agent-can-still-run-up-the-bill", "markdown": "https://wpnews.pro/news/your-ai-app-has-a-rate-limit-your-agent-can-still-run-up-the-bill.md", "text": "https://wpnews.pro/news/your-ai-app-has-a-rate-limit-your-agent-can-still-run-up-the-bill.txt", "jsonld": "https://wpnews.pro/news/your-ai-app-has-a-rate-limit-your-agent-can-still-run-up-the-bill.jsonld"}}