{"slug": "a-rate-limit-that-counts-the-tokens-you-asked-for-not-the-ones-you-got", "title": "A rate limit that counts the tokens you asked for, not the ones you got", "summary": "A developer discovered that Groq's free tier rate limit counts the declared max_tokens rather than actual generated tokens, causing requests with small prompts but large max_tokens values to be rejected with a 413 error. Testing across 14 models revealed that four models failed identical requests when max_tokens was set to 8192, while passing with a lower value, and that the limit is a rolling window shared across models. The developer advises setting max_tokens to actual needs, such as 1500, to avoid failures in agent applications.", "body_md": "Spent a while assuming a 413 meant my prompt was too long. It wasn't. The prompt was 20 tokens.\n\nGroq's free tier caps you at 8,000 tokens per minute, and that budget is charged against the `max_tokens`\n\nyou **declare**, not the number the model actually generates. So this fails:\n\n``` php\nprompt: 20 tokens, max_tokens: 8192\n-> 413  \"on tokens per minute (TPM): Limit 8000, Requested 8271\"\n```\n\nNothing was generated. The request was rejected on the ceiling I asked for.\n\nThe same model with `max_tokens: 16`\n\nand a 4,078-token prompt returns 200 without complaint. Prompt size was never the problem.\n\nI swept the 14 models Groq advertises. Four of them answered a minimal request in ~300ms and then returned 413 on the identical request with `max_tokens: 8192`\n\n:\n\n``` php\nopenai/gpt-oss-120b          478ms   ->  413  (Requested 8271)\nqwen/qwen3.8-27b             324ms   ->  413  (Requested 8212)\nqwen/qwen3.6-27b             330ms   ->  413  (Requested 8210)\nopenai/gpt-oss-safeguard-20b 308ms   ->  413  (Requested 8271)\n```\n\nTwo more passed the same request, which is the second thing worth knowing: it is a rolling window shared across models, not a per-model constant. A model that passes one probe can 413 a minute later — one of mine came back `Limit 8000, Used 4373, Requested 6278`\n\n. Never conclude a model is exempt from a single passing call.\n\nThe practical consequence is for agents. Tool schemas already eat the prompt, and most harnesses declare a generous `max_tokens`\n\nby default. Set it to what you actually need — 1,500 works — or every call dies before the model sees it.\n\nWorth checking whether your provider bills the declared ceiling or the real output. Alibaba's docs, for contrast, say plainly that their TPM \"includes input and output tokens.\"", "url": "https://wpnews.pro/news/a-rate-limit-that-counts-the-tokens-you-asked-for-not-the-ones-you-got", "canonical_source": "https://dev.to/build996/a-rate-limit-that-counts-the-tokens-you-asked-for-not-the-ones-you-got-1oda", "published_at": "2026-09-02 11:34:27+00:00", "updated_at": "2026-09-02 11:53:58.962887+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure"], "entities": ["Groq", "Alibaba", "openai/gpt-oss-120b", "qwen/qwen3.8-27b", "qwen/qwen3.6-27b", "openai/gpt-oss-safeguard-20b"], "alternates": {"html": "https://wpnews.pro/news/a-rate-limit-that-counts-the-tokens-you-asked-for-not-the-ones-you-got", "markdown": "https://wpnews.pro/news/a-rate-limit-that-counts-the-tokens-you-asked-for-not-the-ones-you-got.md", "text": "https://wpnews.pro/news/a-rate-limit-that-counts-the-tokens-you-asked-for-not-the-ones-you-got.txt", "jsonld": "https://wpnews.pro/news/a-rate-limit-that-counts-the-tokens-you-asked-for-not-the-ones-you-got.jsonld"}}