{"slug": "openrouter-batch-api-half-price-inference-by-bundling-requests", "title": "OpenRouter Batch API: half-price inference by bundling requests", "summary": "OpenRouter launched a Batch API that lets providers complete bundled inference requests anytime within a 24-hour window in exchange for charging roughly 50% or less of normal per-token pricing, available today on more than 70 models. Across 230,000+ batches completed during a two-week beta, the median finished in 7 minutes, the 90th percentile in 1.0 hour, and the 99th percentile in 10.3 hours, with batches submitted between 5am and noon Pacific significantly slower than other times of day. The asynchronous API supports chat completions, responses, messages, and embeddings, keeps inputs and results for 30 days, and returns each result independently so a few bad rows never fail the rest of the job.", "body_md": "# Batch API: half-price inference by bundling requests\n\nBrian Thomas ·\n\nFor large batches of work or requests that don’t need to be completed immediately, you can now use the new [batch API](https://openrouter.ai/docs/batch-quickstart). When submitting a batch, a provider gets to choose when during a 24 hour window they will complete the request, and in exchange they generally charge 50% (and sometimes less) of their normal per-token price.\n\nIt works today on [more than 70 models](https://openrouter.ai/models?variant=batch). Learn how to use it in the [Batch API docs](https://openrouter.ai/docs/batch-quickstart).\n\nIn practice, we’ve observed you rarely wait anywhere near 24 hours. Across 230k+ batches that completed over our two week beta period, the median finished in 7 minutes and 90% finished within an hour.\n\nBatch is an asynchronous API for workloads where you can accept highly variable response times, such as labeling a corpus, back-filling embeddings, scoring an eval set, summarizing a backlog of tickets, or running the same prompt across a few thousand rows overnight.\n\n## How the Batch API works\n\nCall `api/v1/batches` with your list of requests and specify what endpoint shape to use. Chat completions, responses, messages, and embeddings are all supported. For example:\n\n```\ncurl https://openrouter.ai/api/v1/batches \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $OPENROUTER_API_KEY\" \\\n  -d '{\n  \"endpoint\": \"/v1/chat/completions\",\n  \"model\": \"google/gemini-3.8-flash\",\n  \"requests\": [\n    { \"custom_id\": \"ticket-0001\", \"body\": { \"messages\": [{ \"role\": \"user\", \"content\": \"Summarize this ticket in one sentence.\" }] } },\n    { \"custom_id\": \"ticket-0002\", \"body\": { \"messages\": [{ \"role\": \"user\", \"content\": \"Summarize this ticket in one sentence.\" }] } }\n  ]\n}'\n```\n\nAfter the request, begin polling `GET /api/v1/batches/:id` until the status is `completed`, `failed`, `expired`, or `cancelled`. Completed batches return their results inline in the same response.\n\n## Most batches finish in minutes\n\nWe looked at the batches that completed during our two week beta period and measured the time from acceptance to finished results. The median was 7 minutes, the 90th percentile was 1.0 hour, and the 99th percentile was 10.3 hours. We also found that the time of day you submit matters more than how many requests you send.\n\nBatches submitted between 5am and noon Pacific are significantly slower than other times of day. The slowest tenth take 2 to 4.5 hours. Submit at any other hour and the 90th percentile drops under 1.1 hours, and after 6pm Pacific it’s under 50 minutes.\n\nA single-request batch finishes in 5 to 11 minutes depending on the hour, and a batch of 1,000 or more requests finishes in 12 to 21 minutes. Large batches do take longer to process. The slowest tenth of batches with more than 100 requests submitted between midnight and noon Pacific took as long as 6.8 hours.\n\n## What’s supported with batches\n\n- Request shapes: any text request body shape you already send to OpenRouter will be supported, including chat completions, responses, messages, and embeddings.\n- Routing: Each batch is executed on a single provider. By default we choose the cheapest batch endpoint for the model after your provider allowlist, data policy, and BYOK settings are applied.\n- BYOK: with a [provider key](https://openrouter.ai/docs/guides/overview/auth/byok) configured, batches on providers that support it route through your key and you pay only the BYOK fee.\n- Per-request results: every result returns independently, so a few bad rows never fail the rest of the job.\n- Retention: inputs and results are kept for 30 days, or until you `DELETE` the batch.\n- Logging: every batch shows up in the [Batches tab of your logs](https://openrouter.ai/logs) with model, provider, status, and cost.\n- Pricing: the discount applies to per-token pricing and varies by model. Web search calls bill at standard rates.\n- Inputs: images and files must be public URLs. Audio, video, and OpenRouter’s own web search plugin aren’t available in batch (see the [limitations section of the docs](https://openrouter.ai/docs/batch-quickstart#limitations) ).\n\n## Get started\n\nPick a [batch-capable model](https://openrouter.ai/models?variant=batch), grab an [API key](https://openrouter.ai/settings/keys), and post your first batch to `https://openrouter.ai/api/v1/batches`. The [Quickstart](https://openrouter.ai/docs/batch-quickstart) has the full request and response shapes.\n\nTell us what you’re batching in [#feedback](https://discord.gg/fVyRaUDgxW) on Discord.", "url": "https://wpnews.pro/news/openrouter-batch-api-half-price-inference-by-bundling-requests", "canonical_source": "https://openrouter.ai/blog/announcements/batch-api/", "published_at": "2026-09-22 16:42:02+00:00", "updated_at": "2026-09-22 16:54:17.049136+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-products", "developer-tools", "large-language-models"], "entities": ["OpenRouter", "Brian Thomas", "Batch API", "google/gemini-3.8-flash"], "alternates": {"html": "https://wpnews.pro/news/openrouter-batch-api-half-price-inference-by-bundling-requests", "markdown": "https://wpnews.pro/news/openrouter-batch-api-half-price-inference-by-bundling-requests.md", "text": "https://wpnews.pro/news/openrouter-batch-api-half-price-inference-by-bundling-requests.txt", "jsonld": "https://wpnews.pro/news/openrouter-batch-api-half-price-inference-by-bundling-requests.jsonld"}}