Filling GPT-6 Astra's 1M-Token Window Costs $10 a Call

OpenAI's GPT-6 Astra, launched September 3, 2026, offers a 1M-token context window at $10 per million input tokens, making a fully filled window cost $10 per request. An engineer argues that stuffing the window is not a substitute for retrieval, citing cost, relevance, debuggability, latency, and citation benefits of retrieval pipelines.

There is a moment in every LLM project where someone says the quiet thing out loud. "The context window is a million tokens now. Why are we still building a retrieval pipeline? Just put the whole codebase in." It is a reasonable question. It has an answer, and the answer is on the price sheet rather than in the architecture diagram. OpenAI announced GPT-6 Astra on 3 September 2026. It takes text and image input, returns text, and carries a 1M token context window . The launch list price for the standard tier is Put the two specs next to each other. One million tokens of context. Ten dollars per million input tokens. Fill the window and you have spent $10 before the model has emitted a single token of answer. A giant context window is not a replacement for retrieval. It is a way to pay for the retrieval you did not do. OpenAI published its own benchmark results at launch. Those are vendor-reported rather than independently verified, and worth reading in that light: GPQA Diamond at 96%, FrontierMath Tier 4 v2 at 97.6%, ARC-AGI-3 at 98.6%. None of those numbers tell you whether to fill the window. The price sheet does. Every number below is the published per-token price multiplied out. Nobody's invoice was consulted. Standard tier, one request that fills the context window: The input side is 250 times the output side. For an ordinary request shape, that ratio runs the other way and output is where your bill lives. The moment you fill a million-token window, that reverses and it is not close. Give the thing traffic. A thousand requests a day, each one stuffing the window: On the fast tier it is $20 per filled window, so double it. The retrieved version of the same feature sends about 8,000 tokens of prompt instead: Same model. Same question. Same answer length, so the output half of the bill is identical either way. The entire difference lives on the input side, and it is 125x. If your corpus is genuinely fixed across requests, check whether cached input pricing applies to your account before you accept the $10 as your real number. Caching helps most exactly where the same bytes go up over and over. It does nothing for a corpus that changes per request, and it does not change any of the other four reasons below. Cost is the loudest argument and it is not the only one. Relevance is a thing you can inspect. When you retrieve the top twelve chunks, you have a list. You can read it. You can check whether the chunk that answers the question is in it. When you stuff a million tokens, you have a haystack and a hope. Failure becomes two separable questions. A wrong answer from a retrieval pipeline splits cleanly: was the right chunk retrieved, and did the model use it. Those have different fixes. Bad retrieval means your chunking, your embedding model, or your query is wrong. Good retrieval with a bad answer means your prompt is wrong. A wrong answer out of a stuffed window is one undifferentiated problem, and the only lever you have is to rewrite the instructions and try again. Latency follows the input. A million tokens has to be sent and processed before the first token of the answer comes back. Eight thousand does not. You do not need a benchmark to know which of those a user waiting on a spinner prefers. Citations. This is the one that changes what you can ship. Retrieved chunks carry ids. Those ids go into the prompt, come back in the answer, get rendered as sources in your UI, and get written to your logs. Six months later somebody asks why the system told a customer the wrong refund policy, and you can answer, because you know which paragraph of which document version was in front of the model. Stuff the window and the honest answer to that question is "all of it". I am deliberately not making a claim here about answer quality degrading over long contexts. That is contested, it depends on the model, and I have not measured it on Astra. The four arguments above hold without it. Here is the whole pipeline in TypeScript. It is about a hundred lines. Set that against the $300,000 a month that the thousand-requests-a-day arithmetic above produces, and the build-versus-buy conversation gets short. Start by loading documents off disk. Swap this for your database, your S3 bucket, your Git repo. js import { readdir, readFile } from "node:fs/promises"; import { join } from "node:path"; export type Doc = { id: string; text: string }; export async function loadDocs dir: string : Promise<Doc { const names = await readdir dir ; return Promise.all names .filter n = n.endsWith ".md" .map async n = { id: n, text: await readFile join dir, n , "utf8" , } , ; } Then chunk. Fixed-size windows with overlap, because the sentence that answers the question has a habit of landing exactly on a boundary. export type Chunk = { id: string; docId: string; text: string }; export function chunkDoc doc: Doc, size = 2000, overlap = 250, : Chunk { const out: Chunk = ; const step = size - overlap; for let i = 0; i < doc.text.length; i += step { const text = doc.text.slice i, i + size ; if text.trim .length 0 { out.push { id: ${doc.id} ${i} , docId: doc.id, text } ; } if i + size = doc.text.length break; } return out; } The chunk id is the character offset inside the document. That is the citation trail, and it costs one string template to keep. Do not throw it away. Character-count chunking is the version you start with. Splitting on headings or paragraphs beats it on prose that has structure, and both beat splitting on a fixed token count that cuts mid-sentence. One embedding call per batch of chunks, at index time. The batch size is not decoration. Hand the endpoint your whole corpus in a single call and the first real run fails on the array limit, so the loop is part of the code, not an optimisation for later. python import OpenAI from "openai"; export const client = new OpenAI ; export async function embed texts: string , batch = 256, : Promise<number { const out: number = ; for let i = 0; i < texts.length; i += batch { const res = await client.embeddings.create { model: "text-embedding-3-small", input: texts.slice i, i + batch , } ; out.push ...res.data.map d = d.embedding ; } return out; } The cost shape here matters more than the number. Embedding is paid once per document version and amortised across every query that document ever serves. The $10 window fill is paid once per request, forever. There is no constant to quote for the first half: it is corpus size times update frequency divided by query volume. A corpus that changes hourly and one that changes yearly land orders of magnitude apart, so compute it from the pricing page https://openai.com/api/pricing/ with your own numbers. export type Indexed = Chunk & { vector: number }; export async function buildIndex chunks: Chunk , : Promise<Indexed { const vectors = await embed chunks.map c = c.text ; return chunks.map c, i = { ...c, vector: vectors i } ; } As a rough rule of thumb, an in-memory array holds up into the tens of thousands of chunks. Measure it on your own data rather than taking that number from me. Past the point where it stops holding up, reach for a vector store. Do not reach for one on day one to look serious. Cosine similarity, sorted, sliced. js export function cosine a: number , b: number : number { let dot = 0; let na = 0; let nb = 0; for let i = 0; i < a.length; i++ { dot += a i b i ; na += a i a i ; nb += b i b i ; } return dot / Math.sqrt na Math.sqrt nb ; } If your embedding model returns unit vectors, the dot product alone would do the job. The full cosine costs two extra loops and is correct either way, which is worth more than the loops cost you at index time. export async function retrieve index: Indexed , query: string, k = 12, : Promise<Indexed { const q = await embed query ; return index .map c = { c, score: cosine q, c.vector } .sort a, b = b.score - a.score .slice 0, k .map r = r.c ; } k is the dial that connects quality to the invoice. Twelve chunks of 2,000 characters is roughly 6,000 tokens, which is about six cents of input on Astra's standard tier. Going to k = 24 doubles that to twelve cents and still uses just over 1% of the window. Tune it on your own evaluation set. The point is that you have a dial. A stuffed window has one setting. Then assemble the prompt so the answer can cite its sources. export function buildContext hits: Indexed : string { return hits .map h, i = ${i + 1} source=${h.id}\n${h.text} .join "\n\n" ; } export const SYSTEM = "Answer only from the numbered sources below.", "Cite the source number after every claim, like 3 .", "If the sources do not contain the answer, say so.", .join "\n" ; That system prompt is doing real work. It turns "the model said something" into "the model said something and pointed at the paragraph it came from", which is the difference between a demo and something you can put in front of a customer. The part that ends the argument in your team's planning meeting. js const IN PER M = 10; // launch list price, USD per 1M input export const estTokens = s: string = Math.ceil s.length / 4 ; export function inputUSD tokens: number : number { return tokens / 1e6 IN PER M; } Four characters per token is close enough to make the decision and wrong enough that nobody should quote it in a budget. Run the model's own tokeniser before that number leaves your team. js const MODEL = process.env.OPENAI MODEL ; const docs = await loadDocs "./corpus" ; const index = await buildIndex docs.flatMap d = chunkDoc d ; const question = "Why did checkout retries double in July?"; const hits = await retrieve index, question, 12 ; const context = buildContext hits ; const stuffed = docs.reduce n, d = n + estTokens d.text , 0 ; const sent = estTokens SYSTEM + context + question ; console.log "stuff ", stuffed, inputUSD stuffed .toFixed 2 ; console.log "retrieve", sent, inputUSD sent .toFixed 2 ; Set OPENAI MODEL to whatever the current model id is on the pricing page. Then run it against your own corpus and read the two lines. If your documents fill the window, the first line is 1000000 and 10.00. The second, with the twelve 2,000-character chunks these settings produce, lands near 6100 and 0.06. Give it a real system prompt and answer-format instructions and you are at 8,000 tokens and eight cents. Either way the ratio between the two lines is the only thing you need to read. Send the retrieved version: js const res = await client.chat.completions.create { model: MODEL, messages: { role: "system", content: SYSTEM }, { role: "user", content: ${context}\n\nQ: ${question} }, , } ; console.log res.choices 0 .message.content ; console.log "sources:", hits.map h = h.id ; That last line is the whole citation trail. Log it next to the answer and next to the cost, and every future question about this system has an answer in your database. The window is not a mistake. There is work it is built for, and pretending otherwise would be as lazy as the "just put it all in" argument. Two of those cases are about volume. If you need one answer about one large document, once, ten dollars is cheaper than an afternoon of your time building an index. And if the whole corpus is 20,000 tokens, retrieval is machinery around a problem you do not have. Send it all. The other two are about the shape of the task. Summarise this whole book. Find every place in this repository that touches the payment flow. Retrieval finds passages that resemble a query, and neither of those has one. Top-k over a codebase will miss the file that matters because it never mentioned the words you searched for. Long agentic runs land in the same category for a different reason: when a missed file turns into a pile of paid steps built on top of a wrong turn, buying the full context up front can be the cheaper option. What those share is low request volume, or a task whose failure mode costs more than the window does. High-volume question answering over a document corpus is the opposite of that on both counts, and it is also the single most common thing teams build. Read the spec again with the price sheet next to it. One million tokens at $10 per million. Read that as a spec and you see capacity somebody handed you. Read it as a price and you see ten dollars of per-call budget, where every token that goes in is a token you chose to pay for. Take your largest prompt in production and print its token count next to its dollar cost, in the log line, at the point of the call. Most teams have never looked at that number and are surprised by it. Then, if you are stuffing anything large, build the hundred lines above against your real corpus and print both figures. You do not need a decision meeting after that. You need the two numbers, and they will make the decision look obvious in retrospect. Chunking, embeddings, top-k and the citation trail are the parts of an LLM system that decide whether your answers are grounded and whether your invoice is survivable. That is what my book AI That Reads covers, in TypeScript, from a flat file on disk It is book 2 of AI in TypeScript , a five-book series that runs from your first LLM call through to agents you can leave running in production.