Before you wire up a search index, a retriever, and a chat model, do one piece of planning that will save you rework later: estimate your token budget for each query.
In a retrieval-augmented generation app, the model does not just read the user's question. It also reads your instructions, the retrieved passages, and any conversation history, and it still needs room to write an answer. If those pieces add up to more than the model's context window, something gets cut off, and it is usually the answer or the most useful passage.
Start with a simple per-query checklist:
Then add a safety buffer. Real documents are uneven, metadata adds tokens, and a question that looks short can retrieve a long passage. A buffer of around 10 to 20 percent is a practical starting point.
A worked example makes the trade-off clear. Suppose your instructions take 800 tokens, the question takes 150, and you retrieve five chunks of 400 tokens each. That is 2,950 input tokens before the answer. If you reserve 350 tokens for output, each query uses 3,300 tokens. At 1,000 queries a day for 30 days, that is 99 million tokens before any buffer. Change the chunk count or size, and the monthly total moves quickly.
That is why the budget should come before the build. Once you know how much room retrieval can safely use, you can choose a sensible number of chunks, a chunk size that fits the window, and a model whose context limit leaves room for output. If the numbers do not fit, trim retrieval, shorten instructions, or summarize history, rather than discovering truncation in production.
For a quick check of your own numbers, try the RAG Token Budget Calculator from AI Calculators. Enter your expected queries, chunk count, chunk size, and output allowance, and compare the per-query total with your model's limit before you commit to an architecture.