When I actually ran the numbers on a 1,000-page document using OpenAI's pricing, the embedding cost was roughly $0.18. That's a rounding error in a production budget. The real costs are hiding in the infrastructure and the LLM generation phase.
To get a real-world view of the spend, I mapped out the entire AI workflow to see where the leakages actually occur:
Upload PDF
│
▼
Extract + clean text ── billing quietly starts here
│
▼
Is this doc already known? ── the check that saves you real money
│
┌──┴──┐
same changed
│ │
skip continue
│ │
▼ ▼
Chunk only what changed
│
▼
Attach metadata
│
▼
Embed (cheap — ~18¢ per 1,000 pages)
│
▼
Store in vector DB ── millions of chunks, running 24×7
│
▼
Index + retrieve ── costs more than embedding, at scale
│
▼
LLM reads chunks, answers ── the real money and latency eater
The most overlooked optimization is the deduplication check. In my experience, about 80% of a knowledge base remains static; only 20% evolves. Treating an updated document as a brand-new file is a rookie mistake that spikes your costs. The professional move is to diff the content and only process the modified sections. This is where precise metadata (versioning, page markers) becomes a financial asset rather than just a technical detail.
The vector database is another deceptive cost center. You aren't paying for the storage of vectors; you're paying for the high-availability infrastructure required to keep those vectors searchable in milliseconds. At a small scale, it's negligible. At production scale, the recurring cost of maintaining the index and the compute for retrieval far outweighs the one-time cost of embedding.
Ultimately, if you're trying to optimize a RAG pipeline from scratch, stop obsessing over embedding tokens and start looking at your indexing overhead and LLM context window usage. That's where the real money is.
Next In-Context Learning vs Generalization: The Reality →