LLM API Costs Dropped 94%: What to Fix in Your Architecture Now

LLM API costs have dropped 94.5% since April 2023, with Gemini 3.1 Flash now costing $0.40 per million output tokens compared to GPT-4's $60 in March 2023, according to BenchLM.ai's pricing index. Most developers have not updated their architectures to reflect these reductions, leaving retrieval-augmented generation pipelines from 2024 potentially inefficient.

GPT-4 launched in March 2023 at $60 per million output tokens. Today, Gemini 3.1 Flash costs $0.40 per million output. That is a 99.3% price reduction in three years. BenchLM.ai’s pricing index shows the average cost across frontier-class models has fallen 94.5% since April 2023. Most developers have absorbed this as a vague “things got cheaper” update. Very few have updated their architecture to match. The RAG Pipeline You Built in 2024 Might Be Dead Weight Retrieval-augmented generation existed, in large part, because stuffing data into context was expensive. At 2023 prices, loading a 500K-token corpus into every request was … The post