We've all been there: you paste an error, then three files, then a fourth file because the AI is hallucinating a function that exists elsewhere, then you spend five minutes explaining your API structure. By the time the LLM actually processes your request, you've burned thousands of tokens on "onboarding" the AI to a codebase it should already understand.
Massive context windows (like 1M+ tokens) are a bit of a distraction. Dumping an entire repo into a prompt is like forcing a senior dev to read every single line of a legacy project before they're allowed to fix a typo in a CSS file. It's possible, but it's wildly inefficient and often leads to the model getting distracted by irrelevant boilerplate.
We're moving from prompt engineering to context engineering. The real win isn't writing a "perfect" prompt; it's curating the exact slice of the codebase the AI needs.
The Efficiency Gap #
A high-performance AI workflow should look like this:
Input:"Fix the auth timeout bug."** Automated Context Retrieval:**The tool identifies the specific middleware, the token validation logic, the user model, and the recent git diffs related to auth.Result: A lean, high-signal prompt that costs less and returns a more accurate fix.
When you strip away the noise, latency drops and the "hallucination rate" usually goes down because the model isn't drowning in 50 irrelevant files.
The Real-World Impact #
If you're doing a deep dive into your own AI workflow, notice how often you're manually attaching files. That's a failure of context engineering. Over-providing context leads to: Higher Latency: Longer prompts take longer to process.Increased Cost: Token spend adds up fast in large projects.Diluted Focus: The "lost in the middle" phenomenon where LLMs ignore the center of a massive prompt.
The next generation of LLM agents won't just be about who has the smartest model, but who has the smartest retrieval system. The tools that win will be the ones that know exactly what to exclude.
If you want to optimize your current setup, start being aggressive about what you feed the AI. Treat your context window like precious real estate.
[Next NotchTune: Controlling Spotify Web via macOS Notch →](/en/threads/3063/)