Good morning, AI enthusiasts!
We recently gave our Context Engineering in 2026 workshop at the AI Engineer World’s Fair, using our production AI tutor to test what actually happens as an agent’s context grows. We spent close to $590 on the Gemini experiments alone, and some of the results contradicted the defaults we had already shipped.
Everything is open source. You can inspect the full tutor codebase and eval harness, run the live experiment demo, see every result, or go through the workshop slides and watch the recording.
If you’re managing context in an agent today, these are the findings I would start with: 1. Measure what is filling the window before trying to shrink it. In our tutor, chat history wasn’t the main problem. Old retrieval and tool outputs were. Retrieval alone could contribute up to 100k tokens, pushing some turns toward 200k. Optimizing conversation history first would have attacked the wrong part of the context.
2. Check your cache economics before summarizing anything. Our production compaction setup sent roughly 41% fewer tokens than keeping the complete history, and still cost about twice as much. Keeping history allowed around 87% of input to receive the cache discount; summarization rewrote that prefix and lost it. On our traces, keep-everything remained cheaper while cached input cost less than roughly
3. Shrink before you rewrite. Capping tool outputs at a stable size cut cost per turn by
4. Test the capability you cannot afford to lose. Our compacted tutor could score
5. Let the constraint choose the technique. If the window doesn’t fit, retrieve instead of stuffing everything in. If cost is the problem, inspect cached-input pricing or switch models before adding summarization. If quality is deteriorating, measure
And keep the simple retrieval tools around. Dense retrieval dropped to 0% recall at a 400k-token corpus in one buried-fact test while BM25 remained at 100%. We also tested GraphRAG against ordinary hybrid retrieval and achieved the correct source 100% of the time with both methods, while GraphRAG consumed 61% more input tokens and cost 44% more per turn. More sophisticated did not mean better for our data.
Our production setup is now much simpler: DeepSeek V4 Flash, hybrid retrieval, full history, stable caps on tool outputs, no tool-output clearing, and summarization pushed so far out that it rarely fires.
In the full article, I go much deeper into the experiments, eval harness, local-model tests, context rot, retrieval, caching, and the things we built that ultimately weren’t worth shipping.
The principle to remember is to name the constraint before choosing the context strategy. Window, cost, and quality degradation are different problems, and the same optimization will not solve all three.
You can read the full article, watch the workshop, or go directly to the code, live experiments, and slides if you want to dig into the results yourself.
The tutor from our Context Engineering workshop is the same one we build in the Full Stack AI Engineering course. While updating that course, we found another retrieval issue: hybrid search can still lose an exact match if you merge results too early.
Semantic search is good at matching meaning. Keyword search is better for exact strings such as product IDs, error messages, names, and code symbols.
Say a user searches for order #8821. Keyword search may put the exact page first, while semantic search returns several broader pages about orders. If you combine both lists immediately and keep only the highest-ranked results, those broader semantic matches can push out the exact result.
Our fix is simple: keep the strongest candidates from each retriever before combining them. We take the top five keyword results and the top five semantic results, then merge and deduplicate them.
This also aligns with one of the workshop findings: in our buried-fact tests, dense retrieval eventually failed to retrieve distinctive terms, while BM25 kept finding them.
The practical takeaway: hybrid search only works if your fusion strategy preserves the different strengths of both retrievers.
If exact identifiers matter in your application, reserve part of the final context for strong keyword matches. — Louis-François Bouchard, Towards AI Co-founder & Head of Community
NVIDIA’s NOOA Proves the Harness Matters More Than the Model, and Everyone’s Watching the Wrong Number by allglenn
NVIDIA’s NOOA reached 82.2% on SWE-bench Verified with GPT-5.5 while using about half the tokens of comparable harnesses. The article explains how much of that gain comes from the surrounding system rather than the model itself. NOOA passes tool outputs by reference rather than repeatedly inserting them into the prompt, which reduces context growth and preserves prompt-cache hits. It also breaks down six harness capabilities NVIDIA considers important, including typed contracts and model-controlled context management.
This article gives a clear structure for understanding the different layers of an agent system. It separates prompt and context engineering from the agent loop, harness, graph structure, and AgentOps, then shows how they fit together. It is especially useful if you are trying to decide whether a problem should be solved with better context, a different workflow, stronger execution controls, or more observability.
- Agentic Analytics with Power BI and Microsoft Fabric: The AI-First Way to Build Enterprise BI by Dave R
This piece shows how coding agents can work directly with Microsoft Fabric and Power BI rather than only generating code around them. It covers agents creating lakehouses, working with semantic models, editing Power BI project files, and scaffolding applications, all while operating within existing governance and security controls. The main value is seeing how agentic workflows can fit into an enterprise data stack without bypassing the systems already in place.
This article separates long context from persistent memory and explains when each is useful. It maps memory across working state, caches, persistent stores, and model weights, then shows how systems such as Redis, vector databases, and graph stores can support retrieval across sessions. It also covers practical risks, including stale memories, conflicting information, poisoning, and privacy concerns, which become important once an agent is expected to remember beyond a single conversation.
- Skills, Hooks and Subagents: Optimizing Claude Code Around What Stays in the Window By Utkarsh Mittal
This article looks at Claude Code’s features through the lens of context cost. It compares instruction files, skills, hooks, subagents, and agent teams, showing how the same repository and model can produce up to a sevenfold difference in cost depending on what remains loaded. The practical takeaway is how to decide what should stay permanently in context, what should load only when needed, and what should be moved into a separate agent or enforced through a hook.
If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards. LAI #139: Fewer Tokens Cost Us More was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.