Artificial Intelligence #
When production AI agents handle tasks such as data analysis and report generation, context management can become a major bottleneck. A 200K-token window sounds generous until a real-world task burns through most of it in minutes. Data analysis and report generation leave the agent failing mid-execution, losing accuracy, or driving up costs. A body of published guidance already addresses this, but the individual techniques are easier to read about than to apply. Applied together, they make a measurable difference: in the reference implementation this post walks through, a single write-and-execute tool cuts the tokens each script adds to context by about 95 percent, and skills on demand shrinks the system prompt by about 97 percent.
This series organizes that guidance into practices you can apply directly. Anthropic’s engineering posts, together with features built into the Strands Agents SDK, converge on four layers of context engineering: isolate context at the architecture level, control it through prompts, externalize it to the filesystem, and verify it with independent agents and safeguards. In this post (Part 2), you will learn how each layer works and what it looks like in working code. We applied these techniques in Deep Insight, an open source multi-agent data analysis solution built on the Strands Agents SDK and Amazon Bedrock AgentCore, which serves as the reference implementation throughout.
Post series #
- Part 1: 5 production challenges in multi-agent systems and how Deep Insight solved them.
- Part 2: Four layers of context engineering for production AI agents: How Deep Insight applies them in working code.
- Part 3: Harness engineering in practice: moving Deep Insight from dev to production. (Coming soon!)
The first post defined five core challenges that production multi-agent systems must solve. It introduced how Strands Agents SDK, Amazon Bedrock and Bedrock AgentCore, and AWS Fargate each address different layers of the problem, with code examples. This second post takes the four layers of context engineering practice that sit on top of that architecture and shows each one implemented. The third post guides you through deploying and operating agents in a secure private network on Amazon Virtual Private Cloud (Amazon VPC), covering everything needed for real-world operations.
What is context engineering? #
Context engineering is the discipline of optimizing the information in a large language model (LLM)’s limited context window so agents perform at their best. Anthropic Claude Sonnet 4.5, available on Amazon Bedrock, offers a 200K context window, but complex production tasks quickly exceed it. For example, a seemingly straightforward request like “Analyze sales data and create a DOCX report” runs through several stages: data and exploration (5 min), category-based sales analysis (3 min), generating five charts for time-series analysis (10 min), statistical validation (3 min), and final report generation (5 min). Together they consume over 150K tokens across 15 minutes.
Processing such tasks in a single context causes problems: the window overflows mid-task, stale context degrades performance, and errors require re-running the whole pipeline. As context grows, attention to important information drops, and inefficiency raises cost under per-token pricing.
“As the number of tokens in the context window increases, the model’s ability to accurately recall information from that context decreases (Context rot).”
— Anthropic, “Effective Context Engineering for AI Agents” (2025.09) Context engineering solves exactly these problems, and it is not only a collection of techniques. Spanning architecture design, prompt writing, tool implementation, and verification, it’s a layered discipline where each layer compensates for what the previous ones miss. This post details the four-layer approach you can apply to manage context efficiently.
How to handle tasks that exceed the context window #
Anthropic has published several techniques for handling long-horizon tasks. Deep Insight groups these techniques into four layers, with each layer addressing the limitations of the others. The following table sets out the four layers.
| Layer | Practice | Core Principle | Limitations | | Layer 1: Architecture | Isolating Context Through Multi-Agent Architecture | Each agent operates in an independent context | When isolation alone is not enough | | Layer 2: Prompt | Output Control and Behavioral Rules | Limiting context intake through prompts | When prompts alone are not enough | | Layer 3: Tools | File-based Context Externalization | Keeping heavy content in files outside the context | When issues still arise | | Layer 4: Verification Agents and Safeguards | Independent Verification and Safeguards | Catching errors and preventing context overflow | Final line of defense |
The sections that follow take each layer in order, stating what the layer prescribes, why the preceding layers alone are insufficient, and how the practice is implemented in Deep Insight.
1. Isolating context through multi-agent architecture #
Give each specialized agent its own context window and pass only compressed results to the orchestrator. Deep Insight does this with a hierarchy of Coordinator, Planner, Plan Reviewer, Supervisor, and Tool Agents (Coder, Validator, Reporter, Tracker, Auditor).
With a single agent, all information, from user requests through data exploration, analysis, chart generation, validation, and reporting, accumulates in one context and easily exceeds the window. Giving each agent its own context keeps even the Coder, which uses the most, under 25K tokens total. Each agent receives only what it needs and signals others with a completion notice and brief summary.
The key to isolation is minimizing shared information. Deep Insight uses a Python dictionary, shared_state, that stores only four things:
messages: The previous agent’s final message (not the full conversation history).clues: Compressed summaries collected from each agent.full_plan: A checklist generated by Planner and updated by Tracker.history: A brief log of agent names and messages.
The clues use a fixed CLUES_FORMAT: instead of each agent’s full conversation (tens of thousands of tokens), only compressed core results are passed.
This pattern is the one recommended in Anthropic’s “How We Built Our Multi-Agent Research System” post, which describes an orchestrator-worker pattern in which a lead agent delegates to specialized subagents, with each agent working only within its own context. You do not need to build it yourself: the agents-as-tools pattern in the Strands Agents SDK provides this isolation at the framework level.
2. Controlling context inflow through prompts #
Layer 1 isolation alone is not sufficient: even with independent contexts, an agent that generates long responses inflates the Supervisor’s context. Layer 2 limits each agent’s output through prompts.
For Deep Insight, we use prompts to limit how much information each agent returns to the Supervisor. Three rules do most of the work, as Deep Insight’s prompts show. “Start with the strongest model and a simple prompt. Then incrementally add specific instructions based on actually observed failure cases.”
— Anthropic, “Effective Context Engineering for AI Agents” (2025.09)
Implementation detail 2-1: Explicit per-agent output token budgets
Each agent’s prompt specifies a maximum response token count, preventing long responses that pollute the Supervisor’s context.
| Agent | Output Token Budget | Reason |
| Coder | 1,000–1,500 tokens | Analysis code is saved to files, so the agent returns only a summary of state, completed tasks, and insights |
| Validator | 800 tokens | Validation results are saved to citations.json . The agent returns only a summary |
| Reporter | 1,000 tokens | The agent generates the report as a DOCX file and returns only a summary of the result |
Anthropic’s ” Writing Effective Tools for Agents” post suggests giving tools a CONCISE or DETAILED response option for context efficiency. Apply the same idea one level up, in each agent’s system prompt. In Deep Insight, all agents share one standardized format:
Implementation detail 2-2: Self-contained code philosophy
The Coder agent’s prompt explicitly states “variables do not persist between scripts.” Every script must include all necessary imports and load data directly. The prompt provides both incorrect and correct examples:
The reason this rule relates to context engineering is that referencing variables from previous scripts requires the agent to “remember” prior conversations, which means past code must remain in context. The self-contained principle means past code can be safely removed from context.
Implementation detail 2-3: Supervisor’s section completion rule
The Supervisor’s prompt includes the rule “Do not move to the next agent until all checklist items in the current agent’s section are marked [x].” We express this rule in the prompt because deciding whether the Coder’s work is complete requires the Supervisor to interpret the plan using information from the Tracker’s updates. This judgment is difficult to express in code alone.
3. “Only pointers in context, actual data in files” #
Layer 1 isolates context between agents, and Layer 2 limits output within agents. But when the Coder agent generates 100 lines of Python code, that code itself enters the context. Even with “respond briefly” prompt instructions, some code can’t be written briefly. Layer 3 uses tool design to move heavy content outside the context, into the filesystem. Four techniques carry most of the benefit, as the following implementation details show.
Implementation detail 3-1: Moving code outside context with Write and Execute Tool
The Coder reads the analysis plan, writes Python, and executes it to generate visualizations. But when the code gets long, the conventional approach (separate file-write and execute tools) adds 100 lines directly to context per script, accumulating over 1,000 lines.
The fix is to collapse the two steps into a single tool, so the code never enters context as a message. Deep Insight’s Coder agent uses one write_and_execute_tool for this.
Here’s part of the write_and_execute_tool. After writing and executing the file, it returns only a summary: “Wrote 105 lines, execution successful, completed analysis of five categories.” The code stays in the file and about three lines of summary remain in context. Based on this example, that saves about 95 percent of the tokens.
“Server-side filtering: filtering data at the execution layer before it reaches context, 98.7% token reduction (150K to 2K).”
— Anthropic, “Code Execution with MCP” (2025.11)
Implementation detail 3-2: Reusing common logic as modules in the Coder agent
Common modules get reused across analyses. Rather than regenerating them each time, define frequently-used functions as a module once and import them, which prevents token waste and context duplication.
Without this, defining functions like to_python_type() or track_calculation() in every analysis produces 200–300 lines of duplicate code across 10 analyses. Deep Insight generates a module called coder_analysis_utils.py before the first task. It includes functions for converting numpy types, tracking calculation metadata, and saving to JSON. Subsequent analyses import it, replacing 20–30 regenerated lines with two import lines, which saves tokens and improves consistency and debugging.
Implementation detail 3-3: Saving and sharing analysis results through structured note-taking
We use structured note-taking to share detailed analysis results without passing them through each agent’s context. Deep Insight stores these results in a file called all_results.txt, which is then read by agents as needed.
For example, the Coder agent appends results to all_results.txt after each analysis task: