How LangChain's Managed Deep Agents let production agents log mistakes, escalate uncertainty, and rewrite their own rules over time.
What is harness engineering for AI agents? #
Harness engineering means improving everything around the model, not the model itself. That includes the system prompt, the sub-agents it delegates to, the tools it calls, and the rules it follows. For production agents, harness engineering specifically means tracking the mistakes an agent makes with real users, then retroactively updating its rules and skills so those mistakes stop recurring. The technique originated in coding agent workflows and maps almost directly onto customer-facing or business automation agents, because both are now built the same way: with markdown files instead of hardcoded logic.
TL;DR #
- Markdown-based harnesses have replaced a lot of hand-coded agent plumbing, with rules, skills, and sub-agent definitions stored as plain markdown files instead of Python or TypeScript functions.
- LangChain’s Managed Deep Agents bundles this harness setup (instructions, skills, guardrails, sub-agents) and handles deployment, with an agent definition file that can be as short as about 30 lines of code.
- Production agents benefit from two improvement loops : a memory loop that captures per-user and shared lessons, and a harness loop that analyzes production traces and suggests fixes to core instructions.
- When an agent is uncertain , it should still attempt an answer but flag that uncertainty and escalate the case to a human inbox for correction instead of silently guessing.
- Corrections made by a human get written back as memories , both a private memory for that specific user and a shared “agents.md” style file that benefits every future conversation.
- The system can generalize from lessons , applying a learned fix to a similar but non-identical problem, rather than only matching exact repeat questions.
- LangSmith traces provide the evidence trail for the second loop: proposed fixes to the harness come with the exact conversation evidence that triggered them.
Remy is new. The platform isn't. #
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
How does this differ from the old way of building agents? #
A couple of years ago, building a production agent meant writing a lot of code: custom functions for every tool, manually wiring guardrails, hardcoding the system prompt into the application logic. That approach worked but made agents brittle and slow to improve, since every change required a code deployment.
The shift described here treats production agents like coding agents such as Claude Code or Codex. Instructions live in a single markdown file (instructions.md) at the root of the agent folder, similar to a claude.md or agents.md file. Skills live in a skills folder as markdown workflows. Rules, sub-agents, MCP tool connections, and hooks for guardrails are all defined declaratively rather than imperatively. The result is an agent definition file that can be dramatically shorter, in the demo shown, around 30 lines of Python, because the platform (LangChain’s Managed Deep Agents, running on LangSmith) stitches the markdown files together automatically at runtime.
This matters for anyone building agents because it means non-engineers, or engineers without deep framework knowledge, can iterate on agent behavior by editing text files instead of touching application code.
What are the core components of a markdown-based harness? #
The mapping between coding agent concepts and production agent concepts is close to one-to-one:
- Rules function as the system prompt, defining the agent’s core behavior.
- Sub-agents handle delegation, equivalent to multi-agent workflows.
- MCP tools provide external connections, the same role tools have always played.
- Skills replace a lot of traditional hardcoded tools with markdown-defined workflows.
- Hooks serve as guardrails, monitoring or blocking actions at the start or end of a conversation, or around individual tool calls.
Because each of these is a file or a small set of files, updating agent behavior often means editing a markdown document rather than redeploying code. That flexibility is what makes the two improvement loops practical at all.
How does the memory loop work? #
The memory loop is about letting an agent learn from the situations where it struggles. The pattern works like this:
- A user interacts with the agent and asks something the agent isn’t confident about.
- Instead of either blocking entirely or confidently guessing, the agent gives its best attempt and explicitly flags its uncertainty.
- That uncertain case gets escalated to a human-managed inbox.
- A human reviews the case and supplies the correct answer.
- The agent writes that correction down as a lesson, updating both a private, per-user memory and a shared memory file used across all users.
In a demonstrated example built around a support help desk agent, a user asked a technical question the agent couldn’t confidently resolve. The agent answered but flagged uncertainty, which pushed the case into a team inbox. A human supplied the fix, confirmed it in the interface, and within seconds the agent updated two things: the specific user’s private memory and a shared “cases” memory file equivalent to an internal agents.md built from accumulated lessons.
The payoff showed up when a second, different user asked a related but not identical question. Rather than needing an exact match to the first case, the agent generalized from the stored lesson and answered confidently and correctly on the first try, with no escalation needed. LangChain’s platform stores these as per-user memories (private, with guardrails ensuring no cross-user visibility) and shared “cases” entries inside a context hub, so retrieval stays targeted instead of dumping every past lesson into the model’s context window.
The practical effect for teams running these agents: escalation volume should trend downward over time as the knowledge base of handled cases grows, which keeps the human review burden manageable rather than growing in step with user volume.
How does the harness-improvement loop work? #
The memory loop fixes individual cases. The second loop operates one level up, improving the core instructions and skills that govern the agent’s overall behavior.
This loop runs on top of LangSmith trace data. Every production conversation generates a trace, and the platform’s engine reviews these traces to identify failure patterns. When it finds one, it surfaces:
- A description of the failure.
- The evidence, linked directly to the LangSmith trace of the actual conversation where it happened.
- A proposed fix to the harness, suggesting a specific change to instructions or skills that would prevent the failure from recurring.
This turns raw production traffic into structured harness-improvement suggestions instead of leaving someone to manually comb through logs. The person managing the agent can inspect the exact trace behind each suggestion before deciding whether to apply the fix.
Is this approach worth adopting for production agents? #
For teams already running agents in front of real users, the case is straightforward: production traffic is the best source of information about where an agent’s harness is incomplete, and most teams don’t systematically capture it. Building an inbox for escalations, a memory store for corrections, and a feedback loop from traces to instruction updates is possible to build independently. Doing it manually means standing up trace logging, a review queue, memory storage with user-level isolation, and some process for turning patterns into instruction edits.
Using a managed platform like LangChain’s Managed Deep Agents collapses that infrastructure into an existing system: LangSmith handles tracing, the platform handles memory storage and retrieval, and the suggestion engine turns trace analysis into actionable harness edits. The tradeoff is the usual one with managed platforms: less infrastructure to build, more dependency on the platform’s conventions. For teams that want full control over storage and retrieval logic, a self-managed version of the same two loops (escalation inbox, memory files, trace-based suggestions) is achievable with any agent framework, since the underlying pattern isn’t platform-specific.
Frequently Asked Questions #
What is a “harness” in the context of AI agents?
The harness is everything around the core language model: system prompts, rules, skills, sub-agent configurations, tool connections, and guardrails. Harness engineering is the practice of improving these components based on observed failures, rather than only trying to improve the underlying model.
What’s the difference between the memory loop and the harness-improvement loop?
The memory loop captures and resolves individual cases where the agent was uncertain, storing both private per-user lessons and shared cases. The harness-improvement loop operates on aggregated production traces to propose changes to the agent’s core instructions and skills, fixing systemic issues rather than one-off cases.
Does the agent block and wait for a human when it’s uncertain?
#
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
No. The pattern described has the agent still attempt an answer even when uncertain, while simultaneously flagging that uncertainty and escalating the case to a human review inbox. This keeps the agent useful in the moment while still capturing the gap for correction.
Can lessons learned from one user’s issue help a different user?
Yes. Shared memories (distinct from private per-user memories) get written into a common file that any future conversation can draw on. In a demonstrated example, a lesson learned from one user’s specific problem let the agent confidently answer a second user’s related but non-identical question without escalation.
Do I need LangChain’s platform to build this kind of self-improving agent?
No. The two-loop pattern, escalation-based memory plus trace-based harness suggestions, is a general strategy that can be implemented with any agent framework. Managed platforms like LangChain’s Managed Deep Agents provide built-in tracing (via LangSmith), memory storage, and automated improvement suggestions, which reduces the amount of custom infrastructure a team needs to build and maintain.