A common way to improve a coding agent is to change its instructions.
You add a rule to AGENTS.md. You write a skill for a recurring task. You tell the agent to check something before editing code.
Then the next run looks better.
But was the instruction responsible?
That's a harder question than it first appears.
Suppose an agent repeatedly makes an error. You add a new instruction explaining how to avoid it.
The agent succeeds on the next attempt.
That result is encouraging, but it doesn't establish that the instruction caused the improvement. The task may have been easier, the agent may have taken a different route, or the original failure may not have been reproducible.
A useful evaluation needs more than a before-and-after impression.
For changes to coding-agent instructions, I'd want to answer several questions:
A smaller tool-call count is interesting, for example, but only if the agent still completes the required work.
I'm working on an open-source project called Agent Harness Lab to explore this workflow.
It supports Claude Code and Codex.
Instead of requiring you to describe a failure manually, it can inspect local project evidence and look for improvement candidates.
The experimental improvement process stages candidate changes, creates evaluation tasks, compares baseline and candidate behavior, and reports the outcome.
The project also provides a simpler entry point:
ahl analyze /path/to/project
This produces a local analysis report without running an agent or modifying project instructions.
The broader workflow is:
ahl improve /path/to/project
That process is deliberately conservative. The project does not currently claim established general performance gains, and automated behavioral adoption is blocked while independent certification requirements remain unmet.
An agent evaluating another agent raises difficult questions.
Could the evaluator reward the wrong behavior? Could it overfit to the specific examples used during development? Could an apparent improvement come from a change in runtime conditions?
Those aren't theoretical details that can be ignored when deciding whether to trust an automated repair.
The current implementation is an experiment in making that process more inspectable, rather than pretending those questions are solved.
The repository includes installation instructions, evaluation contracts, validation documentation, and the remaining safeguards needed before behavioral adoption can be enabled.
GitHub: https://github.com/mihazs/agent-harness-lab
It's MIT-licensed and open to contributions.
If you maintain coding-agent instructions, I'd like to hear how you test them today. Especially when one instruction helps a particular task but makes another task worse.