{"slug": "how-do-you-know-if-your-coding-agent-s-instructions-are-getting-better", "title": "How do you know if your coding agent's instructions are getting better?", "summary": "A developer has released Agent Harness Lab, an MIT-licensed open-source project that tests whether changes to coding-agent instructions actually cause improvements, rather than relying on before-and-after impressions. The tool, which supports Claude Code and Codex, stages candidate instruction changes, generates evaluation tasks, and compares baseline and candidate behavior; a lighter `ahl analyze` command produces a local report without running an agent or modifying project files. The developer says the `ahl improve` workflow is deliberately conservative and blocks automated behavioral adoption until independent certification requirements are met, framing the project as an experiment in making agent self-evaluation more inspectable.", "body_md": "A common way to improve a coding agent is to change its instructions.\n\nYou add a rule to AGENTS.md. You write a skill for a recurring task. You tell the agent to check something before editing code.\n\nThen the next run looks better.\n\nBut was the instruction responsible?\n\nThat's a harder question than it first appears.\n\nSuppose an agent repeatedly makes an error. You add a new instruction explaining how to avoid it.\n\nThe agent succeeds on the next attempt.\n\nThat result is encouraging, but it doesn't establish that the instruction caused the improvement. The task may have been easier, the agent may have taken a different route, or the original failure may not have been reproducible.\n\nA useful evaluation needs more than a before-and-after impression.\n\nFor changes to coding-agent instructions, I'd want to answer several questions:\n\nA smaller tool-call count is interesting, for example, but only if the agent still completes the required work.\n\nI'm working on an open-source project called Agent Harness Lab to explore this workflow.\n\nIt supports Claude Code and Codex.\n\nInstead of requiring you to describe a failure manually, it can inspect local project evidence and look for improvement candidates.\n\nThe experimental improvement process stages candidate changes, creates evaluation tasks, compares baseline and candidate behavior, and reports the outcome.\n\nThe project also provides a simpler entry point:\n\n```\nahl analyze /path/to/project\n```\n\nThis produces a local analysis report without running an agent or modifying project instructions.\n\nThe broader workflow is:\n\n```\nahl improve /path/to/project\n```\n\nThat process is deliberately conservative. The project does not currently claim established general performance gains, and automated behavioral adoption is blocked while independent certification requirements remain unmet.\n\nAn agent evaluating another agent raises difficult questions.\n\nCould the evaluator reward the wrong behavior? Could it overfit to the specific examples used during development? Could an apparent improvement come from a change in runtime conditions?\n\nThose aren't theoretical details that can be ignored when deciding whether to trust an automated repair.\n\nThe current implementation is an experiment in making that process more inspectable, rather than pretending those questions are solved.\n\nThe repository includes installation instructions, evaluation contracts, validation documentation, and the remaining safeguards needed before behavioral adoption can be enabled.\n\nGitHub: [https://github.com/mihazs/agent-harness-lab](https://github.com/mihazs/agent-harness-lab)\n\nIt's MIT-licensed and open to contributions.\n\nIf you maintain coding-agent instructions, I'd like to hear how you test them today. Especially when one instruction helps a particular task but makes another task worse.", "url": "https://wpnews.pro/news/how-do-you-know-if-your-coding-agent-s-instructions-are-getting-better", "canonical_source": "https://dev.to/mihazs/how-do-you-know-if-your-coding-agents-instructions-are-getting-better-3dhk", "published_at": "2026-10-31 20:45:40+00:00", "updated_at": "2026-10-31 20:52:13.988982+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops", "ai-safety"], "entities": ["Agent Harness Lab", "Claude Code", "Codex", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-do-you-know-if-your-coding-agent-s-instructions-are-getting-better", "markdown": "https://wpnews.pro/news/how-do-you-know-if-your-coding-agent-s-instructions-are-getting-better.md", "text": "https://wpnews.pro/news/how-do-you-know-if-your-coding-agent-s-instructions-are-getting-better.txt", "jsonld": "https://wpnews.pro/news/how-do-you-know-if-your-coding-agent-s-instructions-are-getting-better.jsonld"}}