I have been building Chaser Agent and other AI systems across the Chase ecosystem. As those systems became more capable, I needed a better way to inspect what an agent actually did—not just look at its final answer.
So I built Agent Review Studio.
It is an open-source, local-first system for evaluating and refining AI agents and agent harnesses.
An agent run can produce a final answer, source files, extracted claims, proposed actions, memory candidates and a complete execution trace. Looking only at the final response hides most of the engineering evidence.
I wanted one review flow that could answer practical questions:
Agent Review Studio lets an engineer:
The important part is what happens next. A reviewed failure can become a golden evaluation case or regression test. We can update a prompt, retrieval system, tool policy, memory rule or workflow, run the same task again and compare the result with the original baseline.
The launch demo uses a real Chaser Agent research run built around a public Cloudflare source. Agent Review Studio opens the run's files, places extracted claims beside the retained evidence and lets the operator classify what was correct, weak, missing or irrelevant.
The example labels shown in the video are an unsaved demonstration. The +17 result is an automated comparison, and human review is still pending. I kept those boundaries visible because an automated score should not be mistaken for human approval.
Chaser Agent is only one workspace. Fresh installations of the Studio contain no preloaded agent. Other engineers can name the system they are building, describe its objective and import their own datasets and runs.
This is agent evaluation, human labelling, evidence curation and harness refinement.
The Studio does not automatically change model weights. It creates the evaluation infrastructure and trusted improvement data needed to refine prompts, tools, retrieval, memory and orchestration. Reviewed examples can later become candidates for a separate, governed model fine-tuning pipeline.
Version 1.5.0 is open source under Apache-2.0.
I would value feedback from other people building agents: what evidence do you require before deciding that a new harness version is genuinely better?
Disclosure: I used AI assistance while preparing and editing this launch article. I reviewed the claims, product boundaries and final publication myself.