# How to improve agents: simulations, traces and evaluations

> Source: <https://www.agno.com/blog/3-ways-to-improve-agents>
> Published: 2026-10-30 16:21:24+00:00

[Blog](https://www.agno.com/blog)

# 3 ways to improve agents: simulations, traces and evaluations

[Ashpreet Bedi](https://github.com/ashpreetbedi)Founder & CEO

October 30, 20263 min read

Continuous improvement continues to be a buzz-word but we're seeing signs of systems improving themselves. We haven't connected all the dots yet, but I believe we have the primitives in place to make continuous improvement a reality. Today let's talk about how to improve agents, using coding agents.

In general, agent-improvement falls in three buckets:

1. **Simulations:** a coding agent will derive simulations and test cases based on the agent definition, and recursively improve your agent to match its defined purpose.
2. **Traces and conversations:** a coding agent scans usage data, identifies failure modes and improves the agent based on real-world usage.
3. **Evaluations:** the OG way of improving agents. You define a set of evals and then hill-climb against them.

No one approach is better than the other, in fact, these approaches work together. **Simulations help us explore agent behavior, traces help us understand real-world usage, and evaluations help us lock-in a baseline performance.**

Let's use a simple agent called "Daily Brief" as an example. It's job it to find relevant news, check sources, and write a short newsletter for the reader. That sounds straightforward, but what happens on a quiet news day? Or when five articles cover the same announcement? Does it repeat yesterday's news?

We'll skip the building the agent part in this post but if you'd like to build it yourself, follow [this guide](https://docs.agno.com/agent-platform/first-agent).

The question for this post is: once we have our agent, how do we improve it?

| Approach | Start with | Use it to | 
|---|---|---|
| **Simulations** | The agent's intended job and realistic user scenarios. | Discover weaknesses before you have production history. | 
| **Traces and sessions** | Actual sessions, tool calls, results, and user feedback. | Understand failures that happened in use. | 
| **Evaluations** | Repeatable cases with explicit expectations. | Compare changes and prevent regressions. | 

## Improve with simulations

Simulations explore how an agent behaves when people use it in different ways. To improve your agent using simulations, ask your coding agent to read the agent instructions and tools, construct realistic requests, run them, and judge the response. This approach is useful as soon as an agent exists, even before anyone has used it in production.

Agno's agent platform comes with an `/improve` skill that automates this process.

Start your coding agent in your agent platform and give it this prompt:

The user requests are simulated, but the model calls, tools, and storage is all real.

Simulations let you start improving an agent before you have users. Once people use it, you get a much richer set of examples.

## Improve from traces and sessions

No amount of simulations can replace real-world learnings from how users use your product. Users will ask things you didn't think to try. They'll give incomplete instructions, refer to an earlier conversation, and correct answers that looked perfectly reasonable to you. That's what traces and sessions provide.

Imagine someone tells Daily Brief:

This announcement is from last month. Why is it in today's edition?

There's no way a simulation would catch this.

If you're using Agno's agent platform, you can give it access to production data and run the `/improve` skill like:

The skill reads production sessions and traces for that agent and the coding agent uses the development environment to run tests against representative fixtures.

## Improve with evaluations

When you find a problem, write a test that will catch it if it happens again. You can also write tests for things the agent must get right from the start. These are your evaluations, or evals.

Start by saying what should happen. "Write a good brief" is hard to check. "Only include announcements from the requested period" is much clearer.

The Agno platform templates include a `/create-evals` skill to help you write these tests:

## Improving your agents

We'll often use all three approaches on the same problem. For Daily Brief, a reader pointing out an old story gives us something to investigate. Once we've found and fixed the problem, we can save that example as an eval and try a few related situations to see whether the fix holds up.

Over time, our job is to collect useful examples of how our agents behave. Keep the conversations, tool calls, and results from both successful runs and failures, along with the agent version and settings. When we make a change, we can go back to those examples and check whether things actually got better.

Coding agents can do much of the work of investigating and testing changes. Your job is to be clear about what the agent should do, review the evidence, and decide which changes to ship.

Ashpreet