{"slug": "how-to-improve-agents-simulations-traces-and-evaluations", "title": "How to improve agents: simulations, traces and evaluations", "summary": "Agno founder and CEO Ashpreet Bedi outlined three methods for improving AI agents in an October 30, 2026 blog post: simulations, traces and sessions, and evaluations. Bedi wrote that simulations let developers derive test cases from an agent's definition before any production usage, traces and sessions surface failure modes from real user interactions, and evaluations lock in baseline performance to prevent regressions, with Agno's agent platform offering an /improve skill that automates the simulation and trace-review workflows. Bedi stated that \"simulations help us explore agent behavior, traces help us understand real-world usage, and evaluations help us lock-in a baseline performance.", "body_md": "[Blog](https://www.agno.com/blog)\n\n# 3 ways to improve agents: simulations, traces and evaluations\n\n[Ashpreet Bedi](https://github.com/ashpreetbedi)Founder & CEO\n\nOctober 30, 20263 min read\n\nContinuous improvement continues to be a buzz-word but we're seeing signs of systems improving themselves. We haven't connected all the dots yet, but I believe we have the primitives in place to make continuous improvement a reality. Today let's talk about how to improve agents, using coding agents.\n\nIn general, agent-improvement falls in three buckets:\n\n1. **Simulations:** a coding agent will derive simulations and test cases based on the agent definition, and recursively improve your agent to match its defined purpose.\n2. **Traces and conversations:** a coding agent scans usage data, identifies failure modes and improves the agent based on real-world usage.\n3. **Evaluations:** the OG way of improving agents. You define a set of evals and then hill-climb against them.\n\nNo one approach is better than the other, in fact, these approaches work together. **Simulations help us explore agent behavior, traces help us understand real-world usage, and evaluations help us lock-in a baseline performance.**\n\nLet's use a simple agent called \"Daily Brief\" as an example. It's job it to find relevant news, check sources, and write a short newsletter for the reader. That sounds straightforward, but what happens on a quiet news day? Or when five articles cover the same announcement? Does it repeat yesterday's news?\n\nWe'll skip the building the agent part in this post but if you'd like to build it yourself, follow [this guide](https://docs.agno.com/agent-platform/first-agent).\n\nThe question for this post is: once we have our agent, how do we improve it?\n\n| Approach | Start with | Use it to | \n|---|---|---|\n| **Simulations** | The agent's intended job and realistic user scenarios. | Discover weaknesses before you have production history. | \n| **Traces and sessions** | Actual sessions, tool calls, results, and user feedback. | Understand failures that happened in use. | \n| **Evaluations** | Repeatable cases with explicit expectations. | Compare changes and prevent regressions. | \n\n## Improve with simulations\n\nSimulations explore how an agent behaves when people use it in different ways. To improve your agent using simulations, ask your coding agent to read the agent instructions and tools, construct realistic requests, run them, and judge the response. This approach is useful as soon as an agent exists, even before anyone has used it in production.\n\nAgno's agent platform comes with an `/improve` skill that automates this process.\n\nStart your coding agent in your agent platform and give it this prompt:\n\nThe user requests are simulated, but the model calls, tools, and storage is all real.\n\nSimulations let you start improving an agent before you have users. Once people use it, you get a much richer set of examples.\n\n## Improve from traces and sessions\n\nNo amount of simulations can replace real-world learnings from how users use your product. Users will ask things you didn't think to try. They'll give incomplete instructions, refer to an earlier conversation, and correct answers that looked perfectly reasonable to you. That's what traces and sessions provide.\n\nImagine someone tells Daily Brief:\n\nThis announcement is from last month. Why is it in today's edition?\n\nThere's no way a simulation would catch this.\n\nIf you're using Agno's agent platform, you can give it access to production data and run the `/improve` skill like:\n\nThe skill reads production sessions and traces for that agent and the coding agent uses the development environment to run tests against representative fixtures.\n\n## Improve with evaluations\n\nWhen you find a problem, write a test that will catch it if it happens again. You can also write tests for things the agent must get right from the start. These are your evaluations, or evals.\n\nStart by saying what should happen. \"Write a good brief\" is hard to check. \"Only include announcements from the requested period\" is much clearer.\n\nThe Agno platform templates include a `/create-evals` skill to help you write these tests:\n\n## Improving your agents\n\nWe'll often use all three approaches on the same problem. For Daily Brief, a reader pointing out an old story gives us something to investigate. Once we've found and fixed the problem, we can save that example as an eval and try a few related situations to see whether the fix holds up.\n\nOver time, our job is to collect useful examples of how our agents behave. Keep the conversations, tool calls, and results from both successful runs and failures, along with the agent version and settings. When we make a change, we can go back to those examples and check whether things actually got better.\n\nCoding agents can do much of the work of investigating and testing changes. Your job is to be clear about what the agent should do, review the evidence, and decide which changes to ship.\n\nAshpreet", "url": "https://wpnews.pro/news/how-to-improve-agents-simulations-traces-and-evaluations", "canonical_source": "https://www.agno.com/blog/3-ways-to-improve-agents", "published_at": "2026-10-30 16:21:24+00:00", "updated_at": "2026-10-30 16:47:14.367652+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-products"], "entities": ["Agno", "Ashpreet Bedi", "Daily Brief"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-improve-agents-simulations-traces-and-evaluations", "markdown": "https://wpnews.pro/news/how-to-improve-agents-simulations-traces-and-evaluations.md", "text": "https://wpnews.pro/news/how-to-improve-agents-simulations-traces-and-evaluations.txt", "jsonld": "https://wpnews.pro/news/how-to-improve-agents-simulations-traces-and-evaluations.jsonld"}}