Agent quality must be measured during development against the cases you wrote, and after launch against the tasks the agent actually performed. Agent and model evaluations in Agent Platform are now generally available, letting you measure and compare agents and models at both stages, on one engine with consistent metrics. When you use consistent quality scoring on local experiments and live traffic, a drift in production points to a problem with the agent rather than with the way it was measured.
You can reach evaluations from the Agent Platform SDK, agents-cli, the Evaluation section in Agent Platform Google Cloud console, and directly from ADK.
The primary unit of work in evaluation is the experiment, a dataset of eval cases and a set of metrics and scores to run and review as you iterate. The evaluation service comes with a flexible UI to define metrics, kick off new runs, and review the results, all the way down to a single failure, where you can open the agent's full trace and session log to see exactly what happened.
You can run experiments locally for fast iterations, or, if your agent is already deployed on Agent Platform with telemetry enabled, you can grade existing sessions and traces, or run the agent with the user simulator enabled to create traces before your users produce them. Every experiment artifact is stored transparently in Cloud Storage, so you can version and audit your runs for compliance and posterity.
For large evaluation jobs, the system supports [ issue clustering](https://docs.cloud.google.com/gemini-enterprise-agent-platform/optimize/evaluation/view-results#failure-analysis): it groups eval failures into interpretable, actionable clusters against your own
More than 20 [ pre-built metrics](https://docs.cloud.google.com/gemini-enterprise-agent-platform/optimize/evaluation/manage-metrics) ship with the service.
Beyond those, an adaptive rubric is an advanced LLM-judge metric workflow co-developed with our research partners at Google DeepMind. It creates case-specific
We designed and calibrated variants for what you usually want to know about an agent. Task Success grades goal fulfillment across a conversation from observable outcomes and confirmations in the agent's responses. Tool Use Quality evaluates tool selection, argument correctness, and schema compliance. Safety scores the response against content policies spanning hate speech, harassment, dangerous content, sexually explicit material, and PII, returning the policies violated. Trajectory Quality, Final Response Quality, Hallucination, Grounding, and image and video quality metrics are also available. Natural-language guidelines steer rubrics generation toward criteria you specify, with the resulting rubric group reviewable and reusable across agents.
When the pre-built metrics don't fit, you can bring your own. A code-based metric is a Python function, covering exact text matches, JSON-shape checks, and anything else expressible in code. An
However a metric is defined, it lands in the same versioned, org-wide registry, running unchanged in offline experiments and online monitors. We're actively expanding coverage to more specialized tasks and input modalities, and we welcome your feedback.
An agent that's green across your whole test suite can still drift a week after launch, when faced with inputs nobody wrote a case for. That's not a testing failure so much as a fact of production where real tasks are bigger and stranger than any suite you can write up front.
The metrics you built when developing your agent should keep running after your agent launches. If you already collect traces and sessions through Cloud Trace, you can evaluate them directly with just one click in the traces UI. Better still is to set up an online monitor to grade the live traffic as it arrives. To avoid grading every request and keep costs manageable, you can use the built-in sampling and targeted filters. When monitors are active you can see scores over time in built-in dashboards and even set up drift alerts for email, Slack, or other channels.
Every evaluation needs test cases to run against, and writing them all by hand is slow and tends to miss the non-obvious ones. The evaluation service can generate them for you, with a case generator, a user simulator, and an environment simulator.
The case generator seeds synthetic eval cases from the agent's instructions and tools, so you're not starting from a blank page or capping your coverage at what you thought to type out. For ADK agents, it can pair with a user simulator to reach the scenarios that are hard to produce by hand: you define a persona and a short conversation plan, and the simulator plays that user across a full multi-turn exchange, so you can evaluate real back-and-forth without scripting each reply.
The environment simulator stands in for the systems your agent calls. Point it at a tool, give it the response you want — mocked data, a forced error, added latency — and it intercepts that call during the run, so you can test how the agent handles a failing or slow backend without touching production.
Agent and model evaluations are generally available. For supported regions and enterprise security features see regions and security features table. You pay standard rates for the model calls behind LLM-as-a-judge and the other model-based metrics – plus Cloud Storage for the artifacts a server-side run keeps. Code-based and computation metrics add no additional cost. Your datasets and traces always stay in your project.
Evaluation meets you in whatever tool you already build agents with. It's part of the Agent Platform SDK, with the same operations available over the
If you work from the command line, agents-cli makes eval a first-class command next to the ones you already use to deploy agents and inspect telemetry (ADK-Python agents today). You can even run the loop straight from your coding agent: a reusable skill walks Claude Code and similar tools through We can't wait to see what you ship.