# Google AQuA Is Open Source: Catch Silent AI Agent Failures in Production > Source: > Published: 2026-11-14 02:07:15+00:00 Your health check is green. Latency looks normal. Every request returned 200 OK. And somewhere in your production environment, your AI agent has been quietly skipping tool calls for three days. Nobody noticed — because there was no error to notice. This is the class of problem Google built AQuA to surface. Released open-source on November 12 as part of the [google/adk-recipes](https://github.com/google/adk-recipes) repository, AQuA (Ambient Quality Agent) is a monitoring agent that runs beside your production agent in Google Cloud, continuously evaluating quality in ways that traditional observability infrastructure simply cannot. It is not a tracing tool. It is not a log aggregator. It is an agent whose job is to watch your agent. ## The Gap Standard Monitoring Leaves Open Infrastructure monitoring and quality monitoring solve different problems. Error rates and latency tell you whether your agent is *running*. They say nothing about whether it is *working*. A prompt change that causes your agent to stop calling the right tools returns a perfect HTTP 200. A model swap that subtly degrades reasoning quality generates no alert. These are silent regressions, and they compound quietly across deployments. The scale of the problem is not trivial. Research from 2026 found that while 89% of organizations have observability in place, only 62% can actually inspect what their agents do at each individual step. Tool calls fail silently between 3 and 15 percent of the time. The gap — teams who monitor but cannot trace — is exactly where silent failures accumulate. ## How AQuA Works: The Outer Loop AQuA runs on what Google calls the “outer loop” — the quality evaluation layer that sits above the inner loop of individual agent runs. It operates outside your agent’s request path, reading telemetry from Cloud Trace, Cloud Logging, or BigQuery on a schedule, after each deployment, or on demand. The core of AQuA is a five-stage sweep pipeline: - **Sample** — pulls recent agent sessions from your configured telemetry source - **Review** — evaluates each conversation against your`goal.md` file - **Cluster** — groups recurring failures into patterns - **Verify** — confirms each cluster against raw conversation transcripts - **Track** — maintains a running ledger of insights across deployments The `goal.md` file is what gives AQuA context specific to your agent. It is a plain-English description of what your agent should do — appended to every review prompt. You can extend this with deterministic Python metrics via `eval_config.yaml` and publish custom LLM judges using `agents-cli aqua metrics publish`. The [full design rationale is on the Google Developers Blog](https://developers.googleblog.com/the-outer-loop-insights-first-an-ambient-quality-agent-that-diagnoses-your-production-agent/). ## Root Cause Diagnosis: File and Line Precision Beyond continuous sweeping, AQuA offers on-demand root cause analysis. When you need to understand *why* a failure cluster keeps recurring, AQuA compares the failing sessions against an immutable source snapshot captured at deployment time — a frozen copy of your code keyed to the exact revision that produced the problem. The output is specific: relevant files, cited line ranges, and a proposed change. This is a meaningful step beyond “something went wrong after the last deploy.” You get a diagnosis that points at your code. [InfoQ has a deeper technical breakdown](https://www.infoq.com/news/2026/11/google-ambient-agent-prod/) of how the snapshot mechanism works. One constraint worth stating plainly: AQuA never applies fixes or opens pull requests. It stops at the proposal. This is intentional — the tool sits at the boundary between autonomous diagnosis and autonomous action, and Google is not crossing it yet. ## Getting Started AQuA lives at `core/python/ambient-quality-agent` inside the `google/adk-recipes` repository. If you want to try it without touching a cloud project, the local demo requires no credentials and makes no model calls: ``` git clone https://github.com/google/adk-recipes cd core/python/ambient-quality-agent make install && make demo ``` To deploy it alongside a production agent: ``` agents-cli infra single-project --apply-aqua agents-cli deploy --deploy-aqua ``` Deploying AQuA provisions a Cloud Run dashboard behind Identity-Aware Proxy, a BigQuery dataset for insights, and writes the immutable source snapshot to Cloud Storage. See the [official DEV Community walkthrough](https://dev.to/googlecloud/find-the-production-agent-failures-that-every-health-check-misses-39j8) for a step-by-step guide. ## The Current Limitation AQuA is ADK-only. It reads from ADK telemetry sources and deploys into Google Cloud infrastructure. If your stack is built on LangChain, LlamaIndex, or a custom framework, you cannot drop this in today. The [ADK ambient agents documentation](https://adk.dev/runtime/ambient-agents/) explains the broader pattern — but the tool itself is currently scoped to the ADK ecosystem. For teams already in the ADK ecosystem, the case is straightforward: AQuA adds an evaluation layer that takes zero engineer-time between deployments. It runs while your laptop is closed. If your agent degrades after a deploy, AQuA will tell you before your users do.