{"slug": "the-agent-gave-the-right-answer-and-did-the-wrong-thing", "title": "The agent gave the right answer and did the wrong thing", "summary": "A developer's refund agent shipped v2 with identical output to v1, but the execution trajectory was wrong: the fraud check was skipped and the payment was refunded twice, costing $96.40 instead of $48.20. The project FlightRules uses SigNoz traces to enforce trajectory contracts in CI, catching such failures that output-based evaluation misses.", "body_md": "A refund agent ships v2. A customer asks for a refund. The agent replies:\n\nYour refund of $48.20 has been issued and will appear in 3–5 business days.\n\nThat is exactly what v1 said. The amount is right. The tone is right. An LLM judge scores it\n\nidentically. A regression suite comparing outputs sees no diff. Ship it.\n\nUnderneath, this happened:\n\n```\nv1 (approved)                          v2 (shipped)\nrefund.request                         refund.request\n  -> policy.retrieve                     -> order.lookup\n  -> order.lookup                        -> payment.refund\n  -> fraud.check                         -> payment.refund      <- charged twice\n  -> refund.calculate                    -> customer.notify\n  -> payment.refund\n  -> customer.notify\n```\n\nThe refund-policy retrieval is gone. The fraud check is gone. And the payment write happened twice —\n\na timeout, a retry, and the payment service's idempotency ledger recording both entries.\n\nThe customer got the right sentence. The company refunded $96.40 and skipped its own compliance\n\nchecks. Every output-based control in the pipeline was green.\n\nThis is not a hypothetical. It is the failure mode that appears the moment an agent has a retry\n\npolicy, a tool it can skip, and a prompt somebody edited on a Friday. Output evaluation is\n\nstructurally blind to it, because the output is fine.\n\nThe execution *trajectory* is where the bug lives. And a trajectory is exactly what a distributed\n\ntrace is.\n\n[FlightRules](https://github.com/winsznx/flightrules) turns SigNoz traces into deterministic release\n\ncontracts. It mines a **trajectory contract** from runs a human approved, then evaluates every\n\nsubsequent release against it, and fails the release with the traces that prove it. In CI that is\n\none command:\n\n``` bash\n$ flightrules gate check --release refund-agent-v2\nFAIL: This release exceeded one or more trajectory thresholds.\n\nFindings\n  MISSING_PREREQUISITE (skipped_check)\n    fraud.check is required at least 1 time(s) but occurred 0 time(s).\n  DUPLICATE_SIDE_EFFECT (duplicate_write)\n    payment.refund occurred 2 times; the contract permits at most 1 per run.\n\nexit code: 2\n```\n\nExit code 2. The pipeline stops.\n\nFive stages, and the interesting design decisions are all about what is *not* allowed in each.\n\n**1. Instrument.** The demo is a refund agent and five services — policy, orders, fraud, payment,\n\nnotification — instrumented with OpenTelemetry and exporting traces, metrics and logs over OTLP into\n\nSigNoz. The agent emits `gen_ai.*`\n\nattributes for tool identity and operation, plus a small\n\nFlightRules namespace: `agent.side_effect`\n\n, `agent.data_domain`\n\n, `agent.retry.number`\n\n,\n\n`agent.idempotency.present`\n\n.\n\nIt emits **no prompt, no model output, no tool arguments, no tool results and no chain-of-thought**.\n\nThat was a constraint from the start, and it turned out to be the interesting one: if trajectory\n\nenforcement works without any of it, the product is deployable in places prompt logging is not.\n\nIt does work. The forbidden-key redactor *removes* those attributes rather than replacing them with\n\n`[redacted]`\n\n— a redaction marker would still record that the product collected one.\n\n**2. Retrieve.** Everything goes through the SigNoz MCP Server, using the official MCP TypeScript\n\nSDK. The first real discovery of the project came here: ** signoz_get_trace_details cannot return\ncustom span attributes.** It returns a fixed column set. Every contract rule that reads an attribute\n\nThe path that works is `signoz_execute_builder_query`\n\nwith `requestType: \"raw\"`\n\nand `selectFields`\n\ndeclaring `fieldContext: \"tag\"`\n\n. There is a further asymmetry that costs an afternoon if you meet it\n\nby accident: discovery tools accept `\"attribute\"`\n\nas an alias for `\"tag\"`\n\n, but `selectFields`\n\nand\n\n`groupBy`\n\nrequire `\"tag\"`\n\n.\n\n**3. Reconstruct.** Spans become a causal graph, then a canonical form, then a fingerprint. Volatile\n\nidentifiers are normalised out, so two runs of the same behaviour with different order IDs produce\n\nthe same fingerprint. Identical traces produce identical fingerprints — asserted with property tests\n\nover reordered input, reordered rules and reordered attribute keys.\n\n**4. Mine and contract.** Twenty-five approved runs collapse into route families. The dominant family\n\nis proposed as approved; anything rarer is left pending for a human, because approving every family\n\nautomatically would defeat the review step the product exists to make explicit. From the approved\n\nfamily the miner proposes a contract in a small YAML DSL with eleven rule types — required spans,\n\nforbidden spans, cardinality, required edges, required ancestry, forbidden paths, attribute\n\nconstraints, allowed values, retry budgets, approved routes, numeric budgets.\n\n**5. Evaluate and gate.** A pure function over a graph and a contract. Then aggregation across a\n\nrelease, then a gate decision, then a process exit code.\n\nThe single decision everything else follows from: **no model, no clock, no network and no randomness\nparticipates in a pass/fail.**\n\nA model may explain a violation to a human. It may never decide whether a rule passed. The\n\nconsequences show up in odd places:\n\n`completedAt`\n\n, so a timestamp cannot enter the\nhashed region and two evaluations of the same evidence are byte-identical.`evaluateRunSafely`\n\nis the one place an internal error becomes a status, and it structurally\ncannot produce `pass`\n\n.The last one matters more than it sounds. A release gate that can return green after an internal\n\nerror is worse than no gate, because it launders a crash into a permission.\n\nThe subtlest correctness problem in the whole product is what to do when the evidence is missing.\n\nConsider `agent.idempotency.present`\n\non a payment write. If the attribute is absent, has the rule\n\n`equals: true`\n\npassed or failed?\n\nNeither. It is undecided, and the evaluator says so — `insufficient_evidence`\n\n, and the run reports\n\n`insufficient_data`\n\nrather than `pass`\n\n. A contract that could not be checked has not been satisfied.\n\nBut that answer is wrong for a different operator. `not_equals: admin`\n\nover an absent attribute is a\n\ngenuine **pass**: the span demonstrably does not carry the forbidden value. And `exists`\n\nover an\n\nabsent attribute is a genuine **violation**: the rule asks for the attribute itself.\n\nSo absence is resolved per operator. Defaulting it either way breaks something real: default to pass\n\nand an uninstrumented service satisfies `refund-must-be-idempotent`\n\n; default to violation and a\n\nrelease is punished for a telemetry gap.\n\nThis paid for itself. SigNoz returns a non-string tag as `null`\n\n— with no error and no warning — when\n\na query omits its `dataType`\n\n. A rule keyed on `agent.idempotency.present`\n\nwould have read a missing\n\nattribute from a query that reported success. It was caught only because absence produces\n\ninsufficient evidence instead of a pass. Had the evaluator defaulted to green, that would have been\n\na green release built on a query that returned nothing.\n\nA client span whose server span was never exported — an aborted request, a service that died\n\nmid-call — makes that span's subtree unobservable.\n\nThe tempting move is to mark the whole trace incomplete and refuse to conclude anything from\n\nabsence. It is safe-looking and it destroys the product: the canary's genuinely missing fraud check\n\nbecomes \"insufficient evidence\" and the gate goes green on the exact release it exists to catch.\n\nThe opposite move is to treat the trace as complete. Then an aborted request looks like a skipped\n\nstep, and the gate fails releases for network weather.\n\nThe answer is to scope the uncertainty to the one span. The trace stays `complete`\n\n, because the\n\nroute it evidences is fully determined by the client span. The rules anchored on that span check for\n\nits unobservable subtree individually and report `unobservable_subtree`\n\n. Everything else in the trace\n\nis decided normally.\n\nThe test that pins this holds both halves at once, in a single trace: the aborted payment handler\n\nstays insufficient evidence *while* the missing fraud check stays a violation.\n\nFlightRules does not just read SigNoz. It writes to it: the active contract compiles into ten managed\n\nresources — a notification channel, four saved views, a ten-panel dashboard, and four alert rules —\n\ncreated through MCP and **read back by identifier** before the register records them as synced.\n\nThe read-back is not ceremony. It found:\n\n`id`\n\n, `uuid`\n\nor\n`ruleId`\n\nand their name `name`\n\nor `alert`\n\ndepending on resource type. A single `id`\n\nlookup finds\nnothing for a dashboard or an alert, which makes every already-created one look absent — so the\nnext sync creates a second copy.`signoz_update_view`\n\ncorrupts the tenant.`signoz_list_views`\n\nreturns HTTP 500 for the whole\norganisation and None of those would have failed a test that checked the call succeeded.\n\nAlerts get the same treatment. Every managed alert was driven through firing *and* recovery against\n\nthe running deployment, read from SigNoz's own alert history: the violation-rate alert fired at value\n\n220 and recovered to `inactive`\n\nexactly 300 seconds later, matching its configured evaluation window\n\nto the second.\n\nThe hardening phase's job was to break the product before a judge did. Twelve real defects, and the\n\npattern in them is worth more than the list.\n\n**A negative number is not a smaller number.** A span reporting `agent.retry.number: -5`\n\nhad its\n\nvalue summed into the run's retry total. Nine retries plus one mislabelled span totalled four against\n\na limit of four — a clean run. A negative attempt index is not fewer retries; it is not an attempt\n\nindex at all.\n\n**Telemetry reaches terminals.** Span names, tool names and rule summaries all flow into the CLI's\n\nhuman report. A span named with `ESC [ 2 J ESC [ 1 ; 1 H PASS…`\n\nclears the reader's screen and\n\nreprints the opposite verdict — into a CI log that records the escape sequence faithfully, so the\n\ndeception survives review. Every line the CLI writes now passes through a filter applied once at the\n\nentry point.\n\n**Drift created duplicates.** A managed dashboard someone edited by hand failed its read-back, fell\n\nout of the \"unchanged\" path, and then took the *create* branch — leaving a second resource beside the\n\nedited one. And two concurrent syncs of one agent, racing through the saved-view\n\ndelete-and-recreate, each deleted one view and created another. Both now run under a per-agent\n\nPostgreSQL advisory lock covering read-register → sync → persist-register, because the register is\n\nwhat the *next* sync plans from.\n\n**The documented setup did not work.** The reproducibility test — a clone outside the working tree,\n\nvolumes destroyed, from committed files only — found four defects in the exact path a judge follows.\n\nThe best of them: SigNoz enforces a password policy on registration and states it *only in the\nrejection body*, and the command every document recommended,\n\n`openssl rand -base64 18`\n\n, satisfies it`ci-<run_id>`\n\n, which has neither an uppercase letter nor a symbol — `curl -sf`\n\nhad been hiding the reason, because it discards the response body on an HTTP error. The\n\nfix that mattered was not the password; it was making the failure legible.\n\nBoth workflows had never executed on GitHub. Running them found three more defects, every one\n\ninvisible on macOS and fatal on a runner.\n\n**A build that succeeded and then exited 1, silently.** `next build`\n\nprinted\n\n`✓ Compiled successfully`\n\n, then `Skipping validation of types`\n\n, then exited `1`\n\nforty-seven\n\nmilliseconds later having printed nothing at all — no stack, no message, no page named.\n\n`NODE_OPTIONS=--trace-exit`\n\nproduced the entire diagnostic the failure had to offer:\n\n```\n(node:2930) WARNING: Exited the environment with code 1\n    at exit (node:internal/process/per_thread:241:13)\n    at .../next/dist/build/type-check.js:110:17\n```\n\nNext.js probes the filesystem for `typescript/lib/typescript.js`\n\n. TypeScript 7 ships a native\n\ncompiler and has no such file, so Next reports TypeScript as *missing* while `tsc`\n\nworks perfectly.\n\nOff CI it quietly reinstalled TypeScript on every build; with `CI`\n\nset it threw an error that\n\n`type-check.js`\n\ndiscarded before calling `process.exit(1)`\n\n, on the assumption that a worker had\n\nalready logged it — which is false whenever `typescript.ignoreBuildErrors`\n\nis set, because then no\n\nworker is spawned. It was never a Linux problem. `CI`\n\nwas the discriminator, and it reproduces on\n\nmacOS with `CI=true`\n\n.\n\n**One service out of six could not resolve host.docker.internal.** The demo topology's compose file\n\n`host-gateway`\n\nmapping on five services and not on the agent. Docker Desktop injectsThat one was found by changing a readiness probe to report *what SigNoz actually holds* when a\n\nrelease never appears, instead of only that it did not:\n\n```\nrefund-agent-v1 never became queryable after 20 attempts.\n5 span(s) in the window\n  services:     …fraud-service, …notification-service, …order-service, …payment-service,\n                …policy-service\n  span names:   customer.notify.handler, fraud.check.handler, order.lookup.handler,\n                payment.refund.handler, policy.retrieve.handler\n  release ids:  (unset)\n```\n\nFive spans, one per service, no root, no agent. The search ended there.\n\nThe lesson is the same one the whole project keeps relearning: **a system that fails without naming\na cause costs more than a system that fails loudly.** Both fixes are pinned by a test that was\n\n**Make your verification verify.** `make signoz-verify`\n\npassed against a deployment whose credential\n\nwas still the literal string `replace-me`\n\n. It proved the MCP server was reachable and that\n\n`initialize`\n\nsucceeded — but `initialize`\n\nnever presents the credential to SigNoz. Every subsequent\n\ntool call returned 401 against a deployment the script had just declared healthy. It now makes a real\n\nauthenticated call.\n\n**Distrust your own handoff.** Entry verification for the final phase re-checked every figure in the\n\nprevious session's notes instead of trusting them, and found `make verify`\n\nexiting 2 on a file those\n\nnotes had themselves added.\n\n**Write down what the runtime did, not what the docs said.** Sixty-eight source-lock entries, each\n\nwith its claim and whether the installed runtime confirmed it. Several are defects in pinned\n\ndependencies rather than in this product. Every one of them was found by running something.\n\nFlightRules is Apache-2.0 and reproducible from a clean clone: SigNoz deployed by Foundry from a\n\ncommitted casting file, a real agent emitting real telemetry, a contract mined from real approved\n\nruns, and a gate that exits 0 for the approved release and 2 for the canary.\n\nThe canary returns the right answer. That was always the point.", "url": "https://wpnews.pro/news/the-agent-gave-the-right-answer-and-did-the-wrong-thing", "canonical_source": "https://dev.to/winsznx/the-agent-gave-the-right-answer-and-did-the-wrong-thing-4gmg", "published_at": "2026-07-26 21:47:12+00:00", "updated_at": "2026-07-26 22:29:21.558386+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-safety"], "entities": ["FlightRules", "SigNoz", "OpenTelemetry", "MCP"], "alternates": {"html": "https://wpnews.pro/news/the-agent-gave-the-right-answer-and-did-the-wrong-thing", "markdown": "https://wpnews.pro/news/the-agent-gave-the-right-answer-and-did-the-wrong-thing.md", "text": "https://wpnews.pro/news/the-agent-gave-the-right-answer-and-did-the-wrong-thing.txt", "jsonld": "https://wpnews.pro/news/the-agent-gave-the-right-answer-and-did-the-wrong-thing.jsonld"}}