# Observability-Driven Development: Spec-Driven Development Is Only Half the Loop > Source: > Published: 2026-08-25 11:41:22+00:00 It’s 2026 and we all write software the same way now: you describe a feature, your coding agent turns it into a spec, the spec becomes a plan, the plan becomes code, the tests go green, you merge. Spec-Driven Development. It’s genuinely great — I use it every day. Then last week I caught myself doing something embarrassing. My agent had just shipped a feature, all tests passing, and I typed into the terminal: *”looks good, seems fast enough.”* **Seems.** Fast. Enough. Twenty years of engineering and my acceptance criterion was a vibe. Here’s the uncomfortable truth about the SDD era: our agents write more code than ever, faster than ever, and we validate it with the weakest signal available — unit tests and a gut feeling. Tests prove your code does what the spec says *in the lab*. They say exactly nothing about what your service actually does when it runs: the spans it emits, the latency it really has, the errors it quietly swallows, the log lines that never got a trace ID. “Tests pass” is not “it works”. It’s “it works in the lab”. So I started closing the loop. I call it **Observability-Driven Development** — ODD. **🧭 The idea: close the loop** ODD is a simple claim: **Spec-Driven Development is only half the loop.** The other half is observation. The full loop looks like this: 1. **Spec & implement**— your usual SDD wave: spec, plan, code, tests. 2. **Observe a real run** — start the service, drive real requests at it, and read its telemetry: traces, metrics, logs, profiles. 3. **Turn what you see into the next wave** — every finding becomes the input of the next spec. Fix, improve, re-observe. 4. **Repeat. Indefinitely.** If this sounds familiar, it should. TDD’s big move was to pull testing *out of the QA phase* and *into the development loop*. ODD does the same thing to observability: it stops being the thing you bolt on before production (“we should probably add some dashboards”) and becomes the thing that drives what you build next. And it happens to be the perfect complement to agentic coding. Your agent doesn’t just write the code anymore — it starts the service, drives a scenario against it, queries the telemetry backend, and hands you a report full of numbers instead of vibes. **⚖️ The principles** ODD only works if you’re strict about a few rules. These are the ones I’ve landed on: **Evidence over impressions.** Every claim about a service comes from a query and its result — numbers, trace IDs, log lines. Never “it seems faster”. If you can’t paste the query that proves it, it didn’t happen. **Cross-confirm before concluding.** Never conclude from one signal what two could confirm. A latency spike in the metrics should have a matching slow trace; an error log should have a matching failed span. A single-signal anomaly gets labeled as exactly that — a suspicion, not a finding. **The system must be observable locally.** If observation only happens in production, it happens too late and too rarely. Your stack should start with one docker-compose, telemetry included, mocks for anything remote. **What’s missing is a finding too.** An absent span, a log line without a trace ID, a histogram nobody emits — telemetry gaps are first-class deliverables of an observation run. They feed the next instrumentation wave. **Verify by replaying, not by re-measuring differently.** A fix is proven by replaying the **exact** recorded scenario and comparing before/after values. Change one variable and your comparison is worthless. **The memory lives with the code.** Observation reports land in the repo, under a versioned `.odd/` directory. They get reviewed in PRs like any other artifact, the whole team shares them, and the git history reads like a story: observed → fixed → verified. Each new run recalls the previous report as its baseline — the loop accumulates knowledge instead of starting blind every time. **Agents observe, they never fix.** The observation agent’s job ends at the report. What happens next — an SDD wave, a JIRA ticket, a human decision — stays your call. Investigation and modification are separate powers, and that separation is deliberate. **One telemetry, two consumers.** The same OpenTelemetry instrumentation feeds the development loop **and** classic runtime observability — dashboards, alerting, incident response. Instrument once; dev and ops read from the same source of truth. **🚀 oddyssey: the reference implementation** Principles are cheap without tooling, so I built the toolbox [ oddyssey](https://github.com/using-system/oddyssey), an open-source (MIT) CLI toolbox for ODD. Everything rides on OpenTelemetry, and everything is packaged for whatever coding agent you already use — Claude Code, Copilot, Cursor, Codex, Gemini and friends, via The whole loop fits in three prompts: **`/odd-instrument`** — an OpenTelemetry expert agent investigates your codebase and hands back everything a spec-driven wave needs to instrument it properly: stack inventory, per-service approach sourced from the official OTel docs, open decisions, and a verification protocol. **`/odd-observe`** — an observation agent starts your service, drives a reproducible scenario at it, queries the four signals (traces, metrics, logs, profiles), and stores a full report in `.odd/` — findings, evidence, and the replay protocol. **`/odd-verify`** — the same agent replays a stored report’s protocol item by item: before-value, after-value, pass criterion. Fixes are measured, not assumed. Under the hood, a small MCP server does one job: pilot a **complete local Grafana observability stack **— one pinned container, Grafana UI, OTLP endpoint, Tempo, Prometheus, Loki, Pyroscope. Docker is the only prerequisite. Your app exports OTLP to `localhost:4318`, and your agent queries everything through the Grafana proxy — which means the exact same queries work later against any remote Grafana, or against Datadog, Dynatrace, Azure Monitor, CloudWatch, Splunk when you observe production. **🔬 Proof, not promises** Fair question: does this actually work, or is it another README full of adjectives? Here’s the part I like. oddyssey ran its own loop *on itself* — the toolbox instrumented, observed, and verified its own MCP server. Every artifact is committed in the repo, in the open: - **Instrument:** the expert agent investigated the codebase; its report seeded a real SDD wave — [the design spec](https://github.com/using-system/oddyssey/blob/main/docs/superpowers/specs/2026-08-22-mcp-otel-instrumentation-design.md) and [the implementation plan](https://github.com/using-system/oddyssey/blob/main/docs/superpowers/plans/2026-08-22-mcp-otel-instrumentation.md) are in the repo. - **Observe:** the observation agent drove an 18-call scenario against the freshly instrumented server — including an authorized Docker-engine kill, because failure paths emit telemetry too. Result: [a first observation report](https://github.com/using-system/oddyssey/blob/main/.odd/observe-run-reports/2026-08-22-2154-mcp-otel-instrumentation-verification.md) with **4 confirmed findings**— things like a duplicated tool span and noisy probe logging. Real anomalies, backed by trace IDs and queries, in code the tests had already blessed. - **Fix:** the findings became [the next SDD wave’s plan](https://github.com/using-system/oddyssey/blob/main/docs/superpowers/plans/2026-08-23-mcp-otel-fix-wave.md). Two commits. - **Verify:** the agent replayed the **exact** recorded protocol — same scenario, same 18 calls, same Docker kill. [The verification report](https://github.com/using-system/oddyssey/blob/main/.odd/observe-run-reports/2026-08-22-2227-mcp-otel-fix-wave-verification.md) : **9/9 checks pass, all 4 findings fixed.** Cold stack start measured at 4.3 s, status probe at 20 ms — before-values and after-values side by side. No “seems faster”. Numbers, trace IDs, committed reports, reviewable in a PR like any other diff. **🎯 Recap** - SDD gets code written; ODD checks what that code **actually does** when it runs. Together they close the loop: spec → implement → observe → next spec. - Strict rules make it work: evidence over impressions, cross-confirmed signals, replay-based verification, reports versioned with the code. - Agents are the perfect observers — and they observe only, never fix. The report is yours to act on. - [oddyssey](https://github.com/using-system/oddyssey) packages the whole thing for any coding agent: three prompts, a local Grafana stack, OpenTelemetry everywhere. Give the loop a spin on your own service — Docker plus one install command is all it takes. **🤝 Come build the loop** oddyssey is young, MIT-licensed, and very much open for co-conspirators. ODD is a practice before it is a tool — and practices get better when more people run them on real services and report back. Concretely, here’s where you can jump in: - **Run it and tell me what happened.** The most valuable contribution right now is a war story: point `/odd-observe` at one of your services and share what the report caught (or missed) in the [discussions](https://github.com/using-system/oddyssey/discussions). - **Extend the backend coverage. **The observation agent already speaks Grafana, Datadog, Dynatrace, Azure Monitor, CloudWatch, and Splunk through their CLIs — if your backend isn’t on that list, or the queries for yours could be sharper, the skill files are plain markdown and very PR-friendly. - **Sharpen the agents and skills.** Prompts, report templates, ODD principles that don’t survive contact with your reality — challenge them. The whole point of committing reports to git is that the practice itself is reviewable. And if your agent’s observation report surprises you (it will), come tell me what your telemetry said behind your back. ⭐ on [the repo](https://github.com/using-system/oddyssey) if the idea resonates — it’s how more people find the loop. Your tests are passing. But do you know what your service did last night? 🌙 [Observability-Driven Development: Spec-Driven Development Is Only Half the Loop](https://blog.devgenius.io/observability-driven-development-spec-driven-development-is-only-half-the-loop-86618a5ba979) was originally published in [Dev Genius](https://blog.devgenius.io) on Medium, where people are continuing the conversation by highlighting and responding to this story.