Keeping Specs, Tests, And Code In Sync In AI Development

AI coding agents cause specs, tests, and code to drift apart, creating false confidence and circular test coverage. A traceability model using six identifiers—requirement ID, design decision ID, task ID, test ID, commit/PR link, and change file—enforces links between requirements, tests, and code, with CI checks catching drift before merge. The approach, outlined by developer Glukhov, addresses specification rot in AI-assisted development by making traceability queryable data rather than shared understanding.

Keeping Specs, Tests, And Code In Sync In AI Development Stop AI agents drifting from specs, tests, and code. AI coding agents ship features fast, but specs, tests, and code quietly drift apart. This guide covers a traceability model, spec-to-test and spec-to-code mapping, and the CI checks that catch drift before a merge. A spec that nobody re-checks against the running system is worse than no spec at all, because it creates false confidence. Reviewers trust the document instead of the diff, and an AI agent asked to “follow the existing pattern” will happily follow whatever the code actually does, even when that contradicts the requirement it was supposed to satisfy. The fix is not more documentation. It is a small, enforceable link between four things that already exist in most repositories: the requirement, the design decision behind it, the tests that prove it, and the commits or pull requests that changed it. Once that link exists as data rather than as a shared understanding, you can query it. You can ask which requirements have no test coverage, which tests no longer map to any requirement, and which files changed in a pull request without a matching requirement ID. That query is the actual deliverable of this article, and the rest of the post walks through how to build it with tools you likely already run. The Drift Problem: Why Specs, Tests, And Code Fall Out Of Sync Drift shows up in four recognisable shapes, and AI-assisted teams tend to hit all four faster than teams that write every line by hand. Spec changes, code does not. A requirement gets clarified in a follow-up conversation or a comment thread, but nobody regenerates or edits the implementation to match. Code changes, spec does not. An agent or a developer fixes a bug or refactors a module, and the spec keeps describing the old behavior as if it were still current. Tests cover implementation, not intent. Unit tests assert what the code currently does, which is circular: they pass by construction even when the code satisfies the wrong requirement. Pull requests do not reference requirements. Reviewers approve a diff on the strength of “looks reasonable” because there is no explicit claim to check it against. Recent process research on AI development frameworks identifies specification drift as a recurring risk precisely because agents regenerate code quickly and repeatedly, and each regeneration is a fresh opportunity for spec and implementation to diverge a little further. The Spec-Driven Development vs Vibe Coding https://www.glukhov.org/ai-devtools/vibe-coding/spec-driven-development-vs-vibe-coding/ debate is really an argument about this same failure mode: a spec that nobody enforces degenerates into the same drift you get without one, just with extra ceremony. Modern spec-kit-style workflows increasingly frame this as specification rot : the spec keeps looking authoritative while quietly losing its connection to what the system actually does. The core definition of spec-driven development https://www.glukhov.org/app-architecture/documentation/what-is-spec-driven-development/ treats the spec as the source of truth, but a source of truth only stays true if something keeps checking it against reality. A Traceability Model For AI-Assisted Development A workable traceability model needs six identifiers that connect a business requirement all the way down to the lines of code and the pull request that implemented it. Most teams already have three or four of these; the missing ones are usually the design decision ID and the explicit link back from tests and commits. | Identifier | Lives in | Example | |---|---|---| | Requirement ID | requirements.md or spec tool | REQ-014 | | Design decision ID | ADR / decision record | ADR-0032 | | Task ID | task breakdown or issue tracker | TASK-014-3 | | Test ID | test file or test name | test req 014 password reset | | Commit / PR link | Git history | PR 482 | | Changed files | Git diff | auth/reset.go , auth/reset test.go | The relationships between these identifiers form a graph rather than a straight line, because one requirement can spawn several tasks, and one pull request can touch several requirements at once. REQ-014" -- ADR "Design Decision ADR-0032" ADR -- TASK "Task TASK-014-3" TASK -- CODE "Code Change auth/reset.go" TASK -- TEST "Test test req 014 password reset" CODE -- PR "Pull Request 482" TEST -- PR PR -- COMMIT "Commit history" Storing this graph as structured data, not prose, is what lets you query it later. GitHub’s Spec Kit ecosystem has moved in exactly this direction: extensions like spec-kit-trace scan REQ-XXX tokens embedded in spec files and test files and generate a deterministic matrix from that literal text match, deliberately avoiding fuzzy name-based guessing that produces silent false positives. Spec-To-Test Mapping: Turning Acceptance Criteria Into Test Cases Every acceptance criterion in a spec is, by construction, a behavioral assertion: given this state, when the actor does this, then the system should respond that way. That is already the shape of a test case, which is why the strongest SDD workflows generate tests from the same acceptance criteria that generate the code, instead of asking the code-generating agent to also invent its own tests after the fact. A widely used format for writing these criteria is EARS Easy Approach to Requirements Syntax , which forces each requirement into an unambiguous, testable pattern such as “When <trigger , the system shall <response .” That structure maps cleanly onto four categories of test that every requirement should carry: Positive tests — the happy path the requirement explicitly describes. Negative tests — inputs or states the requirement says must be rejected. Boundary tests — the edges of ranges, limits, and thresholds mentioned in the acceptance criteria. Migration tests — behavior for data or state that predates the requirement, so an old record does not silently bypass a new rule. | Requirement type | Test category to add | Common miss | |---|---|---| | “System shall reject X” | Negative | Only the accept path is tested | | “Limit is N items” | Boundary | N-1, N, and N+1 are not all covered | | “New field replaces old field” | Migration | Old records with no new field crash silently | | “Within 60 seconds” | Boundary + timing | Test asserts logic, not the actual time budget | Unit tests written this way still matter as the fast, cheap layer of the pyramid; the practical patterns for structuring them are covered in the Go unit testing guide https://www.glukhov.org/app-architecture/testing-architecture/unit-testing-in-go/ and the Python unit testing guide https://www.glukhov.org/app-architecture/testing-architecture/unit-testing-in-python/ . What traceability adds on top is a literal, stable requirement token embedded in the test name or a test comment, so a later query can prove — not assume — that REQ-014 has coverage. Spec-To-Code Mapping: From Design Plans To A Trace Table Spec-to-test mapping proves behavior; spec-to-code mapping proves scope. It answers a different question: which files were actually supposed to change for this requirement, and did the diff stay inside that boundary or spill into unrelated modules? A design plan that lists affected files up front — even a rough list — gives you something to diff the real pull request against later. Comments in code should only reference a requirement ID when doing so adds information a reviewer cannot get from the spec itself; a comment repeating the requirement text verbatim is noise, but // enforces REQ-014 boundary: max 5 reset attempts per hour earns its place because the number is otherwise invisible in the diff. A generated trace table turns this into something reviewable in seconds rather than something a reviewer has to reconstruct by reading both documents side by side: | Requirement | Design decision | Files changed | Tests | Status | |---|---|---|---|---| | REQ-014 | ADR-0032 | auth/reset.go , auth/reset test.go | test req 014 4 | Covered | | REQ-015 | ADR-0032 | auth/reset.go | none | Gap | | REQ-016 | — | auth/notify.go | test notify basic | Orphan spec link | That single table surfaces two of the most common failure patterns at a glance: REQ-015 changed code with zero matching tests, and the test attached to REQ-016 does not actually reference a requirement ID, which means either the spec is missing or the test was misfiled. The Pull Request Workflow: Reviewing Spec, Code, And Test Diffs Together A pull request built around traceability reviews three diffs side by side instead of one: what changed in the spec, what changed in the code, and what changed in the tests. The review question stops being “does this look right?” and becomes the far more specific “which requirement does this change satisfy, and does the evidence prove it?” A short, concrete reviewer checklist works better here than a long one, because reviewers skip long checklists under deadline pressure: - Does the PR description name the requirement ID s it satisfies? - Does every changed file appear in the design plan’s affected-files list, or is the extra scope explained? - Does at least one new or existing test reference each requirement ID touched by this PR? - If the spec changed, did the code and tests change in the same PR, or is there a tracked follow-up? Automating Traceability In CI Manual review catches drift only as often as reviewers remember to look for it, which is why the checks above belong in CI rather than in a wiki page nobody re-reads. The same GitHub Actions cheatsheet https://www.glukhov.org/developer-tools/ci-cd/github-actions-cheatsheet/ patterns you already use for build and test jobs apply directly here — traceability checks are just another job in the same pipeline. Practical automation ideas, roughly in order of effort: CI checks for spec files — fail the build if a spec file was edited without a corresponding code or test change in the same PR, or vice versa. Require requirement IDs in PR titles or descriptions — a lightweight regex check REQ-\d+ blocks merges that don’t name what they implement. Agent-generated trace summaries — have an agent produce a short summary of which requirements a PR touches, for a human to confirm rather than write from scratch. Test coverage by acceptance criterion , not just by line — line coverage tells you code ran; requirement coverage tells you a claim was checked. Stale spec warnings — flag specs that have not been touched in N commits touching their linked files, since long-silent specs are the ones most likely to have quietly rotted. Extensions built on top of GitHub’s Spec Kit already implement several of these mechanically: one scans literal REQ-XXX tokens across spec and test files to build a matrix and flag orphaned tests, and a stricter V-Model-oriented pack goes further, generating a paired test specification for every development specification and producing multiple traceability matrices for teams working under regulatory frameworks such as IEC 62304 or ISO 26262. You do not need that level of ceremony for most projects, but the underlying idea — a deterministic, script-generated matrix rather than a hand-maintained spreadsheet — scales down just as well as it scales up. Using AI Agents For Traceability, Not As An Oracle AI agents are well suited to the mechanical parts of traceability and poorly suited to being the final judge of whether a requirement was actually satisfied. Three tasks fit an agent’s strengths directly: Compare spec and diff — ask the agent to list every requirement mentioned in the spec files touched by a PR, and every one it did not find corresponding code for. Find uncovered requirements — ask the agent to scan the test suite for requirement tokens and report which requirements in the spec have none. Detect code not described by spec — ask the agent to flag changed files or functions that touch requirement-bearing modules but do not correspond to any requirement ID in the diff. The failure mode to guard against is trusting the agent’s summary as ground truth instead of as a reviewer’s starting point. An agent can misread a comment, miss a requirement token split across two files, or confidently declare coverage for a test that only exercises the code path superficially. Treat every agent-generated trace report the way you would treat a junior reviewer’s pass: useful, fast, and still subject to a second look before it gates a merge. This is the same caution that applies to decision records for AI-driven development https://www.glukhov.org/app-architecture/documentation/decision-records-ai-driven-development/ — the record only stays trustworthy if something other than the agent that wrote it eventually checks it. A Minimal Traceability Template You Can Copy You do not need a heavyweight framework to start. A five-file template, checked into the repository next to the code it describes, covers the essentials: docs/ requirements.md REQ-IDs with EARS-style acceptance criteria design.md ADR-IDs, affected files, architecture decisions tasks.md TASK-IDs mapped to one or more REQ-IDs tests.md which test files/functions reference which REQ-IDs traceability.md generated table: REQ - ADR - TASK - files - tests - PR requirements.md , design.md , and tasks.md are written or edited by humans and agents together, the same way the spec-driven development workflow https://www.glukhov.org/app-architecture/documentation/spec-driven-development-workflow/ already describes. tests.md and traceability.md should be generated, not hand-maintained, even if the generator is a short script that just greps for REQ-\d+ across the test directory and the spec files — hand-maintained trace tables are themselves a form of drift risk, because nobody updates a spreadsheet under deadline pressure. Conclusion Spec-driven development is not finished the moment code comes out of an agent; it is only useful once code, tests, and specs keep each other honest over time, through PRs, refactors, and requirement changes that arrive months apart. A traceability model built from six plain identifiers, enforced by a handful of CI checks, and reviewed with a short PR checklist gets you most of the benefit without the overhead of a full compliance framework. Start with the minimal template, wire the cheapest CI check first — requirement IDs in PR descriptions — and add the trace table and stale-spec warnings once that habit sticks. Traceability is one piece of a larger testing and documentation discipline covered across the App Architecture in Production https://www.glukhov.org/app-architecture/ cluster, and it sits alongside the tooling questions explored in the AI developer tools https://www.glukhov.org/ai-devtools/ cluster for teams choosing which agent workflows to standardize on.