{"slug": "don-t-build-an-ai-that-replays-yesterday-s-spec-the-gap-between-spec-and-source", "title": "Don't build an AI that replays yesterday's spec — the gap between spec and source of truth is the real context", "summary": "An AI agent's accuracy depends more on its context than its prompting, specifically the gap between a polished specification and the actual \"source of truth\" that evolves during implementation. It claims that feeding an AI only the spec causes it to reproduce outdated \"past truth,\" while feeding it the reasons for the drift between the spec and reality allows it to approach \"today's truth.\" The author concludes that organizations should stop polishing specifications and instead focus on accumulating the reasons for this drift to build more effective AI agents.", "body_md": "📌 The full version (with interactive SVG figures, the drift curve, the five-whys hub, the document-vs-context split, and the Harness concentric layers) is hosted on my blog:\n👉 https://okikusan-public.pages.dev/context-is-the-gap.en\nThis dev.to post is the condensed version. The visualisations live on the original.\nMore and more often, an AI agent's accuracy is decided by its context, not its prompting.\nBut \"context\" here is not a polished spec. What really moves the needle is the gap between the spec and the source of truth, and the reasons behind the drift.\nAn AI fed only the spec replays \"past truth.\" Feed it the drift reasons too, and it approaches \"today's truth.\" The blind spot of Spec-Driven Development and the real core of Harness Engineering, laid out.\nThe spec describes what should be. A snapshot of agreement at a moment, internally coherent, neatly polished.\nAs implementation and operations evolve, the actual \"truth\" drifts elsewhere:\nThese are the Source of Truth (SoT). The spec inevitably drifts away from the SoT over time. This is not laziness — it's structural.\nThe problem is not that the gap exists. It's that the gap is never explained.\nTypical failures:\nThis is not the AI's fault. The context you fed it is frozen at a point in time, and the AI is faithful to that point. The cleaner the spec, the more confidently the AI quotes \"past truth.\"\nReverse-engineering alone is not enough either. Code reveals \"what is implemented and how,\" but never \"why it became that.\"\nKeeping these \"whys\" is exactly the Externalisation step in Nonaka's SECI model. The twist: you're externalising the process, not the conclusion. That's how judgement patterns become reproducible in other contexts.\nTolerating contradiction is the core. If you treat context as a \"thinking process,\" contradictions are natural. Human judgement wobbles constantly; organisational decisions get overwritten. Whether you can keep that without sanding it down decides whether your AI agent can reproduce \"your kind of judgement.\"\nAgent = Model + Harness (Karpathy framing). SDD alone is not enough — you need to design the SDD outer rings.\n\"Issue Driven Development (IDD)\" pairs well with this. SDD = the spec is the truth. IDD = the drift reasons are the truth. Let them coexist.\nIn May 2026, on the Linux kernel 7.1 RC4 release, Linus Torvalds publicly declared the security mailing list \"almost entirely unmanageable\" due to the flood of AI-generated vulnerability reports1. What was a stream of 2-3 reports per week two years ago has ballooned to 5-10 reports per day.\nLinus himself does not dismiss AI in security work — he asks researchers to \"understand the code and contribute a patch,\" not just the alert. That's a miniature of AI-agent operations in general. The value of an AI is not output volume — it is how much it lowers the human's verification, correction, and review load.\nA spec-only AI mass-produces plausible-looking output. It reads right, but it's drifted from the SoT and a human has to check every line to use it — the textbook case of \"Slop\" (low-quality, generic, templated AI output). Only the AI fed the drift reasons becomes the kind that actually lowers human verification load.\nWhat sharpens an AI agent is no longer the model or the prompt. It is whether you can accumulate the gap between spec and Source of Truth, and the reasons for the drift.\nMany organisations pour energy into \"polishing the spec\" because of the SDD boom. But the real differentiation lies elsewhere: not in polishing the spec, but in accumulating the gap with the SoT. To stop building AIs that replay \"past truth,\" stop polishing — start accumulating.\n📌 Full version with interactive SVGs: https://okikusan-public.pages.dev/context-is-the-gap.en\n- FIG.0 — THE GAP (spec vs SoT drift curve)\n- FIG.1 — SPEC-ONLY VS SPEC + GAP (two AIs)\n- FIG.2 — FIVE WHYS (the accumulating hub)\n- FIG.3 — DOCUMENTS VS CONTEXT (polish vs accumulate)\n- FIG.4 — HARNESS LAYERS (spec at the core, drift reasons on the outside)\nIf this resonates, a 🦄 / ❤️ / 💬 helps a lot. Feedback welcome.\nThe Register (2026-05-18): Linus Torvalds says AI-powered bug hunters have made Linux security mailing list 'almost entirely unmanageable' / Tom's Hardware (2026-05-18): Linus Torvalds says flood of duplicate AI-generated vulnerability reports... ↩", "url": "https://wpnews.pro/news/don-t-build-an-ai-that-replays-yesterday-s-spec-the-gap-between-spec-and-source", "canonical_source": "https://dev.to/hiro_kiku_public_88cd7eff/dont-build-an-ai-that-replays-yesterdays-spec-the-gap-between-spec-and-source-of-truth-is-the-9m2", "published_at": "2026-05-19 22:45:49+00:00", "updated_at": "2026-05-19 23:05:02.025210+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "developer-tools"], "entities": ["Harness"], "alternates": {"html": "https://wpnews.pro/news/don-t-build-an-ai-that-replays-yesterday-s-spec-the-gap-between-spec-and-source", "markdown": "https://wpnews.pro/news/don-t-build-an-ai-that-replays-yesterday-s-spec-the-gap-between-spec-and-source.md", "text": "https://wpnews.pro/news/don-t-build-an-ai-that-replays-yesterday-s-spec-the-gap-between-spec-and-source.txt", "jsonld": "https://wpnews.pro/news/don-t-build-an-ai-that-replays-yesterday-s-spec-the-gap-between-spec-and-source.jsonld"}}