{"slug": "when-does-a-specification-become-a-program", "title": "When Does a Specification Become a Program?", "summary": "Gabriella Gonzalez's essay 'A sufficiently detailed spec is code' argues that producing a specification precise enough for an AI agent requires the same intellectual work as programming, so specification does not escape the hard part. Mario Zechner's AI Engineer talk 'Building Pi in a World of Slop' adds that unspecified details are filled by the model, making decisions that are no longer the developer's. The article explores how regenerative software blurs the line between specification and implementation, noting that perfect specifications are unattainable and that production feedback must inform the durable description.", "body_md": "Someone asked me recently where the boundary lies between a specification and an implementation.\n\nThe question has been floating around a lot this year. Gabriella Gonzalez put it directly in an essay titled [A sufficiently detailed spec is code](https://haskellforall.com/2026/03/a-sufficiently-detailed-spec-is-code). Her argument is partly economic: if producing a specification precise enough for an agent requires doing essentially the same intellectual work as programming, we haven't escaped the hard part.\n\nMario Zechner made a related point in his AI Engineer talk [Building Pi in a World of Slop](https://youtube.com/watch?v=RjfbvDXpFls): whatever you leave unspecified gets filled in by the model, and those decisions are no longer yours.\n\nBoth objections matter for regenerative software.\n\nImagine starting with an English description of a system. It isn't precise enough, so we add requirements. Then invariants. Examples. Schemas. Conditions. State machines. At some point pseudocode seems clearer than another paragraph of prose. Eventually we may include executable fragments because they express some requirement more accurately than English can.\n\nKeep going and the specification starts looking suspiciously like a program.\n\nBut there are really two questions hiding here.\n\nHow much work does it take to specify a system precisely enough that we trust what gets generated?\n\nAnd once our durable description contains prose, tests, schemas, formal constraints, executable examples, and maybe actual code, what distinguishes that description from the implementation itself?\n\nThe first question is about whether specification actually saves intellectual labor. The second is about where the identity of a regenerative system lives.\n\nI'm interested in both, but especially the second.\n\n[There will be no perfect specification](#there-will-be-no-perfect-specification)\n\nOne seductive version of AI-native development starts with an oracle.\n\nDescribe the desired system perfectly. Give that description to a sufficiently capable machine. Receive a correct implementation. Delete it whenever you like and regenerate another.\n\nThe fantasy is hiding inside the word perfectly.\n\nWe have never been very good at describing complex systems completely before building and operating them. There is little reason to expect that changing the recipient of the description from a programmer to an LLM suddenly fixes that.\n\nThe first version of a specification represents our current understanding of the problem. Reality immediately begins editing it.\n\nA dependency fails in a way nobody anticipated. A customer discovers a workflow we never modeled. Something that is logically correct takes eleven seconds in production and is therefore unusable. A regulatory requirement appears. An implementation passes every acceptance test while a business metric quietly deteriorates.\n\nThose discoveries change what we know the system must do.\n\nI wrote about this in [Production Is a Compiler Input](https://aicoding.leaflet.pub/3mjx4erlboc2l). Production gives us evidence that should flow back into the inputs to the next implementation. The durable description of the system has to be able to learn from that evidence.\n\nThis is also where Zechner's gap-filling concern becomes important. Every specification is incomplete. The generator will make decisions inside those gaps.\n\nHumans have always filled gaps too, but there is an important difference in scale. A human programmer's decisions were at least made by someone present in the process and could plausibly be reviewed. An agent can fill thousands of unspecified details in seconds, most of which nobody will ever inspect individually.\n\nSome of those decisions are harmless implementation choices. Others will turn out to matter.\n\nThe problem is knowing which of those decisions should remain ephemeral and which ones should become part of what the system knows about itself.\n\nThat distinction only becomes visible over time.\n\n[Specifications are allowed to contain code](#specifications-are-allowed-to-contain-code)\n\nSuppose a requirement is most accurately expressed as an executable function.\n\nMaybe there is a financial calculation with enough edge cases that ten lines of code capture its meaning more faithfully than two pages of English. Maybe a property is naturally expressed in temporal logic. Maybe the clearest description of acceptable behavior is a collection of executable examples.\n\nThere is no advantage in translating those into prose simply to preserve a neat border between specification language and programming language.\n\nAn English sentence can describe an implementation detail. A piece of executable code can define an obligation.\n\nWe already accept this in testing. A test is code, but nobody concludes that the test suite and the application under test are therefore the same artifact.\n\nIf an executable property says:\n\n```\ndecode(encode(x)) == x\n```\n\nthen every implementation of the encoder and decoder is constrained by that property.\n\nOne implementation might use JSON. Another CBOR. Another a custom binary representation. The executable assertion remains valid across all of them.\n\nI wrote about this in [Evaluations Are the Real Codebase](https://aicoding.leaflet.pub/3mb526js42k26). Durable evaluations survive implementation changes because they express obligations at boundaries that outlive the current code.\n\nThe useful distinction is not whether something runs. It is what authority that artifact has over future implementations.\n\n[A specification is probably not a document](#a-specification-is-probably-not-a-document)\n\nThe word specification still carries an unfortunate image with it: a large document.\n\n`Requirements Document v7-final-FINAL.docx`\n\nI don't think that shape survives. I mean I seriously hope it doesn’t. Sheesh.\n\nA real system accumulates many different kinds of assertions.\n\nA checkout service may have an obvious behavioral requirement: a valid payment should result in an order.\n\nIt may also have an operational constraint saying p95 response time must remain below 400ms under a particular traffic shape. There may be an invariant that the same payment authorization cannot create two orders, a jurisdiction-specific rule, examples of known edge cases, and a business constraint saying that abandonment above some threshold is unacceptable even if the software is technically functioning.\n\nDifferent representations are appropriate for different kinds of knowledge.\n\nNatural language is good at some of them. Schemas are good at others. Property tests, formal constraints, examples, diagrams, measured production evidence, and executable reference functions all have their place.\n\nI suspect the useful abstraction looks closer to a knowledge graph than a requirements document: a collection of structured assertions with relationships, provenance, evidence, and different projections for different consumers.\n\nA human might ask it for a readable design document. An agent might ask it for all constraints governing checkout. An evaluator might ask which obligations can be mechanically checked. A production system might attach new evidence to an existing assertion.\n\nThe document becomes one view onto the underlying knowledge.\n\n[How do you read a specification like this?](#how-do-you-read-a-specification-like-this)\n\nIf the specification is a graph of assertions, constraints, evidence, provenance, and executable fragments, an obvious question follows:\n\nHow does a human read it?\n\nProbably the same way we read almost no large structured system directly.\n\nYou don't read the raw representation from top to bottom. You query it.\n\nAsk for everything related to performance. Everything that constrains checkout. Every requirement introduced after a particular incident. Every business rule that depends on a specific regulation. Every assertion that has weak evidence. Every part of the system whose current implementation depends on knowledge discovered in production.\n\nOr ask an LLM to explain a subsystem to you.\n\nGenerate a dependency graph. Produce a requirements document for a particular team. Show the relationship between business objectives and the evaluations that enforce them. Trace a production failure backward through the constraints and decisions that led to the current implementation.\n\nThe raw representation is storage, not necessarily an interface.\n\nThis is part of what makes the graph model appealing to me. It separates the canonical knowledge from the way any particular person needs to consume it.\n\nThere is an old idea hiding in here. More than twenty years ago, XML and XPath promised something similar: keep information in a structured form, then query and transform it into whatever view you need. When I wrote the first edition of [The Passionate Programmer](https://pragprog.com/titles/cfcar2/the-passionate-programmer-2nd-edition/), I did it in a proper XML editor with first class support for XML as a data model. It was an amazing glimpse into a future that didn’t quite pan out. The reality of XML was often much uglier than the promise, but the underlying idea was powerful.\n\nLLMs make that idea much more interesting.\n\nYou no longer need to know the exact query language or schema before asking a useful question. You can ask, \"Why does this service have a 100ms latency requirement?\" or \"Show me everything we learned from the outage last March,\" and have the system traverse the underlying knowledge for you.\n\nThe specification stops being a thing you read.\n\nIt becomes a thing you interrogate.\n\n[Detail does not collapse the distinction](#detail-does-not-collapse-the-distinction)\n\n[Hillel Wayne made a useful distinction](https://buttondown.com/hillelwayne/archive/a-sufficiently-comprehensive-spec-is-not/) in his response to this debate: a specification describes a set of acceptable implementations. Making the specification more precise shrinks that set.\n\nThat is a cleaner way to think about detail.\n\nSuppose we completely specify an API's externally visible behavior: schemas, error conditions, authorization rules, latency envelope, idempotency guarantees, ordering constraints, examples, invariants, and compatibility requirements.\n\nThere may still be many programs that satisfy it.\n\nOne implementation could use PostgreSQL. Another could use FoundationDB. One could be written in Rust, another in Go. One could use actors internally, another conventional request handlers. They might have almost no source code in common.\n\nMore precision narrows the implementation space. It does not automatically collapse specification and implementation into the same artifact.\n\nAnd if some small portion of a system becomes so tightly constrained that only one sensible implementation remains, that does not create a philosophical crisis. That implementation can simply become part of the durable system knowledge.\n\nThe more interesting question is what needs to survive if we decide to generate a different implementation later.\n\n[What mature implementations know](#what-mature-implementations-know)\n\nThere is a harder problem.\n\nExisting systems already contain knowledge that their specifications don't.\n\nI wrote about this recently in [The Implementation Remembers](https://aicoding.leaflet.pub/3mobohx4fq22x). Mature codebases accumulate odd retries, strangely specific timeout values, defensive validations, exceptional workflows, and ugly-looking conditionals. Some are junk. Others are compressed history.\n\nThat history matters enormously if we want to regenerate software.\n\nSuppose a query has an inexplicable index hint. An agent regenerating the service might reasonably remove it. Then production latency explodes because the hint was added three years ago after discovering pathological planner behavior at a particular data distribution.\n\nThe index hint preserves the discovery, but only indirectly. What we learned was something about acceptable behavior under real conditions.\n\nPerhaps the real lasting fact is:\n\nUnder the observed production cardinality, this query must remain below 100ms at p95.\n\nOnce that fact is represented explicitly, a future implementation is free to solve the problem differently.\n\nThe index hint might survive. It might be replaced by a different index, a rewritten query, a cache, or an entirely different storage engine.\n\nThe important thing is that the production lesson no longer depends on preserving the particular code that first embodied it.\n\nThis is where regeneration becomes interesting to me. Mature software contains a great deal of encoded experience. We need mechanisms that progressively move that experience into artifacts which survive the current implementation.\n\n[What is actually permanent?](#what-is-actually-permanent)\n\nThis is why I would not make specification carry the entire conceptual load.\n\nEarlier in this series, in [The Phoenix Primitives](https://aicoding.leaflet.pub/3mjfruwwuck2d), I described four things that need to survive regeneration: specification, evaluation, context boundaries, and provenance.\n\nThey answer different questions.\n\nThe specification records what we require.\n\nEvaluations provide evidence that an implementation satisfies those requirements.\n\nContext boundaries define the obligations between independently regenerable parts.\n\nProvenance records how the current state arose.\n\nNone of these needs to be a particular kind of file or language.\n\nA schema can define a boundary. Python can express an evaluation. English can state a requirement. Production telemetry can supply evidence. A conversation can create provenance.\n\nThe architecture comes from the relationships among them.\n\n[Where the boundary matters for Phoenix](#where-the-boundary-matters-for-phoenix)\n\nThe set-of-programs definition tells us something useful about what a specification is, but Phoenix has a slightly different architectural problem.\n\nWe need to know what survives.\n\nThe counterfactual I find most useful is:\n\nIf I throw this implementation away and generate another one, does this information have to survive?\n\nIf it does, the information belongs somewhere in the durable system.\n\nIt may be specification. It may be an evaluation, a boundary definition, provenance, a reference implementation, or production evidence. Phoenix does not require all durable knowledge to fit under the word specification.\n\nThat distinction matters because some executable artifacts should survive regeneration and some prose should not.\n\nA 300-line state machine may encode a protocol that every future implementation has to preserve. Meanwhile, a page of English explaining the internal organization of today's cache implementation may safely disappear with the code.\n\nThe format tells us very little. The regeneration test tells us much more.\n\n[This changes the economics of precision](#this-changes-the-economics-of-precision)\n\nThis is where Gonzalez's objection bites hardest.\n\nIf maintaining the durable description requires humans to manually reproduce every implementation decision in another form, we have gained nothing. We’ve created two codebases and called one a specification.\n\nHistorically, that is close to what formal specification often meant in practice.\n\nYou could write the program, or you could write an elaborate model of the program and then write the program too. Keeping the two synchronized created a permanent tax. Most teams understandably allowed the code to become the most precise description of the system.\n\nGenerative systems may change that cost structure.\n\nAgents can extract candidate constraints from implementations. They can compare production behavior with declared expectations. They can generate evaluations from requirements and flag disagreements. They can connect production evidence to assumptions. They can produce multiple human and machine-readable views from the same underlying facts.\n\nHumans still have to decide what matters.\n\nThat is the intellectual work we cannot automate away by renaming code a specification.\n\nBut we may be able to automate much of the clerical work involved in keeping different representations synchronized.\n\nIf that happens, richer external representations of system knowledge become economically plausible in a way they usually were not before.\n\nThis does not make Gonzalez's objection disappear. It changes where the cost lies.\n\nThe difficult work becomes deciding which facts are authoritative, which observations deserve promotion into durable knowledge, which constraints are accidental, and which freedoms we want future implementations to retain.\n\nThat seems like a better problem for humans to be spending their time on.\n\n[Regeneration needs a learning loop](#regeneration-needs-a-learning-loop)\n\nThere is one more consequence of all this.\n\nIf specifications are incomplete, implementations will inevitably contain decisions that do not yet exist in the durable system.\n\nSome of those decisions will later prove important.\n\nA regenerative architecture therefore needs a path in the other direction.\n\nGeneration takes durable system knowledge and produces an implementation.\n\nOperation produces evidence.\n\nThat evidence changes our understanding of the system.\n\nSome of what we learn gets promoted into the durable layer and constrains the next generation.\n\nThis is the loop I care about:\n\n```\nknowledge\n\n    ↓\n\nimplementation\n\n    ↓\n\nproduction\n\n    ↓\n\nevidence\n\n    ↓\n\nrevised knowledge\n```\n\nWithout that return path, regeneration becomes amnesia.\n\nYou can generate fresh software forever while repeatedly forgetting everything previous implementations learned the hard way.\n\nThe useful system is the one that gets harder to fool every time it runs.\n\n[Has anyone built the complete version?](#has-anyone-built-the-complete-version)\n\nI haven't seen a substantial production system that can throw away arbitrary pieces of itself and reliably regenerate them from a complete external body of system knowledge without meaningful loss.\n\nThe ideas are ahead of the tooling.\n\nBut “complete regeneration or nothing” is not a useful adoption threshold.\n\nA system can become progressively more regenerative.\n\nMove one important invariant out of implicit code and into a durable evaluation.\n\nMake one service boundary explicit enough that either side can be replaced.\n\nRecover one production lesson from an implementation and represent the constraint it exposed.\n\nTake one component and reach the point where deleting its implementation is boring.\n\nEach step changes where the system's knowledge lives.\n\nOver time, less of its identity is trapped in the current source tree.\n\nAnd this gets back to the question in the title. A specification does not become a program merely because it gets detailed, formal, or executable. The boundary that matters appears when we ask what must survive regeneration and what can be allowed to disappear with this particular implementation.\n\nA sufficiently detailed specification may contain prose, schemas, tests, formal models, executable examples, and actual code.\n\nThe important question is what remains after the program burns.", "url": "https://wpnews.pro/news/when-does-a-specification-become-a-program", "canonical_source": "https://aicoding.leaflet.pub/3msssevtkqc2c", "published_at": "2026-08-11 14:14:09+00:00", "updated_at": "2026-08-18 20:12:24.478145+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-products"], "entities": ["Gabriella Gonzalez", "Mario Zechner", "Haskell for All", "AI Engineer"], "alternates": {"html": "https://wpnews.pro/news/when-does-a-specification-become-a-program", "markdown": "https://wpnews.pro/news/when-does-a-specification-become-a-program.md", "text": "https://wpnews.pro/news/when-does-a-specification-become-a-program.txt", "jsonld": "https://wpnews.pro/news/when-does-a-specification-become-a-program.jsonld"}}