Agents made data pipelines cheap. Being wrong is still expensive. Here is who decides, what gets verified and how autonomy is earned.
TL;DR
AI agents made producing data pipelines almost free. They did not make being wrong any cheaper. The bottleneck has moved from writing code to specifying meaning and verifying outcomes.
ADDLC is a lifecycle designed around that shift: 3 actors, 3 loops, 7 phases, 4 gates.
The operating rule fits in one line: agents propose, controls enforce, humans decide.
Autonomy is granted per class of change, by reversibility and blast radius ‐ not per agent, and not by enthusiasm. It is earned with evidence and lost with escaped defects.
Build and Run are segregated by design, for humans and agents alike. Master and reference data get a stewarded change channel of their own.
The model comes with gate criteria, an autonomy scale from Manual to Self-organizing, metrics, a maturity model and a self-assessment. It builds on DAMA-DMBOK, Data Mesh, Data Management at Scale rather than replacing them.
Numbers from applying this model to the reference and streaming market data of one large platform, before and after. They are observational, not a controlled experiment, and part of the gain comes from organizational changes made at the same time.
Human effort per integration: about 4‐5x lower, with a target of roughly 15x.
Lead time from request to data in production: about 5x shorter.
Integrations delivered per quarter by the same team: about 8x more.
Share of incidents first reported by consumers rather than by monitoring: about 17x lower.
Specs and packages accepted at the first review: nearly 2x higher, approaching all of them.
Incident triage done by agents: from none to a target of about three quarters.
- The cost curve flipped
In the last year the economics of data engineering inverted. A coding agent with access to a warehouse, a dbt project and a catalog drafts an ingestion job, staging models, tests and documentation in minutes. What used to be the expensive part of a data integration ‐ writing the code ‐ is now the cheap part.
What did not get cheaper is being wrong. A mis-specified join, a metric that silently changes meaning, a wrong identifier mapping in reference data: each costs as much as before, and often more, because agents produce change faster than people can review it.
Teams that adopt agents without changing their process see the same symptoms:
Sprawl. Creating a new table is cheaper than understanding an existing one, so every request gets its own pipeline.
Self-certified quality. The agent writes the transformation and the tests that confirm it. Both pass. Both encode the same misunderstanding.
Documentation nobody trusts. Generated descriptions multiply, nobody reviewed them, and readers learn to ignore all of them.
Shadow autonomy. The agent runs with an engineer’s production credentials, so the real permission model is “whatever the person who started the session could do”.
Artifact chaos. Agents create copies instead of versions: several near-identical files for one diagram, with the file name as the only version control.
These are lifecycle problems. DevOps spread through shared practices and, later, a shared yardstick ‐ the four key delivery metrics from the DevOps Research and Assessment program, popularized by the book Accelerate. Agentic data work needs the same, and it needs it before every vendor defines it in terms of its own product.
Definition. The Agentic Data Development Lifecycle (ADDLC) is the end-to-end process by which data products are framed, specified, built, verified, released, operated and retired when AI agents perform most of the execution and humans retain decision rights over intent, meaning and acceptance.
- What already exists, and what is missing
Coalesce, “The Agentic Data Development Lifecycle in Practice” (Mikkel Dengsøe, October 2026). The closest prior art: a product workflow in four stages ‐ Define, Build, Govern, Observe ‐ with a clear split between what the agent does and what people decide, and spec-driven development in Git. It is anchored in one product and does not define gate criteria, accountable deciders, autonomy policy, metrics or maturity.
Thoughtworks, “Making Your Data Ready for Agentic AI” (Pramod Sadalage and Prem Chandrasekaran, martinfowler.com, August 2026). Five attributes of AI-ready data, a context layer (domain, semantic and capability models) and a staged autonomy ladder. It treats agents mainly as consumers and curators of data rather than as builders of pipelines.
“The Self-Organizing Data Stack” (Ananth Packkildurai, Data Engineering Weekly, October 2026). An end state in which an engine maintains the dependency graph through five loops, with assets as contracts and a firm rule: never let an agent grade its own work. It describes a possible destination; the path to it is out of scope.
Holistics, “Agentic Data Engineering” (Huy Nguyen, September 2026). What agents can do across the stack and why the semantic layer is the contract between engineering and analytics. No lifecycle.
“From Data Quality to Quality of Agentic Data Use” (Ania Cravero, Jorge Díaz-Villegas, Zihao Xiao, Applied Sciences, September 2026). Six dimensions of agentic data use and a staged control lifecycle for an agent acting under delegated authority. Rigorous, but focused on executing a data task rather than on a team’s operating model.
The ADLC family (IBM, EPAM, AWS AI-DLC and others). Lifecycles for building agents, or for software built by agents. A comparative review in Towards AI found six definitions and zero consensus, and none of them targets data engineering.
The gap: there is no vendor-neutral model that (1) assigns decision rights between humans, agents and controls, (2) defines gates with evidence criteria, (3) segregates Build from Run, (4) governs autonomy explicitly, (5) defines metrics and maturity, and (6) treats master and reference data as the special case it is. ADDLC is an attempt to fill that gap.
- The bottleneck moved
In the classic SDLC the scarce resource was engineering time, and the lifecycle was organized to spend it well. In agentic data work, three other resources are scarce:
Judgment about meaning. What exactly is an “active instrument”? Which source wins when two vendors disagree on a listing date? Is a negative trade size an error or a correction? These are decisions, not outputs, and an agent cannot make them on anyone’s behalf.
Verification capacity. Agents produce changes faster than people can check them. Unverified change is not velocity ‐ it is deferred incidents.
Operational attention. Every new asset is something somebody has to run. Generation without a path to operation moves cost from Build to Run, where it is more expensive.
ADDLC is organized to protect these three resources. Its operating rule:
Agents propose. Controls enforce. Humans decide.
DAMA-DMBOK defines data governance as “the exercise of authority and control (planning, monitoring, and enforcement) over the management of data assets” and keeps it separate from data management itself ‐ oversight apart from execution, which DMBOK compares to the separation of audit and accounting in finance. ADDLC applies that separation literally: governance is exercised by humans and enforced by deterministic controls, while management is increasingly executed by agents. The computational half of Zhamak Dehghani’s federated computational governance is exactly what “controls enforce” requires: policies as code, applied automatically, everywhere.
- Ten principles
The specification is the system of record; code is derived data. Martin Kleppmann distinguishes systems of record from derived data that can be recomputed from them. ADDLC applies the idea by analogy: the specification ‐ contract, semantics, quality expectations, acceptance examples ‐ is the system of record, and pipelines, DAGs and even tests are derived artifacts an agent can regenerate. When code and spec disagree, that is a defect: fix the code or formally revise the spec ‐ neither wins silently. Unlike derived data, generated code is not a deterministic function of its spec, which is why verification exists.
Meaning is human-owned. Grain, keys, survivorship rules, metric definitions, glossary terms and quality thresholds are decisions with an accountable owner. Agents draft them, owners approve them, and nobody infers them at runtime.
No agent certifies its own work. The actor that built a change cannot be the only one that verifies it. Verification relies on independent evidence: tests derived from the spec, reconciliation against an independent source and ‐ where an agent verifies ‐ a separate agent that sees the spec and the output but not the builder’s reasoning.
Autonomy is granted per change class, by reversibility and blast radius. “How autonomous is our agent?” is the wrong question. The right one is “which classes of change may be executed without a human, and why?”. A reversible documentation update and an irreversible change to golden-record matching rules deserve different answers, even from the same agent.
Context is a product. Glossary, domain model, contracts, standards, agent instruction files and skills are what an agent knows about your organization. They need owners, versions, reviews and tests, like any other product. This is DMBOK’s metadata management with a new and very demanding consumer.
Every action leaves evidence. Every agent action is attributable (which agent identity, on whose behalf), explainable (which spec, which decision) and reconstructable (inputs, tool calls, outputs). Lineage extends from “where did this data come from” to “why did this change happen”.
Build and Run are segregated ‐ for humans and for agents. Build agents have no write access to production data. Run agents cannot change code or models. They use different identities, credentials and queues. It separates identities and permissions, not accountability: a team that builds and runs its own product still uses separate Build and Run identities, because non-human workers are otherwise happy to do everything.
Master-data changes are data operations, not deployments. Correcting a record, remapping an identifier or patching history goes through a stewarded change channel that is isolated, auditable and reversible. Nobody ‐ human or agent ‐ edits master data through a code path or an ad hoc SQL session.
Reuse before you generate. Before building, an agent must show that no existing asset already fulfills the need, or that extending one is worse than creating a new one. Otherwise near-free generation turns into permanent sprawl.
Measure flow and trust, not output. Lines of generated code and the number of pipelines created are vanity metrics. ADDLC measures lead time, human effort, failure rates, detection and verification yield ‐ per change class.
- The model at a glance: 3 actors, 3 loops, 7 phases, 4 gates
The figure at the top of the article shows the whole model on one page.
Three actors
Humans hold decision rights: intent, meaning, acceptance and exceptions. Typical roles are the data product owner, data steward, data engineer (increasingly an architect and supervisor of agents rather than an author of code), DataOps operator and consumer.
Agents hold execution rights ‐ and only execution rights. They draft, build, test, document, triage and propose fixes. Build, verify and run agents are separate identities with separate permissions.
Controls hold enforcement: CI pipelines, contract validators, policy as code, access control, quality gates and promotion rules. Controls are deterministic. They do not use judgment, and they cannot be persuaded.
Three loops
Intent loop (business and governance): Frame and Specify. Humans lead, agents prepare.
Delivery loop (the Build side): Build, Verify and Release. Agents lead, controls gate, humans accept.
Run loop (the Run side): Operate, then Evolve and Retire. Operators decide, agents triage and prepare, controls watch.
The specification is the hinge between all three. Delivery cannot start without an approved spec, and Run reopens the spec when reality disagrees with it.
Four gates: Intent, Meaning, Evidence, Operability.
ADDLC does not replace the data engineering lifecycle described by Joe Reis and Matt Housley ‐ generation, storage, ingestion, transformation, serving. It wraps it. Data flows through those stages inside Build and Operate, and the undercurrents ‐ security, data management, DataOps, architecture, orchestration, software engineering ‐ become context that agents consume and controls enforce.
- The seven phases
For each phase: what agents do, what humans decide, what controls enforce, and what the phase must leave behind. Phase 1 ‐ Frame
Purpose: decide whether the work is worth doing, for whom, and how risky it is.
Agents draft an intent brief from tickets, requests or call transcripts, find similar past requests and existing assets, and estimate effort and running cost.
Humans confirm the consumer and the value hypothesis, name the accountable owner and assign a risk tier.
Controls enforce the intake template: no work item without an owner and a tier.
The risk tier combines data criticality, reversibility and blast radius. A one-off analytical extract is low. A new feed into a golden record that every product consumes is high. The tier sets the strictness of every later gate; the autonomy ceiling of the change class applies on top.
Phase 2 ‐ Specify
Purpose: agree on what the data means and what “good” looks like before any code exists.
Agents profile candidate sources, search the catalog for reusable assets and write a reuse report, check licensing and entitlements, and draft the specification package: a data contract (for example in the Open Data Contract Standard), semantic definitions linked to glossary terms, the mapping to the canonical model, quality expectations with thresholds per dimension (completeness, validity, uniqueness, timeliness, consistency, accuracy), and acceptance examples ‐ concrete input-output cases that any correct implementation must reproduce.
Humans ‐ the owner and the steward ‐ decide meaning: grain, keys, source precedence, survivorship, thresholds, what is out of scope. They resolve every ambiguity the agent escalates.
Controls validate the contract against its schema and check that every field is linked to a glossary term or explicitly marked local, and that every expectation has a threshold and a severity.
Exit artifact: approved specification package.
Gate: Meaning gate.
This is where most of the human time in ADDLC should go. The new code review is the spec review.
Phase 3 ‐ Build
Purpose: produce an implementation of the spec.
Agents generate ingestion into Bronze, cleaning and conforming into Silver, and business and semantic models in Gold. Tests are derived from the spec in a separate context that never sees the implementation, acceptance examples run unchanged, and agents also generate documentation and lineage. They work in a sandbox on sampled, masked or synthetic data.
Humans review architecture only where a deterministic template-conformance check flags a departure from a known pattern ‐ the agent does not get to decide that itself ‐ and answer escalations.
Controls encode standards as templates and agent skills, run linters and naming rules, apply policy as code (no personal data in Gold without classification, no new source without a contract) and keep production credentials out of the build environment.
Exit artifact: a change set with generated tests, documentation and lineage, linked to a spec version.
Certify, don’t delete. Quality rules decide what is published, not what exists. Every conformed record stays one layer below Gold with its quality flags; Gold, and the agents that read it, get only what passed; failures go to the steward queue with evidence instead of disappearing.
Phase 4 ‐ Verify
Purpose: produce independent evidence that the implementation satisfies the spec.
Verification combines lines of evidence that do not share the builder’s assumptions:
Deterministic checks derived from the spec ‐ contract tests, schema checks, data tests such as dbt tests, acceptance examples.
Reconciliation against an independent source or a known invariant ‐ totals, counts, identities, temporal rules. Packkildurai calls this adversarial verification: schema, null and key tests can all pass while a join silently double-counts after historical entity merges.
An independent verifier agent with its own instructions and, ideally, a different model. It sees the spec and the output, never the builder’s reasoning, and tries to break the claim. This is the weakest line of evidence ‐ models share training data and blind spots ‐ so it supplements the other three and never replaces them.
Human acceptance on a sample by the consumer ‐ or, for master and reference data, the steward ‐ for medium and high risk tiers.
Exit artifact: evidence pack ‐ test results, coverage against the spec, reconciliation report, verifier findings, sample sign-off, cost of the run.
Gate: Evidence gate.
Phase 5 ‐ Release
Purpose: put the change into production safely and hand it over to the people who will run it.
Agents prepare the promotion, backfill and rollback plans and the consumer notification, and assemble the handover package.
Humans approve the release where the tier requires it. DataOps decides on operational acceptance.
Controls promote through environments via CI/CD, publish the versioned contract, perform an atomic cutover in Packkildurai’s sense (consumers are switched only if no contract changed during verification) and apply automatic rollback criteria.
Exit artifact: a released, versioned data product and an accepted handover package.
Gate: Operability gate.
The handover package is what makes Build and Run separable. It contains: ownership and escalation path; data flow; runbook; alerts mapped to each known failure mode and routed to the right queue; operational skills ‐ instructions that let run agents triage this product; operator access; quality rules and tests; contract, classification and license; expected capacity and cost.
Early-life support: the authors stay on call for 30 days or one full event cycle, whichever is longer. For market data, a cycle includes expirations, holidays, index rebalances and vendor maintenance windows ‐ many failure modes simply do not occur in a quiet month.
Phase 6 ‐ Operate
Purpose: keep the product within its commitments.
Agents (read-mostly) watch the five observability pillars described by Barr Moses, Lior Gavish and Molly Vorwerck ‐ freshness, volume, distribution, schema, lineage. They deduplicate alerts, gather evidence, run root-cause analysis, classify incidents and draft fixes.
Humans ‐ operators and stewards ‐ decide what the incident is and which channel fixes it.
Controls route alerts, enforce noise budgets and suppression rules with expiry dates, and guard the stewarded change channel.
The critical decision in Operate is classification, because each class has a different owner and a different fix path:
Defect ‐ the implementation is wrong. It goes back to the Delivery loop as a change.
Data correction ‐ the code is right, the data is wrong (a vendor error, a late correction). It goes through the stewarded change channel.
Decision ‐ a guard fired on a legitimate but unusual event, such as a mass change on a corporate-action day. It needs a human call, prepared by an agent with evidence.
Spec gap ‐ reality produced a case the spec never described. It reopens Specify.
In feed operations the last two classes are easy to underestimate. Routed to developers as bugs, they get closed, not fixed.
Phase 7 ‐ Evolve and Retire
Purpose: keep the estate small, coherent and current.
Agents detect overlapping assets and propose consolidation, propose contract versions for schema evolution, flag assets whose consumers and commitments have expired, and draft deprecation notices.
Humans approve consolidation and retirement and set deprecation windows.
Controls enforce contract versioning, deprecation windows and consumer impact analysis from lineage.
Exit artifacts: consolidation and retirement decisions, new contract versions, context updates.
Any change Evolve produces ‐ a consolidation, a new contract version ‐ re-enters the Delivery loop under Build identities. Run proposes; Build implements.
Phase 7 also closes the learning loop. Every rejected gate, escaped defect and incident updates the context ‐ a skill, a standard, a glossary entry, a test template. If the same failure happens twice, the context failed, not the agent.
- The four gates
A gate is passed with evidence, not with checkboxes. Each gate has exactly one accountable decider, a defined evidence pack, automated checks that run before any human looks, and explicit outcomes.
Gates need quality control of their own. A reviewer handed a polished, agent-assembled pack drifts toward rubber-stamping. Track approval rates and review depth per gate, and periodically re-review a sample of approved packs.
Intent gate ‐ is this worth doing, who owns it, how risky is it?
Outcomes: go; go with a smaller scope; reuse an existing asset; no. Meaning gate ‐ do we agree on what this data means and what “good” looks like?
Accountable: data product owner. For master and reference data, the data steward’s approval is also required.
Evidence: contract, semantic definitions linked to the glossary, canonical-model mapping, quality expectations with thresholds and severities, acceptance examples, licensing and entitlement check, reuse report.
Outcomes: approved; approved with listed open questions that block release but not build; rejected.
Evidence gate ‐ does the implementation demonstrably satisfy the spec?
Accountable: for medium and high tiers, a release approver who did not author the change; for low tiers, the change-class owner ‐ the person accountable for that class of change. For classes at the Delegated level (section 8), controls enforce the gate automatically and the change-class owner stays accountable, auditing after the fact.
Evidence: the evidence pack from Verify.
Outcomes: pass; pass with dated exceptions; fail ‐ back to Build with findings.
Operability gate ‐ can the Run side operate this without its authors?
Accountable: DataOps lead.
Evidence: the handover package plus an operational test ‐ replay historical incidents, fire a test alert end to end, walk through the access scenario, and let a run agent triage a seeded failure using only the provided skills.
Outcomes: accepted; accepted with dated gaps; accepted for specific failure classes only; not accepted.
One rule makes the Operability gate work: if the Run side does not accept, the Build side keeps the pager.
Minimum viable ADDLC. For low-risk change classes the lifecycle collapses to a spec template, spec-derived tests, an automated Evidence gate and a named change-class owner ‐ minutes, not meetings. If the gates cost more human time than the work they protect, the risk tiering is wrong. Ownership follows the same split as decision rights: whoever owns data governance ‐ usually the CDO ‐ owns the autonomy policy, change-class owners sit in the domains, and DataOps owns the Operability gate.
Fitting into what you already run. Gates ride on existing mechanics: the Meaning gate on a spec pull request with the steward as code owner, the Evidence gate on required CI checks, the Operability gate on the production-readiness review. Roles are hats, not headcount, and the platform team owns the controls and the agent identities.
- Autonomy levels
ADDLC borrows the shape of SAE J3016 (the levels of driving automation) and defines six autonomy levels. A level is assigned to a change class, never to an agent as a whole.
Manual. No agent involvement.
Assist. The agent suggests; a human authors.
Draft. The agent authors; a human reviews every change before it merges.
Supervised. The agent executes the change class end to end; humans decide at gates and sample-review the internals.
Delegated. The agent releases changes in the class without a human; controls enforce; humans audit after the fact; rollback is automatic.
Self-organizing. Agents propose and reorganize the asset graph within policy, in the spirit of Packkildurai’s engine. Humans own meaning and policy only.
The ceiling for a change class is set by three factors: reversibility (can we undo it completely, and how fast?), blast radius (how many consumers can it affect?) and data criticality. The Thoughtworks article reaches a similar conclusion through reversibility classes and staged autonomy.
Example ceilings:
Descriptions and documentation on Bronze and Silver: Delegated.
Descriptions of Gold and glossary-linked fields: Supervised ‐ that is meaning.
Incident triage and evidence gathering, read-only: Delegated.
New Bronze ingestion from a known source type and template: Supervised.
New or changed Silver transformation: Supervised.
Change to a certified Gold metric definition: Draft.
Matching, survivorship or identifier-mapping rules in master data: Draft, as a hard ceiling.
Correction of master records in production: Assist, through the stewarded change channel ‐ agents prepare, humans approve.
Changes to the controls themselves ‐ CI rules, test thresholds, policy as code, agent permissions: Assist. An agent that can edit its own gates has no gates.
Autonomy is earned with evidence and lost with escapes. A change class moves up one level only after a run of consecutive changes that passed every gate first time with no escaped defects. The bar scales with risk: 20 clean changes bound the failure rate only to about 15%, which is acceptable for documentation and far too weak for identity rules. Any escaped defect above an agreed severity demotes the class by one level automatically, and the count restarts.
- Master and reference data: where ADDLC gets strict
Analytics teams can often afford a looser version of this lifecycle. Teams that own master and reference data cannot, for three reasons:
Fan-out. A reference record is read by every product. A wrong instrument classification is wrong everywhere at once.
Silent failure. Mapping errors pass every structural test. The row is valid; it is attached to the wrong entity.
Identity is semantic. Tickers are reused, instruments are renamed, corporate actions merge and split entities, and vendors disagree on listing dates. Identity resolution is a stream of judgments, not a join.
ADDLC adds four requirements for this domain:
Matching and survivorship rules are meaning. They are decided at the Meaning gate and capped at Draft. Agents propose candidate matches with confidence scores; matches above the agreed threshold are applied only under rules a steward approved; everything below goes to the steward queue with evidence. Candidate matches stay in the layer below Gold until survivorship produces the golden record.
A stewarded change channel. Corrections and patches are data operations. They are isolated from code deployment, validated by the same contracts as the pipeline, attributable (who, what, why, on whose request), reversible (versioned or bitemporal storage, so any correction can be undone and any past state reconstructed), and subject to four-eyes approval for high-criticality entities. Integrators and operators ‐ and their agents ‐ cannot unilaterally break master data.
Entitlements travel with the data. License terms and entitlements are metadata that agents must read and propagate. Moving licensed data to a new consumer is a Meaning-gate decision.
Reconciliation is mandatory in Verify. Every release that touches identity is reconciled against at least one independent source before cutover.
A hypothetical walkthrough
Onboarding a new exchange’s instrument reference feed ‐ a composite scenario, not a case study:
Frame. Consumers and value are clear. The risk tier is high: the feed populates the canonical instrument model.
Specify. The agent profiles the feed, maps vendor fields to the canonical instrument model, flags the attributes it cannot map with confidence, and drafts expectations (for example, every active instrument has exactly one primary listing) and acceptance examples from well-known instruments. The steward resolves the ambiguities and approves the matching rules.
Build. The agent generates Bronze ingestion from the standard vendor template, Silver conformance and matching, and Gold publication, with tests derived from the spec.
Verify. Deterministic tests run; instrument counts and identities are reconciled against an independent source; a verifier agent hunts for instruments matched to two entities; the steward signs off a sample.
Release. A shadow run alongside the old path, then an atomic cutover. The handover package includes a run skill for the typical failure classes of this feed type.
Operate. On the first corporate-action day a mass-change guard fires. The run agent classifies it as a decision and attaches evidence, and the steward approves the change through the stewarded channel ‐ no developer is paged.
- Risk, regulation and licensing
For regulated firms, most of ADDLC maps onto controls they already run: Segregating Build from Run is segregation of duties, extended to non-human identities.
The stewarded change channel and bitemporal history are what accuracy and integrity requirements for critical data, such as BCBS 239 for banks, expect.
Evidence packs and decision records are the audit trail an examiner will ask for. In the EU, the AI Act’s logging requirements for high-risk systems point the same way.
Your model provider is an ICT third party ‐ for EU financial entities, under the Digital Operational Resilience Act (DORA ‐ the EU regulation, not the DevOps metrics) ‐ and belongs in third-party risk management like any other critical vendor.
Market data needs one more warning. A model provider that processes licensed data is itself a consumer of that data. Exchange and vendor licenses often restrict redistribution and derived data, so sending licensed fields to a third-party model can breach them. Check licenses at the Meaning gate before any licensed field reaches a model outside your perimeter, and prefer deployments where the provider neither retains nor trains on your inputs.
- Metrics
Accelerate gave software delivery four numbers everyone could compare. ADDLC proposes four families, always reported per change class and autonomy level, against a baseline taken before agents were introduced.
Flow
Lead time for data: from the Intent gate to data in production, broken down by phase, so you can see where time goes.
Human effort per change: hours by role ‐ owner, steward, engineer, operator. This is where the economic case has to be proven: more integrations from the same team, at an equal or lower change failure rate.
Throughput: changes or integrations released per period, per team.
Stability and trust
Change failure rate: the share of releases that cause an incident, rollback or correction.
Data downtime, as defined in Data Quality Fundamentals: number of incidents × (time to detection + time to resolution).
Consumer-detected share: the share of incidents first reported by consumers rather than by monitoring. If users find your incidents, Operate is not working, however green the dashboards look.
Escaped defects per gate: defects found after a gate that the gate should have caught.
Agent effectiveness
First-pass yield per gate: the share of agent-produced packages that pass the Meaning, Evidence or Operability gate without rework.
Human review minutes per change ‐ read together with escaped defects: minutes trending to zero with a flat defect rate is rubber-stamping, not efficiency.
Escalation rate and quality: did the agent ask the right questions, with evidence?
Cost per change: tokens plus compute plus human time.
Context health
Coverage: the share of assets with an owner, a contract and glossary-linked semantics.
Freshness: the age of skills and standards relative to the last relevant incident.
Learning rate: the share of incidents and rejected gates that produced a context update.
Generated lines of code, pipelines created and “share of code written by AI” are not success metrics. They reward sprawl.
- Maturity model
Ad hoc. Individuals use coding agents on their own initiative. No shared context, no agent identities, no evidence. Agent output is indistinguishable from human output ‐ including its mistakes.
Assisted. Shared context exists: agent instruction files, standards, templates. Agents draft code and documentation; humans review everything. Gates are informal.
Spec-driven. Specifications and contracts are the system of record. All four gates are defined, with evidence packs. Build and run agents have separate identities. Baseline metrics exist.
Governed autonomy. Autonomy is set per change class and moves with evidence. Run agents triage incidents. Master-data changes go only through the stewarded channel. Context is owned, versioned and tested. Metrics drive autonomy decisions.
Self-organizing. An engine maintains the asset graph ‐ synthesizing, consolidating, repairing ‐ within policy. Humans own meaning and policy.
For most organizations, and for every master-data domain, Governed autonomy is the right target. Self-organizing is optional, and some change classes should never go beyond Draft, whatever the maturity level. 13. Anti-patterns
Prompt-and-pray pipelines. Code generated from a chat message, with no spec to verify against. If nothing defines correct, nothing can be verified.
Generation sprawl. A new asset for every request, because generating is cheaper than understanding. Fix: reuse before you generate, with a reuse search at the Intent gate and a reuse report at the Meaning gate.
Self-certification. The same agent, in the same context, writes the code and the tests. Fix: tests derived from the spec, an independent verifier, reconciliation.
Documentation slop. Long, generated, unreviewed text. It costs readers more time than it saves writers and teaches people to ignore documentation altogether. Fix: generated documentation goes through the gate of its change class, and shorter wins.
Shadow autonomy. Agents running on a person’s production credentials. Fix: separate agent identities with scoped, short-lived credentials. Be especially strict when an agent reads untrusted content ‐ vendor emails, web pages, tickets ‐ while also having access to private data and the ability to communicate externally. Simon Willison calls this combination the lethal trifecta.
Dual authority. A generator regenerates an artifact that humans also edit by hand, and the next run silently erases their edits. Fix: exactly one authoritative source per artifact ‐ the trusted-source principle DMBOK applies to master and reference data, extended to every artifact an agent touches.
Context rot. Instruction files written once and never maintained. Fix: context has owners and is updated by the learning loop.
Autonomy by enthusiasm. Granting the Delegated level because a demo looked good. Fix: earned autonomy (section 8).
- How ADDLC maps to the canon
Most of ADDLC is existing practice, reassigned to non-human executors:
DAMA-DMBOK. Data governance becomes gates and decision rights. Metadata management becomes the context layer. Data quality runs through Specify, Verify and Operate. Reference and master data management gets the stewarded change channel. Data integration and interoperability lives in Specify and Build. Data security covers agent identities and entitlements. Data modeling and design provides the domain and semantic models. Data storage and operations is the Run loop.
Data Mesh (Zhamak Dehghani). Data as a product: the handover package is what makes a product operable by someone other than its authors. Domain ownership: owners decide at the Intent and Meaning gates. Self-serve platform: agents with standard skills become the platform’s interface. Federated computational governance: “controls enforce”.
Data Management at Scale (Piethein Strengholt). Metadata-driven automation, data contracts and domain interfaces ‐ the preconditions for letting agents execute safely at scale.
Fundamentals of Data Engineering (Joe Reis, Matt Housley). The data engineering lifecycle runs inside Build and Operate, and its undercurrents become agent context and controls.
Data Quality Fundamentals (Barr Moses, Lior Gavish, Molly Vorwerck). The observability pillars and data downtime anchor Operate and the stability metrics.
Designing Data-Intensive Applications (Martin Kleppmann). Systems of record versus derived data inspire the first principle; immutability and replayability make verification, rollback and reversible corrections possible.
ISO standards. ADDLC is not an ISO standard, but it is meant to sit alongside them: ISO/IEC 5338 defines life cycle processes for AI systems, ISO/IEC 42001 the management system around AI, the ISO/IEC 5259 series data quality for analytics and machine learning, and ISO 8000 data quality and master data. ADDLC covers what they leave open ‐ how data products are built and run when agents do most of the execution.
Current building blocks that fit ADDLC well: the Open Data Contract Standard for contracts, OpenLineage for data lineage, OpenTelemetry-based tracing for agent actions, semantic layers such as dbt MetricFlow, the Model Context Protocol for governed agent access to tools, spec-driven development tooling such as GitHub Spec Kit, and AWS AI-DLC as an analogous lifecycle on the software side.
- Getting started: the first 90 days
ADDLC is meant to be adopted incrementally. A pragmatic sequence:
Weeks 1‐2: baseline and inventory. Measure today’s lead time, human effort per integration and incident profile. Find out where agents are already in use, and with whose credentials.
Weeks 3‐6: context and identities. Write agent instruction files and encode standards as skills, link the glossary, give agents their own identities, and introduce the spec template and the Meaning gate.
Weeks 7‐10: one change class, full lifecycle. Pick a frequent, bounded class ‐ for example, new ingestion from a known source type ‐ and run it at Draft or Supervised level through all four gates. Shape the evidence pack on real changes.
Weeks 11‐13: handover and decision. Run the Operability gate with an operational test and run-agent skills. Compare the metrics with the baseline. Decide, with evidence, whether to promote the class and which class comes next.
Keep the Meaning gate strict and everything else light at first. Most of the value of ADDLC comes from agreeing on meaning before generating code.
- What ADDLC is not
Not a tool or a platform. Any stack can implement it.
Not a replacement for SDLC, DataOps or data governance. It is a profile of them for agentic execution.
Not an argument for full autonomy. It is a way to decide, one change class at a time, how much autonomy is justified.
Not only for analytics. It applies to ingestion, integration, master and reference data, and data operations. Low-latency streaming paths need adapted gates ‐ shadow runs instead of backfills, for example ‐ but the decision rights stay the same.
- Help shape it
The specification lives at github.com/savvakrasavasavva/addlc: 67 numbered requirements, three conformance levels, the MRD profile, templates and a self-assessment. Disagreement backed by practice is the most useful contribution: open an issue with what you ran, what broke and what you would change.
ADDLC grew out of practical work on market-data reference and master data. The early results above come from one platform; whether the model holds elsewhere is exactly what the metrics in section 11 are for. The model will be wrong in places, and the fastest way to improve it is disagreement backed by practice.
I am particularly interested in:
change classes and autonomy ceilings from your domain;
gate criteria that worked, or failed, in real teams;
metric baselines from before and after agents;
where the model breaks for streaming, unstructured data or ML features.
Leave a response here. Material improvements will be folded into the next revision.
Acceptance examples cover normal, edge and known-bad cases
License and entitlement check done
Open questions listed, each with an owner
Evidence gate
All spec-derived tests pass; coverage of expectations reported
Reconciliation against an independent source or invariant
Independent verifier findings resolved or explicitly accepted
Consumer (or steward) sample sign-off for medium and high tiers
Cost of build and verification recorded
Operability gate
Owner, escalation path and on-call defined
Runbook covers each known failure class
Alerts fire end to end into the correct queue; noise within budget
Run-agent skills triage a seeded failure correctly
Operator access verified
Rollback plan tested
Early-life support window agreed
Appendix B. Self-assessment
Answer yes or no. Your level is the highest one for which every answer at that level and below is yes. If any Assisted answer is no, you are at Ad hoc.
Cravero, A., Díaz-Villegas, J., Xiao, Z. “From Data Quality to Quality of Agentic Data Use: A Conceptual Framework for Agentic Data Engineering”. Applied Sciences 16(17), 8887, 2026. https://doi.org/10.3390/app16178887
ISO/IEC 42001:2023. Information technology ‐ Artificial intelligence ‐ Management system.
ISO/IEC 5259 series. Artificial intelligence ‐ Data quality for analytics and machine learning (ML).
ISO 8000 series. Data quality.
I lead data management ‐ data quality, ops, stewardship and integration. ADDLC is a personal proposal: examples are composite, results are rounded and relative, and the views are mine.