Contestable Agents for Data Governance: Recording Human Stewardship of AI-Generated Metadata Across Structured and Unstructured Systems

A new ArXiv paper (2610.31553v1) proposes "contestable governance," a pattern in which LLM agents produce scored, cited arguments for metadata decisions that human stewards can accept, reject, modify, or defer before commit, with a separate governance record retaining the evidence, interventions, and outcome. The design routes high-confidence proposals (score ≥ 0.85 with no conflicts) to auto-commit while sending uncertain ones to a steward review queue, and it stores governance records in a dedicated store rather than inline with operational data so they can be queried for audit trails, drift detection, and agent retraining. A reference implementation classifies HVAC telemetry points against Brick Schema labels, using a versioned glossary that can be replayed to surface semantic drift when definitions change.

Organizations deploying LLMs to classify operational data face a governance gap. Agents tag telemetry points, link warranty claims to service bulletins, and propose metadata for unstructured documents. When a human overrides an agent decision, most systems discard the agent's reasoning and store only the final value. You lose the audit trail, the correction context, and the signal needed to prevent classification drift. A new ArXiv paper 2610.31553v1 introduces contestable governance : a pattern where agents produce scored, cited arguments for metadata decisions, stewards can accept or modify those arguments before commit, and a governance record retains the evidence, interventions, and outcome. The plumbing treats human corrections as first-class events, not silent overrides. Industrial systems hold operational knowledge in two forms: Governance programs manage these separately. Linking a telemetry point to its manual section, classifying a warranty failure mode, or tagging an HVAC zone requires domain specialists. Agents can propose classifications, but unreviewed proposals spread errors. A perfunctory approval leaves no record of what the reviewer actually checked. The failure mode: agent-generated metadata drifts as equipment changes, terminology evolves, or edge cases accumulate. You discover the drift when downstream analytics break or compliance audits fail. The pattern introduces three primitives: Agent Proposal → Triage → High Confidence → Auto-Commit → Uncertain → Steward Review → Governance Record High-confidence proposals score above threshold commit automatically but still generate a governance record. Uncertain proposals enter a review queue. The steward sees: The steward can: The governance record schema: @dataclass class GovernanceRecord: record id: str timestamp: datetime target entity: str telemetry point ID, claim number, document hash agent proposals: List ArgumentProposal steward decision: StewardDecision final metadata: Dict str, Any precedent flag: bool version: int @dataclass class ArgumentProposal: agent id: str proposed value: Any confidence score: float citations: List Citation reasoning: str @dataclass class StewardDecision: steward id: str action: Literal "accept", "reject", "modify", "defer" justification: str added citations: List Citation The record lives in a separate governance store, not inline with the operational data. You query it for audit trails, drift detection, and agent retraining. Agents reference a versioned glossary that supplies: When the glossary updates, you can replay governance records to detect which decisions would change under new definitions. This surfaces semantic drift before it propagates. The paper presents a reference implementation for classifying HVAC telemetry points using Brick Schema labels. Input : A telemetry point with raw metadata: { "point id": "AHU-3-SAT", "description": "Supply air temp sensor zone 2", "unit": "degF", "equipment parent": "AHU-3" } Agent task : Propose a Brick Schema class e.g., brick:Supply Air Temperature Sensor . Workflow : brick:Supply Air Temperature Sensor Uncertain case : { "point id": "RTU-7-OAT", "description": "Outside temp", "unit": "degC", "equipment parent": "RTU-7" } Agent proposes brick:Outside Air Temperature Sensor with 0.68 confidence ambiguous: could be brick:Outdoor Air Temperature Sensor . Steward reviews, confirms the correct class based on equipment manual, and adds a citation. The governance record captures both the agent's uncertainty and the steward's resolution. The triage logic routes proposals based on confidence score and similarity to prior decisions: | Condition | Path | Governance Record | |---|---|---| | Score ≥ 0.85, no conflicts | Auto-commit | Minimal proposal + outcome | | Score < 0.85 or conflicts present | Steward review | Full arguments + intervention | | Steward marks as precedent | Precedent store | Indexed for future similarity search | The precedent gate controls which verified decisions become training examples. Only decisions where: This prevents auto-committed high-confidence decisions from polluting the precedent store with obvious cases. You need metrics to detect when agent metadata quality degrades: Drift indicators : Alerting thresholds : Dashboard queries : -- Detect classification drift by equipment type SELECT equipment type, COUNT FILTER WHERE action = 'reject' AS rejections, COUNT AS total proposals, AVG confidence score AS mean confidence FROM governance records WHERE timestamp NOW - INTERVAL '30 days' GROUP BY equipment type HAVING COUNT FILTER WHERE action = 'reject' / COUNT 0.15; The paper's second use case links warranty claims to service bulletins and telemetry patterns. Input : A warranty claim with fields: { "claim id": "WC-2024-8832", "failure description": "Compressor short cycling", "equipment serial": "RTU-2024-4471", "claim date": "2024-09-15" } Agent tasks : Agent output : Steward review : The governance record now links the claim, the telemetry evidence, and the correct bulletin version. When SB-2024-02 is revised again, you can query governance records to find all claims that referenced it. The feedback loop: Retraining trigger : When override rate exceeds threshold, you can: The key: you have structured data governance records instead of unstructured feedback emails, tickets, manual corrections in spreadsheets . | Approach | Pros | Cons | |---|---|---| | Inline metadata + governance log | Single source of truth for current state | Governance records grow unbounded | | Separate governance store + metadata snapshots | Clean separation, easier to archive old records | Requires snapshot reconciliation logic | | Event-sourced metadata | Full history, time-travel queries | Complex rebuilds, storage overhead | The paper recommends separate governance store with periodic metadata snapshots. You keep the operational database clean and query the governance store only for audits and retraining. Access control layers : Citation integrity : All citations include document hashes or version identifiers. If a cited document changes, the governance record flags the citation as stale. Steward identity : Each governance record logs the steward's identity and timestamp. You can audit who approved which decisions and detect patterns e.g., one steward consistently overriding a specific agent . Minimal stack : Integration points : Scaling considerations : Steward fatigue : If too many proposals land in the review queue, stewards rubber-stamp decisions. Mitigation: tune confidence thresholds, improve agent prompts, surface precedent matches to reduce cognitive load. Glossary staleness : Agents cite outdated terms, stewards correct them, but glossary never updates. Mitigation: automated glossary update proposals when override rate for specific terms exceeds threshold. Citation rot : Linked documents move, get revised, or deleted. Mitigation: periodic citation validation job, flag stale citations in governance records. Precedent pollution : Auto-committed decisions leak into precedent store. Mitigation: strict precedent gate only steward-reviewed, initially-uncertain decisions qualify . Conflicting precedents : Two stewards resolve similar cases differently. Mitigation: precedent conflict detection, escalation to domain lead for reconciliation. Use contestable governance when : Avoid when : The pattern shines in regulated industries energy, manufacturing, healthcare where metadata errors propagate into compliance failures. It adds complexity: you maintain a separate governance store, build a steward UI, and tune confidence thresholds. The payoff is structured feedback that improves agent performance and prevents silent drift. If you are already logging agent decisions, contestable governance is the next step: capture not just the outcome but the reasoning, the human intervention, and the evidence. You turn corrections into training data instead of lost context.