{"slug": "governance-beyond-security-knowledge-context-ontology-on-the-lakehouse", "title": "Governance beyond security: knowledge, context & ontology on the lakehouse", "summary": "Databricks Inc. argues that data governance artifacts such as classification tags, model cards, and data contracts should be treated as semantic metadata to build a foundation for AI, enabling cheaper models and greater trust. The company's Data Empowerment Program (DEP) outlines a five-pillar vision where governance metadata in Unity Catalog serves as runtime instructions for 'build agents' and 'analytic agents,' with de-identification and testing agents prioritized to reduce risk and manual work.", "body_md": "Read your governance artifacts as semantics, and the audit work you already do becomes the foundation for a best-in-class AI strategy – with cheaper models and more trust.\n\nAsk most organizations what data governance for AI means, and you’ll hear a security answer: lock it down, restrict access, pass the audit. In healthcare, security is non-negotiable — but it’s incomplete. Security tells you who can touch data. It says nothing about what the data **means**, whether it can be **trusted**, or whether an AI model should ever **learn from it**.\n\nOur Data Empowerment Program (DEP) starts from a different premise: governance is **knowledge, context, and ontology**; not just controls. Artifacts most teams treat as compliance overhead, such as classification tags, de-identification policies, model cards, and data contracts are raw material for enterprise data semantics.\n\nWhen seen this way, you are not choosing between governance and AI, but instead, governance helps build AI. New approaches to governance need to be implemented in the AI era. The only question is whether you do the work later just to pass the audit, or now, to lay the foundation your AI runs on.\n\nOur goal is to show that the security and governance work you already do is the foundation your AI runs on. Govern the data well enough, and AI can run on cheaper models with more trust.\n\nStart with the lens that every governance artifact contributes to semantics. Every classification tag is a concept. Every model card is context. Every data contract is a shared definition. Every lineage link is a relationship. Read that way, the security stack you already run is the first draft of your ontology, and the catalog is where it lives.\n\nGovernance then stops being one thing and becomes five facets of a single discipline: the data itself and how it’s controlled, the AI built on top of it, the people who need to understand it, the products that carry it into the business, and the shared context that ties all four together. It’s the same lens, but from five fronts.\n\nWith DEP, we envision semantics through five pillars:\n\nOur five-pillar vision ends as only slideware unless the platform can carry it out into something operational. Once your governance artifacts live as structured, machine-readable metadata, they stop being just documentation and start being instruction sets for agents.\n\nWhen we refer to \"agent\", we are thinking about it in two ways: “build agents” that assemble and deliver data products, and “analytic agents” that answer business questions on top of them; each bound to a single data product.\n\nLet’s start with build agents. Build agents automate the delivery lifecycle of data products from source mapping through ETL, testing, and de-identification to a production release. Everything they need lives in Unity Catalog as governed metadata: source-to-target mappings, business definitions, classification tiers, deidentification policies, data contracts, and model cards. The platform derives from tags, comments, certified flags, lineage, and glossary-linked terms. The catalog isn’t where you just document governance; it’s the runtime the agents execute against.\n\nEach agent works in a loop. It reads instructions from the catalog; does one concrete task such as generating pipeline code, running a test suite, producing de-identified data, or deploying a certified dataset; and then writes the evidence back as test outcomes, quality scores, lineage, or change capture data. This repeats.\n\nIn practice, we sequence the De-ID and Testing agents first. They eliminate the highest risks and the heaviest manual work up front. Starting where the payback is fastest helps build momentum early. As we continue through the loop, no agent acts on data the catalog doesn’t describe.\n\nModern catalogs make this approach scalable because it can auto-generate column and table descriptions for a steward to approve, classify sensitive fields automatically, and capture column-level lineage without anyone maintaining it by hand. The **human role shifts from authoring the metadata to approving it**, which is exactly the kind of judgment work humans should be doing.\n\nBuild agents operate within an end-to-end lifecycle designed to release two assets simultaneously: the governed data product (mapping, curation, pipeline) and the analytic agent running on top of it (semantic layer, prompt configs, eval suites).\n\nThis approach marks a fundamental shift from pipeline-centric engineering (moving data from point A to B) to context-centric engineering (making data understandable and actionable for LLMs). Rather than certifying code quality alone, the gates in this lifecycle validate semantics, context, and ownership.\n\nTwo core properties distinguish this framework from a traditional SDLC:\n\nHuman stewards serve as the accountability layer for both properties: agents propose, people approve. While managing five gates across two tracks might look like creating lengthy bottlenecks, most gates can clear in mere hours. Approvals take place directly inside standard developers' tooling. Automated test suites attach data quality results, eval scores, and lineage before a ticket is opened. A formal gate meeting is an exception to investigate, not the standard operating procedure.\n\nThe mechanism that makes these gates objective rather than arbitrary is the AI Certification. Recorded directly in Unity Catalog, this certification acts as an automated, query-able scorecard rather than a manual legal attestation. It governs release eligibility across four core dimensions:\n\nCertification and gates prove that an agent was trustworthy at release. But the question governance leaders ask isn't \"how does it work\"—it's \"who is accountable when it gives the wrong answer?\" The answer must be a specific name, not a steering committee.\n\nTo solve this, each analytic agent (e.g., a Databricks Genie Agent) is bound to a single governed data product with one designated owner. When an agent returns an incorrect result because an underlying metric was misdefined, the issue doesn't belong to the AI engineering team. Instead, it goes straight to the Data Product Owner, who corrects the catalog definition. Binding an agent to a domain-scoped, certified data product is also the single largest accuracy lever available: a focused agent querying certified metadata consistently outperforms a global model guessing across an entire enterprise estate.\n\nCrucially, this shared metric definition is enforced rather than merely documented. Once a certified metric is defined in the catalog, the answer agent is required to compute directly from it. This turns static documentation into active runtime logic.\n\n*Accountability is held because of a firm limit on what the AI is allowed to do unattended: *no agent promotes code to production, modifies policy, or operates on unclassified data without human intervention. While certification scores are calculated automatically, the final release gate always requires a human signature. If the catalog doesn't explicitly describe a data asset, the system defaults to suppression rather than guessing. At runtime, this fail-closed policy enforces clear boundaries:\n\nDefining these guardrails on paper is easy, but making them work in practice requires replacing vague governance committees with four distinct, accountable roles:\n\nThe lifecycle we described has a hard prerequisite hiding inside it: every one of those test and evaluation stages needs realistic data to run against – and in healthcare, you can’t test real PHI. So, the challenge becomes the need for realistic test data everywhere without compromising security.\n\nDe-identification is how we keep data analytically useful and safe. Where does the de-identification agent get its knowledge? Not from a hand-maintained spreadsheet. It works from security policies the enterprise tools already produce. The flow is three steps:\n\nFor the security & IAM team, this is a two-way street. InfoSec policies stop being PDFs and become executable: classification tiers and retention rules drive de-identification automatically. In return, security gains a continuously updated view of sensitive data, fail-closed protection for anything newly discovered, and residual scans that generate audit evidence on every run. The access model stays the same from end to end. Because any agent data retrieval inherits the querying user's catalog grants, RAG approaches can't surface an embedding of a row the user isn't entitled to see. The same ABAC rules span SQL and vector search alike, and agents act with the querying user's entitlements, not a privileged service account. Every agent prompt is logged with the lineage used to answer it, under the same governance as the data itself.\n\nThat is the real unlock: one permission model over the data, the models, the embeddings, and the audit trail — not a data catalog stitched to a separate model registry, stitched to a separate vector store. Governance work becomes the AI foundation instead of a parallel project.\n\nNotice what the lifecycle has been doing this whole time: every stage, every gate, every certification has been producing metric. Roll the four certification dimensions into a single AI-readiness score per dataset, and make it operational, not aspirational. Semantics hits 100% only when every column carries a glossary-linked definition and the table has a signed data contract; Ownership hits 100% only when a named owner is responding to issues.\n\nThe outcome that follows the scoring is the business case for the whole DEP program: metrics prove the AI's results, proof earns trust, and trust is what converts a pilot into daily usage. No business user adopts an agent because the architecture diagram is elegant. They adopt it because the numbers were right last week and someone accountable fixed them when they weren't. The score explains why the numbers come out right in the first place: the higher the score, the less the model has to guess. It isn't inferring what a column means, compensating for duplicates, or hallucinating joins - because the catalog already told it.\n\nEvery week brings a bigger, more expensive model. Here's what the hype cycle misses: when the catalog already supplies the meaning, quality, and context, the model doesn't have to. Smaller or open-weights models serve most needs for reporting and analytics on governed data.\n\nFrontier models are often used to mask underlying metadata gaps. When schemas and business rules are explicitly cataloged, smaller domain-specific models deliver identical accuracy at a fraction of the token cost.\n\nThis is a cost-to-quality choice, not a quality ceiling. Right-size the everyday work, and reserve frontier spend for the problems that truly need it, and cost never forces AI to pause. Fix the data. Right-size the model. Keep the accuracy. That's what governance roots buy an AI strategy: not cheaper AI — unstoppable AI.\n\nDon't attempt an enterprise-wide overhaul at once. Prove the model by taking one data product through the full lifecycle:\n\nOnce the loop is running, repeat the process one certified data product at a time. Security tells you who can access your data, but governance tells you what it means and whether an AI can trust it.\n\n**Governance isn't the gate in front of a data-driven organization. Done right, it's the ground under it.**\n\nSubscribe to our blog and get the latest posts delivered to your inbox.", "url": "https://wpnews.pro/news/governance-beyond-security-knowledge-context-ontology-on-the-lakehouse", "canonical_source": "https://www.databricks.com/blog/governance-beyond-security-knowledge-context-ontology-lakehouse", "published_at": "2026-09-03 20:20:00+00:00", "updated_at": "2026-09-03 20:23:12.294313+00:00", "lang": "en", "topics": ["ai-agents"], "entities": ["Databricks Inc.", "Data Empowerment Program", "Unity Catalog"], "alternates": {"html": "https://wpnews.pro/news/governance-beyond-security-knowledge-context-ontology-on-the-lakehouse", "markdown": "https://wpnews.pro/news/governance-beyond-security-knowledge-context-ontology-on-the-lakehouse.md", "text": "https://wpnews.pro/news/governance-beyond-security-knowledge-context-ontology-on-the-lakehouse.txt", "jsonld": "https://wpnews.pro/news/governance-beyond-security-knowledge-context-ontology-on-the-lakehouse.jsonld"}}