Enterprise AI agents belong where your data, governance, and policies already live.
by Kaan Kuguoglu and John Karlsson Most enterprise AI pilots clear the same low bar: connect an LLM to your data, drop in a vector database, demo it to leadership. The hard part shows up later. Security flags the governance holes. Latency in multi-step agents kills the user experience. The bill from your model provider keeps climbing. These problems usually trace back to one decision: pulling data out of governed systems and into an AI stack that was never built to enforce your policies.
This post argues for a different architectural direction: move models and agents to the data, not the other way around. Instead of building a parallel AI infrastructure and wiring it back to your lakehouse, you treat agents as native workloads that run inside your data platform, under the same governance, security, and observability controls you already trust for your data.
The lakehouse gave you one place to govern your data. The next question is whether your agents live inside that boundary or outside it. There are two emerging paradigms.
Agents and LLMs run in a separate AI stack. Data is exported or queried over the network into external vector databases, SaaS LLMs, or bespoke serving layers. Governance, security, and observability are re-implemented for AI on the side.
Agents, models, tools, retrieval, and agent memory run inside the same platform as the data itself, under a unified governance and security layer. AI becomes another workload on your existing data stack.
Data has gravity. Compute is cheap to relocate; data is not, especially as volumes grow and modalities multiply. Pulling it out introduces a familiar set of penalties:
Of all these penalties, governance deserves special attention because it's the one that can't be patched after the fact. Most approaches to AI governance treat it as a filter applied after the agent has already accessed data, e.g., redacting sensitive fields from the response, blocking certain topics at the output layer, and auditing logs after the fact. This works for simple Q&A demos. It breaks down the moment agents start computing on data.
Consider an agent that calculates a financial summary across rows subject to row-level security. The aggregation itself (the sum, the average, the trend) is a derived value that is shaped by which rows are included. If governance isn't enforced before the query runs, the result already encodes data the user shouldn't have influenced. No amount of downstream redaction can undo that calculation. The policy decision needed to happen at query planning time, not at response rendering time.
This is the fundamental gap in perimeter-based or post-hoc governance: it assumes data can be safely censored after it reaches the agent. In practice, once an aggregation or transformation has occurred, the governance intent is already lost. Retroactive controls are fundamentally incomplete.
There's a second cost to getting this wrong, and it shows up on the bill. The trouble isn't only what the agent is allowed to touch, though that's part of it. When governance is resolved after the fact instead of at the source, the agent ends up doing the reconciling itself: traversing audit logs, joining fragments across external systems, pulling the same record from several places to work out whether it can use it, re-reasoning over every partial result. None of that is the task. It's the agent compensating for a governed answer that was never handed up front. Blocked or redacted outputs only feed the spiral, since the agent reads them as failures and tries again. Sessions stretch out, each hop loads more context into the model, and one request quietly turns into thousands of billed tokens. That's the token-burning loop, and post-hoc governance is what sets it spinning.
Data-native agents address these challenges by embedding policy enforcement directly into query planning and computation. Every intermediate result reflects the same governance constraints. Governance must be evaluated before and during execution, which you can’t afford to apply as an afterthought after the computation is complete. Custom Guardrails in Unity AI Gateway are the concrete form of this: composable, deterministic policies the gateway enforces on every request and response, not filters the model is asked to obey after the fact. Deciding policy at planning time, against data that already lives in one governed place, also means the agent gets a clean answer in a single pass instead of spelunking across systems to assemble or justify one.
So far, we've focused on how agents read data. But production agents also write: conversation history, task progress, user preferences, cached results, and tool outputs for later audit. As agents take on more, the state layer matters as much as the data layer. Leave it outside the governance boundary, and every promise of end-to-end auditability has a hole in it.
State is the agent's short-term scratchpad: the live conversation, the task in flight, the cache it just filled. Memory is what outlives the session: which customers an agent has dealt with, what a user prefers, which past outputs are worth reusing. Both need transactional storage, and both leak governance the moment they leave the platform. A memory like "user X is a high-value EU customer" is itself sensitive data, bound by the same access and residency rules as the record it summarizes. And it's a transactional workload: Delta tables are built for big analytical scans, but agent state needs fast per-row reads and writes, keyed lookups, and atomic updates.
The usual workaround is an external Postgres or Redis. But that drops you back into the problem this post argues against: agent state leaves the governed perimeter into a system your governance layer can't see, with its own security and lifecycle to manage. You've built a data-native agent with an ungoverned dependency.
And the problem grows the moment one agent becomes many. A swarm working toward a larger outcome (a planner delegating to specialists, a supervisor reconciling their results) needs shared memory, and staying aligned on it is where these systems break. When each agent keeps private state and passes context peer-to-peer, there's no single source of truth. Agents diverge, writes collide, and every handoff is another ungoverned channel that multiplies as the swarm grows. Exchanging memory turns out to be the hard part.
Lakebase closes these gaps. It's fully managed, PostgreSQL storage inside the Databricks platform, under the same governance plane as everything else. Agent state becomes a governed asset: it inherits the platform's access controls, lives alongside the agent's data and tools, and needs no separate infrastructure team. And because every agent reads and writes that same transactional layer, it doubles as the swarm's single source of truth. State stays consistent, atomic updates keep two agents from corrupting the same task, a policy on one agent's view carries to whatever it writes for the next, and any memory traces back through the swarm: which agent wrote it, which read it, how a deep conclusion reaches its source.
The governance argument is the sharpest, but the advantages go further. When agents run inside the data stack, the advantages compound across every operational dimension: security, quality, observability, deployment, latency, and cost. Tools and data dependencies are configured and logged alongside the model, so the entire system is versioned, auditable, and reproducible by default.
The following comparison summarizes how these two paradigms differ across the dimensions that matter most for production systems:
| Means of Trust & Control | Data-Native Agents | External Agents |
|---|---|---|
| Governance | A single control plane for data and agents, with policy enforcement embedded directly into query planning and execution. | Needs replicating governance in every AI component: warehouses, vector DBs, SaaS LLMs all need their own ACLs, masking rules, and token policies. Fragmented governance, lineage, and often a lack of FGAC. |
| Security | Data and models stay inside your cloud perimeter/VPC. | Data often leaves your secure perimeter, creating additional attack surfaces. |
| Agent Quality | End-to-end tracing enables systematic, platform-native evaluation. | Evaluation is fragmented and manual, with logs spread across multiple external vendors. |
| Data Quality | Consistency is built in via shared data pipelines; freshness is platform-managed. | Disconnected data quality controls require brittle, bespoke sync processes to keep agents' data fresh. |
| Observability & Monitoring | Holistic evaluation and monitoring capture all steps centrally, alongside model and agent versions. | Logs are scattered across tools, making end-to-end troubleshooting difficult and slow. |
| Agent Memory | Conversation history, user preferences, and learned context persist in Lakebase, governed by Unity Catalog, joinable to the underlying business data, and bound to the same identity model as the rest of the stack. | Memory lives in a separate Redis or Postgres instance, with its own access model, no lineage to source data, and no way for governance to see what the agent has retained about a user. |
| Business Context | The same Unity Catalog that governs the data models its meaning. Metrics, Glossary, Domains, and Genie Ontology give agents your business definitions automatically, ranked by authority and respecting source ACLs. | Business meaning lives outside the perimeter, in BI tools, spreadsheets, or domain experts' heads. Each agent has to rebuild it, often from column names alone. |
| Deployment | Simplified CI/CD with the full stack (data, models, agents) versioned together. | Requires separate CI/CD pipelines and complex coordination across multiple vendors. |
| Latency | Low-latency because agents run close to the data, minimizing network hops. | High latency due to multiple network round-trips for each retrieval and tool call. |
| Cost | Consolidated serving and storage (data is stored once), avoiding egress costs. | Fragmented pricing and substantial egress costs for moving data at scale. |
The Data Intelligence Platform puts this into practice. Instead of building a separate "AI stack" and wiring it to your data, you build AI agents inside the data stack itself. A data-native agent on Databricks is one that:
Some enterprises are already building this way. Their agents run inside the same boundary as their data, with Unity Catalog policies covering both, state in Lakebase, and traffic through Unity AI Gateway. None of them got there with a single re-platforming project. They closed the seam between the lakehouse and the AI stack one pilot at a time, then stopped opening new ones.
For teams earlier on the path, the work isn't dramatic. Most of the pieces already exist on the platform side: Model Serving, Unity Catalog, Lakebase, MLflow. What's missing is the decision to treat them as the agent's home rather than as data sources for a parallel stack. That decision tends to be the hardest part. The most useful place to start is an inventory of what's already running outside the perimeter. That's where the next governance incident is going to come from, and that's the work that makes everything else possible.
If you want to go deeper on how to operationalize data-native agents with the right guardrails and patterns, these resources are a great next step: To explore the platform capabilities discussed in this post:
Subscribe to our blog and get the latest posts delivered to your inbox.