Half the market is arguing about whether RAG or a semantic layer is the right foundation for enterprise AI.
They are not competing. They answer different questions, and most teams need both.
Every question an agent receives breaks into one of two forms:
Treating them as rivals is how teams end up with a system that can quote the pricing policy but cannot tell you this quarter's realised price.
| RAG | Semantic layer | |
|---|---|---|
| Good at | Retrieving relevant prose | Resolving definitions and joins |
| Fails on | Aggregation, math, current state | Anything not modelled as data |
| Permissions | Flattened at ingest, rebuilt at query time | Compiled per person, per query |
| Answer stability | Varies with retrieval ranking | Identical by construction |
| Audit story | Cites a chunk | Reproduces the exact SQL |
The permissions row is the one that ends pilots. A retrieval index that ingested everything has, by construction, assembled your most sensitive object — and reconstructing entitlement at query time is guesswork.
Neither a document chunk nor a metric definition is worth much until something compiles it into a governed query and runs it.
That is the piece most architectures are missing: intent → context resolution → constrained planning → governed execution. RAG can feed the first step. It cannot perform the last three.
Point an agent at raw tables and the best models score in the low teens on real enterprise data. Give the same model compiled, governed context and it clears the high nineties. The retrieval quality was never the bottleneck.
The full breakdown — the precise division of labour, why hybrid architectures win, and how compile-time governance closes the gap RAG cannot — is here:
👉 RAG vs. Semantic Layer: Why AI Needs Deterministic Governance Originally published at colrows.com/blogs/rag-vs-semantic-layer