cd /news/artificial-intelligence/rag-cag-and-kag-choosing-how-enterpr… · home › topics › artificial-intelligence › article
[ARTICLE · art-173301] src=ainexusdaily.vercel.app ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

RAG, CAG and KAG: Choosing How Enterprise AI Uses What the Business Knows

A chief AI officer's 29-page whitepaper (DOI 10.5281/zenodo.24399422) argues that enterprises should run RAG, CAG and KAG behind a single governed knowledge platform and route each question to the cheapest reliable path rather than standardizing on one technique. The paper states CAG's widely cited "40x faster" figure is measured against re-reading the whole corpus per question, not against RAG, and that CAG's accuracy edge over RAG is marginal and disappears as content grows. It recommends fixing retrieval first, since contextual chunking, hybrid search and reranking cut retrieval failures by more than half in published tests, and authorizing knowledge before it reaches the model, isolating caches per tenant and reporting cost per correct, authorized answer.

recommendation

Apply contextual chunking, hybrid search and reranking before buying new tooling; published tests show the combination cuts retrieval failures by more than half, and most reviewed production systems have not done all three.

more in playbook: Retrieval and RAG
read34 min views1 publishedNov 20, 2026
RAG, CAG and KAG: Choosing How Enterprise AI Uses What the Business Knows
Image: Ainexusdaily (auto-discovered)

A CAIO's perspective on enterprise knowledge architecture. Full whitepaper (PDF, 29 pages, 11 figures): DOI 10.5281/zenodo.24399422. Across energy, manufacturing, healthcare and financial services, I keep seeing the same pattern. The model is rarely the problem. Enterprise assistants disappoint beca

use the right business knowledge is not in front of the model when it answers, or because nobody can prove the model was allowed to see what it used. RAG, CAG and KAG are three ways of solving the first problem; governance decides the second. For a CAIO or CTO, choosing between them is less a technology bet than a decision about where knowledge preparation happens, who owns it, and what each correct answer costs. RAG (Retrieval-Augmented Generation) searches a document collection when a question arrives and passes the model the few passages that matter. It is the usual starting point because it copes with large collections, frequent change and per-user permissions. CAG (Cache-Augmented Generation) loads a small, stable body of knowledge into the model once, keeps the model's processed state (the KV cache) and reuses it for later questions. It can work well for a shared handbook that is queried all day. It does not suit large, fast-changing or permissioned content.

KAG (Knowledge-Augmented Generation) first turns documents into a graph of entities and relationships linked back to the source text, then answers by planning and running a sequence of lookups and calculations. It earns its higher cost on questions that chain several facts, depend on dates, or apply rules. Mature programs do not pick one. They build a single governed knowledge platform and route each kind of question to the cheapest path that answers it reliably.

That is the direction I recommend, and the rest of this article explains how to get there without overinvesting early. Most headline numbers do not survive a reading of the papers. CAG's famous "40x faster" is measured against re-reading the whole corpus for every question, not against RAG. Its accuracy edge over RAG is marginal and disappears as the content grows.

Graphs are an investment for specific questions, not a general upgrade. Graph methods win on questions that chain facts, dates and rules, and lose to plain retrieval on simple lookups. Bigger context windows do not retire retrieval. Models lose accuracy well before their advertised limit, especially when the answer is not phrased like the question.

No approach is reliable on its own. Even the best systems in public benchmarks miss or get wrong a meaningful share of questions. Evaluation, and the ability to say "I don't know", belong in the design from day one. Five recommendations for the next twelve months Fix retrieval before buying anything new.

Contextual chunking, hybrid search and reranking cut retrieval failures by more than half in published tests, and most production systems I review have not done all three. This is the cheapest accuracy gain available. Turn on caching where the terms fit, and make CAG earn its place. Prefix caching is close to free money for stable instructions.

For small, stable, shared content, run CAG side by side with a well-tuned RAG pipeline before committing. Fund a graph only where real questions demand it. Look at what users actually ask. If enough of it is relational, time-bound or rule-bound, start with low-cost graph methods and reserve KAG for rule- and calculation-heavy domains such as compliance, claims or engineering change. Route rather than choose. A router lets each path be justified, measured and retired on its own business case, which is how you avoid a platform built around last year's favourite technique. Govern all three paths the same way. Authorize before any knowledge reaches the model, isolate caches per tenant, track where every fact came from, and report cost per correct, authorized answer to the business, not cost per token.

The picture below shows where this article lands: one governed platform in which RAG, CAG and KAG are options chosen by an orchestration layer. Figure 1. Enterprise knowledge architecture: applications on top, an orchestration and knowledge router choosing RAG, CAG or KAG, and a governed knowledge fabric over enterprise data sources. When I brief a board, I describe a language model as a brilliant new hire who studied everything public up to a certain date and then stopped reading: fluent and fast, but unaware of your company, your latest policies, and anything that happened since.

Everything a model learned in training is stored in its weights. Researchers call this parametric memory. It has three limits that matter to every enterprise: Limit What it means in practice Example Frozen in time The model knows nothing after its training cutoff It cannot tell you last quarter's revised pricing Never saw your data Contracts, maintenance logs, SOPs, customer records were never in training It cannot answer "what is our warranty term for Product X?" Cannot cite Even a correct answer arrives without a source An auditor cannot verify where the answer came from When a model lacks the relevant facts it will often still produce a fluent, confident answer. Unsupported answers of this kind are called hallucination.

The fix is to turn a closed-book exam into an open-book one: give the model the relevant pages at the moment it answers, and ask it to answer from those pages and cite them. That is the idea behind RAG, first published by Facebook AI researchers in 2020. Knowledge lives in a searchable index outside the model, where it can be updated without retraining. In public benchmarks, giving a model the right documents lifts accuracy substantially, yet even the best systems still miss a meaningful share of questions.

External knowledge is the difference between a model that guesses and one that grounds its answers. How you deliver that knowledge, and how you govern it, decides whether the result is enterprisegrade. All three approaches answer the same question — how does the right knowledge reach the model? — but they do it at different times and in different forms: RAG fetches it when asked, CAG prepares it in advance, and KAG organizes it into connections first.

Imagine three ways to help an expert answer customer questions: RAG is the research librarian. Every time a question arrives, the librarian searches a huge, constantly updated library, pulls the three or four most relevant pages, and hands them to the expert. It copes with very large libraries and stays current as long as new and changed documents are indexed promptly. But if the librarian grabs the wrong pages, the expert answers from the wrong pages.

CAG is the open binder on the desk. For a small, stable reference — say a 150-page product handbook — the expert reads it once in the morning and keeps it open. Every question that day is answered without fetching anything. Fast and simple, but only works if the binder is small, does not change during the day, and everyone asking is allowed to see all of it.

KAG is the investigator's evidence board. Before any questions arrive, an analyst pins every person, company, contract and event to a board and draws the connections between them, each string labelled and tied to a source document. Questions like "which suppliers of Plant A have contracts expiring within 90 days?" are answered by following the strings. Powerful for connected questions, but building and maintaining the board is real work.

Figure 2 · the three approaches as everyday analogies Read each column top to bottom: the difference is when the knowledge work happens — at question time (RAG), once in advance (CAG), or as a map built in advance and walked per question (KAG). The same colours mark the three approaches in every diagram that follows. The same idea in one table RAG CAG KAG In one phrase Find what is relevant Reuse what is prepared Connect facts, then reason When knowledge is selected At question time Ahead of time, once Ahead of time (the graph), then planned at question time Form the knowledge takes Retrieved text passages The model's saved "reading" of the whole corpus (KV cache) Entities, relationships and rules, linked back to text Best for Large, changing, permissioned libraries Small, stable, shared references with heavy reuse Multi-step, relational, time-bound or rule- bound questions Main business risk Wrong or missing answers when retrieval misses Stale or over-shared content High build and upkeep cost; wrong links in the graph Typical owner Platform or search team Platform team, with content owners Data and domain teams together Instead of asking "RAG, CAG or KAG?", ask three independent questions of every request — what should be selected, what can be reused, and what must be structured — because a single answer can need all three. Axis The question it answers Primary technique Typical failure if ignored Selection Which information is relevant_and authorized_ for this request?

Retrieval: keyword, semantic, SQL, API, graph lookup (RAG) The model answers from the wrong or forbidden material Reuse Which work can be shared across requests without breaking freshness, permissions or model compatibility? Caching: prompt/prefix and KV caches (CAG) Paying again and again to re-read the same handbook Structure Which entities, relationships, time limits, rules and calculations must be explicit rather than buried in prose? Knowledge graphs, logical forms, deterministic operations (KAG) Plausible but wrong joins, dates and arithmetic Seen this way, hybrids are natural rather than exotic. A cached handbook does not stop the system retrieving a newly issued notice.

A graph traversal does not stop it retrieving the supporting clause. A retrieval pipeline can still cache its stable instructions. The three axes become concrete once you sort enterprise information by what it is. Most knowledge fits one of four categories, and each has a natural home: Category Examples Preferred treatment Unstructured evidence Contracts, policies, manuals, tickets, reports Retrieval (RAG), including multimodal retrieval for drawings and scanned pages Stable shared context System instructions, approved reference sets, a product handbook Prefix caching, or bounded- corpus CAG where it tests well Relational knowledge Suppliers and the parts they supply, asset hierarchies, dependencies, ownership Graph queries (GraphRAG / KAG) or relational joins Authoritative operational state Inventory, balances, open orders, production events, credit limits Governed APIs, SQL and semantic-layer queries against the system of record The fourth category is the one most often mishandled.

A vector index or a knowledge graph should never become an accidental copy of transactional data: the copy goes stale within minutes, and it bypasses the authorization rules of the system that owns the data. RAG is the enterprise default for grounding models in large, changing, permissioned knowledge. In my experience its quality is set far more by retrieval engineering (how documents are parsed, chunked, searched and reranked) than by the choice of vector database, which is where too many programs spend their first procurement cycle. Every RAG system has two halves.

An offline half prepares the library: it parses documents, splits them into chunks, converts each chunk into numbers that capture its meaning (embeddings) and stores them in search indexes. An online half answers questions: it searches those indexes, picks the best evidence, and asks the model to answer from it. Figure 3 · RAG reference pipeline: offline ingestion and online serving The top lane runs whenever documents change; the bottom lane runs for every question and snakes from the question to a cited answer. The dashed line is the only link between them: the search step looks up the indexes the offline lane built.

The best public evidence that chunk context, hybrid search and reranking compound comes from Anthropic's 2024 contextual retrieval study. Adding chunk context, then keyword search, then reranking each reduced how often the right passage went missing, and together they cut those failures by roughly two thirds. The one-off cost of adding context was trivial with prompt caching. These were the vendor's own datasets, so test on yours, but the direction matches what I see in client pipelines: most RAG quality problems are retrieval problems, and they are cheap to fix.

I treat CAG as a deployment pattern rather than an architecture: if a small, stable body of knowledge is going into the prompt anyway, compute the model's reading of it once and reuse it. Every major model vendor now offers this as a managed feature. Its practical ceiling is set by how much context a model can actually use well, not by the window size on the datasheet. The pattern was named in a 2024 paper with the deliberately provocative title "Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks" (Chan et al.).

It describes three phases: Preload. Fit the whole document collection into the context window, run the prefill once, and store the KV cache. Infer. For each question, load the cache, append the question, and generate.

No retrieval runs. Reset. Truncate the question tokens from the cache so it returns to the clean knowledge state. "Reset" only clears the last question.

It does nothing about corpus updates: any change to the knowledge means recomputing the cache. Figure 4 · CAG reference flow: preload once, serve many The top lane is paid once per corpus version; the bottom lane runs per question. The loop on the right is the reset that returns the cache to its clean state. The only way new knowledge enters is to rebuild the top lane.

The study is widely quoted and rarely read. It used one small open model, small corpora and a soft similarity metric. CAG edged out RAG on most of the small tests by very thin margins, and lost on the largest corpus, the one closest to a real knowledge base. The authors themselves call CAG impractical for significantly larger datasets.

The famous "40x faster" compares CAG with re-reading the whole corpus on every question, not with RAG. A simple keyword RAG pipeline was faster than CAG at every size tested, and CAG's response time grows with the corpus while RAG's stays flat. What the paper proves is that if a small corpus goes into the prompt, it should be cached. It does not prove that caching beats retrieval at enterprise scale.

My guidance: consider bounded-corpus CAG when the authorized content fits comfortably within the context the model uses well, leaves room for the question, history and answer, and holds up on quality, latency and cost in a side-by-side test against tuned RAG. Good candidates change rarely, are queried heavily, and need no per-user permissions inside the corpus: HR and benefits handbooks, product FAQs, support playbooks, standard operating procedures, a single contract or regulation, and agent system prompts with tool documentation. Poor fits are large or fast-changing catalogues, multi-tenant document sets, permissioned content, and audit regimes that require citing the exact passage used. Graph methods pay off on questions that chain entities, depend on dates, apply rules or need a view across a whole portfolio, and they lose to plain RAG on simple lookups.

For a technology leader the more important point is that which graph method you choose matters more than whether you use a graph at all: the cost of answering a question varies by orders of magnitude across methods. Consider: "Which supplier-related events contributed to production losses at Plant A last quarter, and which purchase orders were affected?" Answering it means connecting a supplier → a delivered component → an equipment asset → a maintenance order → a production interruption → a financial impact → purchase orders. Those links live across SAP, maintenance records, contracts and incident reports. A vector retriever finds documents that sound related; it does not establish the relationships between the underlying business entities. That gap is what KAG targets. KAG comes from Ant Group, which published it in 2024 and open-sourced it as OpenSPG/KAG. It targets two RAG weaknesses: the gap between "sounds similar" and "is relevant", and blindness to "knowledge logic" such as numbers, time relations and expert rules. It combines a domain schema, two-way links between graph and source text, and a planner that turns each question into executable steps.

In short: KAG is GraphRAG plus a symbolic planner-executor plus a domain schema. Its distinctive contribution is the logical form — an auditable, step-by-step plan that can include arithmetic and deduction. Figure 5 · KAG reference flow: builder and solver The builder runs offline and keeps the graph and the text chunks linked both ways. The solver turns each question into a plan, executes it against graph, text and tools, and loops back to re-plan when evidence is missing.

On academic multi-hop benchmarks KAG posted the largest gains of any graph method, though those figures come from its authors and are often quoted wrongly. The more persuasive evidence is from Ant Group's own production use: in government-services Q&A, precision rose from roughly two thirds with naive RAG to above 90%, with similar improvements in health-insurance policy questions. The authors are candid about the cost: many model calls to build the graph and to plan each answer, planning that roughly doubles output, slow entity standardization, and unresolved extraction consistency. KAG buys precision on rule-bound domains with engineering effort; it is not a free upgrade.

The open-source framework is maturing. Recent releases cut graph-construction cost substantially and added MCP support, which positions KAG as a knowledge service that agents can call rather than a standalone application. Domain variants are appearing in areas such as building-engineering standards, which is exactly where I expect KAG to earn its keep: dense, rule-heavy reference material where a wrong clause is expensive. Two independent studies published in 2025 and 2026 compared graph methods with plain RAG on the same questions.

Their findings line up, and they are what I use to set expectations with business sponsors: Question type Better choice Simple fact lookup Plain RAG, slightly Detail questions on a long document Plain RAG, clearly Multi-hop questions Graph methods, slightly Time-dependent questions Graph methods, by a wide margin Themes and summaries across a corpus Graph methods, clearly Questions with no answer in the data Plain RAG: graph methods often answer anyway instead of abstaining Three details matter for planning. Automatically built graphs captured only around two thirds of the entities needed to answer the questions. Each approach answered a meaningful set of questions the other missed, and combining both beat either alone. And model-based judges flipped their verdicts when two answers swapped position, which undermines many published "win rate" claims.

The case for graphs is real but narrow, and it is strongest in a hybrid. No approach wins everywhere, and nobody has yet compared all three on the same data. What the independent studies do agree on is enough to decide: good retrieval matches long-context prompting at a fraction of the tokens, graphs win on multi-step, time-based and summary questions but lose on simple lookups, and combining approaches beats any single one. For a CTO, the more important differences are in cost.

When I review the bill for a production assistant, the spend is rarely where the team expects. It is driven less by the answer than by everything sent alongside the question: instructions, conversation history, retrieved evidence, cached content, tool results and, in agentic systems, every intermediate step. Each approach shapes that mix differently. Figure 6 · where the tokens go in one request, by approach (illustrative) The numbers are illustrative, not measured, but the shapes are typical.

RAG keeps each request small by sending only a few reranked passages. CAG sends the whole corpus every time; caching makes that affordable, but only while the cache stays warm and the corpus stays small. Graph approaches add planning calls and intermediate results. Agents are the real multiplier: each step re-sends the instructions and the growing context, so cost grows much faster than the number of steps unless context is pruned between them.

Token component What drives it How to control it Instructions and tool definitions Prompt length; agents carrying every tool's schema on every call Cache them; give each route only the tools it needs; load tool definitions just in time Conversation history Long sessions replayed in full Summarize or truncate; keep durable facts in a memory store, not the transcript Retrieved evidence (RAG) Number and size of passages Rerank to fewer, better passages; adapt how many to the question; compress Cached prefix (CAG) Corpus size times question volume Billed at the cached-read rate; watch rewrites on expiry; keep volatile content out of the prefix Graph results and plans (KAG) Traversal breadth, community summaries, planning steps Prefer local over global search; cap hops; return paths and IDs, not whole documents Tool outputs and intermediate steps Agent iterations; verbose tool responses Step and token budgets per request; trimmed, structured tool outputs Reasoning tokens Model "thinking" settings Set reasoning effort per route; reserve deep reasoning for hard question classes Output Answer length and format A clear answer contract; concise formats; cite by source ID Treat tokens as a managed resource, not a by-product. In practice that means: Budgets. Set a token budget per use case and a hard cap per request, enforced by the orchestrator, with a defined behaviour when it runs out (answer what is supported, list the gaps). Routing to the right model.

Send simple, well-grounded questions to smaller, cheaper models and reserve frontier models for the question classes that need them. Output tokens typically cost several times more than input tokens, so verbose answers and heavy reasoning are disproportionately expensive. Metrics. Track input versus output tokens, cache hit rate, tokens per request and per correct answer, and steps per agent task, by use case.

Alert on runaway agent loops. Showback. Report token spend by business unit and use case next to the value it produced. Cost per correct, authorized answer is the number that keeps the conversation with the CFO grounded.

  1. Combining them: hybrid architecture and context engineering The research groups behind all three approaches have each moved towards hybrids, and the 2026 practice of context engineering treats RAG, CAG and KAG as complementary services that an orchestrator combines to assemble the smallest sufficient evidence for each step. This is where I would point any enterprise architecture. Most programs I review treat the semantic layer as a BI artefact.

In an AI architecture it is something more important: the contract between the business's numbers and the model. The model never calculates a governed metric itself and never queries raw tables. It asks the semantic layer, which returns the number together with its definition and provenance. Figure 7 · how a governed semantic layer answers a numeric question for an AI assistant The model's job is limited to steps 1, 2 and 7: understanding the question, mapping it to a governed metric, and explaining the result.

Everything that must be exact (joins, formulas, currency conversion, fiscal calendars, access rules) happens deterministically inside the layer. That division is what makes numeric answers auditable. The operating principles I recommend: One definition per metric, owned and versioned. "Downtime cost", "on-time delivery" and "active customer" each have a named business owner, a written definition and a version.

When the definition changes, old versions stay queryable for audit. Expose a service, not a schema. Publish metrics through a governed API or MCP tool with typed parameters (metric, entity, period, filters). The model fills in the parameters; the layer writes the query.

Free-form SQL generated against raw ERP tables is the single most common source of confidently wrong numbers. Enforce access under the user's identity, inside the query. Row- and column-level rules come from the same policy engine as the rest of the platform, so a user sees the same numbers in the assistant as in their reports, and no more. Return metadata with every number.

Definition version, as-of timestamp, source system and the query used travel with the result, and the answer cites them. Declare freshness per metric. Some numbers are read live from the ERP, others refresh hourly or daily from the warehouse. The answer should say which.

Share identities with the graph. The semantic layer and the knowledge graph use the same entity and master-data IDs. The graph answers "which suppliers affect plant 1000"; the semantic layer answers "what that cost"; the IDs make the two answers line up. Separate certified from exploratory.

Only certified metrics may feed answers that go to customers, regulators or the board. Exploratory queries are allowed but clearly labelled. Put metric changes through the release gate. Definition changes re-run the evaluation set, including a check that natural-language questions still resolve to the right metric.

Cache results safely. Key cached results by metric version, user scope and as-of time, and invalidate them on data refresh. Responsibility Owner Metric definitions and certification Business data owners, typically finance and operations Semantic model, API and MCP tooling Data platform team Access policies Security and data governance Question-to-metric resolution quality AI platform team, through the evaluation set In SAP landscapes much of this layer already exists. CDS views and the virtual data model in S/4HANA, and the analytical models in SAP Datasphere and Business Data Cloud, carry SAP's business semantics and authorizations.

AI should consume these rather than rebuild definitions. Where a metric spans SAP and non-SAP systems, define it once in the enterprise semantic layer and expose it to agents as a governed tool. Reference architecture: a four-tier enterprise knowledge platform This is the target architecture I recommend: a cached core, hybrid retrieval, a selective graph layer and a semantic layer, with a router in front and governance across every path. It is not a vendor standard.

Its main advantage is that it turns an irreversible architecture bet into a reversible routing decision. Figure 8 · four-tier enterprise knowledge platform My default for every new use case: start with a hybrid retrieval baseline unless the whole corpus is small, stable and shared. Then add a cached core, a graph layer and a router only when the workload proves each one pays for itself. This keeps early investment small and makes every later addition a measured decision rather than a bet.

Figure 9 · decision tree, from use case to architecture Only the first question can end the tree early. After the retrieval baseline, each "yes" adds a layer and the flow continues, so a mature system often ends up with all four tiers of Section 8, each justified by its own evidence. Recommended starting routes for the router: stable handbook questions use the cached core; anything mentioning recent amendments or dates uses retrieval; exact numeric aggregation calls deterministic computation (SQL or a calculator), never the model's arithmetic; entity-chain questions use the graph; corpus-wide "themes" questions use summaries or GraphRAG-style global search; weak evidence triggers a targeted search or an honest "insufficient evidence" — not unlimited autonomous retries. In my experience, the failures that reach the board are rarely about model capability or retrieval accuracy.

They come from weak access control, poisoned or stale sources, and uncontrolled agent actions, and each architecture fails in its own way. This is the section I would ask every CISO and data-protection officer to read. If an employee may read a plant's maintenance records but not confidential supplier pricing, the pricing must never enter that employee's model context. Filtering the final answer is not enough; unauthorized data must not be supplied in the first place.

The same applies to relationships: a user allowed to see two entities is not automatically allowed to discover a confidential relationship between them, and a derived relationship or summary can reveal what no single visible document does. This is where the three architectures differ most: RAG and graph retrieval can filter by permission at query time, so each user's context holds only what they are entitled to see. CAG preloads the whole corpus, so every user of a cache sees all of it. It suits only content everyone may see, or needs a separate cache per access tier — multiplying write costs and memory.

Graphs must propagate permissions from source chunks to the entities, edges and community summaries derived from them. Figure 10 · three enforcement stages, one policy decision point Stage 1 is where RAG, CAG and KAG differ most. Retrieval can filter per user inside the search. A graph has to filter nodes, edges and any summaries built from them.

A cache cannot be filtered at all once it is built, so the only option is a separate cache per access tier. When a document is deleted or a user loses access, the change has to propagate to everything built from it: index entries, embeddings, cached retrieval results, KV caches and prefix caches that contain it, graph nodes and edges extracted from it, and community summaries that mention it. Rebuilding all of that takes time, so the order matters. Serving policy changes first and immediately, so requests stop using the revoked knowledge even while the rebuild runs: Access change recorded (deletion, revocation, legal hold release) Serving-policy denial takes effect at once for the affected scope Affected artifacts retired from serving: index entries, caches, graph elements and summaries are marked unusable Rebuild the affected index segments, caches and graph regions Validate that the rebuilt artifacts no longer contain the removed content Publish a new, version-consistent manifest across index, cache and graph The gap between steps 2 and 6 is where most leakage happens in practice, which is why revocation propagation time belongs on the evaluation dashboard (Section 11).

Compare workloads, not labels. An architecture that wins on an academic benchmark may lose on your SAP maintenance data, and a cache that cuts response time may not improve a single answer. Every organization needs its own evaluation set, run the same way across every option. I treat this as the single most important investment in the program, because it is what lets every later decision be made on evidence.

Teams often postpone evaluation until it is too late to change the architecture. This is a starting kit that fits inside the first phase of the roadmap: Element Minimum setup Question set 200–300 questions reviewed by a domain expert, with about a quarter held back for final testing Question mix (starting point) About 30% single-hop factual, 20% multi-hop, 10% time-based, 10% numeric or rule-based, 10% summary, 10% unanswerable (should abstain), 10% permission-boundary cases Core quality metrics Groundedness, answer relevance, citation precision, correct abstention Core operational metrics p95 latency; cost per correct, authorized answer Diagnostic An oracle-evidence test: give the model hand- picked evidence to separate retrieval failures from generation failures Judging A model as judge with randomized answer order, plus a human-reviewed sample, because judges favour position and length Statistics Paired comparisons on the same questions, with confidence intervals, reported per question class Regression gate Every change to prompts, models, chunking, indexes, caches or graphs re-runs the set; a drop beyond an agreed margin blocks release Add adversarial tests early: injected instructions inside documents, poisoned nearduplicates, questions that try to elicit restricted relationships, and stale-version traps. Asset-intensive and regulated industries get the most from combining all three approaches, because they already keep an implicit ontology (asset registers, product hierarchies, policy rules) and ask genuinely multi-hop questions — but the authoritative numbers must always come from systems of record, not from the graph or the model. Industry CAG (cached core) RAG (hybrid retrieval) KAG / GraphRAG (graph layer) Where I would start Energy / oil & gas HSE procedures, permit-to- work rules, SOPs Well reports, drilling and maintenance logs, engineering documents (often visually rich) Field, facility, equipment and component hierarchies; failure modes; procedure links Multimodal RAG over engineering documents, then an asset graph Manufacturing Product-line FAQs, safety procedures Manuals, service bulletins Bill-of-materials and asset/failure graphs RAG for service, graph for root-cause and supplier impact Healthcare / life sciences Clinical protocols, formularies, benefit summaries Literature, payer policies, clinical notes (strict access control) Drug–gene– disease graphs, claim-to-policy relationships Permission- aware RAG; KAG for payer- policy rules Financial services / insurance Product and policy FAQs Research, filings, contracts KYC/AML entity networks, riders and exclusions, regulatory lineage RAG for research; graph for entity risk and policy rules Public sector Citizen- service FAQs Permission- aware search with data residency Rule-bound eligibility Cached FAQs plus rules- driven eligibility Legal One contract or regulation under review Contract and clause search Clause and precedent citation graphs RAG with exact-passage citation Retail / customer support Returns, warranty and shipping policies Orders, products and inventory via retrieval and live tools Supplier and part networks Cached policies plus live order lookup IT operations / cybersecurity Runbooks Logs and tickets (just-in- time search suits logs) Configuration- item, change and incident dependency graphs Cached runbooks plus a dependency graph ERP systems are the clearest case for combining all three approaches, because they hold transactional truth that changes by the minute, business semantics encoded in the data model, and fine-grained authorizations — none of which a vector index or a cached prefix can safely replace.

The rule of thumb for SAP landscapes: documents go to RAG, stable reference goes to the cached core, relationships go to the graph, and live facts come from SAP itself. SAP's AI stack now contains a native counterpart for each layer (status as of November 2026; check exact availability for your edition and region): Approach SAP building block What it does RAG SAP HANA Cloud vector engine and document grounding in the generative AI hub and Joule Generally available since 2024; the document-grounding module implements RAG on top of it KAG / graph SAP HANA Cloud knowledge graph engine and SAP Knowledge Graph Generally available since 2025; SAP positions SAP Knowledge Graph as the semantic bridge between Joule agents and business data and processes Unified data SAP Business Data Cloud (with Databricks) The trusted data layer that grounds Joule agents across SAP and non- SAP sources Agents and routing Joule, Joule Agents and Joule Studio Custom agents grounded on business documents and data, with MCP and agent-to-agent collaboration, and governed execution of multi-step and third- party agents CAG No SAP-branded equivalent Prompt caching depends on the model provider reached through the generative AI hub; confirm support and pricing per model For an SAP-centric client the practical question is therefore not whether to use RAG, CAG or KAG, but when to use SAP's native layer versus an external stack — typically SAPnative where the knowledge and the users live inside SAP and Joule, and an external hybrid platform where knowledge spans SAP, other ERPs, document systems and nonSAP applications, connected to SAP agents through MCP or A2A. A 90-day pilot can prove or disprove each layer in order — retrieval baseline first, then caching, then a graph, then routing — with a go/no-go decision based on targets agreed in advance by accountable owners. Figure 11 · 90-day pilot roadmap: six phases, one gate Phases are shown at equal width, not to scale.

The coloured phases test RAG, CAG and KAG in the same colours used throughout the paper. Investment is usually decided at two points: after phase 2 (is the baseline good enough to build on?) and at the end (scale, extend or stop). Business KPIs agreed in phase 1, such as time saved per case, escalation rate or cost per resolved request, should be reported at both. This is a pilot sequence.

Enterprise-wide rollout, regulated-domain validation and legacy integration usually take longer. "RAG vs CAG vs KAG" is a false choice: the durable asset is a governed, evaluated knowledge platform that places each question on the cheapest path that answers it correctly, and that survives the next change of model or retrieval technique. RAG established that models do not need to hold all knowledge in their weights. CAG showed that knowledge preparation and repeated inference can be optimized.

KAG moved the focus to relationships, logic and domain reasoning. None of them alone solves data quality, authorization, business semantics, freshness, provenance, cost or agent governance. The question for the next generation of enterprise AI is not how much a model can retrieve or remember, but this: can the system access the right knowledge, understand its business meaning, verify its validity, respect its boundaries, and use it to produce a trustworthy outcome? If I were starting this program on Monday, I would do three things in the first month: commission a question-mix analysis of what users actually ask, stand up a small evaluation set owned jointly by the business and the platform team, and fix the retrieval basics in whatever RAG system is already running.

Everything else in this article (caching, graphs, routing, agents) becomes a decision you can make on evidence once those three are in place. Everything in this article comes out of the work we do with clients at Elixir Global, an AI-first enterprise consulting firm headquartered in Atlanta, with offices in Houston, Dubai, Bengaluru and Singapore. Together with our subsidiaries S2 Integrators (SAP delivery) and Kyyte (procurement and SAP), we spend most of our time where this article lives: at the point where AI meets the systems of record that run the business. Our approach follows the same order this article recommends.

We start with the questions people actually ask and an evaluation set the business owns. We fix retrieval before adding anything new, put a governed semantic layer in front of SAP and other ERPs so numbers are never improvised, and add caching, graphs and routing only where the evidence says they pay. AgentVault is where that thinking becomes reusable. AgentVault™ is our proprietary library of more than 500 enterprise agents, covering supply chain, procurement, finance and document-heavy processes such as IDoc handling and invoice OCR.

The agents are designed to sit on top of a governed knowledge platform rather than carry their own copies of enterprise data: they retrieve documents, read live values through governed services, and call graph and semantic-layer tools over MCP. They integrate with SAP Joule, and because they are built on open components, clients can adopt the latest models without vendor lock-in or dependence on any single platform roadmap. Governance is built in, not bolted on. AgentVault agents follow the same guardrails described in Section 10, including SOX and FedRAMP controls where they apply, and our BYOAI-Gov™ framework and GRC tooling help clients find governance gaps early, before they reach production.

We have deployed agents and set up AI governance in federal, oil and gas, manufacturing, life sciences, utilities and consumer goods environments, and we work alongside SAP, Tricentis and Anthropic as a Claude Partner Network member. Download the full whitepaper (PDF): RAG, CAG and KAG: Choosing How Enterprise AI Uses What the Business Knows (Zenodo, DOI 10.5281/zenodo.24399422) Related white paper: The Token Economy: How Smarter AI Computation Will Reshape Enterprise Costs and Value (DOI 10.5281/zenodo.24386748) Related white paper: Governing Rogue AI Agents with Valid Credentials (DOI 10.5281/zenodo.24378767) Ganesh Raju is Chief AI & Innovation Officer (CAIO) and CTO at Elixir Global, an AI-first enterprise consulting firm headquartered in Atlanta with offices in Houston, Dubai, Bengaluru and Singapore. He leads enterprise AI strategy and AgentVault, Elixir's portfolio of governed, production-ready AI agents for SAP and enterprise operations, and the Migra AI-native data migration platform. He brings more than 15 years of experience in artificial intelligence, machine learning and data engineering.

As Head of Big Data, Data Science & AI at Linaro he led Arm enablement of the open-source big data and data science ecosystem, served as ODPi PMC member and TSC lead and as a Linux Foundation TSC member, and led the Apache Bigtop community. He has contributed to TensorFlow, Apache Spark and Hadoop. He is the author of AI Blockchain Design and Deployment (BPB Publishing), founder and CEO of Akshaya.io, founder of RapidEVChargE, an investor, and an advisor to Canon. He is a former Vice Chair of IEEE Houston and a participant in the Wharton Executive Education Chief Technology Officer program.

He is based in Houston, Texas. Connect on LinkedIn · GitHub · Blog Views are the author's own. Elixir Global offers products and services related to the subject of this paper. Research results cited are from published studies; figures marked illustrative are not measurements.

RAG #CAG #KAG #EnterpriseAI #AgenticAI #KnowledgeGraphs #AIArchitecture #AIGovernance #GenerativeAI #CAIO #CTO #SAP

Key Takeaways #

  • •RAG, CAG and KAG are three ways of solving the first problem; governance decides the second.
  • •This story was reported by Dev.to , covering developments in thedev space.
  • •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage.

📖 Continue reading the full article:

Read Full Article on Dev.to →

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @rag 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/rag-cag-and-kag-choo…] indexed:0 read:34min 2026-11-20 · —