The RAG Pattern That Stops LLMs From Inventing Fields That Don't Exist

Attribute Knowledge RAG (AK-RAG) prevents large language models from inventing nonexistent fields by indexing governed attribute catalogs instead of documents, forcing field selection through retrieval and classification rather than free generation. This architecture addresses governance failures in regulated industries like banking and healthcare, where invented fields can cause compliance and operational risks.

The RAG Pattern That Stops LLMs From Inventing Fields That Don't Exist Standard RAG retrieves documents and lets the LLM generate field names freely — which means it will invent fields your schema never had. AK-RAG Attribute Knowledge RAG indexes your governed attribute catalog instead, and the LLM can only emit attribute ids that actually exist. Here's the architecture and why it matters for regulated-industry AI. Table of Contents Every team building RAG for a regulated enterprise hits the same failure mode eventually. The system retrieves the right context — clinical guidelines, product documentation, data dictionaries — and the LLM produces a confident, fluent response that references fields, thresholds, and identifiers that don’t exist in the actual data model. In a consumer app this is a nuisance. In a bank’s credit underwriting pipeline, a healthcare payer’s cohort identification system, or a compliance team’s AML query layer, it is a governance failure. The LLM invented a field. That field never went through data governance. The output is meaningless at best and misleading at worst. Standard RAG was not designed to prevent this. It retrieves document chunks and hands them to the LLM as context. The LLM then generates free-form text, including any field names, identifiers, and thresholds it infers from that context. There is no guarantee that what the LLM generates corresponds to anything that actually exists in your data catalog. Prompt-level mitigations — instructions to only use fields from the provided context, few-shot examples, output validation — help at the margins but don’t solve the structural problem. The problem is that the retrieval unit is wrong. Attribute Knowledge RAG AK-RAG is a reference architecture that fixes this at the retrieval unit level. Instead of embedding documents, it indexes your governed attribute catalog as individual knowledge objects — one embedding per attribute, not one embedding per document page. The LLM’s job is phrase extraction and clarification dialog, not field name generation. The only identifiers that can appear in the final output are attribute id values that exist in the indexed catalog. The LLM cannot invent fields because field selection happens through a governed retrieval and classification pipeline, not through free generation. Architecture AK-RAG has two distinct pipelines: an ingestion pipeline that builds the attribute index, and a query pipeline that translates natural language into a governed DSL. Both are designed around a core principle: the attribute catalog is the knowledge layer, not the document corpus . Ingestion Pipeline The ingestion pipeline takes your enterprise attribute metadata — typically maintained in Excel, CSV, or exposed via an API — and transforms it into a searchable, versioned index: Excel / CSV / API | v Contract Validation & Normalization - Split synonyms - Parse ranges and thresholds - Coerce types, standardize units/dates | v NDJSON One Attribute = One Document | v ┌──────────────────────────────────────┐ │ Embedding Provider │ │ sentence-transformers dev │ │ openai · gemini · bedrock prod │ └──────────────────────────────────────┘ | v ┌──────────────────────────────────────┐ │ Search / Index Backend │ │ local BM25 + token vector + RRF │ ← offline, no cluster dev │ opensearch BM25 + HNSW kNN + RRF │ ← production │ faiss / chroma │ ← dense vector alternatives └──────────────────────────────────────┘ Each attribute becomes a single NDJSON document. Here’s what that looks like for a clinical attribute in the healthcare domain: { "attribute id": "clinical.hba1c", "type": "numeric", "business name": "HbA1c Level", "technical field": "hba1c pct", "domain": "clinical", "definition": "Most recent HbA1c percentage on record. Indicates glycemic control...", "synonyms": "glycated hemoglobin", "blood sugar control", "a1c", "hemoglobin a1c" , "operators": " ", " =", "<", "<=", "=" , "unit": "%", "example values": 7.0, 8.0, 9.0, 10.0 , "governance": { "phi": true, "hipaa category": "clinical", "de identification required": false, "minimum cell size": 11, "allowed channels": "care management", "quality reporting", "analytics" , "consent required": false }, "embedding text": "numeric | HbA1c Level | hba1c pct | glycated hemoglobin | blood sugar control | a1c | glycemic control | diabetes | percentage | clinical" } The embedding text field is hand-composed to maximize retrieval signal: it concatenates the type, business name, technical field name, synonyms, definition keywords, and domain tags. This is not a document summary — it is a retrieval-optimized representation of what this attribute means in natural language. Indexes are versioned by date and promoted via an atomic alias swap: attributes v2026 06 26 previous attributes v2026 07 06 new attributes current → attributes v2026 07 06 This means a new metadata release never partially updates a live index. If post-deployment smoke tests fail, rollback is a single alias pointer change. Query / Conversation Pipeline The query pipeline is a six-step process. The LLM participates in two of the six steps — phrase extraction and clarification dialog generation. Everything else is deterministic: User Natural Language Input | v ┌──────────────────────────────────────┐ │ LLM Step 1: Parse │ │ Extract attribute phrases only │ │ "diabetic patients over 65 with │ │ elevated HbA1c" → │ │ "diabetic", "over 65", │ │ "elevated HbA1c" │ └──────────────────────────────────────┘ | v For each phrase independently: ┌──────────────────────────────────────┐ │ Step 2: Hybrid Retrieval │ │ BM25 lexical ──┐ │ │ ├──► RRF fusion │ │ kNN vector ──┘ │ │ Top-k candidates per phrase │ └──────────────────────────────────────┘ | v ┌──────────────────────────────────────┐ │ Step 3: Decision Policy │ │ │ │ score ≥ EXACT THRESHOLD 0.92 │ │ → exact: use attribute as-is │ │ score ≥ NEAR THRESHOLD 0.75 │ │ → near: show top options │ │ multiple near matches │ │ → ambiguous: ask user to choose │ │ no match above threshold │ │ → none: ask for alternative │ └──────────────────────────────────────┘ | v ┌──────────────────────────────────────┐ │ Step 4: Clarify LLM │ │ Generate clarification dialog for │ │ near / ambiguous phrases │ │ User selects from governed options │ └──────────────────────────────────────┘ | v ┌──────────────────────────────────────┐ │ Step 5: Governance Check │ │ PHI · HIPAA category │ │ consent required │ │ allowed channels │ │ minimum cell size │ │ Blocks any attribute that violates │ └──────────────────────────────────────┘ | v ┌──────────────────────────────────────┐ │ Step 6: Emit Governed DSL │ │ Only selected attribute ids │ │ No free-form field generation │ └──────────────────────────────────────┘ The final DSL output contains only attribute id values that exist in the catalog, plus provenance metadata: { "version": "1.0", "type": "attribute filter", "filters": { "attribute id": "diagnosis.diabetes type2", "business name": "Type 2 Diabetes Diagnosis", "technical field": "diabetes t2 flag", "source phrase": "diabetic" }, { "attribute id": "utilization.readmission 30d", "business name": "30-Day All-Cause Readmission", "technical field": "readmit 30d flag", "source phrase": "readmitted in the last 30 days" } , "unresolved": { "phrase": "elevated HbA1c", "outcome": "ambiguous", "options": { "attribute id": "clinical.hba1c gt 8", "business name": "HbA1c Level 8.0%" }, { "attribute id": "clinical.hba1c gt 9", "business name": "HbA1c Level 9.0%" } } } unresolved phrases are surfaced back to the user — the system never silently picks one. This is not a UX courtesy; it is an accuracy requirement. “Elevated HbA1c” maps to two different governed clinical thresholds with different population implications. Guessing wrong has downstream consequences for care management targeting, quality reporting, and regulatory submissions. The Governance Gap The reason standard RAG fails for governed enterprise data is that it treats the retrieval problem as a document retrieval problem, when it is actually an attribute selection problem. When an analyst asks “show me diabetic patients with high HbA1c who were readmitted,” the correct system behavior is to select the right diagnosis.diabetes type2 , clinical.hba1c gt 9 , and utilization.readmission 30d attribute ids from the catalog, confirm any ambiguous thresholds with the user, and produce a filter expression using only those fields. The system does not need to retrieve paragraphs of clinical documentation to do this. It needs to retrieve governed attribute objects that match the analyst’s natural language phrases. Standard RAG would embed the clinical documentation — HEDIS technical specifications, ICD-10 codebooks, clinical guideline PDFs — chunk them, and let the LLM synthesize field names from the retrieved text. This produces outputs that sound correct to a non-technical reader but are unreliable as machine-executable data operations. The LLM might generate hba1c high instead of hba1c gt 9 . It might infer a 30-day readmission window from the retrieved text without knowing whether the enterprise system distinguishes all-cause from condition-specific readmissions. It might produce a filter expression that references fields which simply don’t exist in the target data model. AK-RAG sidesteps this entirely because the retrieval unit is already a governed attribute. The system cannot generate hba1c high because hba1c high is not an attribute id in the catalog. The system cannot guess at a readmission time window because the decision policy forces clarification when a phrase maps to multiple governed attributes. The governance layer catches any remaining violations — PHI fields in unauthorized channels, attributes below the minimum cohort size threshold — before the DSL is emitted. Why Hybrid Retrieval BM25 + kNN + RRF ? The pattern uses hybrid retrieval because neither lexical nor semantic search alone is sufficient for enterprise attribute lookups. BM25 excels at exact field name matches, acronyms, and clinical abbreviations. A query phrase of a1c should directly match the synonyms array on clinical.hba1c . Vector search would handle this too but degrades on rare clinical abbreviations without domain-specific embedding training. kNN vector search handles paraphrase and semantic variation. “Readmitted to hospital last month” is semantically equivalent to “30-Day All-Cause Readmission” but shares almost no tokens. A pure BM25 search would miss this; semantic search finds it. Reciprocal Rank Fusion RRF combines the two ranked lists without requiring the scores to be on the same scale. BM25 scores and cosine similarities cannot be directly averaged — they measure different things. RRF fuses by rank position, which is stable across query types, embedding model changes, and index rebuilds. The decision thresholds default: EXACT THRESHOLD=0.92 , NEAR THRESHOLD=0.75 are configured in .env and should be calibrated on your specific attribute catalog with a labeled evaluation set. What This Means for Developers The AK-RAG reference implementation https://github.com/crazyaiml/attribute-knowledge-rag is a working Python vertical slice. You can run it locally without an OpenSearch cluster using the offline local search backend: Install with Claude LLM + sentence-transformers dev default pip install -e ". claude,sentence " Validate your attribute contract akrag validate data/sample attributes.csv Convert to NDJSON akrag to-ndjson data/sample attributes.csv build/attributes.ndjson Run hybrid retrieval locally akrag query build/attributes.ndjson \ "high HbA1c" \ "readmitted in the last 30 days" The pluggable provider matrix means you can start with local search and sentence-transformers during development and swap to OpenSearch + OpenAI embeddings for production without changing any application code — all three layers LLM, embedding, search are selected via environment variables: Dev: fully offline, no API costs LLM PROVIDER=ollama EMBEDDING PROVIDER=sentence SEARCH PROVIDER=local Production AWS LLM PROVIDER=bedrock EMBEDDING PROVIDER=bedrock SEARCH PROVIDER=opensearch The same pipeline works for any enterprise domain. The healthcare cohort-definition use case in the demo is one instance. The pattern is domain-agnostic — the six steps parse → retrieve → classify → clarify → govern → emit DSL are identical whether the catalog is clinical attributes, banking risk factors, KYC fields, or insurance underwriting parameters. For banking specifically, the same governance layer that enforces phi , hipaa category , and minimum cell size maps directly to PII/NPI masking, regulatory reporting eligibility, and fair-lending minimum-cohort-size rules: | Healthcare Governance Field | Banking Equivalent | |---|---| phi: true | PII / NPI masked field | hipaa category: clinical | regulatory category: credit risk | minimum cell size: 11 | Fair lending minimum cohort size | allowed channels: care management | allowed channels: underwriting, analytics | consent required: true | SAR reporting restriction | The attribute document shape is the same. The decision policy is the same. The DSL emitter is the same. You swap the catalog. The SuperML Take The failure mode AK-RAG addresses is one of the most common and least discussed problems in production enterprise AI: the LLM that confidently generates field names that don’t exist. Everyone who has shipped a RAG system against a structured data model has seen this. It gets patched with prompt engineering, output validation, and retry logic — but it keeps surfacing because the root cause is architectural, not prompt-level. The root cause is that standard RAG is optimized for document retrieval and free-form synthesis. That is excellent for knowledge-base chatbots, customer support systems, and research assistants. It is the wrong architecture for any system that needs to translate natural language into governed data operations — cohort definitions, compliance queries, credit policy evaluations, AML rule lookups. These systems need deterministic attribute selection, not probabilistic field name synthesis. AK-RAG’s key insight is that the embedding unit should be the attribute, not the document. A clinical data dictionary has hundreds of attributes. Each attribute has a business name, synonyms, a technical field name, governance metadata, and valid operators. That is exactly what should be embedded and retrieved. When you embed at the attribute level, the retrieval result is always a set of candidate attribute objects. The LLM selects from that set — it cannot generate outside it. The governance layer checks the selected attributes before the DSL is emitted. The system is constrained end to end. For teams building AI systems in banking, healthcare, or insurance, the practical question is not “should we use AK-RAG?” but “do we have an attribute catalog to index?” Most regulated enterprises already have the catalog — it lives in a data dictionary, a metadata management tool, or a governance platform. The ingestion pipeline accepts Excel, CSV, or API input. The attribute document schema is defined in the repo. Getting a working prototype running against your own catalog should take a day, not a sprint. The open-source reference implementation at github.com/crazyaiml/attribute-knowledge-rag https://github.com/crazyaiml/attribute-knowledge-rag includes the full pipeline: contract validation, NDJSON generation, offline hybrid retrieval, decision policy, governance check, and DSL assembly. It runs without a cluster in dev mode. Production deployment swaps local search for OpenSearch BM25 + HNSW kNN and the embedded sentence-transformers for a production embedding provider. The architecture is provider-agnostic by design — the LLM, embedding model, and search backend are all pluggable. If your enterprise AI system is producing field names your data model has never heard of, the fix is not a better prompt. It is a better retrieval unit. Sources Enterprise AI Architecture Want more enterprise AI architecture breakdowns? Subscribe to SuperML.