# How I Built a NIST AI RMF-Compliant RAG System for Regulated Domains

> Source: <https://dev.to/lakshman-ai/how-i-built-a-nist-ai-rmf-compliant-rag-system-for-regulated-domains-jch>
> Published: 2026-08-18 11:15:09+00:00

**By Lakshman Pandey | August 2026**

I shipped a production RAG (retrieval-augmented generation) system serving UK arts and culture clients. This article documents how the system implements NIST AI Risk Management Framework controls, with real decisions, trade-offs, and measurable outcomes.

**TL;DR:**

**Stack:**

**Risk Profile: LOW**

The NIST framework has 4 functions: GOVERN, MAP, MEASURE, MANAGE. Here's how the production system implements each.

**Requirement:** Define roles, responsibilities, and decision-making authority for AI risk management.

**Implementation:**

**Decision Authority:** Solo architect with client stakeholder approval loops.

**Data Governance (ADR-001):**

**Stakeholder Roles:**

**Policy:** All user data stays in EU. API calls to Claude/Voyage are transient (no data stored in US).

**Measurement:** Langfuse audit trail logs every query's origin and destination.

**Requirement:** Identify risks specific to the AI system's context, design, and use case.

**Implementation:**

**Risk Inventory:**

| Risk | Severity | Source | Mitigation |
|---|---|---|---|
| Hallucination | Medium | LLM generating answers beyond retrieved context | Prompt constraints (answer only from sources) + eval suite thresholds |
| Embedding Quality Drift | Medium | Voyage model updates degrade retrieval | Phase 2 eval baseline (context recall 0.98) prevents regression |
| Data Drift | Low | Corpus content changes over time | Scheduled re-eval (monthly) against golden questions |
| Vendor Outage | Low | Supabase/Voyage API downtime | Documented fallback to Ollama (local, offline) |
| PII Leakage | Low | User data in prompts | (Future) Microsoft Presidio redaction at ingestion |
| Prompt Injection | Low | User query attempts to jailbreak system | Input validation + output validation (present but basic) |

**Risk Rating:** OVERALL = LOW-RISK

**Requirement:** Define metrics to assess AI system performance and risk throughout the lifecycle.

**Implementation:**

**Eval Framework (Phase 2):**

Built Ragas-based evaluation suite with 18 golden questions:

| Metric | Baseline | Threshold | Current Status |
|---|---|---|---|
| Faithfulness | 0.42 | > 0.50 | Pending re-run with Voyage |
| Context Precision | 0.69 | > 0.65 | ✅ Passing |
| Context Recall | 0.98 | > 0.95 | ✅ Passing |
| Answer Relevancy | 0.64 | > 0.60 | ✅ Passing |

**Why these metrics?**

**Observability (Phase 3):**

Langfuse integration traces every production query:

```
{
  "trace_name": "rag_query",
  "input": "What are the current content guidelines?",
  "output": "Based on sources [1][2]...",
  "input_tokens": 450,
  "output_tokens": 85,
  "cost_usd": 0.0031,
  "latency_ms": 1250
}
```

**Cost Per Query:** $0.0005 (embedding) + $0.003 (generation) = $0.0031

**Latency Target:** < 2 seconds (currently ~1.2s)

**Requirement:** Manage identified risks through safeguards, monitoring, and response.

**Implementation:**

**Current Safeguards:**

**Code Evidence:**

```
# phase3-deployment/app.py, lines 52-58
response = requests.post(
    "https://api.anthropic.com/v1/messages",
    json={
        "model": "claude-haiku-4-5-20251001",
        "messages": [{
            "role": "user",
            "content": (
                'Answer using ONLY the sources below. '
                f'If answer not in sources, say so.\n\n{context}\n\nQ: {question}'
            )
        }]
    }
)
```

**Future Safeguards (Phase 4):**

**Decision:** Cloud-managed Supabase pgvector (EU)

**Trade-off:**

**Why this trade-off wins:**

**Decision:** Cloud API (Voyage) vs local (Ollama)

**Trade-off:**

**Why this trade-off wins:**

**Decision:** Manual HTTP calls (requests lib) vs SDK

**Trade-off:**

**Why this trade-off wins:**

**GOVERN:** ✅ Documented roles, EU data residency, stakeholder approval

**MAP:** ✅ Risk inventory, low-risk classification, identified mitigations

**MEASURE:** ✅ Eval framework (Phase 2), Langfuse tracing (Phase 3), cost monitoring

**MANAGE:** ⚠️ Basic error handling, prompt constraints; future human-in-the-loop + spend ceiling

**Compliance Status:** COMPLIANT with NIST for low-risk use case. Future enhancements (Phase 4) will strengthen MANAGE function.

Data Residency First: For UK public-sector clients, EU hosting is table-stakes. Chose Supabase before other factors.

Evaluate Everything: Phase 2 eval framework caught that naive keyword-matching underperforms vector search. Measuring > assuming.

Direct API > SDKs for Stability: Python 3.14 broke 4 versions of the Anthropic SDK. Direct HTTP calls worked immediately.

Cost Transparency Builds Trust: Langfuse tracing makes per-query costs visible. Clients appreciate this.

Document Decisions, Not Just Code: ADRs explain WHY, not just HOW. Critical for onboarding + architectural clarity.

Lakshman Pandey is a Senior Technical Lead specializing in AI Solutions Architecture for content-rich, regulated domains (UK public sector, cultural institutions, education). 13+ years full-stack development (Drupal, Python, Node.js). Currently building RAG systems that balance innovation with governance requirements.

GitHub: code-lakshman/ai