Letting AI touch CAPA: the validation question nobody actually answers

A developer prototyping an AI assistant for CAPA triage under 21 CFR 820 and ISO 13485 found that traditional software validation does not apply to probabilistic LLMs. The team created a validation pattern using a golden-test corpus, acceptance criteria, continuous monitoring, and human-in-the-loop controls to move from experiments to a controlled assistive deployment.

I started prototyping an AI assistant for CAPA triage because our backlog was burying the QA team. The idea felt obvious: have a model read an incoming nonconformance or complaint, suggest probable root-cause categories, surface related docs from the technical file, and propose a draft corrective action for a human to edit. It sped up screening in a sandbox — until the uncomfortable question landed: how do we validate this so a regulator or my internal auditors can accept it? If you work under 21 CFR 820 and ISO 13485, you already know CAPA is not a place for unproven gamble: 21 CFR 820.100 expects documented procedures and verification of effectiveness; ISO 13485 expects controlled corrective/preventive processes. But those standards are written for human processes and classic software validation. AI changes the shape of "verification" and "evidence." Below are what I learned by prototyping, plus a pragmatic validation pattern we used to move from experiments to a controlled assistive deployment. The core problem: what is "validated" when a model is probabilistic? Traditional software validation is: input X - expected deterministic output Y. With an LLM, output is probabilistic, context-sensitive, and can change with model updates. That means you can't treat the model like a black-box SOP that always produces the same correction plan. You can, however, treat the model as "controlled assistance" — a source of suggestions that are subject to human review and traceability. But treating it that way still requires validation artifacts. What I prototyped practical, not theoretical - Ingested a de-identified sample of historic CAPA and nonconformance reports from our QMS we scrubbed PHI and supplier-sensitive text . - Built an embeddings index of our controlled documents design history, risk assessments, past CAPAs so the assistant could surface context. - Created a "shadow mode" pipeline: the AI wrote suggestions but we did not act on them — we logged suggestions and compared them to the original human outcomes. - Measured alignment: did the AI's root-cause category match the human category? Did its top-3 suggested references include docs humans actually used? This revealed the usual: helpful hits, plausible-but-wrong rationales, and occasional hallucinations with confident-sounding citations that didn't exist. Validation pattern that made sense for us I didn't want to "validate the model" too brittle ; I wanted to validate the controlled-assistance process and produce evidence that regulators expect. We defined the following, iterated, and kept everything under change control: - Define scope and boundaries - Which CAPA types can the assistant touch? e.g., administrative triage only; not manufacturing hold decisions - Allowed data: no PHI or regulated supplier IP to public LLMs. - Create a golden-test corpus - De-identified historical cases with human-accepted root cause, references used, and final corrective actions. - Include edge cases and ambiguous inputs. - Acceptance criteria measurable - Example metrics: top-1 root-cause agreement vs human, presence of at least one correct doc in top-5 references, rate of hallucinated citations per 100 suggestions. - Define pass/fail thresholds based on risk higher scrutiny for anything that influences product disposition . - Continuous monitoring and drift detection - Catalog model version, prompt templates, and embedding index version. - Run periodic re-tests against the golden corpus and new live-decision audits. - If performance drops, initiate change-control and revalidation. - Human-in-the-loop controls - The assistant's output is saved as a draft in the CAPA record with clear provenance model version, timestamp, prompt . - A named human reviewer must accept, edit, and sign off before any action is taken. - Documentation and traceability - SOPs for assistant use, validation plan, test reports, and periodic review logs. - Maintain an auditable trail linking suggestions to final CAPA content. Implementation notes useful to engineers - Treat prompts, test suites, and evaluation code as source-controlled artifacts. We used a small CI job to run the golden-corpus tests when we changed prompts or upgraded the model. - Snapshot your embedding index and store the version used by the assistant in each CAPA record. That preserves "why" the assistant surfaced a doc. - Redact or tokenise PHI and supplier secrets before any call to an external LLM. If you must send sensitive data, prefer enterprise/private-hosted models under a DPA and SOC2 controls. - Log everything to an append-only audit store we wrote logs to our QMS via webhook so each draft suggestion became part of the CAPA's history . - Define roles: who can accept AI suggestions, who can only view, and who can edit model prompts/policy. Where auditors and notified bodies will push back From my experience, reviewers care about: - Evidence that the assistant doesn't change CAPA outcomes without oversight. - Reproducible records showing model versioning and the decision path. - Data protection controls especially for clinical complaints . - Clear SOPs on when the assistant can be used and when it must be bypassed. If you can't demonstrate those artifacts, expect questions — and be ready to show your golden-corpus test results and revalidation plan. Final thoughts AI can materially reduce triage time and surface relevant evidence from large technical files. But the validation conversation isn't a one-off checkbox; it's an ongoing program: test corpus, monitoring, change-control, and human oversight. In the end, we accepted AI where it reduced admin work and never where it could directly change product disposition without a human sign-off. What I'd like to learn from this community: do you have a concrete list of test cases or metrics you used to convince internal auditors or a notified body that an AI-assisted CAPA workflow was "validated enough"? If so, can you share the kinds of test cases or acceptance criteria that worked in your audits?