{"slug": "100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself", "title": "100% Recall, 38.5% Precision: What Happened When My AI Auditor Audited Itself", "summary": "ThirdLine, an open-source AI auditing system, achieved 100% recall and an F1 score of 0.909 on a deterministic evaluator when auditing synthetic banking agents, but precision fell to 38.5% and F1 to 0.556 when run with GPT-4o-mini, causing its own model card to fail its threshold. The system, which audits AI agents for hallucination, bias, drift, and security failures, uses a human-in-the-loop gate that prevents the orchestrator from approving findings, as described by the author. The results highlight limitations of using AI to audit AI, especially given that revised model-risk guidance from the Federal Reserve, OCC, and FDIC on April 17, 2026, explicitly excludes generative and agentic AI from its scope.", "body_md": "The most useful output from my AI-auditing system was a red status label: Requires Review.\n\nThirdLine had caught all five defects I deliberately planted in a synthetic fleet of banking agents. Recall was 100%. On the deterministic evaluator, F1 reached 0.909. Then I ran the audit end to end with GPT-4o-mini on the same test fleet. Precision fell to 38.5%, F1 fell to 0.556, and the system’s own model card failed it against the threshold I had written.\n\nThirdLine is a system that audits other AI agents, checking them for hallucination, bias, drift, and security failures before those failures reach production. This piece walks through how the audit pipeline and its human-review gate work, and what the results revealed about the limits of using AI to audit AI.\n\nThat failure became the real project. ThirdLine is open source under the MIT license, the full implementation discussed below is available on GitHub.\n\nOn April 17, 2026, the Federal Reserve, OCC, and FDIC issued revised model-risk guidance. A footnote explicitly states that generative and agentic AI are outside its scope because they are novel and rapidly evolving. The same footnote immediately adds that banks should still use their risk-management and governance practices to determine appropriate controls.\n\nThat is not an exemption from governance. It is a gap in prescriptive model-validation guidance.\n\nTraditional validation works best when the object under review is comparatively stable: defined inputs, defined outputs, a measurable error distribution, and a controlled release process.\n\nAn agent can retrieve different context, change the prompt it sends downstream, call tools, retry failed steps, and follow a different execution path for the same user request. The unit under audit is no longer just a model. It is a system with state, permissions, dependencies, and runtime behavior.\n\nI built ThirdLine to test one possible response: an orchestrator and five specialist evaluators auditing five synthetic banking agents across hallucination, bias, drift, robustness, and reliability.\n\nThe pipeline discovers the fleet, collects evidence, evaluates interactions, maps failures to controls, drafts findings, and places every finding into a human review queue.\n\nThe obvious objection is that an AI auditor can be wrong too. I agree. The design only makes sense if the auditor can produce evidence but cannot grant itself authority.\n\nFigure 1: ThirdLine’s six-step audit pipeline, from fleet discovery to the human review queue. Image by author.\n\nA human-in-the-loop gate has to be more than a warning\n\nMany agent workflows call something “human in the loop” when they really mean “log a warning and continue.”\n\nThat is monitoring, not a control boundary.\n\nIn ThirdLine, the orchestrator’s final step writes each finding to a queue with a PENDING status and a severity-based service-level deadline. The orchestration component contains no branch that changes the finding to APPROVED.\n\n``` php\ndef _step_hitl_gate(    self,    findings: list[dict],    run_id: str,) -> list[dict]:    queue_entries = []\nfor finding in findings:        sla_hours = {            \"CRITICAL\": 4,            \"HIGH\": 24,            \"MEDIUM\": 72,            \"LOW\": 168,        }\nnow = datetime.now(timezone.utc)        deadline = now + timedelta(            hours=sla_hours.get(finding[\"severity\"], 24)        )\nentry = {            \"queue_id\": str(uuid.uuid4()),            \"finding_id\": finding[\"finding_id\"],            \"severity\": finding[\"severity\"],            \"status\": \"PENDING\",            \"sla_deadline\": deadline.isoformat(),            \"run_id\": run_id,        }\nqueue_path = (            self._queue_dir / f\"{entry['queue_id']}.json\"        )        queue_path.write_text(            json.dumps(entry, indent=2, default=str)        )        queue_entries.append(entry)\nreturn queue_entries\n```\n\nThe important choice is not the JSON file. It is the absence of an approval capability inside the component that generated the finding.\n\nSeverity also changes the operational expectation. A critical finding is due in four hours; a low-severity item can wait seven days.\n\nBut reviewing the implementation forced me to narrow my own claim.\n\nThe current API exposes approval and rejection endpoints without an authentication dependency. Reviewer identity is supplied as a string in the request body, with “auditor” as its default value. The orchestrator also marks an audit run COMPLETE after placing its findings in the queue, even though those findings remain pending.\n\nThe prototype therefore enforces a workflow boundary inside the orchestrator. It does not yet prove that the caller approving the finding was an authorized human.\n\nThat distinction matters in a regulated system:\n\n“The agent cannot approve its own output” is weaker than “only a separately authenticated reviewer with the correct role can approve it.”\n\nBefore calling this production-ready, I would add identity-aware authentication, role-based authorization, separation-of-duties checks, reviewer identity derived from an access token, transactional state transitions, and an explicit WAITING_FOR_REVIEW run state.\n\nThe architecture had the right instinct. The security boundary was incomplete.\n\nThe second mechanism is a SHA-256 hash-chained audit ledger.\n\nEach finding event — drafted, approved, or rejected — stores a hash derived from the previous ledger entry and the current event:\n\n```\nchain_hash = sha256(previous_hash + finding_hash)\n```\n\nIf someone edits an old entry without updating the rest of the chain, verification fails at that point. This is better than a plain mutable log because partial historical edits become detectable.\n\nIt is not a blockchain, and it is not a complete answer to privileged tampering.\n\nA user with permission to rewrite the entire JSON ledger can modify history and recompute every later hash. The chain will verify because the attacker replaced both the data and the evidence used to verify it.\n\nHash chaining proves internal consistency. By itself, it does not prove that the chain is the original one.\n\nFor a defensible production design, I would anchor periodic chain heads outside the application’s write boundary. Options include signed checkpoints, a signing key held in a key-management system, append-only storage retention, or an external transparency service.\n\nThat would turn the current local tamper signal into evidence that a privileged application user cannot silently recreate.\n\nThis also changed the language I would use to describe the feature. Tamper-evident is accurate. Cryptographically verified is too broad unless the root used for verification is independently protected.\n\nThe headline result was 100% recall in both evaluation modes. Every injected defect was detected.\n\nThat sounds excellent until precision enters the picture.\n\nThe deterministic run produced 83.3% precision, 100% recall, and an F1 score of 0.909. With five injected defects and no misses, those values correspond to five true positives and one false positive.\n\nThe GPT-4o-mini run also found all five defects, but precision dropped to 38.5% and F1 to 0.556. That corresponds to five true positives and eight false positives.\n\nEvaluation mode True positives False positives Recall Precision F1 Deterministic 5 1 100% 83.3% 0.909 GPT-4o-mini 5 8 100% 38.5% 0.556\n\nEight false alarms for five real defects is not a cosmetic issue.\n\nIn an audit queue, false positives consume reviewer time, delay real findings, and teach operators to distrust the system. High recall protects against missed risk; low precision creates alert fatigue. A governance tool needs both.\n\nThis also exposed an error in my first headline. I had written “100% recall, 55% precision.”\n\nThe 55.6% number was F1, not precision. The actual precision was 38.5%.\n\nThat is exactly the kind of flattering metric substitution an audit system should prevent — including when the person making the mistake is its builder.\n\nThe gap between deterministic and LLM-based evaluation is the most important result in the project. The controlled evaluator benefited from known signatures and synthetic failure patterns. Real model outputs varied in length, phrasing, and borderline behavior.\n\nRules that looked calibrated in a controlled environment became noisy when the generator was allowed to behave like a generator.\n\nFigure 2: Detection performance in deterministic and GPT-4o-mini evaluations. Recall remained at 100%, while precision fell from 83.3% to 38.5%. Image by author.\n\nMy auditor failed its own audit\n\nThirdLine includes a “Validate the Validator” module with minimum thresholds:\n\nThe GPT-4o-mini run passed recall at 100% and judge consistency at 97.9%. It failed F1 at 0.556 and failed the false-positive-rate limit at 61.5%.\n\nThe generated model card therefore labeled the system Requires Review and stated that it did not meet its minimum governance standards for deployment.\n\nThat is the result I trust most.\n\nThere is another important nuance. The model card identifies PSI-based drift detection as a known limitation: output-length variance from real LLM responses can create false drift findings.\n\nBut the self-audit code does not autonomously isolate PSI as the root cause. Its calibration check compares observed precision with a fixed confidence assumption of 0.85, while the PSI explanation is documented separately as a suspected limitation.\n\nThe accurate claim is:\n\nThe self-audit exposed overconfidence and unacceptable false positives; follow-up analysis pointed to PSI calibration as one contributor.\n\nSaying the system “diagnosed its own PSI bug” would overstate what the code does.\n\nI also added an eval-as-test gate to GitHub Actions. The workflow generates a synthetic fleet, runs the audit and meta-evaluation, and fails when F1 falls below 0.70.\n\nThat is useful because evaluator quality becomes a release condition instead of a dashboard metric. It is still a narrow gate. The checked-in workflow does not exercise the live GPT-4o-mini path, so passing CI does not establish that the behavior that failed the model card is ready.\n\nFirst, I would calibrate each evaluation dimension separately.\n\nA single global result hides whether the false positives originate mainly in drift detection, hallucination judging, or another evaluator. Precision-recall curves by dimension would make tuning decisions visible instead of anecdotal.\n\nSecond, I would replace output-length PSI with representations closer to semantic behavior: embedding distributions, task-specific outcome features, and drift tests conditioned on prompt category.\n\nLength can be a useful signal. It is a fragile proxy for meaning.\n\nThird, I would test the reviewer boundary as aggressively as the model boundary.\n\nCan an unauthenticated caller approve a finding? Can one identity both create and approve it? Can queue state be changed without a ledger event? Governance claims should have adversarial tests, not just architecture diagrams.\n\nFinally, I would externally anchor the ledger and run the real-model evaluation as a scheduled quality gate. A deterministic test is valuable for regression control. It should not become a comforting substitute for the behavior that actually failed.\n\nThe implementation, evaluation code, and generated model card are available in the [ThirdLine repository](https://github.com/AasthaPJoshi/ThirdLine-of-Defense). The limitation report is as important as the feature list.\n\nThe broader lesson is not that AI should never audit AI. It is that a probabilistic auditor needs non-probabilistic boundaries around authority, identity, evidence, and release decisions.\n\nWhen your AI auditor raises a finding, what proves that the reviewer was authorized and what proves that the audit history was not rewritten afterward?\n\n[100% Recall, 38.5% Precision: What Happened When My AI Auditor Audited Itself](https://pub.towardsai.net/100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself-d000c7e8f626) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself", "canonical_source": "https://pub.towardsai.net/100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself-d000c7e8f626?source=rss----98111c9905da---4", "published_at": "2026-07-31 22:01:01+00:00", "updated_at": "2026-07-31 22:42:32.953415+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "ai-policy", "ai-agents"], "entities": ["ThirdLine", "GPT-4o-mini", "Federal Reserve", "OCC", "FDIC"], "alternates": {"html": "https://wpnews.pro/news/100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself", "markdown": "https://wpnews.pro/news/100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself.md", "text": "https://wpnews.pro/news/100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself.txt", "jsonld": "https://wpnews.pro/news/100-recall-38-5-precision-what-happened-when-my-ai-auditor-audited-itself.jsonld"}}