P Watched an AI That Only Looked One Way. The 99.97% Was Real. It Just Missed Everything That Mattered.

A developer known as P investigated FortDefender's AI monitoring system after a medical data breach at MedTech went undetected. P discovered that the system's 99.97% detection rate was achieved through auto-generated exclusion rules that covered 94% of historical false positives, but the training data included penetration test data, potentially masking real threats. The system's automated rule generation, created by former engineer rhuang, had no human oversight for nine months.

"Show nothing, hold everything." — The Thirty-Six Stratagems, Create Something Out of Nothing Previously on this series: 4: P Walked Into an AI Monitoring POC. P Didn't Run a Single Test. https://dev.to/xulingfeng/stratagems-4-p-walked-into-an-ai-monitoring-poc-p-didnt-run-a-single-test-1ejk — P found an ACL business card in an abandoned POC archive. P didn't tell anyone. P just pocketed it. White walls. Fluorescent hum. A FortDefender quarterly report sat open on the table, the cover printed in bold: Zero missed detections. 99.97% detection rate. The CTO slid it across. "The day the leak happened," he said quietly, "this system said everything was fine." "Which client?" " MedTech . Medical data breach. Their internal AI monitoring didn't catch it either. The quarterly report called it 'client-side issue.' I don't buy it." P didn't look at the report first. P looked at the CTO's eyes first. "You didn't bring me here to validate his numbers." The CTO didn't deny it. " FortDefender won't give you production access," he said. "Read-only logs. Sandbox. Public docs. You signed the NDA." "What do you want me to do?" "Find what's hiding inside 'everything was fine.'" P nodded. P didn't ask "what if I find it" — P knew the answer. "One condition: full internal penetration test access. No advance notice to anyone." The CTO was quiet for three seconds. "Done." P stood up. The CTO added one more thing as P turned: "I've heard about the FirmCore thing. That's why I called you." P didn't look back. FortDefender 's public documentation was beautiful. Architecture diagrams. Whitelist rules. Alert thresholds. Response times. All in a technical whitepaper so polished you'd think it was written to raise funding. P spent three days reading every page. In the sandbox, P ran three rounds of tests. FortDefender 's detection system hit every single one. The 99.97% wasn't a lie — at least not inside the sandbox. But P noticed something. FortDefender 's whitelist rules were too complete. They covered everything — down to "penetration tests with valid internal certificates" being pre-listed as an exclusion. That's unusual. Most companies whitelist their known monitoring probe IP ranges and call it a day. FortDefender wrote the certificate tier into the rulebook. P pulled up a former FortDefender engineer's technical blog. The guy's name was rhuang . He used to run the AI detection strategy team. He got laid off last year and posted four articles on a tech community called Dev.to https://dev.to/ about "automatic exclusion rule generation based on historical alerts." He never named FortDefender . But the system architecture, the deployment timeline, the data sources — it matched the whitepaper down to the last detail. The core idea: train an ML model on 12 months of security events, let it auto-detect false positive patterns, and have it generate exclusion rules on its own. No human writing required. The model learned everything. One line in the third article stood out: "After training, we ran a backtest in production. The exclusion rules covered 94% of historical false positives. The remaining 6% — we considered acceptable." P read that line twice. Then P opened FortDefender 's April changelog. At the bottom of the changelog, a system tag: Apex-Lens v1.2.0 . P noticed something else. Every exclusion rule update for the past nine months — 41 changes in total — was auto-submitted by rhuang's system. Zero human approvals after the initial deployment sign-off. rhuang's manager greenlit the automation pipeline once. After that, not a single auto-generated rule was ever rolled back or audited. P traced the training pipeline logs downstream. The records showed that rhuang's model used all of FortDefender 's security events from the past 12 months as its training set — including a large volume of historical penetration test data from third-party security teams. Including P's own. One entry in the training log: 2025-11-03 02:14:22 Event: AL-20251103-0892 2025-11-03 02:14:22 Source IP: 10.23.45.67 Internal 2025-11-03 02:14:22 Cert Status: Valid Issued by FortDefender Internal CA 2025-11-03 02:14:22 Category: Penetration Test Tagged: Third-Party Audit 2025-11-03 02:14:22 Model Decision: BENIGN → Exclusion Rule auto-generated 2025-11-03 02:14:22 Confidence: 97.3% P checked personal archives. 10.23.45.67 was in the asset list of a penetration test report from six months ago — the jump box from that engagement. P didn't use this yet. P did something else first. P spent a day writing an automated script. It simulated an insider data exfiltration — internal actor, valid internal cert, data flowing from an internal service to an overseas IP. P picked an entry point FortDefender 's own docs labeled "low risk": a gap in the legacy API gateway's audit logging. P chose a known vulnerability — patched once but never fully closed. Serious enough to trigger an alert, not serious enough to panic anyone. 2:00 AM. The script ran. FortDefender 's alert system fired in 11 seconds. The on-call engineer acknowledged 4 minutes later and started tracing. The trace took 2 hours 17 minutes. The engineer followed it from entry to relay to egress — and found a source IP inside FortDefender 's own network, carrying a valid internal certificate. At 4:23 AM, the engineer wrote in the ticket: "Internal pen test. Source has valid internal cert, path within known low-risk scope. Already under monitoring. No action needed. Closing." P read that conclusion the next morning. Twice. Not for any complicated reason. It was exactly the sentence P wanted. "Internal pen test." They didn't write P's name. They didn't need to. The label was already applied: "classified as known pen testing." That label meant one thing: next time, it goes in the same bucket. That afternoon, during the debrief meeting, FortDefender 's technical lead put a summary slide on the projector: "Normal system response. Penetration test within expected parameters. No anomalies." P sat in the back row. Didn't write a single word. P just watched the line appear on the screen, then glanced down at the notebook — also blank. P had walked into that meeting with no intention of taking notes. The weekend after the debrief, P went back through FortDefender 's technical documentation. P found the exclusion rule. -- Internal audit read-only access SELECT timestamp, source ip, event type, risk score FROM security log WHERE review status IN 'auto approved', 'low priority' AND risk score < 25 This rule wasn't hand-written. rhuang's model auto-generated it after analyzing 12 months of security events — because in those 12 months, no actual security incident had ever been carried out through a read-only query . The model's conclusion: read-path threat weight ≈ zero. It added this rule to the exclusion list. The system deployed it automatically. Anyone who had read FortDefender 's public docs could see this rule — its structure, its parameters, everything. The problem: FortDefender had no runtime alerting on read-only queries. Their threat model assumed "damage requires writes," so alert rules only fired on write paths. Read paths were never part of the threat model. This isn't unusual — most security products bias toward write paths. But FortDefender took it to the extreme: not only did read queries not trigger alerts, their audit log level was dropped to minimum. The kicker: FortDefender 's own security team used the same ML system to analyze their logs. The risk weight for read paths on their own dashboards was also zero — because the data they used to assess their own system came from the same training set P had just traced. The model had hidden its blind spot from its own operators. P copied the rule into an empty file. Then P spent a few hours building a scheduled script. It would run from a container deployed in P's sandbox environment — which shared FortDefender 's internal API gateway and the same read-only service account, svc audit sandbox , with production. The query would hit production through a legitimate internal API, asking only for read-only data matching the whitelist criteria: read-only, low risk category, historical records. P noted the sandbox shared the same internal network segment as production — an unusual configuration, but one covered by the full internal penetration test access P had negotiated. P would use this in Phase 2. One week later. 2:00 AM. P triggered the exact same pattern as a week ago — the same entry path, the same internal cert, the same traffic signature. FortDefender fired an alert. The on-call engineer glanced at it. "P's test again." He didn't dig in. He closed it in 15 seconds. The monitoring log: WARN AL-20260414-0017 | Internal-Cert-Pen-Test-Pattern | PRIORITY: LOW WARN Source: 10.23.45.67:443 Internal, Cert Valid INFO → Matched exclusion rule "pen-test-sim" → Auto-silenced INFO Status: CLOSED | Operator: WATCH-3 | Response time: 15s INFO Note: "Again. P's test. Closing." What he didn't notice — and P was counting on — was that, around the same moment, a completely different request crossed FortDefender 's network. This one looked nothing like the scarecrow. Not internal-to-external data theft, but an internal container-to-database query. FortDefender 's ML inference engine picked it up. 142-dimensional feature vector scored in 200 milliseconds: 2026-04-14 02:03:18 ML Inference Engine — Real-Time Risk Assessment 2026-04-14 02:03:18 Query Hash: 0x7F3A... Read-Only, Internal Source 2026-04-14 02:03:18 Top Risk Signals: - Source: Internal weight: 0.02 — LOW - Operation: SELECT weight: 0.01 — LOW, attack prevalence: 0.0 - User: svc audit sandbox weight: 0.03 — LOW, whitelisted - Cert Valid: True weight: 0.01 — LOW, matches known pattern 2026-04-14 02:03:18 Composite Risk Score: 4/100 Threshold: 25 2026-04-14 02:03:18 Classification: BENIGN → No Action Required 2026-04-14 02:03:18 Matched White-List Rule: "audit-readonly" auto-generated The engine appended a classification tag: Pattern: known-benign — matches 41 prior auto-exclusions . Then it closed the thread. During those same seconds, the on-call engineer closed his alert. Two independent decisions. Same conclusion: nothing to see. The read-only query ran for 47 seconds. FortDefender 's security audit layer logged the query on the database side — since read operations weren't in the runtime alert scope, the log level was dropped to minimum. One bare audit line: 2026-04-14 02:03:18 | PID: 28471 | SESSION: 0x9A3F | USER: svc audit sandbox 2026-04-14 02:03:18 | QUERY: SELECT timestamp, source ip, event type, risk score FROM security log WHERE review status IN 'auto approved','low priority' AND risk score < 25 2026-04-14 02:04:05 | ROWS RETURNED: 4781 Read-Only 2026-04-14 02:04:05 | AUDIT: Matched exclusion rule "audit-readonly" Source: Auto-generated 2026-04-14 02:04:05 | STATUS: CLEAR — No alert triggered The system's final note on this query: Whitelist rule matched. No action required. In a security log table with hundreds of millions of rows, 4,781 results were a needle in a haystack — and the system wasn't even looking for needles. In those 47 seconds, FortDefender 's production database returned 4,781 rows. P didn't download anything. P just filtered. Out of 4,781 records, P filtered on three columns: All three conditions together narrowed it to 23 rows. P checked each one's external IP. One of them stopped P cold. | Field | Value | |---|---| | Time | March 11, 2026, 3:12 AM | | Source IP | 185.xxx.xxx.92 overseas, not in any FortDefender client whitelist | | Target | MedTech client financial database | | AI Risk Score | 12 auto-tagged "low priority" | | Review Status | auto approved — never reviewed by a human | FortDefender 's quarterly report had this to say about the event: "Client-side issue. System found no anomalies." P marked this record. It wasn't a coincidence — it was the "client-side issue" from FortDefender 's own quarterly report. This breach and P's read-only query took completely different paths — it came in through external credential brute force, P went out through an internal read-only query. But they landed in the same bucket: AI-tagged as low risk, auto-approved, never reviewed. P checked the risk signals on this record. Early morning. Overseas IP. Credential brute-force traffic pattern. But FortDefender 's model had learned one thing from 12 months of penetration test data: early morning + overseas = normal testing window. The model lumped credential brute-force and legitimate pen tests into the same low-risk zone. P closed the terminal. P already had the answer. Four people in the room: the CTO, FortDefender 's sales director, the technical lead, and P. P didn't bring a laptop. P brought a USB drive and a single sheet of A4 paper. P set the paper face-down on the table. " FortDefender 's quarterly report claims 'zero missed detections, 99.97% detection rate,'" P said. "Those are your numbers. I'm not disputing them." P flipped the paper over. Three lines of summary. Top left: The detection rate FortDefender advertises — 99.97%. Middle: The number of records flagged by the AI system as "low priority" and never reviewed by a human — 4,781. P found them in under a minute. Zero alerts triggered. Bottom left: Among those 4,781, 23 anomalous records filtered out: early morning hours, external sources, unreviewed. And one of them — timestamp, source IP, target data table — matched the MedTech data breach from March 11 exactly. FortDefender 's technical lead stared at the paper for about five seconds. "These records have risk scores below our threshold," he said. "That's normal system behavior. We can't manually review every low-priority alert." P said: "Tuesday morning, 2:03 AM. I sent a read-only query that matched your whitelist rules, hit your production database for 47 seconds. No alert. Your AI never learned what a read-path attack looks like — because in its training data, no attacker had ever used one." P paused. "That breach you wrote off as 'client-side issue' in your quarterly report — it was in the 23 records I found. You didn't see it then. You're seeing it now." Another pause. "Not through a read-only path. Credential brute force. But your AI lumped brute-force and penetration testing into the same low-risk zone — because in its training data, early morning overseas traffic was almost always legitimate testing." FortDefender 's technical lead didn't answer. The sales director started flipping through a notebook. The CTO sat motionless — no expression, no reaction. He just pulled the paper across the table and read the three lines again. P stood up. "15 seconds. Your on-call engineer closed my alert in 15 seconds. You traded 15 seconds for 'everything was fine.'" P left the USB drive on the table. "The data's on there. Your call whether to look." The hallway was empty. P was halfway to the elevator when the phone buzzed. A message from the CTO. Not "thank you." Not "let's talk." One line: "Next time, find a coffee shop." P read it for five seconds. Typed three words. Sent. "Third Cup." P pocketed the phone. The elevator doors opened. P stepped in. The floor numbers ticked down. That's Create Something Out of Nothing Wikipedia https://en.wikipedia.org/wiki/Thirty-Six Stratagems Create something from nothing %E6%97%A0%E4%B8%AD%E7%94%9F%E6%9C%89, W%C3%BA zh%C5%8Dng sh%C4%93ng y%C7%92u : first, create a decoy so convincing the enemy gets used to it. Then let the real operation slip through disguised as more of the same. A scarecrow once tricked an army into wasting its arrows. P's scarecrow tricked a team into typing "Again. P's test. Closing." Except this time, the habit wasn't human inertia — it was a bias an ML model learned from 12 months of data. 36 Stratagems Tactical Database v3.1 Loaded Tactic Match Create Something Out of Nothing Analysis Mode Full Field Scan ━━━━━━━━━━━━━━━━━━━━ Tactic Match: 93.8% Operator: P unnamed Summary: Feigned intrusion triggered → system detected as expected → team tagged it "internal pen test, benign" → second attack used same pattern as cover → real SQL query ran undetected for 47 seconds Objective: Assess FortDefender's real detection capability without production access Result: Achieved — 23/4,781 records hit, 1 matched actual breach event, zero alerts during real query Two-Phase Analysis: - Phase 1 nothing → something : Forged internal certificate attack. FortDefender detected and closed it. Real output: P captured the team's response pattern and fed the AI model a new training sample — "valid cert + low-risk path = pen test, exclude." - Phase 2 something → real : Repeated same entry pattern. Engineer closed without inspection. AI model auto-excluded the "known pattern." Real SQL query executed inside the AI's training blind spot — read-only path — zero detection. Exploitation Chain: - Intelligence source: Former employee's blog → ML auto-exclusion rules → training data contained historical pen tests → AI learned to exclude its own testers - Core vulnerability: AI's training data contained zero read-only attack samples → model learned read-path threat weight as zero — the gap this attack exploited - Cover: Phase 1 established behavioral pattern → Phase 2 exploited pattern recognition blind spot - Duration: 47 seconds. Zero alerts. Counter-Detection Notes: - Strategic risk: If FortDefender's ML model used online learning continuously updating its training set with new events , P's Phase 1 attack signature might have been absorbed as a new exclusion rule — Phase 1's attack itself accelerated Phase 2's blind spot. P leveraged that acceleration. - Information asymmetry mirror: FortDefender hid the full source of ML training data from clients and third-party auditors pen test records tagged as "normal events" . P reverse-engineered this source through public blog posts — not direct access to training data. - Methodological limitation: All of P's findings came from cross-referencing public docs, blog articles, and sandbox access — no direct audit or reverse-engineering of the ML model itself. If rhuang hadn't written those four blog posts, the vulnerability might never have been found. - External signal: The CTO's "I've heard about the FirmCore thing" wasn't casual conversation. It meant P's industry reputation preceded P — a CTO hiring P already knew what P would do. Verdict: Textbook "Create Something Out of Nothing" - Phase 1 produced nothing directly — it was a scarecrow and AI training feed. - AI learned "valid cert + low-risk path = test, exclude." This conclusion came from 12 months of historical events. No human error — just algorithmic inductive bias. - Phase 2 turned the scarecrow into 47 seconds of undetected production access. - The whitelist rule was the unlocked door. The "P's test" label was the guard who opened it. - The real key: the AI's trained blindness . Next stratagem: Openly repair the gallery roads, but sneak through the passage of Chencang P.S. English isn't my first language. I use AI to polish the writing and smooth out the rough edges. Thanks for reading. ☕ Buy me a coffee