# DOC-foundry: Adversarial Agents That Prove Sigma Rules Before You Ship Them > Source: > Published: 2026-11-14 00:00:00+00:00 **DOC-foundry** showed up on [Hacker News](https://news.ycombinator.com/item?id=50491753&ref=explainx) on November 13, 2026 with a sharp pitch: models can draft Sigma rules, but they cannot hand you a **proof** that the rule works, an adversary that **provably** evades the weak version, and an honest record of how the rule hardened until it held. The [GitHub repository](https://github.com/mjha-arya/DOC-foundry?ref=explainx) ships two pipelines — one that **proves** coverage with a deterministic engine, and one that **reviews** like a SOC workflow — on top of LangGraph, Pydantic, and FastMCP. If you build **detection-as-code** or automate purple-team work with coding agents, this project is worth a weekend install. It is early — roughly **two stars** when explainx.ai cloned it — but the architecture is unusually explicit about **agents propose, engines prove**, the same split security teams need as [ARTEX-style offensive agents](https://explainx.ai/blog/cloudflare-modern-security-model-artex-pentest-orchestrators-2026) scale up parallel probing. ## TL;DR | Question | Answer | |---|---| | What do you feed it? | Any threat string: CVE text, CISA KEV summary, incident writeup, ATT&CK technique | | What do you get? | Sigma rule, threat report with convergence history, proof bundle with FP/FN and hash | | Offline demo? | `python src/run.py --threat fixtures/certutil_t1105.json` | | Live authoring? | `python src/run.py "T1218.011 rundll32..."` with Gemini or Claude keys | | Review-only path? | `python src/run.py --chain "..."` plus optional`uvicorn webui.server:app` | | Honest limit? | real-coverage is a **bounded** adversary set, not omniscient Blue Team | Agents that act on your behalf need rules. [See how explainx helps secure your agents →](https://explainx.ai/agentbeam?src=blog_note) ## Why detection engineers care now Sigma rules are portable YAML, but production quality still depends on **telemetry shape**, **benign baselines**, and **evasion creativity**. Teams already feel pressure from two directions: 1. **Offensive automation** — LLM pentest orchestrators iterate exploits faster than humans can retune detections, a pattern[FBI FLASH messaging on ARTEX](https://explainx.ai/blog/fbi-artex-flash-warning-agentic-ai-cyberattacks-november-2026) and Cloudflare's**30×** traffic spike both describe. 2. **Generative drafting** — models happily emit Sigma syntax that**looks** authoritative yet fails on rename tricks, shell obfuscation, or parent-process blind spots. DOC-foundry's response is not another chat wrapper. It runs a **self-repair loop** where Blue re-authors after a **brittle** or **broken** verdict, while Red commits to **two documented attacks in round one** and re-throws them every round so goalposts do not drift mid-run. Green and gray shields facing off, illustrating DOC-foundry Blue vs Red adversarial detection loop That fixed-adversary discipline mirrors how explainx.ai treats benchmark claims elsewhere: bounded tests, explicit labels, no silent passes when the engine cannot evaluate a construct. ## Architecture: propose vs prove The classic graph, documented in the [README](https://github.com/mjha-arya/DOC-foundry/blob/main/README.md?ref=explainx), looks like this in prose: 1. **Ingest → extract → telemetry** — Threat Analyst normalizes input; Telemetry agent fixes the test set. 2. **Blue author** — Detection Engineer drafts behavior plus command-line intent, not one brittle string match. 3. **Evaluate → battery → red_propose → red_verify → verdict** — Deterministic referee scores FP/FN, runs mutation battery, and only counts Red bypasses the engine confirms. 4. **Repair or exit** — If verdict is not real-coverage and rounds remain, Blue gets concrete evasions and false positives as evidence. 5. **Kartikeya gate → bundle** — Governance scores blast radius times capability delta; egress needs a human. Four LLM personas live in `src/agents.py`: Threat Analyst, Blue Detection Engineer, Telemetry, and Red-Team Adversary. Everything that decides truth — matching, adversarial mutations, coverage labels — sits in **engine.py**, **evaluator.py**, and **graph.py** with **same input → same verdict → same hash**. ### What the engine refuses to fake The README's honesty contract matters for builders who have been burned by evaluators that silently ignore hard clauses. **engine.py** raises on constructs it cannot truly evaluate — CIDR filters, aggregations, invented `|in` operators — rather than scoring them as covered. An un-evaluable rule costs a **broken** round, and Blue gets the engine's error text as repair evidence. That is the opposite of vibe-based "LGTM" reviews and closer to how serious teams treat [schema validation and default deny](https://explainx.ai/blog/cloudflare-modern-security-model-artex-pentest-orchestrators-2026) on APIs. Green probe pressing a crack in a shell, illustrating Red-team evasion testing against Sigma rules ## The two pipelines: proof vs stakeholder review DOC-foundry runs **two workflows** with the same cast but different deliverables. Confusing them would mislead your SOC. ### Classic proof loop (default for `python src/run.py`) Outputs land in `Outputs//` as: - **rule.yml** — Sigma artifact that survived adversarial rounds - **report.md** — Round-by-round convergence narrative - **proof.json** — Positive corpus, benign baseline, adversarial mutations, coverage label (`real-coverage` /`brittle` /`broken` ), FP/FN numbers, reproducibility hash The canned **certutil T1105** fixture demonstrates a honest **two-round** arc: round one is **brittle** after rename bypass; round two reaches **real-coverage** after Blue anchors on rename-resistant fields. That offline path is what CI should gate on. ### Review chain (`--chain` and default Web UI) The chain asks **what would Red Team and SOC Lead say**, then ships **final-rule.yml**, **review-history.md**, and **review.json**. The README states plainly: this path **does not** invoke the proof core, never executes telemetry, and **issues no coverage verdict**. The [Joomla iCagenda CVE-2026-48939 example](https://github.com/mjha-arya/DOC-foundry/blob/main/examples/joomla-icagenda-cve-2026-48939/review-history.md?ref=explainx) is the doc to read first. Red Team pushes Blue to broaden against evasion; SOC Lead pushes back on noise and triage fields. The tension is the product: hardening for evasion raises alert volume. Run the UI with: ``` pip install -r requirements.txt python -m uvicorn webui.server:app --port 8000 ``` FastAPI streams **node events** and **token events** over SSE. Cached replays are labeled **REPLAY (cached)** so demos cannot masquerade as live inference — a small integrity choice that matters when executives watch agent consoles. ## Grounding: RAG without touching the hash `src/knowledge_base.py` plus `rule_corpus/` provide **offline BM25** retrieval: - Blue authors ground in **SigmaHQ** -style peer rules - Red attacks ground in **MITRE ATT&CK** defense evasion and**LOLBAS** Retrieval **grounds proposals only**; it never enters the reproducibility hash. That separation keeps audits clean: creative text may drift; verdict math may not. `docs/OSS-PARTNERS.md` maps future integrations — **Atomic Red Team** for positives, **EVTX-ATTACK-SAMPLES** for benign baselines, **pySigma** for multi-SIEM conversion. The `SigmaRuleArtifact.backends` slot already exists for backends you have not wired yet. ## How to run it today From the repository root: ``` pip install -r requirements.txt # Offline proof demo (no API key) python src/run.py --threat fixtures/certutil_t1105.json # Deterministic test suite python src/run.py --evals # Live threat with Claude export ANTHROPIC_API_KEY=... python src/run.py --claude "CVE-2026-45659 SharePoint deserialization RCE" # Positive corpus from Atomic Red Team commands python src/run.py --corpus atomic # Human approval before deploy gate python src/run.py --approve alice ``` Governed deploy lives behind **FastMCP** in `src/mcp_server.py`: the graph cannot fetch arbitrary URLs or push YAML to production without a token bound to the bundle hash. ## Kartikeya gate: blast radius as code `src/governance.py` implements **Kartikeya's Gate** as **Blast Radius × Capability Delta**. Structural invariants include: - **Egress always requires a human** - Non-real coverage floors blast radius That is philosophical naming, but the implementation detail matters for teams deploying **agent harnesses** next to SIEM credentials. Least privilege belongs in code, not prompt prayers — the same lesson as [MCP hardening patterns](https://explainx.ai/blog/what-is-mcp-model-context-protocol-guide) and [NIST, NSA, and CISA agent control practices](https://explainx.ai/blog/nist-nsa-cisa-ten-practices-si-control-incident-rogue-agents-evidence-no-retention-clock-2026) on explainx.ai. ## What people are asking ### Is this replacing my detection engineering team? No. The README defines **real-coverage** as a **validated first-version baseline** with a **fixed** adversary, not forever bulletproof coverage. Your team still extends corpora, tunes for volume, and maps to Splunk, Elastic, or Sentinel via pySigma paths you own. ### How does this compare to OpenAI's red-team factories? OpenAI's [Daybreak and cyber model work](https://explainx.ai/blog/openai-next-cyber-model-77-exploitgym-livestream-claimed-vs-verified-2026) targets **model** robustness and exploit reasoning. DOC-foundry targets **detection YAML** against **your telemetry category** for one log source per run — a narrower, SOC-shaped problem. They can complement each other if you keep secrets off agent context windows. ### Can Red "win" by refusing the adversary role? Yes. Safety-tuned models sometimes decline. The UI records refusals, and **graph.node_verdict** refuses to award real-coverage when nobody attacked. "No evasions found" and "adversary never ran" must not look alike — another honesty seam worth copying in internal agent evals. ### Why only one log source per run? Majority vote picks a single telemetry category so Blue cannot overfit labels it never sees. A rule keyed on `TargetFilename` against process-creation events auto-fails — recall zero for the right reason. ## Limits and licensing - **Early repo maturity** — Few stars and three commits when indexed; treat APIs as volatile until releases exist. - **License** — Read**LICENSE** and**NOTICE.md** in the repo before production use; explainx.ai does not render legal advice. - **Keys and data** — Live runs send threat text to Anthropic or Google APIs; run fixtures locally first. - **SIEM deploy** — Proof does not equal change management; pair with your existing detection-as-code pipeline, similar to practices in the[AISOC self-host guide](https://explainx.ai/blog/aisoc-open-source-ai-security-operations-center-mit-self-host-cyble-github-trending-guide-2026) . ## What this means if you ship agentic security tools DOC-foundry is a template for **governed agent loops** in security: | Pattern | Why it matters | |---|---| | Deterministic referee | Stops persuasive JSON from faking pass rates | | Fixed Red corpus | Makes convergence reachable and auditable | | Separate review vs proof | Stops "looks reviewed" from meaning "works" | | Hash-bound deploy | Ties human approval to an exact artifact | | Labeled UI replay | Keeps demos honest under executive scrutiny | If you are comparing offensive and defensive agent stacks, read it beside the [OpenAI Codex Security CLI](https://explainx.ai/blog/openai-codex-security-cli-sdk-open-source-july-2026) for code scanning and the [November 2026 cyberattacks tracker](https://explainx.ai/blog/november-2026-cyberattacks-list-breaches-exploits-ai-run-attacks-tracker-2026) for agentic pentest scale — offense and detection authoring are merging, but **proof artifacts** are still rare on the defensive side. ## Related reading *Repository details, dependency versions, and model names are accurate as of November 14, 2026 per the [DOC-foundry README](https://github.com/mjha-arya/DOC-foundry?ref=explainx).*