{"slug": "doc-foundry-adversarial-agents-that-prove-sigma-rules-before-you-ship-them", "title": "DOC-foundry: Adversarial Agents That Prove Sigma Rules Before You Ship Them", "summary": "DOC-foundry, a project by GitHub user mjha-arya, launched on Hacker News on November 13, 2026 with a pipeline that pairs LLM agents drafting Sigma detection rules against a deterministic engine that proves coverage and a fixed Red-Team adversary that must document two attacks in round one and re-throw them each round. Built on LangGraph, Pydantic, and FastMCP, the tool ingests threat strings such as CVE text, CISA KEV summaries, incident writeups, or ATT&CK techniques and outputs a Sigma rule, a threat report with convergence history, and a proof bundle with false-positive/false-negative counts and a hash. The repository had roughly two stars when explainx.ai cloned it, and its authors describe real-coverage as a bounded adversary set rather than an omniscient Blue Team.", "body_md": "**DOC-foundry** showed up on [Hacker News](https://news.ycombinator.com/item?id=50491753&ref=explainx) on November 13, 2026 with a sharp pitch: models can draft Sigma rules, but they cannot hand you a **proof** that the rule works, an adversary that **provably** evades the weak version, and an honest record of how the rule hardened until it held. The [GitHub repository](https://github.com/mjha-arya/DOC-foundry?ref=explainx) ships two pipelines — one that **proves** coverage with a deterministic engine, and one that **reviews** like a SOC workflow — on top of LangGraph, Pydantic, and FastMCP.\n\nIf you build **detection-as-code** or automate purple-team work with coding agents, this project is worth a weekend install. It is early — roughly **two stars** when explainx.ai cloned it — but the architecture is unusually explicit about **agents propose, engines prove**, the same split security teams need as [ARTEX-style offensive agents](https://explainx.ai/blog/cloudflare-modern-security-model-artex-pentest-orchestrators-2026) scale up parallel probing.\n\n## TL;DR\n\n| Question | Answer | \n|---|---|\n| What do you feed it? | Any threat string: CVE text, CISA KEV summary, incident writeup, ATT&CK technique | \n| What do you get? | Sigma rule, threat report with convergence history, proof bundle with FP/FN and hash | \n| Offline demo? | `python src/run.py --threat fixtures/certutil_t1105.json` | \n| Live authoring? | `python src/run.py \"T1218.011 rundll32...\"` with Gemini or Claude keys | \n| Review-only path? | `python src/run.py --chain \"...\"` plus optional`uvicorn webui.server:app` | \n| Honest limit? | real-coverage is a **bounded** adversary set, not omniscient Blue Team | \n\nAgents that act on your behalf need rules. [See how explainx helps secure your agents →](https://explainx.ai/agentbeam?src=blog_note)\n\n## Why detection engineers care now\n\nSigma rules are portable YAML, but production quality still depends on **telemetry shape**, **benign baselines**, and **evasion creativity**. Teams already feel pressure from two directions:\n\n1. **Offensive automation** — LLM pentest orchestrators iterate exploits faster than humans can retune detections, a pattern[FBI FLASH messaging on ARTEX](https://explainx.ai/blog/fbi-artex-flash-warning-agentic-ai-cyberattacks-november-2026) and Cloudflare's**30×** traffic spike both describe.\n2. **Generative drafting** — models happily emit Sigma syntax that**looks** authoritative yet fails on rename tricks, shell obfuscation, or parent-process blind spots.\n\nDOC-foundry's response is not another chat wrapper. It runs a **self-repair loop** where Blue re-authors after a **brittle** or **broken** verdict, while Red commits to **two documented attacks in round one** and re-throws them every round so goalposts do not drift mid-run.\n\nGreen and gray shields facing off, illustrating DOC-foundry Blue vs Red adversarial detection loop\n\nThat fixed-adversary discipline mirrors how explainx.ai treats benchmark claims elsewhere: bounded tests, explicit labels, no silent passes when the engine cannot evaluate a construct.\n\n## Architecture: propose vs prove\n\nThe classic graph, documented in the [README](https://github.com/mjha-arya/DOC-foundry/blob/main/README.md?ref=explainx), looks like this in prose:\n\n1. **Ingest → extract → telemetry** — Threat Analyst normalizes input; Telemetry agent fixes the test set.\n2. **Blue author** — Detection Engineer drafts behavior plus command-line intent, not one brittle string match.\n3. **Evaluate → battery → red_propose → red_verify → verdict** — Deterministic referee scores FP/FN, runs mutation battery, and only counts Red bypasses the engine confirms.\n4. **Repair or exit** — If verdict is not real-coverage and rounds remain, Blue gets concrete evasions and false positives as evidence.\n5. **Kartikeya gate → bundle** — Governance scores blast radius times capability delta; egress needs a human.\n\nFour LLM personas live in `src/agents.py`: Threat Analyst, Blue Detection Engineer, Telemetry, and Red-Team Adversary. Everything that decides truth — matching, adversarial mutations, coverage labels — sits in **engine.py**, **evaluator.py**, and **graph.py** with **same input → same verdict → same hash**.\n\n### What the engine refuses to fake\n\nThe README's honesty contract matters for builders who have been burned by evaluators that silently ignore hard clauses. **engine.py** raises on constructs it cannot truly evaluate — CIDR filters, aggregations, invented `|in` operators — rather than scoring them as covered. An un-evaluable rule costs a **broken** round, and Blue gets the engine's error text as repair evidence.\n\nThat is the opposite of vibe-based \"LGTM\" reviews and closer to how serious teams treat [schema validation and default deny](https://explainx.ai/blog/cloudflare-modern-security-model-artex-pentest-orchestrators-2026) on APIs.\n\nGreen probe pressing a crack in a shell, illustrating Red-team evasion testing against Sigma rules\n\n## The two pipelines: proof vs stakeholder review\n\nDOC-foundry runs **two workflows** with the same cast but different deliverables. Confusing them would mislead your SOC.\n\n### Classic proof loop (default for `python src/run.py`)\n\nOutputs land in `Outputs/<rule-slug>/` as:\n\n- **rule.yml** — Sigma artifact that survived adversarial rounds\n- **report.md** — Round-by-round convergence narrative\n- **proof.json** — Positive corpus, benign baseline, adversarial mutations, coverage label (`real-coverage` /`brittle` /`broken` ), FP/FN numbers, reproducibility hash\n\nThe canned **certutil T1105** fixture demonstrates a honest **two-round** arc: round one is **brittle** after rename bypass; round two reaches **real-coverage** after Blue anchors on rename-resistant fields. That offline path is what CI should gate on.\n\n### Review chain (`--chain` and default Web UI)\n\nThe chain asks **what would Red Team and SOC Lead say**, then ships **final-rule.yml**, **review-history.md**, and **review.json**. The README states plainly: this path **does not** invoke the proof core, never executes telemetry, and **issues no coverage verdict**.\n\nThe [Joomla iCagenda CVE-2026-48939 example](https://github.com/mjha-arya/DOC-foundry/blob/main/examples/joomla-icagenda-cve-2026-48939/review-history.md?ref=explainx) is the doc to read first. Red Team pushes Blue to broaden against evasion; SOC Lead pushes back on noise and triage fields. The tension is the product: hardening for evasion raises alert volume.\n\nRun the UI with:\n\n```\npip install -r requirements.txt\npython -m uvicorn webui.server:app --port 8000\n```\n\nFastAPI streams **node events** and **token events** over SSE. Cached replays are labeled **REPLAY (cached)** so demos cannot masquerade as live inference — a small integrity choice that matters when executives watch agent consoles.\n\n## Grounding: RAG without touching the hash\n\n`src/knowledge_base.py` plus `rule_corpus/` provide **offline BM25** retrieval:\n\n- Blue authors ground in **SigmaHQ** -style peer rules\n- Red attacks ground in **MITRE ATT&CK** defense evasion and**LOLBAS**\n\nRetrieval **grounds proposals only**; it never enters the reproducibility hash. That separation keeps audits clean: creative text may drift; verdict math may not.\n\n`docs/OSS-PARTNERS.md` maps future integrations — **Atomic Red Team** for positives, **EVTX-ATTACK-SAMPLES** for benign baselines, **pySigma** for multi-SIEM conversion. The `SigmaRuleArtifact.backends` slot already exists for backends you have not wired yet.\n\n## How to run it today\n\nFrom the repository root:\n\n```\npip install -r requirements.txt\n\n# Offline proof demo (no API key)\npython src/run.py --threat fixtures/certutil_t1105.json\n\n# Deterministic test suite\npython src/run.py --evals\n\n# Live threat with Claude\nexport ANTHROPIC_API_KEY=...\npython src/run.py --claude \"CVE-2026-45659 SharePoint deserialization RCE\"\n\n# Positive corpus from Atomic Red Team commands\npython src/run.py --corpus atomic\n\n# Human approval before deploy gate\npython src/run.py --approve alice\n```\n\nGoverned deploy lives behind **FastMCP** in `src/mcp_server.py`: the graph cannot fetch arbitrary URLs or push YAML to production without a token bound to the bundle hash.\n\n## Kartikeya gate: blast radius as code\n\n`src/governance.py` implements **Kartikeya's Gate** as **Blast Radius × Capability Delta**. Structural invariants include:\n\n- **Egress always requires a human**\n- Non-real coverage floors blast radius\n\nThat is philosophical naming, but the implementation detail matters for teams deploying **agent harnesses** next to SIEM credentials. Least privilege belongs in code, not prompt prayers — the same lesson as [MCP hardening patterns](https://explainx.ai/blog/what-is-mcp-model-context-protocol-guide) and [NIST, NSA, and CISA agent control practices](https://explainx.ai/blog/nist-nsa-cisa-ten-practices-si-control-incident-rogue-agents-evidence-no-retention-clock-2026) on explainx.ai.\n\n## What people are asking\n\n### Is this replacing my detection engineering team?\n\nNo. The README defines **real-coverage** as a **validated first-version baseline** with a **fixed** adversary, not forever bulletproof coverage. Your team still extends corpora, tunes for volume, and maps to Splunk, Elastic, or Sentinel via pySigma paths you own.\n\n### How does this compare to OpenAI's red-team factories?\n\nOpenAI's [Daybreak and cyber model work](https://explainx.ai/blog/openai-next-cyber-model-77-exploitgym-livestream-claimed-vs-verified-2026) targets **model** robustness and exploit reasoning. DOC-foundry targets **detection YAML** against **your telemetry category** for one log source per run — a narrower, SOC-shaped problem. They can complement each other if you keep secrets off agent context windows.\n\n### Can Red \"win\" by refusing the adversary role?\n\nYes. Safety-tuned models sometimes decline. The UI records refusals, and **graph.node_verdict** refuses to award real-coverage when nobody attacked. \"No evasions found\" and \"adversary never ran\" must not look alike — another honesty seam worth copying in internal agent evals.\n\n### Why only one log source per run?\n\nMajority vote picks a single telemetry category so Blue cannot overfit labels it never sees. A rule keyed on `TargetFilename` against process-creation events auto-fails — recall zero for the right reason.\n\n## Limits and licensing\n\n- **Early repo maturity** — Few stars and three commits when indexed; treat APIs as volatile until releases exist.\n- **License** — Read**LICENSE** and**NOTICE.md** in the repo before production use; explainx.ai does not render legal advice.\n- **Keys and data** — Live runs send threat text to Anthropic or Google APIs; run fixtures locally first.\n- **SIEM deploy** — Proof does not equal change management; pair with your existing detection-as-code pipeline, similar to practices in the[AISOC self-host guide](https://explainx.ai/blog/aisoc-open-source-ai-security-operations-center-mit-self-host-cyble-github-trending-guide-2026) .\n\n## What this means if you ship agentic security tools\n\nDOC-foundry is a template for **governed agent loops** in security:\n\n| Pattern | Why it matters | \n|---|---|\n| Deterministic referee | Stops persuasive JSON from faking pass rates | \n| Fixed Red corpus | Makes convergence reachable and auditable | \n| Separate review vs proof | Stops \"looks reviewed\" from meaning \"works\" | \n| Hash-bound deploy | Ties human approval to an exact artifact | \n| Labeled UI replay | Keeps demos honest under executive scrutiny | \n\nIf you are comparing offensive and defensive agent stacks, read it beside the [OpenAI Codex Security CLI](https://explainx.ai/blog/openai-codex-security-cli-sdk-open-source-july-2026) for code scanning and the [November 2026 cyberattacks tracker](https://explainx.ai/blog/november-2026-cyberattacks-list-breaches-exploits-ai-run-attacks-tracker-2026) for agentic pentest scale — offense and detection authoring are merging, but **proof artifacts** are still rare on the defensive side.\n\n## Related reading\n\n*Repository details, dependency versions, and model names are accurate as of November 14, 2026 per the [DOC-foundry README](https://github.com/mjha-arya/DOC-foundry?ref=explainx).*", "url": "https://wpnews.pro/news/doc-foundry-adversarial-agents-that-prove-sigma-rules-before-you-ship-them", "canonical_source": "https://explainx.ai/blog/doc-foundry-adversarial-sigma-detection-langgraph-proof-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-14 00:00:00+00:00", "updated_at": "2026-11-14 02:22:53.986486+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-safety", "developer-tools"], "entities": ["DOC-foundry", "Hacker News", "LangGraph", "Pydantic", "FastMCP", "mjha-arya", "Sigma", "explainx.ai"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/doc-foundry-adversarial-agents-that-prove-sigma-rules-before-you-ship-them", "markdown": "https://wpnews.pro/news/doc-foundry-adversarial-agents-that-prove-sigma-rules-before-you-ship-them.md", "text": "https://wpnews.pro/news/doc-foundry-adversarial-agents-that-prove-sigma-rules-before-you-ship-them.txt", "jsonld": "https://wpnews.pro/news/doc-foundry-adversarial-agents-that-prove-sigma-rules-before-you-ship-them.jsonld"}}