Every AppSec engineer eventually hits a mathematical scaling wall.
In a high-growth environment, a single application security engineer is often personally responsible for securing dozens of microservices, multiple public-facing API gateways, complex native mobile apps, and continuous CI/CD pipelines. When you are outnumbered 100-to-1 by software engineers shipping code daily, 70% of your offensive testing time is eaten by repetitive friction: subdomain sweeps, manual API parameter fuzzing, writing headers, and inspecting boilerplate responses.
Naturally, when the AI wave hit, many of us tried to offload this friction by feeding targets to LLMs.
But if you’ve ever tried to run a penetration test using a standard conversational LLM chatbot, you know it fails catastrophically in production. The reasons are always the same:
Frustrated by these limitations, I set out to build something different.
This is the story of how I evolved my automated security testing from a brittle, free-roaming prompt loop into Okwute: a deterministic, 4-node directed graph pipeline (Mapper → Generator → Executor → Validator) that runs on a headless, self-hosted harness to achieve zero-false-positive autonomous penetration testing.
.claude
Harness (Context on Disk) My initial attempt at solving LLM amnesia was building an engine I called the ** .claude Harness**.
Instead of letting the agent hold execution state in its active context, I decided to decouple the intelligence layer from the ephemeral chat session. The core breakthrough was "loop engineering", creating an execution engine that lives between prompts by anchoring all memory, phase progression, and vulnerability proofs in deterministic filesystem artifacts on a self-hosted workspace.
Instead of isolating learning inside independent target silos, we designed an enterprise-ready workspace topology with a robust Three-Tier Memory System under a central .claude/
root runtime:
.claude/
├── shared/
│ └── org-knowledge.md # Tier 1: Organization-wide security baselines & false-positive filters
├── products/
│ └── <product-slug>.md # Tier 2: Shared product-family dynamic memory (e.g., login_api)
├── working/
│ └── <session-id>/
│ └── memory.md # Tier 3: Isolated session scratchpad & exploration sandbox
└── targets/
└── payment-gateway/ # Isolated target workspace
├── state.json # Authoritative state-machine ticker & configs
├── findings.md # Cumulative, append-only verified vulnerability proofs
├── triage.json # Human-in-the-loop triage overrides
└── reports/ # Generated executive & technical audit reports
This directory layout decouples ephemeral execution logs from enterprise-grade intelligence:
product_type
. If an agent discovers sequential ID exploitation patterns on a target within a product family, every other target in that family inherits that heuristic, speeding up warm-starts.The system ran as an autonomous state machine. Instead of running an infinite script that would drain API credits, it executed one bounded, deterministic "tick" via a /scan-target
command.
During each tick, the harness would:
state.json
to determine the current testing phase (recon
, enumeration
, vuln-discovery
, chain-development
, reporting
).By writing every single observation, endpoint, and finding immediately to a structured state.json
and a Markdown audit trail (findings.md
), the context could survive host restarts, process crashes, and API disconnects.
To allow multiple instances of this harness to run without stepping on each other's toes, we implemented a lightweight three-tier memory system using SHA-256 context hashing in the file frontmatter:
`context_hash = SHA-256 (file_content ∖ {context_hash line})`
If a background worker attempted to write an update to a product's shared memory, it would first re-evaluate the hash on disk. If the current hash matched the initial state, the write was committed; if not, a conflict was declared, and the worker safely aborted.
It was an incredible step forward. But as we scaled it, we hit a massive conceptual roadblock: LLMs love to roam free. If you give an LLM agent access to a generic "run terminal command" tool, it will eventually try to skip steps, guess endpoints, or spiral into endless recursive loops trying to fix a single broken curl payload.
We needed a system that enforced absolute discipline.
In August 2026, at DevCon (presented also at Black Hat USA 2026 and DEF CON 34), PortSwigger's Director of Research, James Kettle, published a paradigm-shifting whitepaper titled " Can AI do novel security research? Meet the HTTP Terminator".
In his paper, Kettle tackled a profound question: can an autonomous AI system actually invent new attack techniques, bypass complex security layers, and discover zero-days on live, production systems?
To prove it, he built and open-sourced the HTTP Terminator, an autonomous research engine designed to hunt for HTTP desync vulnerabilities. But his ultimate success was built on a series of harsh, real-world lessons about the cognitive limits of SOTA LLMs. He realized that while LLMs are brilliant at high-level reasoning and pattern recognition, they are notoriously unreliable at state-tracking, structured execution, and exact payload formatting.
Left to their own devices, agents in a freeform loop behave like over-caffeinated interns. They trigger WAF rate-limits, get trapped in recursive debugging loops, hallucinate vulnerabilities, and frequently give up entirely upon seeing defensive headers like Connection: close
.
To tame this chaos, Kettle structured the HTTP Terminator around a Four-Phase Discovery Loop:
Kettle's ultimate takeaway was that "AI vs. Human" is the wrong framing. Instead, modern security engineering is a three-way collaboration: AI vs. Code vs. Human.
He found that starting with an AI-heavy loop is useful for speed, but to achieve consistent, production-grade accuracy, you must gradually move cognitive responsibility to deterministic code. In his exploitation engine, he split templates in half, ensuring deterministic code validated the evidence while the LLM handled planning.
This core design principle was the absolute catalyst for Okwute.
Okwute takes Kettle's four-phase research methodology and compiles it into a rigid, relational, and mathematically-enforced Directed Acyclic Graph (DAG). We eliminated the loose phase drift of legacy tools by locking down each step inside its own database boundary. The Mapper, Generator, Executor, and Validator are the physical implementation of Kettle's Ideation, Evaluation, Weaponization, and Cascade loops, bound forever by strict read/write contracts on SQLite.
I retired the free-flowing state machine of the .claude
harness and completely re-engineered the platform around a rigid 4-node pipeline managed by SQLite.
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ MAPPER │─────▶│ GENERATOR │─────▶│ EXECUTOR │─────▶│ VALIDATOR │
│ (Pure Recon) │ │ (Test Cases) │ │ (Burp Suite) │ │ (PoC Proof) │
└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
The pipeline operates on a strict rule: Each node reads only from its upstream table and writes only to its designated downstream table. No node may skip, jump ahead, or bypass the graph.
Here is how the 4 nodes execute natively in our self-hosted environment:
/run-mapper <target> [<domain-or-url>]
subfinder
), port scanning (nmap
), and passive traffic analysis (httptoolkit
MCP).endpoints
table of the target's SQLite database (appsec.db
)./run-generator <target>
endpoints
table. Utilizing the LLM's deep contextual reasoning, it evaluates the endpoint parameters, headers, and underlying technology stack. It then plans highly specific test cases (e.g., SQLi, IDOR, CL.0 HTTP Request Smuggling desync paths, unvalidated postMessage
handlers).test_cases
table./run-executor <target>
test_cases
table and fires them through a headless mcp-proxy
) that bridges Burp's REST API into the Model Context Protocol (MCP).execution_results
. If a response shows an anomaly (status or length delta compared to the baseline), it flags it and creates an active Repeater tab in Burp Suite for manual traceability./run-validator <target>
execution_results
. It parses the raw HTTP response and compares it logically against the Generator's expected_behavior
. It actively attempts to reconstruct a working, step-by-step PoC request-response pair.verified_findings
table on disk and synced to our shared cross-target SQLite memory (appsec_memory.db
) to prioritize future test cases on similar tech stacks.Because all memory lives in SQLite, we have an immutable audit trail. We never delete data; we simply progress rows through a state lifecycle (unprocessed
$\to$ processed
).
Here is a look at how clean and simple the database model is:
| Table Name | Owner (Writes) | Primary Purpose |
|---|---|---|
endpoints |
||
| Mapper | Mapped attack surface (URLs, methods, parameters) | |
test_cases |
||
| Generator | Planned payloads and expectation criteria | |
execution_results |
||
| Executor | Raw HTTP responses and anomaly flags | |
verified_findings |
||
| Validator | Verified vulnerabilities with reproducible PoCs |
The transition from the old .claude
harness to Okwute forced us to completely rethink how we managed cross-target security memory.
In our early iterations, we attempted to maintain cross-target knowledge inside flat, prose Markdown files (products/<product-slug>.md
). But as multiple agents executed in parallel, this filesystem-centric model created a severe bottleneck: concurrent file writes caused clobbered updates, and our SHA-256 context hashing system would trigger frequent merge conflict halts, entirely defeating fully autonomous runs.
We solved this concurrency crisis by building a Statistical Cross-Target Memory Engine ( appsec_memory.db) managed by SQLite. Instead of prose notes, the engine computes real-time mathematical hit-rates for successful exploit vectors:
`Exploit Hit Rate = confirmed_count / tested_count`
If a specific JWT header spoofing vector yields an 85% success rate on a fintech target, that statistical weight is committed atomically to the centralized database, allowing day-one prioritization when the pipeline spins up on a new target sharing the same product_type
.
While relational tables solve the multi-agent race conditions, they are notoriously hostile to human operators who need to quickly review, organize, and enrich security findings.
To bridge this gap, we designed a hybrid interface that ports our relational memory directly into a hyper-linked Obsidian Vault using a dedicated Python command-line utility (memory_db.py
).
python3 .opencode/scripts/memory_db.py sync-vault /workspace/obsidian-security-vault/
This synchronization engine dynamically translates binary SQL tables into a beautifully structured, human-readable Obsidian workspace:
Obsidian_Vault/
├── Index.md # Auto-generated, linked directory map of all product types
└── Products/
├── fintech_api.md # Auto-generated vector hit-rates and metrics (Overwritten on sync)
└── fintech_api - Notes.md # Hand-authored qualitative research and overrides (Preserved forever)
The system automatically splits product-family memory into a dual-file architecture:
<product_type>.md
):<product_type> - Notes.md
):This design gives us the best of both worlds: strict relational concurrency for our parallel agents, and a beautifully visualized Markdown canvas for the human product owner.
While the core SQLite database acts as our decentralized state engine, how the pipeline is actively triggered and managed can be adapted across three distinct operational control planes, depending on the environment:
For hands-off, continuous scanning, the entire pipeline is packaged inside a self-hosted Forgejo Actions (or GitHub Actions) CI/CD runner. This headless pipeline runs overnight on a dedicated label-node, managing process state for Burp and the AI harness dynamically.
For rapid, interactive local testing, security engineers can drive Okwute directly from terminal-native agent environments like Claude Code or OpenCode.
/run-generator
or /run-validator
) straight from their local terminal to triage new API endpoints or debug specific payload modifications in real time, combining autonomous execution with on-demand interactive control.To abstract CLI friction entirely, Okwute's pipeline CLI outputs can be fed directly into a custom-engineered Web UI Control Plane.
AI is not going to replace the human-in-the-loop security engineer, but the security engineers who automate their workflows using deterministic graph engineering will completely outscale those who do not.
By moving away from open-ended chat inputs and building structured, database-backed state machines like Okwute, we can let AI do what it does best, reason, classify, and generate patterns, while forcing the underlying infrastructure to remain secure, disciplined, and deterministic.
Stop chatting with your models. Start building compilers for them.
What are your thoughts on agentic security pipelines? Have you explored wrapping Burp Suite or headless proxies into autonomous decision loops? Let’s discuss in the comments!