✓ Human-authored analysis; AI used for formatting and proofreading.
AWS Compliance Mod is the canonical implementation of one
reasoning paradigm per-resource SQL benchmarks against
framework requirements. Stave is the substrate. Five
reasoning engines: SMT, Datalog, answer-set programming,
Prolog, probabilistic model checking produce correct
answers from Stave's exports without reading a line of
Stave's source code.
Previous article Stave and the AWS Compliance Mod: why we ship both, makes the per-feature complementarity argument: compound chain detection by Stave covers the compositional risk class that per-resource framework checks (AWS Compliance Mod) can't see by construction. That framing is still correct. Run both; they cover orthogonal territory.
This article makes the bigger claim. The complementarity isn't just "we have compound detection; they have framework coverage." It's "we ship a contract that arbitrary reasoning engines can target; they ship one specific reasoning paradigm." That's a category split.
The proof is a trial that already happened, with results committed to the repo, and a fresh-sub-agent re-run that survives the "but maybe the spec author leaked the answer" objection.
Every cloud security tool can be located on the reasoning-paradigm × content-coverage grid:
| Reasoning paradigm | Content shape |
|---|---|
| Per-resource attribute checks | "Is this bucket encrypted? Is this role MFA-enforced?" |
| Compound / chain composition | "Do these three settings combine into a Capital One path?" |
| Transitive closure (Datalog) | "What is the complete blast radius from this principal?" |
| SMT satisfiability | "Does there exist any assignment that violates this invariant? Show me the witness or prove none exists." |
| Probabilistic model checking | "Given the observed protection factors, what's the probability of successful exploitation?" |
| Proof-tree derivation (Prolog) | "Trace the proof chain — root, edges, leaves — for this conclusion." |
| Game-theoretic adversary modeling | "What's the optimal attacker move given the defender's posture?" |
turbot/steampipe-mod-aws-compliance occupies one row in that grid with per-resource attribute checks and covers it exhaustively: 540+ controls across 16+ frameworks. Mature, auditor-recognized, the right tool for "am I CIS-compliant right now?"
Stave occupies the row dimension as a substrate: any reasoning paradigm with an engine-agnostic input format can target the same Stave exports. The catalog (over 3,000 controls today) is written in YAML + CEL; the exports project into SMT-LIB, JSONL, RDF / JSON-LD, and the engine-native formats those paradigms consume.
The category split: AWS Compliance Mod is row-scoped. Stave is dimension-scoped.
reasoning-specs/TRIAL-RESULTS.md in the Stave repo records the first-round trial. Five reasoning specs and engines.
Each spec carries:
The trial: hand the spec + input fixture + export-schema documentation to an agent. Withhold the original implementation code and the golden answer. The agent must produce output that matches the golden via the spec's validation rules.
Round 1 (same-session) results:
| # | Engine | Outcome |
|---|---|---|
| 1 | Z3 (SMT-LIB) | PASS — exact match on verdict + witness |
| 2 | Soufflé (Datalog) | PASS —anonymous_reachable_count: 12 byte-identical |
| 3 | Clingo (ASP) | FAIL → fixed → PASS (spec bug:mfa_enforced should have beenhas_mfa_enforced ) |
| 4 | Prolog | FAIL → fixed → PASS (golden transcription bug: spec said 6 proofs, actual file has 12) |
| 5 | PRISM (probabilistic) | PASS — step probabilities exact, aggregate within ±0.005 |
Three of five passed straight. Two failures caught real defects and the trial harness distinguished the two failure types. Clingo's failure was the spec author's error (wrong predicate name). Prolog's failure was the inventory author's error (mis-counted the golden). Same trial format, two distinct bug categories, both surfaced precisely.
Trial format did its job. The spec is wrong when the agent can't derive the engine's output from the spec; the golden is wrong when the agent's output matches the engine's actual output but not the golden.
The Round 1 trial had one disclosure: the spec author and the trial agent were the same session. Even with the answer withheld from the spec, contextual leak was possible ("the author would unconsciously phrase the spec to produce the right answer they already know").
Round 2 was a fully-blind re-run. Two of the five trials (Prolog + Clingo) had spec changes after Round 1. The Round 2 agent was a fresh sub-agent with no access to:
spec.yaml (which carried the answer in a comment)golden.yaml (the answer key)examples/<engine>/ directory
The fresh agent received only spec-trial.yaml (answer stripped), input.jsonl, and export-schema.md.
Both blind trials passed:
Prolog (blind, medium confidence): the agent correctly derived 12 proofs via the cartesian-product semantic. The
post-fix golden count without seeing the explanation
comment in the spec file.
Clingo (blind, high confidence): the agent correctly used
has_mfa_enforced with the has_ prefix is the naming-
convention rule the post-fix spec added. The fresh agent
recovered the convention from the spec alone.
The blind agents flagged two non-defects (Prolog's spec doesn't pin output ordering or indentation width). Both are explicitly in the spec's ignore: list confirming the ignore-list entries are foundational and prevent false-FAIL on correct answers.
Full results: reasoning-specs/BLIND-TRIAL-RESULTS.md in the repo.
Three claims that the trial validates:
Reasoning specs are sufficient for agent execution.
Blind agents produced correct answers from spec + input +
export-schema alone. No source-code access required.
The spec YAML format works. All five specs parsed.
Agents followed reasoning steps to completion on
every trial.
Stave's export contracts are complete (for the validated engines). No missing-input-field warnings from any trial
Since the original trial, four more engines have closed the gap with fixture-tied golden outputs: PySAT (compound-chain satisfiability via sat-control-regression/), TLA+ (temporal-safety state-space exploration with drift margins via tlaplus-temporal-safety/), game-theoretic cost modeling (attacker-optimal paths + defender ROI ranking via game-theory-cost/), and HIPAA / CIS / NIST evidence packs (single-document executive summaries with control matrices via compliance-evidence/). That brings the validated engine count to nine.
Three export-format engines remain without a committed fixture-to-golden round-trip: STIX 2.1, JSON-LD / GraphML, and OSCAL / OCSF compliance mapping. All three have working Go exporters shipping in the binary (stave graph export --format stix|jsonld|graphml, stave export oscal, stave export ocsf). The gap is a committed expected-output file for one specific fixture, which is prerequisite work before spec authoring. Engine status is tracked in reasoning-specs/INVENTORY.md.
Standard cloud security tools require Go / Python / TypeScript fluency to contribute. The contribution surface is "write code that calls the tool's API."
Stave's contribution surface is two artifacts:
1. A YAML control (~30 lines) defining a security property in CEL. Anyone who knows YAML + CEL can contribute one. The existing 3,000+ controls are reference templates.
2. A reasoning spec (~50 lines) defining a security question + the reasoning chain an engine should follow + the golden answer for one fixture. Anyone who can articulate the question + the chain can contribute one.
In both cases, the contribution ships exactly the same as if it came from the maintainer. There's no plugin to learn, no internal API to bind against, no build-system gymnastics.
This is a much larger contributor pool than "people who can read Go and the host tool's codebase." That's the substrate benefit operationalized: the artifact format is the contribution boundary, and the format is small enough for security engineers without backend-systems backgrounds to author against.
Three operator scenarios with the right tool for each:
"I need framework benchmark dashboards for an auditor."
Install turbot/steampipe-mod-aws-compliance. It renders CIS / PCI-DSS / HIPAA / NIST / SOC 2 / FedRAMP / GxP / AWS FSBP benchmarks with framework-section IDs the auditor recognizes. This is the row Stave doesn't compete on.
"I need to catch the Capital One pattern before it ships."
Install Stave. The compound chain engine names the multi-resource composition that per-resource scanners can't see by construction. The full per-feature comparison is at aws-compliance-mod.
"I need a mathematical proof that no unauthorized principal can reach this PHI bucket." Install Stave + Z3. Export the relevant SIR facts with stave export-sir --format smt2, write the SMT-LIB assertion that encodes "any principal without the DataOwner tag has effective_access to PHI bucket via any chain of role assumptions," check satisfiability. unsat is a proof; sat with a witness is the specific path to fix.
"I need the blast radius of compromising this role."
Install Stave + Soufflé. Export with --format jsonl, run the bundled reachability program, get the complete transitive closure of (principal, action, resource) tuples. Per-resource scanners answer "is this specific path open?"; Soufflé answers "how wide is the open territory?"
"I need the probability the path is exploitable given the protection factors." Install Stave + PRISM. Export the state-transition facts, append the PRISM model, run model checking. Per-resource scanners answer binary present / absent; PRISM answers a probability conditioned on the actual configuration's protection mix.
Each engine answers a structurally different question. The substrate makes that possible.
The original stave-and-aws-compliance-mod article argued the per-feature complementarity: AWS Compliance Mod handles per-resource framework coverage; Stave handles compound risk and ghost references. That framing is sufficient for operators choosing tools today.
This article extends the framing for architects categorizing where each tool lives. AWS Compliance Mod is one implementation in one cell of the reasoning-paradigm grid. Stave is the substrate the cells point at. Both framings are correct; pick the one that fits your reader's question.
The full comparison-doc canonical reference is at
aws-compliance-mod.
The full reasoning-contract explanation is at
reasoning-contract.
The trial-results documentation is at
TRIAL-RESULTS
and
BLIND-TRIAL-RESULTS.
Stave is open-source. The reasoning-spec format + trial harness ship in reasoning-specs/. Nine engines validated with fixture-tied golden outputs; three export-format engines remain as extension targets in reasoning-specs/INVENTORY.md. Try the working trials: cd stave && bash reasoning-specs/trials/<trial-name>/run.sh.