# Wilson 95% lower bounds on rule survival across 29

> Source: <https://promptcube3.com/en/threads/8753/>
> Published: 2026-09-03 20:50:03+00:00

# Wilson 95% lower bounds on rule survival across 29

The setup: two independent extractors parsed Missouri statutes, and on numeric-threshold presence they hit a **0.43 false-negative rate**. Almost half the time one misses what the other catches. That number alone should make anyone running [RAG](/en/tags/rag/) over legal corpora pause.

What they actually built:

1. Take the Duquenne-Guigues implication basis of the extracted contexts (for anyone who's dealt with FCA, this is the minimal non-redundant rule set).

2. Measure per-attribute disagreement between extractors.

3. Replay that disagreement against the basis in **1,000 Monte Carlo trials**.

4. An implication only "survives" if a one-sided Wilson 95% lower bound on its survival rate hits **0.95**.

Each surviving rule ships with its premise spans and a minimal counterexample. That's the audit trail. That's the part I like — you can actually point at a rule and say "here's the exact statutory text, and here's the smallest thing that would break it."

The empirical results are messy in a useful way:

**29,365 Missouri sections + 502 Indian central-Act sections**- Preregistered held-out gate:
**10 statute families across 7 Titles exact**,** 16 across 11 with 5% tolerance** - Under one globally deployed error model,
**93.2% of held-out chapters fall below the informativeness floor** - 2x2 factorial pins that collapse on
**calibration-rate transfer**, not selection

Translation: the rules themselves are fine. The extraction error doesn't transfer cleanly across chapters, so a globally-tuned error model quietly kills your rules. If you calibrate per-chapter, or accept some error tolerance, the certificate survives. Otherwise you're shipping dead logic.

There's also a retracted claim in the released audit trail, which I'm pointing out because more authors should do this.

``` python
# The Wilson 95% one-sided lower bound (the core gate)
def wilson_lower(k, n, z=1.6448536269514722):
    p = k / n
    denom = 1 + z*z/n
    centre = p + z*z/(2*n)
    adj = z * ((p*(1-p) + z*z/(4*n))/n) ** 0.5
    return (centre - adj) / denom
```

Two takeaways I'm carrying forward into my own LLM-extraction work:

**Don't trust a single extractor's output without a survival check.** Even at 0.43 FNR, rules with premise+counterexample chains can still certify — but only if you measure per-attribute noise, not aggregate accuracy.**Global error models are a trap for legal/scientific text.** Domain drift between chapters is real, and a single threshold turns 93% of your signal into noise.

If you're doing prompt engineering for legal RAG, structured extraction, or building LLM agents over regulatory corpora, the per-chapter calibration point is the one to internalize. The Duquenne-Guigues machinery is overkill for most use cases, but the discipline — measure disagreement, bound survival, ship counterexamples — isn't.

Has anyone else been hitting this calibration-transfer wall with extraction pipelines? I'm curious whether the 93.2% collapse rate holds up outside US statutes.

[Next NVIDIA is pushing local inference speeds up by 1. →](/en/threads/8721/)