Keith, Etidorhpa, with drafting, editorial, and verification assistance from Claude (Anthropic)
Abstract
Structural fairness checks — keyword- or regex-based sieves that score prompts or content deterministically and offline, without invoking another model — are common in AI-governance pipelines operating under strict deployment constraints. Such checks typically combine several weighted sub-rules into a single dimension score, but standard rubric formats do not surface how much of a dimension's outcome rests on any single sub-rule. A dimension dominated by one brittle rule silently inherits that rule's every false positive and false negative at nearly full strength. This paper reports an audit of a NIST AI RMF-aligned prompt-governance rubric that identified (i) a documentation/implementation drift in which the rubric specification documents four rules for the fair_bias_managed
trustworthiness dimension while only two are implemented, and (ii) a reproducible false positive in the implemented dimension's regex-based rule, which is negation-blind and scores an explicitly anti-discriminatory prompt identically to an actually discriminatory one. To quantify this class of risk, we repurpose Jain's fairness index — originally a resource-allocation-evenness metric for shared computer systems [1] — as a meta-metric over a rubric's own internal rule weights rather than over any resource allocated to end users. Applied to the audited rubric, the cross-dimension (composite) weight distribution is close to perfectly even ($J \approx 0.956$ at $n=7$), but the fair_bias_managed
dimension's internal rule weights are highly concentrated ($J \approx 0.775$ at $n=2$), with one rule carrying $\approx 77%$ of the dimension's decision weight — effectively closer to $100%$ given the second rule's near-total inactivity in practice. The metric measures weight distribution, not rule correctness, is computable from a rubric's configuration alone, and requires no runtime cost or labeled data. No modification was made to the audited compiler or scorer; this is a descriptive record of the findings and the metric they motivated.
Index Terms — Jain's fairness index, rule-weight concentration, structural bias sieve, NIST AI RMF, prompt governance, negation blindness, doc/code drift, trustworthiness scoring.
Structural fairness checks are common in AI-governance pipelines that must evaluate prompts or content deterministically, offline, and without calling another model — a requirement in regulated or air-gapped deployment settings. Such checks typically combine several weighted sub-rules into a single dimension score (for example, "no demographic-targeting language" combined with "contains bias-mitigation vocabulary"). The standard rubric format used for this purpose does not surface how much of a dimension's outcome rests on any single sub-rule, so a dimension dominated by one brittle rule inherits that rule's every false positive and false negative at nearly full strength, invisibly, because the dimension's weight distribution is never itself measured or reported.
This paper reports three findings from an audit of one such system — a NIST AI RMF-aligned prompt-governance rubric implemented as a deterministic compiler and scorer — and introduces a metric motivated by those findings:
- a documentation/implementation drift in the rubric's
fair_bias_managed
trustworthiness dimension, in which the specification documents four scoring rules but only two are implemented; - a reproducible false positive in the implemented dimension's regex-based rule, which is negation-blind and scores an explicitly anti-discriminatory prompt identically to an actually discriminatory one; and
- a reusable concentration metric — a repurposing of Jain's fairness index [1] — that quantifies how much of a weighted scoring dimension's outcome rests on a single sub-rule, independent of that rule's correctness.
No modification was made to the audited compiler or scorer. This paper is a descriptive record of the findings and the metric they motivated, retained for reference and for anyone deciding whether to act on the findings.
The audited system is a deterministic prompt compiler and scorer (the governance
CLI, src/Governance.Cli
) that compiles a raw request into a nine-section governed prompt and scores it against seven NIST AI RMF trustworthiness characteristics (src/Governance/Scoring/PromptScorer.cs
, reference/rubric.yaml
). A preview mode (compile --preview
) exposes the deterministic input-data selection pane prior to a full compile, confirming that the compiler executes as a genuine dry-run/real-run pair rather than a simulation.
A representative compilation — compile
on "write a Python function that dedupes a list"
, with context "the list may contain unhashable dict entries"
— produced a nine-section prompt containing eight unresolved <TBD>
placeholders and a composite score of NIST 0.968
. Manually resolving every <TBD>
and re-scoring with score --file
reduced the placeholder count from eight to one (the remaining <TBD>
is a literal reference to the sentinel within the injected execution contract's own explanatory text, not an unfilled section), but left both the composite score and the fair_bias_managed
dimension score unchanged at 0.77
.
Tracing this invariance to the rubric definition for fair_bias_managed
in reference/rubric.yaml
revealed the documentation/implementation drift: the YAML specifies four rules for this dimension, while the C# implementation (PromptScorer.cs:155-160
) wires up only two. Recomputing the dimension score from the two implemented rules ($1.0 / 1.3 \approx 0.769 \approx 0.77$) reproduced the observed value exactly, confirming which rules are active at runtime.
The one substantive active rule, neutral_framing
, is implemented as a single regular expression: based on (race|gender|religion|ethnicity|nationality) | only (men|women|whites|blacks) | exclude (men|women)
. Direct testing established that this rule has no negation handling: a prompt requesting a hiring rubric that explicitly avoids excluding women scored fair_bias_managed: 0.00
and was flagged NEEDS HUMAN REVIEW
, identical to the score an actually discriminatory prompt would receive.
This finding motivates a broader question of how "fairness" is defined and measured, both within the fairness literature — statistical parity, equalized odds, calibration, individual fairness, counterfactual fairness, and the representational-versus-allocational-harm distinction in NLP — and as an engineering property in other domains, including CPU scheduling, network queueing, multi-resource cluster scheduling, concurrency/liveness, and mechanism design. Most of the latter domains admit closed-form, directly computable fairness metrics, because the resource under allocation is already numeric. This observation motivates the metric developed in Section IV: applying a closed-form engineering fairness metric — Jain's fairness index — not to a resource the compiler allocates to end users, but to the rubric's own internal rule weights.
Determinism, and independence from an auxiliary model call, is a stated design constraint of the audited system (README.md
: "No APIs, no tunnel, no DB"), and is typical of structural fairness sieves generally. The rubric's own authors anticipated that such a sieve could not be trusted outright: PromptScorer.cs
's class-level doc comment states a caveat carried from the source system — "low scores are more trustworthy than high scores — a low score means a named rule fired; a high score means no rule caught anything." That caveat is why a failing fair_bias_managed
score routes to NEEDS HUMAN REVIEW
rather than automatic rejection.
The concentration metric developed here does not contradict that design; it gives a number for how much review-triggering power sits behind a single, unverified rule.
Given a scoring dimension composed of $n$ sub-rules with weights $w_1, \ldots, w_n$, Jain's fairness index is:
$$J(\mathbf{w}) = \frac{\left(\sum_{i=1}^{n} w_i\right)^{2}}{n \sum_{i=1}^{n} w_i^{2}}$$
$J$ ranges from $1/n$ (all weight concentrated on one rule) to $1$ (weight distributed perfectly evenly across all rules). A low $J$ is a structural early-warning signal: it flags that a "fairness score" is really a proxy for one narrow check, independent of whether that check happens to be correct on any given input.
This paper applies the index in two independent contexts within the same rubric: across the seven top-level trustworthiness dimensions (the composite score), and across the sub-rules internal to a single dimension, fair_bias_managed
. The metric's assumptions are identical in both cases: weights are non-negative and known at configuration time, and $J$ is computed directly from those configured weights, independent of any input data or labeled test set.
Jain's fairness index was originally developed to quantify evenness of resource allocation in shared computer systems [1]. This paper repurposes it as a meta-metric over a fairness sieve's own internal rule weights, rather than over any resource the sieve allocates to end users. Computing $J$ over a dimension's configured sub-rule weights makes concentration risk visible before any individual rule is tested against real inputs, at design time, with no dependency on labeled data.
Applied to the audited system's fair_bias_managed
dimension — two implemented sub-rules, a demographic-targeting regex at weight 1.0 and a bias-mitigation-vocabulary keyword match at weight 0.3 — $J = 1.3^{2} / (2 \times 1.09) \approx 0.775$: the demographic-targeting regex alone carries $1.0/1.3 \approx 77%$ of the dimension's decision weight. Independently, direct testing established that this regex is negation-blind: it fires identically on "exclude women from consideration"
and "does not exclude women from consideration"
, flipping the dimension score from 1.00 to 0.00 on an explicitly anti-discriminatory prompt. The concentration index predicted the blast radius of that failure mode before it was found empirically: $J \approx 0.775$ (out of a maximum of $1.0$ at $n=2$) meant essentially the whole dimension's reliability rode on one rule's correctness.
Two independent applications of the formula follow, computed from the actual configured weights in the audited system.
Source: docs/nist-governance-spec.md
, verified against the printed NIST 0.968
-style composite output of governance compile
.
| Dimension | Weight |
|---|---|
| valid_reliable | 0.18 |
| accountable_transparent | 0.18 |
| explainable_interpretable | 0.15 |
| privacy_enhanced | 0.15 |
| fair_bias_managed | 0.14 |
| safe | 0.10 |
| secure_resilient | 0.10 |
def jain(x):
n = len(x)
s = sum(x)
ss = sum(v * v for v in x)
return (s * s) / (n * ss)
composite_weights = {
"valid_reliable": 0.18,
"accountable_transparent": 0.18,
"explainable_interpretable": 0.15,
"privacy_enhanced": 0.15,
"fair_bias_managed": 0.14,
"safe": 0.10,
"secure_resilient": 0.10,
}
x = list(composite_weights.values())
jain(x) # 0.9562057754828839
1 / len(x) # 0.14285714285714285 — the ideal (perfectly even) 1/n share
composite_weights["fair_bias_managed"] # 0.14 — close to the ideal share
$J \approx 0.956$ against a perfect-evenness ceiling of $1.0$ at $n=7$. The composite weight for fair_bias_managed
(0.14) sits almost exactly at the ideal $1/7 \approx 0.1429$ share. At this level, the rubric is not structurally biased against fairness in how importance is allocated across the seven trustworthiness characteristics.
Source: the rules actually implemented in src/Governance/Scoring/PromptScorer.cs:155-160
(not the four documented in reference/rubric.yaml
, per the documentation/implementation drift noted in Section II):
["fair_bias_managed"] =
[
new("neutral_framing", 1.0, c => !DemographicTargeting().IsMatch(c.Prompt)),
new("bias_mitigation_language_present", 0.3, c =>
BiasMitigationLanguage().IsMatch(c.Prompt)),
],
rule_weights = [1.0, 0.3] # neutral_framing, bias_mitigation_language_present
jain(rule_weights) # 0.7752293577981652
1.0 / sum(rule_weights) # 0.7692307692307692 — share carried by neutral_framing alone
$J \approx 0.775$ against a perfect-evenness ceiling of $1.0$ at $n=2$. neutral_framing
alone carries $\approx 77%$ of the dimension's decision weight; bias_mitigation_language_present
carries the remaining $\approx 23%$ and, per the rubric's own comment, "essentially never fires in operational prompts" — so the effective concentration in practice is closer to $100%$ on one rule.
Jain's index at the composite level indicates the rubric does not structurally favor one trustworthiness characteristic's weight over another. Jain's index at the dimension-internal level indicates the opposite for fair_bias_managed
specifically: its score is a near-monopoly of one regex, which is precisely why a single false positive can swing the entire dimension from $1.00$ to $0.00$ in one step. A dimension with more evenly distributed internal rule weight — for example, robustness
, with five rules at weights 1.0 / 1.5 / 1.0 / 1.0 / 0.3 — dilutes any single rule's brittleness the same way diversifying a portfolio dilutes single-asset risk. fair_bias_managed
is architecturally the least diversified dimension in the rubric — a second, independent explanation, alongside the negation-blindness itself, for why it is the noisiest one.
This metric measures the distribution of weight across sub-rules, not the correctness of any individual rule. A dimension can have perfectly even weight ($J=1$) across several equally bad rules. It is a concentration-risk metric, not a substitute for adversarial testing or for establishing the underlying accuracy of each sub-rule.
Any AI-governance or content-moderation system that scores trustworthiness dimensions as a weighted sum of sub-checks can compute this index per dimension at design time, with no runtime cost and no dependency on labeled data — it is a structural property of the configured weights, computable from the rubric definition alone. A low $J$ does not prove the dominant rule is wrong; it identifies where to invest adversarial testing effort (negation, paraphrase, code-switching, sarcasm) before deployment, and it gives auditors a single number to track as a rubric evolves. Adding a third, differently weighted rule to a $J=0.775$ dimension is a concrete, measurable way to reduce single-rule dependency, distinct from simply adding more rules without knowing whether the addition meaningfully redistributes decision weight.
Analysis, drafting, editorial revision, and calculation verification for this paper were performed in collaboration with Claude (Anthropic).
[1] R. Jain, D. Chiu, and W. Hawe, "A Quantitative Measure of Fairness and Discrimination for Resource Allocation in Shared Computer Systems," DEC Research Report TR-301, 1984.
[2] National Institute of Standards and Technology, "Artificial Intelligence Risk Management Framework (AI RMF 1.0)," NIST AI 100-1, Section 3 (Characteristics of Trustworthy AI).
[3] reference/rubric.yaml
— the audited system's rubric configuration (documents four rules for fair_bias_managed
; two are implemented).
[4] src/Governance/Scoring/PromptScorer.cs
— the deterministic scorer (implements the two live rules for fair_bias_managed
, lines 155-160).
[5] docs/nist-governance-spec.md
— the cross-dimension composite weight table.
dotnet run --project src/Governance.Cli -- compile "write a Python function that dedupes a list"
dotnet run --project src/Governance.Cli -- score --file <filled-in-prompt.txt>
dotnet run --project src/Governance.Cli -- compile "write a hiring rubric that does not exclude women from consideration"