Correction, July 13: the headline finding did not survive its own pre-registered confirmation study. Study AA inverted it, and the foreword explains what happened. The original post follows unchanged. Study Z, the twenty-sixth in our pre-registered benchmark series, finally measured the thing almost every AI document editor does without evidence: shipping a standing context block with every request. Company background, client records, a twelve-rule styleguide. Does the model actually use it? The answer is a clean yes: 216 of 216 on facts and rules across three models, zero cross-client contamination in 324 cells, and no burial effect anywhere in the styleguide. The hazard we found instead lives in the styleguide itself. When a standing rule collides with what the user asked for, models don't break either one. They pick a reading. And the strongest model picks the most literal reading most often.
Your RAG Eval Is Checking the Receipt, Not the Patient