cd /news/ai-safety/compute-the-comparison-without-inven… · home › topics › ai-safety › article
[ARTICLE · art-164388] src=dev.to ↗ pub= topic=ai-safety verified=true sentiment=· neutral

Compute the comparison without inventing the study

The RigorMed team published a self-written synthetic teaching example showing how a small standard-library Python worksheet can compute named differences between two group summaries without asserting a causal effect. In the invented table, both groups improved by 18 points, so the difference in changes is zero, while sample size, uncertainty and causal effect are explicitly marked as not supplied. The team notes it did not run a model evaluation or apply a formal risk-of-bias instrument, and frames the worksheet as a way to organize source-linked questions rather than certify a study.

by read3 min views1 publishedNov 5, 2026

A claim-check worksheet can start with arithmetic and stop before making an unsupported research conclusion. That boundary matters when a table is handed to an AI: a fluent explanation can silently turn a difference into an effect.

We are the RigorMed team. This example is self-written synthetic teaching material. It contains two group summaries, no participants and no observed research results. We executed only the small standard-library Python calculation below. We did not install or execute the full third-party scientific-critical-thinking Skill, run a model evaluation, or start a production research task.

Imagine two voluntarily formed groups using and not using a learning tool:

Group Baseline score Final score
Tool use 60 78
No tool use 52 70

The final scores differ by 8. But the baseline scores also differ by 8. If the question concerns changes, comparing only the final values hides a relevant part of the table.

Here is the entire calculation:

from dataclasses import dataclass

@dataclass(frozen=True)
class GroupSummary:
    baseline: int
    final: int

    @property
    def change(self) -> int:
        return self.final - self.baseline

use = GroupSummary(baseline=60, final=78)
nonuse = GroupSummary(baseline=52, final=70)

worksheet = {
    "use_change": use.change,
    "nonuse_change": nonuse.change,
    "baseline_difference": use.baseline - nonuse.baseline,
    "final_difference": use.final - nonuse.final,
    "difference_in_changes": use.change - nonuse.change,
    "sample_size": "not supplied",
    "uncertainty": "not supplied",
    "causal_effect": "not identified from these summaries",
}
print(worksheet)

Actual local output:

{'use_change': 18, 'nonuse_change': 18, 'baseline_difference': 8, 'final_difference': 8, 'difference_in_changes': 0, 'sample_size': 'not supplied', 'uncertainty': 'not supplied', 'causal_effect': 'not identified from these summaries'}

The last three fields are deliberately not numeric. Missing sample size is not zero participants. An absent uncertainty estimate is not zero uncertainty. The program computes the named differences; it has no evidence with which to decide why they occurred.

The zero is the arithmetic difference between two changes in this invented table. It does not by itself establish equivalence, rule out an effect, or validate a causal model. We have not supplied a design or assumptions that identify a causal effect, and voluntary group membership can leave important differences unaddressed.

Calling this number a difference in changes is a description. Treating it as a causal estimate would require a separate argument about the design, measurement, selection and relevant assumptions. Picking a sophisticated method name cannot supply that argument retrospectively.

For a lawful source paper, a useful AI request is:

Identify the comparison that each conclusion uses. Separate observations, interpretations and causal claims. For each, preserve the source passage or table location and list the design information needed to support the interpretation. Do not invent sample sizes, uncertainty estimates, allocation details or analytical assumptions. Mark what remains unidentified.

Review the sheet against the original paper. An abstract may omit details found in methods or supplements, so a missing field in an abstract should remain a reading gap until checked. A claim-check sheet organises questions; it does not certify a study or replace the researcher's appraisal.

The task-oriented structure was informed by K-Dense's public scientific-critical-thinking guidance, version 1.5 as read on November 5, 2026, including its design, confounding and observation-versus-inference notes. The numbers, code and worksheet above are our own teaching example. We have not applied a formal risk-of-bias instrument or produced a clinical decision.

Researchers who want to organise lawful papers into source-linked reading notes can describe that task in the RigorMed research workspace. Model work uses the displayed credits. Checking the original sources and deciding what the design supports remain the researcher's work.

── more in #ai-safety 4 stories · sorted by recency
── more on @rigormed 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/compute-the-comparis…] indexed:0 read:3min 2026-11-05 · —