{"slug": "mean-versus-median-reading-room-level-drift-in-stagingbench", "title": "Mean versus median: reading room level drift in StagingBench", "summary": "A developer analyzed the StagingBench open dataset of AI virtual staging tools, grouping aligned architecture drift scores by room to compare mean versus median. The analysis found that room-level means and medians diverge in both directions—Room 01 averaged 20.33 against a median of 15.75 due to high outliers, while Room 02 averaged 26.33 against a median of 27.10—leading to the conclusion that both summaries should be reported rather than a single number.", "body_md": "A room's average drift can differ noticeably from its middle score. That difference is worth examining before compressing a benchmark into a single number.\n\n[StagingBench](https://stagingbench.org) is an independent test of AI virtual staging tools on 10 real room photographs. Its CSV records a tool and room pair in each row. To compare the mean and median per room, we can group the eligible aligned architecture drift values by room identifier.\n\nThe mean includes every value through their sum. The median describes the middle of the sorted values, averaging the central pair when necessary. Neither is a substitute for inspecting the underlying distribution. Used together, they reveal whether a summary is sensitive to values away from the middle.\n\nPut the CSV from the open dataset in the current directory as `scores.csv`, then run this script. It uses the standard library and excludes unknown rows by skipping any row whose `alignment_ok` field is not `True`. It includes eligible rows regardless of framing status, because the question here concerns drift summaries.\n\n``` python\nimport csv\nfrom collections import defaultdict\nfrom statistics import mean, median\n\nscores = defaultdict(list)\nwith open(\"scores.csv\", newline=\"\") as source:\n    for row in csv.DictReader(source):\n        if row[\"alignment_ok\"] != \"True\":\n            continue\n        scores[row[\"room\"]].append(\n            float(row[\"aligned_architecture_drift\"])\n        )\nfor room, values in sorted(scores.items()):\n    print(f'{room}: n={len(values)}, '\n          f'mean={mean(values):.2f}, median={median(values):.2f}')\n```\n\nThe printed means and medians use two decimal places. The observation count remains visible alongside them, which matters because different rooms contribute different numbers of eligible scores. A room with no eligible values does not enter the grouped collection. Room 09 is such a case in this dataset and is not printed.\n\nRoom 01 has 12 eligible values. Its mean is 20.33 and its median is 15.75. The sorted values include 25.4, 27.1, 28.2 and 52.8 at the upper end. The last value is far above the central pair, 15.3 and 16.2.\n\nThat is a concrete reason to report both summaries. The mean describes the full collection, including the high value. The median tells a reader that the middle of the collection sits lower. Calling either number the room's complete story would discard useful information.\n\nRoom 05 also has 12 eligible values. Its mean is 20.17 and its median is 17.65. Its upper values include 34.9, 36.2 and 38.0. Again, the average lies above the middle, with the larger observations contributing to that separation.\n\nRoom 07 has 11 eligible values, a mean of 26.55 and a median of 24.40. Its values extend up to 46.3. The gap is another feature of this observed collection, not evidence that every tool responds to the room in the same way.\n\nThe direction can reverse. Room 02 has 11 eligible values, a mean of 26.33 and a median of 27.10. Its values include 11.6 at the low end. Room 03, also with 11 values, has a mean of 22.21 and a median of 22.90, with low observations of 7.5 and 9.2.\n\nThese examples show why \"the average exceeds the median\" should be an observation to check, not an assumption carried into every room.\n\nThe summaries can also nearly coincide. Room 06 has 10 eligible values, a mean of 21.12 and a median of 21.20. Their closeness says something about these two summaries. It does not establish that all individual measurements are close together.\n\nThis remains a small test of 10 rooms, with one run per tool and room cell. The room groups summarize observed outputs from different tools. They do not estimate how stable any individual tool would be across repeated generations. See the [methodology](https://stagingbench.org/methodology.html) for how the drift is measured.\n\nDifferent group sizes also deserve attention. Room 04 has only four eligible measurements. A compact summary of that group carries much less observational coverage than a group containing all 12 tools. Missing eligible measurements are not zero drift.\n\nThe practical reporting choice is straightforward: keep the mean, median and count together, and consult the individual values when the summaries diverge. That makes the arithmetic inspectable while keeping its claims proportional to the evidence.", "url": "https://wpnews.pro/news/mean-versus-median-reading-room-level-drift-in-stagingbench", "canonical_source": "https://dev.to/stagingbench/mean-versus-median-reading-room-level-drift-in-stagingbench-211h", "published_at": "2026-11-20 15:38:38+00:00", "updated_at": "2026-11-20 15:44:21.138532+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "computer-vision"], "entities": ["StagingBench"], "also_reported_by": [], "takeaway": "Room-level drift summaries diverge: Room 01's mean is 20.33 versus a median of 15.75, and Room 02's mean is 26.33 versus a median of 27.10, so report both mean and median with the observation count.", "recommendation": {"advice": "Report mean, median and sample count per group when summarizing benchmark scores, since outliers can push the mean above or below the median.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/mean-versus-median-reading-room-level-drift-in-stagingbench", "markdown": "https://wpnews.pro/news/mean-versus-median-reading-room-level-drift-in-stagingbench.md", "text": "https://wpnews.pro/news/mean-versus-median-reading-room-level-drift-in-stagingbench.txt", "jsonld": "https://wpnews.pro/news/mean-versus-median-reading-room-level-drift-in-stagingbench.jsonld"}}