Most multi-model comparisons never tell you what they cost to run. Ours did, because in MultiO every message carries a receipt — so the study's own bill is part of its method, and we published it.
The programme: the same 24 questions, unchanged, sent to five mid-tier models — GPT, Claude, Gemini, Grok and Llama — then a Council read per question to flag where the answers materially split, plus a rerun control (two questions, three fresh samples each through one model). The total metered cost was $0.87 — 866,245 micro-USD on the study account's own receipts.
The breakdown is the interesting part:
Two honesty details sit inside those numbers. Ten of the panel calls (the first two questions in the completed run) were exact-cache replays of the lost first attempt's answers — same models, same questions, generated minutes earlier — and were charged 0; the Council judging on them ran fresh. And the judging itself needed a metered extraction follow-up in 17 of 24 runs when the first pass missed the structured format. Neither fact changes the findings — 52 material divergences, models materially agreeing on only 8 of 24 questions — but both are in the published method because the receipt is the point: if a tool meters usage, the study run on it should be auditable down to the micro.
That is also the practical takeaway for anyone cross-checking models on a budget: the answers are not the expensive part. Asking five models a question is cheap; the cost is in the structured read that tells you where they split — and that read is exactly the part a single-model workflow skips entirely, at $0 and at the price of never seeing the disagreement.
Full study, method, cost breakdown and all caveats (including that one judge was also a panel member, the rotating judging sets, and a memory-bleed incident we found and disclosed): https://multio.ai/research/model-divergence
Disclosure: I build MultiO (multio.ai) — One subscription. Every top AI model… Auto sends each message to the most cost-efficient model that can handle it. Compare asks up to three models side by side, and every message shows its exact token charge on a receipt. Free trial: 3,000,000 tokens, no card required.