Show HN: A tool to optimize AI model cost without benchmark score trade-offs

Experimental MathematicsJeff Kline · Interactive

Compare scores #

Why does the setting change the answer? #

Each setting has its own scores to match, so each has its own lowest cost. Paying more may buy a higher score.

How to read this #

Matching means scoring at least as well on every benchmark.

A blend is a weighted average of models. Its cost and every score use the same shares, which total 100%. Splitting real work this way is untested. The single-slide creation task compares single models only, because its preference ratings cannot be averaged.

When nothing cheaper matches, nothing at the same cost does better. No measured model or blend at that cost scores higher on one benchmark without scoring lower on another. Your setting may still score too low for your work.

Each task uses its own benchmark, cost unit and date. Frontend development uses automation and terminal-task benchmarks as a stand-in. HLE tests advanced academic questions, not proof writing or research. A setting such as “High” may mean different amounts of computing at different providers.

Savings here are benchmark estimates. Latency, quotas, switching and other deployment costs are excluded. Test alternatives on your own tasks.

How this was made #

This page applies the approach of Zhou et al. (2022), which used data envelopment analysis (DEA) to compare language models’ resource use with their performance. Here, the resource is API cost and performance is each task’s benchmark scores. The single-slide task uses the free disposal hull (FDH) variant, which compares individual models only.

I directed the work and edited the prose. I used this page’s own results to choose models for building it: GPT-6 Astra at first, then GPT-6.1 Sol at several reasoning settings. Claude Opus 5.5 also did substantial work.

Cite this page: Jeffery Kline. 2026. Could you get the same AI results for less? Experimental Mathematics, version 1.0, November 2, 2026. https://jeff-kline.github.io/posts/same-ai-results-for-less/

Method from: Zachary Zhou, Alisha Zachariah, Devin Conathan and Jeffery Kline. 2022. Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis. In Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems, pages 11–20. Association for Computational Linguistics. doi:10.18653/v1/2022.eval4nlp-1.2

BibTeX #

@misc{kline-2026-same-ai-results,
        title = "Could You Get the Same {AI} Results for Less?",
        author = "Kline, Jeffery",
        year = "2026",
        month = nov,
        howpublished = "Experimental Mathematics, interactive page, version 1.0",
        url = "https://jeff-kline.github.io/posts/same-ai-results-for-less/",
        note = "Calculations November 2, 2026, from benchmark data published September to October 2026"
}

@inproceedings{zhou-etal-2022-assessing,
        title = "Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis",
        author = "Zhou, Zachary  and
            Zachariah, Alisha  and
            Conathan, Devin  and
            Kline, Jeffery",
        editor = "Deutsch, Daniel  and
            Udomcharoenchaikit, Can  and
            Opitz, Juri  and
            Gao, Yang  and
            Fomicheva, Marina  and
            Eger, Steffen",
        booktitle = "Proceedings of the 3rd Workshop on Evaluation and Comparison of {NLP} Systems",
        month = nov,
        year = "2022",
        address = "Online",
        publisher = "Association for Computational Linguistics",
        url = "https://aclanthology.org/2022.eval4nlp-1.2/",
        doi = "10.18653/v1/2022.eval4nlp-1.2",
        pages = "11--20"
}