{"slug": "show-hn-a-tool-to-optimize-ai-model-cost-without-benchmark-score-trade-offs", "title": "Show HN: A tool to optimize AI model cost without benchmark score trade-offs", "summary": "Jeff Kline published an interactive tool, \"Could you get the same AI results for less?\" (Experimental Mathematics, version 1.0, November 2, 2026), that identifies cheaper models or weighted model blends matching a chosen model's benchmark scores without trading off performance. The tool applies data envelopment analysis, the method from Zhou et al. (2022), using API cost as the resource and each task's benchmark scores as performance, with a free disposal hull variant for the single-slide creation task that compares individual models only. Kline cautions that savings are benchmark estimates excluding latency, quotas and switching costs, and that splitting real work across blended models is untested.", "body_md": "[Experimental Mathematics](https://jeff-kline.github.io/)Jeff Kline · Interactive\n\n# Could you get the same AI results for less?\n\n## Compare scores\n\n| Display is rounded; calculations use full precision. |  |  | \n|---|---|---|\n| Benchmark | Your score | Cheaper match | \n|---|---|---|\n\n## Why does the setting change the answer?\n\nEach setting has its own scores to match, so each has its own lowest cost. Paying more may buy a higher score.\n\n| Same model and task; separate scores for each setting. |  |  | \n|---|---|---|\n| Setting | Scores to match | Lowest cost / 1,000 tasks | \n|---|---|---|\n\n## How to read this\n\n**Matching means scoring at least as well on every benchmark.**\n\n**A blend is a weighted average of models.** Its cost and every score use the same shares, which total 100%. Splitting real work this way is untested. The single-slide creation task compares single models only, because its preference ratings cannot be averaged.\n\n**When nothing cheaper matches, nothing at the same cost does better.** No measured model or blend at that cost scores higher on one benchmark without scoring lower on another. Your setting may still score too low for your work.\n\n**Each task uses its own benchmark, cost unit and date.** Frontend development uses automation and terminal-task benchmarks as a stand-in. [HLE](https://lastexam.ai/) tests advanced academic questions, not proof writing or research. A setting such as “High” may mean different amounts of computing at different providers.\n\n**Savings here are benchmark estimates.** Latency, quotas, switching and other deployment costs are excluded. Test alternatives on your own tasks.\n\n## How this was made\n\nThis page applies the approach of Zhou et al. (2022), which used data envelopment analysis (DEA) to compare language models’ resource use with their performance. Here, the resource is API cost and performance is each task’s benchmark scores. The single-slide task uses the free disposal hull (FDH) variant, which compares individual models only.\n\nI directed the work and edited the prose. I used this page’s own results to choose models for building it: GPT-6 Astra at first, then GPT-6.1 Sol at several reasoning settings. Claude Opus 5.5 also did substantial work.\n\nCite this page: Jeffery Kline. 2026. Could you get the same AI results for less? Experimental Mathematics, version 1.0, November 2, 2026. [https://jeff-kline.github.io/posts/same-ai-results-for-less/](https://jeff-kline.github.io/posts/same-ai-results-for-less/)\n\nMethod from: Zachary Zhou, Alisha Zachariah, Devin Conathan and Jeffery Kline. 2022. [Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis](https://aclanthology.org/2022.eval4nlp-1.2/). In *Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems*, pages 11–20. Association for Computational Linguistics. [doi:10.18653/v1/2022.eval4nlp-1.2](https://doi.org/10.18653/v1/2022.eval4nlp-1.2)\n\n## BibTeX\n\n```\n@misc{kline-2026-same-ai-results,\n        title = \"Could You Get the Same {AI} Results for Less?\",\n        author = \"Kline, Jeffery\",\n        year = \"2026\",\n        month = nov,\n        howpublished = \"Experimental Mathematics, interactive page, version 1.0\",\n        url = \"https://jeff-kline.github.io/posts/same-ai-results-for-less/\",\n        note = \"Calculations November 2, 2026, from benchmark data published September to October 2026\"\n}\n\n@inproceedings{zhou-etal-2022-assessing,\n        title = \"Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis\",\n        author = \"Zhou, Zachary  and\n            Zachariah, Alisha  and\n            Conathan, Devin  and\n            Kline, Jeffery\",\n        editor = \"Deutsch, Daniel  and\n            Udomcharoenchaikit, Can  and\n            Opitz, Juri  and\n            Gao, Yang  and\n            Fomicheva, Marina  and\n            Eger, Steffen\",\n        booktitle = \"Proceedings of the 3rd Workshop on Evaluation and Comparison of {NLP} Systems\",\n        month = nov,\n        year = \"2022\",\n        address = \"Online\",\n        publisher = \"Association for Computational Linguistics\",\n        url = \"https://aclanthology.org/2022.eval4nlp-1.2/\",\n        doi = \"10.18653/v1/2022.eval4nlp-1.2\",\n        pages = \"11--20\"\n}\n```\n\n", "url": "https://wpnews.pro/news/show-hn-a-tool-to-optimize-ai-model-cost-without-benchmark-score-trade-offs", "canonical_source": "https://jeff-kline.github.io/posts/same-ai-results-for-less/index.html", "published_at": "2026-11-03 12:38:49+00:00", "updated_at": "2026-11-03 12:54:36.502154+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-tools"], "entities": ["Jeff Kline", "Experimental Mathematics", "Zhou et al.", "GPT-6 Astra", "GPT-6.1 Sol", "Claude Opus 5.5"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-a-tool-to-optimize-ai-model-cost-without-benchmark-score-trade-offs", "markdown": "https://wpnews.pro/news/show-hn-a-tool-to-optimize-ai-model-cost-without-benchmark-score-trade-offs.md", "text": "https://wpnews.pro/news/show-hn-a-tool-to-optimize-ai-model-cost-without-benchmark-score-trade-offs.txt", "jsonld": "https://wpnews.pro/news/show-hn-a-tool-to-optimize-ai-model-cost-without-benchmark-score-trade-offs.jsonld"}}