{"slug": "intelligence-per-watt-measuring-intelligence-efficiency-of-local-ai", "title": "Intelligence per Watt: Measuring Intelligence Efficiency of Local AI", "summary": "A study by researchers including Jon Saad-Falcon, submitted to arXiv, proposes intelligence per watt (IPW) as a unified metric for local AI efficiency, finding that local language models with up to 20B parameters answer 88.7% of 1M real-world queries accurately, IPW improved 5.3x from 2023 to 2025, and local accelerators achieve at least 1.4x lower IPW than cloud accelerators running identical models, suggesting local inference can redistribute demand from centralized infrastructure.", "body_md": "# Computer Science > Distributed, Parallel, and Cluster Computing\n\n[Submitted on 11 Nov 2025 (\n\n[v1](https://arxiv.org/abs/2511.07885v1)), last revised 7 Aug 2026 (this version, v5)]# Title:Intelligence per Watt: Measuring Intelligence Efficiency of Local AI\n\n[View PDF](/pdf/2511.07885)\n\n[HTML (experimental)](https://arxiv.org/html/2511.07885v5)\n\nAbstract:Large language model (LLM) queries are predominantly processed by frontier models in centralized cloud infrastructure. Demand growth strains this paradigm faster than providers can scale. Two advances create an opportunity to rethink it: small, local LMs (<=20B active parameters) now achieve competitive performance to frontier models on many tasks, and local accelerators (e.g., Apple M4 Max) can host these models at interactive latencies. This raises the question: can local inference viably redistribute demand from centralized infrastructure? This requires measuring both whether local LMs can accurately answer real-world queries and whether they can do so efficiently on power-constrained devices (e.g., laptops). We propose intelligence per watt (IPW), task accuracy per unit of power, as a unified metric for the capability and efficiency of local inference across model-accelerator configurations. We evaluate 20+ state-of-the-art local LMs, 8 hardware accelerators (local and cloud), and 1M real-world single-turn chat and reasoning queries. For each query, we measure accuracy (local LM win rate against frontier models), energy, latency, and power. We find three key results. First, local LMs successfully answer 88.7% of these queries, with accuracy varying by domain. Second, longitudinal analysis from 2023-2025 shows IPW improved 5.3x, driven by both algorithmic and accelerator advances, with locally-serviceable query coverage rising from 23.2% to 71.3%. Third, local accelerators achieve at least 1.4x lower IPW than cloud accelerators running identical models, revealing significant headroom for local accelerator optimization. These findings demonstrate that local inference can meaningfully redistribute demand from centralized infrastructure for a substantial subset of queries, with IPW serving as the critical metric for tracking this transition.\n\n## Submission history\n\nFrom: Jon Saad-Falcon [[view email](/show-email/c69de5f7/2511.07885)]\n\n**Tue, 11 Nov 2025 06:33:30 UTC (5,373 KB)**\n\n[[v1]](/abs/2511.07885v1)**Fri, 14 Nov 2025 00:53:12 UTC (5,538 KB)**\n\n[[v2]](/abs/2511.07885v2)**Thu, 26 Feb 2026 17:09:14 UTC (5,538 KB)**\n\n[[v3]](/abs/2511.07885v3)**Thu, 21 May 2026 03:40:21 UTC (5,134 KB)**\n\n[[v4]](/abs/2511.07885v4)**[v5]** Fri, 7 Aug 2026 02:40:27 UTC (5,621 KB)\n\n### Current browse context:\n\ncs.DC\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/intelligence-per-watt-measuring-intelligence-efficiency-of-local-ai", "canonical_source": "https://arxiv.org/abs/2511.07885", "published_at": "2026-08-22 05:54:49+00:00", "updated_at": "2026-08-22 06:13:15.210341+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-infrastructure"], "entities": ["arXiv", "Jon Saad-Falcon", "Apple M4 Max"], "alternates": {"html": "https://wpnews.pro/news/intelligence-per-watt-measuring-intelligence-efficiency-of-local-ai", "markdown": "https://wpnews.pro/news/intelligence-per-watt-measuring-intelligence-efficiency-of-local-ai.md", "text": "https://wpnews.pro/news/intelligence-per-watt-measuring-intelligence-efficiency-of-local-ai.txt", "jsonld": "https://wpnews.pro/news/intelligence-per-watt-measuring-intelligence-efficiency-of-local-ai.jsonld"}}