{"slug": "scaling-laws-meet-model-architecture-toward-inference-efficient-llms", "title": "Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs", "summary": "Researchers have developed a conditional scaling law that predicts optimal large language model architectures for inference efficiency, training over 200 models from 80 million to 3 billion parameters. The study found that adjusting hidden size, MLP-to-attention ratio, and grouped-query attention can yield up to 2.1% higher accuracy and 42% greater inference throughput compared to LLaMA-3.2 under the same training budget. The findings address the underexplored trade-off between model accuracy and inference cost as LLMs grow increasingly powerful and widely deployed.", "body_md": "# Computer Science > Machine Learning\n\n[Submitted on 21 Oct 2025 (\n\n[v1](https://arxiv.org/abs/2510.18245v1)), last revised 13 May 2026 (this version, v3)]# Title:Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs\n\n[View PDF](/pdf/2510.18245)\n\n[HTML (experimental)](https://arxiv.org/html/2510.18245v3)\n\nAbstract:Scaling the number of parameters and the size of training data has proven to be an effective strategy for improving large language model (LLM) performance. Yet, as these models grow increasingly powerful and widely deployed, the cost of inference has become a pressing concern. Despite its importance, the trade-off between model accuracy and inference efficiency remains underexplored. In this work, we examine how key architectural factors, hidden size, the allocation of parameters between MLP and attention (mlp-to-attention ratio), and grouped-query attention (GQA), influence both inference cost and accuracy. We introduce a conditional scaling law that augments the Chinchilla framework with architectural information, along with a search framework for identifying architectures that are simultaneously inference-efficient and accurate. To validate our approach, we train more than 200 models spanning 80M to 3B parameters and 8B to 100B training tokens, and fit the proposed conditional scaling law. Our results show that the conditional scaling law reliably predicts optimal architectural choices and that the resulting models outperform existing open-source baselines. Under the same training budget, optimized architectures achieve up to 2.1% higher accuracy and 42% greater inference throughput compared to LLaMA-3.2.\n\n## Submission history\n\nFrom: Song Bian [[view email](/show-email/00d752f1/2510.18245)]\n\n**Tue, 21 Oct 2025 03:08:48 UTC (236 KB)**\n\n[[v1]](/abs/2510.18245v1)**Sun, 1 Mar 2026 01:23:07 UTC (633 KB)**\n\n[[v2]](/abs/2510.18245v2)**[v3]** Wed, 13 May 2026 04:16:31 UTC (633 KB)\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/scaling-laws-meet-model-architecture-toward-inference-efficient-llms", "canonical_source": "https://arxiv.org/abs/2510.18245", "published_at": "2026-05-31 03:41:57+00:00", "updated_at": "2026-05-31 04:15:27.071266+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "neural-networks", "ai-research", "ai-infrastructure"], "entities": ["Chinchilla"], "alternates": {"html": "https://wpnews.pro/news/scaling-laws-meet-model-architecture-toward-inference-efficient-llms", "markdown": "https://wpnews.pro/news/scaling-laws-meet-model-architecture-toward-inference-efficient-llms.md", "text": "https://wpnews.pro/news/scaling-laws-meet-model-architecture-toward-inference-efficient-llms.txt", "jsonld": "https://wpnews.pro/news/scaling-laws-meet-model-architecture-toward-inference-efficient-llms.jsonld"}}