Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs

Researchers have developed a conditional scaling law that predicts optimal large language model architectures for inference efficiency, training over 200 models from 80 million to 3 billion parameters. The study found that adjusting hidden size, MLP-to-attention ratio, and grouped-query attention can yield up to 2.1% higher accuracy and 42% greater inference throughput compared to LLaMA-3.2 under the same training budget. The findings address the underexplored trade-off between model accuracy and inference cost as LLMs grow increasingly powerful and widely deployed.

Computer Science Machine Learning Submitted on 21 Oct 2025 v1 https://arxiv.org/abs/2510.18245v1 , last revised 13 May 2026 this version, v3 Title:Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs View PDF /pdf/2510.18245 HTML experimental https://arxiv.org/html/2510.18245v3 Abstract:Scaling the number of parameters and the size of training data has proven to be an effective strategy for improving large language model LLM performance. Yet, as these models grow increasingly powerful and widely deployed, the cost of inference has become a pressing concern. Despite its importance, the trade-off between model accuracy and inference efficiency remains underexplored. In this work, we examine how key architectural factors, hidden size, the allocation of parameters between MLP and attention mlp-to-attention ratio , and grouped-query attention GQA , influence both inference cost and accuracy. We introduce a conditional scaling law that augments the Chinchilla framework with architectural information, along with a search framework for identifying architectures that are simultaneously inference-efficient and accurate. To validate our approach, we train more than 200 models spanning 80M to 3B parameters and 8B to 100B training tokens, and fit the proposed conditional scaling law. Our results show that the conditional scaling law reliably predicts optimal architectural choices and that the resulting models outperform existing open-source baselines. Under the same training budget, optimized architectures achieve up to 2.1% higher accuracy and 42% greater inference throughput compared to LLaMA-3.2. Submission history From: Song Bian view email /show-email/00d752f1/2510.18245 Tue, 21 Oct 2025 03:08:48 UTC 236 KB v1 /abs/2510.18245v1 Sun, 1 Mar 2026 01:23:07 UTC 633 KB v2 /abs/2510.18245v2 v3 Wed, 13 May 2026 04:16:31 UTC 633 KB References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender IArxiv Recommender What is IArxiv? https://iarxiv.org/about arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .