Z.ai CEO Jie Tang announced that GLM 5.3's performance gains come from reinforcement learning on long-horizon environments, not parameter count, and introduced a new post-training scaling law with fiv…
Scaling machine learning models is more complex than simply increasing size, requiring careful engineering to avoid pitfalls, according to a technical essay by Beren Millidge. The author explains that…
A developer's analysis of scaling laws for large language models shows that the Chinchilla rule of about 20 training tokens per parameter is often treated as a fixed target, but recent scrutiny sugges…
Giles Thomas, a developer, tested the Chinchilla scaling rule by comparing overtrained GPT-2 style models (trained on 40 tokens per parameter) against a model scaled up in parameters and tokens equall…
Giles Thomas, in a blog post, reports that deliberately overtraining GPT-2-style models did not improve their performance on instruction fine-tuning evaluations, contrary to his hypothesis that OpenAI…
Google researchers have found that scaling laws similar to those for large language models apply to wearable sensor data, with validation loss scaling predictably with compute, model size, and data si…
AI teams often hit a 'latent capability ceiling' where larger models no longer improve task accuracy, with scaling data showing diminishing returns beyond 100B parameters and 61% of tasks exhibiting n…
A developer built a novel triple-hybrid LLM combining Mamba, Attention, and a 32-expert Mixture of Experts architecture from scratch for approximately $50, completing Titan v1 and the first training c…
Microsoft's MAI-Thinking-1 report introduces an "efficiency gain" (EG) metric that compares candidate model architectures against a baseline by measuring how much compute is needed to reach the same l…
Researchers have developed a conditional scaling law that predicts optimal large language model architectures for inference efficiency, training over 200 models from 80 million to 3 billion parameters…
A new analysis argues that the era of AI scaling producing clearly better models is ending, with frontier labs facing exponentially increasing costs for linear returns. The author claims that the inte…
Amazon researchers have developed a new scaling law that connects large language model architecture directly to inference efficiency, enabling faster text generation without sacrificing accuracy. The …