Inkling: A New Open-Weight 975B Moe with a Few Surprises

Thinking Machines Lab released Inkling, an open-weight 975B-parameter sparse Mixture-of-Experts model with 41B active parameters and a 1M-token context window. The model outperforms GLM-5.2 on IFBench and SimpleQA Verified but trails on reasoning and coding-agent benchmarks, and features unusual architectural choices including small convolution layers, an additional RMSNorm after the embedding layer, and a learned input-dependent relative-position bias instead of RoPE.

Inkling: A New Open-Weight 975B MoE with a Few Surprises Interesting surprise open-weight LLM drop from Thinking Machines Lab yesterday. Their nearly 1T-parameter Inkling model https://thinkingmachines.ai/news/introducing-inkling/ looks pretty solid on the reported benchmarks. Compared with GLM-5.2, Inkling performs better on evaluations such as IFBench 79.8% versus 73.3% and SimpleQA Verified 43.9% versus 38.1% . It performs worse on several reasoning and coding-agent benchmarks, including HLE without tools 29.7% versus 40.1% , SWE-Bench Pro Public 54.3% versus 62.1% , and Terminal-Bench 2.1 63.8% versus 82.7% . These are release-time numbers, and some rows combine externally reported values with internal harness results. I would therefore not over-interpret small differences. Overall, Inkling looks like a good all-rounder that is intended for further fine-tuning and specialization. That direction makes sense given that Thinking Machines Lab also develops Tinker, its model customization and fine-tuning platform. Architecture-wise, Inkling is a 975B-parameter sparse Mixture-of-Experts model with 41B active parameters and a context window /glossary/ context-length of up to 1M tokens. Some side-by-side observations: - Inkling has about 231B more total parameters than GLM-5.2 975B versus 744B , although their active footprints are almost identical at 41B and 40B parameters. - Inkling is less sparse than Kimi K2.5. It activates 4.2% of its parameters per token, compared with 3.2% for Kimi K2.5 41B of 975B versus 32B of 1T . - Inkling uses a regular Transformer decoder rather than the hybrid Mamba-Transformer approach used by Nemotron 3 Ultra. I am curious about token throughput. The larger active footprint than Kimi K2.5 and the use of conventional GQA rather than MLA /glossary/ mla or a recurrent hybrid stack suggest that raw decoding speed may not be Inkling’s main advantage. However, throughput also depends heavily on quantization, expert parallelism, attention kernels, batching, and hardware. I have not seen directly comparable provider measurements yet. The overall design follows the recent large- MoE /glossary/ moe trend, but the architecture has a few interesting surprises: - Small convolution layers in several places. Each decoder layer applies short kernel-4 convolutions after the key and value projections and on the attention and MLP branch outputs. My intuition is that these provide cheap local token mixing and an explicit short-range inductive bias alongside attention. - An additional RMSNorm directly after the token This is separate from the pre-attention embedding layer /glossary/ token-embeddings . RMSNorm /glossary/ rmsnorm inside every transformer block. At first glance, it looks almost redundant, but it is explicitly enabled in the configuration https://huggingface.co/thinkingmachines/Inkling/blob/main/config.json and present in the Transformers implementation https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling inkling.py . Whether it materially helps would require an ablation. - A learned, input-dependent relative-position bias instead of RoPE /glossary/ rope . Thinking Machines says https://thinkingmachines.ai/news/introducing-inkling/ architecture that the relative-position approach “performs better and extrapolates better to longer sequences” than RoPE. Regarding the last point, my intuition is that the sliding-window-heavy architecture helps. Of the 66 decoder layers, 55 use local attention with a small 512-token window. A learned relative-position bias may provide enough positional information /glossary/ positional-encoding within these windows. In the 11 global layers, the released implementation applies the learned bias only over the preceding 1,024 tokens. Attention beyond that range is effectively content-based with respect to this positional bias. This is somewhat similar to the intuition behind NoPE /glossary/ nope , which other architectures use in selected global-attention layers. Overall, Inkling is an interesting variation on the DeepSeek-V3-style MoE recipe and a solid release. Some may find it underwhelming because it does not lead every benchmark. I find the broad and mixed benchmark profile refreshingly honest. It may also indicate that the model is less benchmark-specialized than several recent releases. And, as always, it is good to see another strong open-weight base model /glossary/ base-model available for fine-tuning and independent study. Read Next 200,000 Subscribers Short note celebrating Ahead of AI reaching 200,000 subscribers. /blog/2026/ahead-of-ai-reached-200000-subscribers.html GPT 5.6 Has 72 Possible Configurations. What's A Good Default? Short note on how GPT 5.6 model and effort choices map onto training-time and inference-time scaling, producing 72 configurations. /blog/2026/gpt-5-6-configurations.html Build a Reasoning Model From Scratch Is Out Short note announcing the release of Build a Reasoning Model From Scratch and linking the publisher and Amazon pages. /blog/2026/build-a-reasoning-model-from-scratch-is-out.html