# Inkling: A New Open-Weight 975B Moe with a Few Surprises > Source: > Published: 2026-07-16 14:03:05+00:00 # Inkling: A New Open-Weight 975B MoE with a Few Surprises Interesting surprise open-weight LLM drop from Thinking Machines Lab yesterday. Their nearly 1T-parameter [Inkling model](https://thinkingmachines.ai/news/introducing-inkling/) looks pretty solid on the reported benchmarks. Compared with GLM-5.2, Inkling performs better on evaluations such as IFBench (79.8% versus 73.3%) and SimpleQA Verified (43.9% versus 38.1%). It performs worse on several reasoning and coding-agent benchmarks, including HLE without tools (29.7% versus 40.1%), SWE-Bench Pro Public (54.3% versus 62.1%), and Terminal-Bench 2.1 (63.8% versus 82.7%). These are release-time numbers, and some rows combine externally reported values with internal harness results. I would therefore not over-interpret small differences. Overall, Inkling looks like a good all-rounder that is intended for further fine-tuning and specialization. That direction makes sense given that Thinking Machines Lab also develops Tinker, its model customization and fine-tuning platform. Architecture-wise, Inkling is a 975B-parameter sparse Mixture-of-Experts model with 41B active parameters and a [context window](/glossary/#context-length) of up to 1M tokens. Some side-by-side observations: - Inkling has about 231B more total parameters than GLM-5.2 (975B versus 744B), although their active footprints are almost identical at 41B and 40B parameters. - Inkling is less sparse than Kimi K2.5. It activates 4.2% of its parameters per token, compared with 3.2% for Kimi K2.5 (41B of 975B versus 32B of 1T). - Inkling uses a regular Transformer decoder rather than the hybrid Mamba-Transformer approach used by Nemotron 3 Ultra. I am curious about token throughput. The larger active footprint than Kimi K2.5 and the use of conventional GQA rather than [MLA](/glossary/#mla) or a recurrent hybrid stack suggest that raw decoding speed may not be Inkling’s main advantage. However, throughput also depends heavily on quantization, expert parallelism, attention kernels, batching, and hardware. I have not seen directly comparable provider measurements yet. The overall design follows the recent large-[MoE](/glossary/#moe) trend, but the architecture has a few interesting surprises: - **Small convolution layers in several places.** Each decoder layer applies short kernel-4 convolutions after the key and value projections and on the attention and MLP branch outputs. My intuition is that these provide cheap local token mixing and an explicit short-range inductive bias alongside attention. - **An additional RMSNorm directly after the token** This is separate from the pre-attention[embedding layer](/glossary/#token-embeddings).[RMSNorm](/glossary/#rmsnorm)inside every transformer block. At first glance, it looks almost redundant, but it is explicitly enabled in the[configuration](https://huggingface.co/thinkingmachines/Inkling/blob/main/config.json)and present in the[Transformers implementation](https://github.com/huggingface/transformers/blob/main/src/transformers/models/inkling/modeling_inkling.py). Whether it materially helps would require an ablation. - **A learned, input-dependent relative-position bias instead of**[RoPE](/glossary/#rope).[Thinking Machines says](https://thinkingmachines.ai/news/introducing-inkling/#architecture)that the relative-position approach “performs better and extrapolates better to longer sequences” than RoPE. Regarding the last point, my intuition is that the sliding-window-heavy architecture helps. Of the 66 decoder layers, 55 use local attention with a small 512-token window. A learned relative-position bias may provide enough [positional information](/glossary/#positional-encoding) within these windows. In the 11 global layers, the released implementation applies the learned bias only over the preceding 1,024 tokens. Attention beyond that range is effectively content-based with respect to this positional bias. This is somewhat similar to the intuition behind [NoPE](/glossary/#nope), which other architectures use in selected global-attention layers. Overall, Inkling is an interesting variation on the DeepSeek-V3-style MoE recipe and a solid release. Some may find it underwhelming because it does not lead every benchmark. I find the broad and mixed benchmark profile refreshingly honest. It may also indicate that the model is less benchmark-specialized than several recent releases. And, as always, it is good to see another strong open-weight [base model](/glossary/#base-model) available for fine-tuning and independent study. ## Read Next [200,000 Subscribers Short note celebrating Ahead of AI reaching 200,000 subscribers.](/blog/2026/ahead-of-ai-reached-200000-subscribers.html) [GPT 5.6 Has 72 Possible Configurations. What's A Good Default? Short note on how GPT 5.6 model and effort choices map onto training-time and inference-time scaling, producing 72 configurations.](/blog/2026/gpt-5-6-configurations.html) [Build a Reasoning Model From Scratch Is Out Short note announcing the release of Build a Reasoning Model From Scratch and linking the publisher and Amazon pages.](/blog/2026/build-a-reasoning-model-from-scratch-is-out.html)