cd /news/large-language-models/thinking-machines-releases-inkling-2… · home › topics › large-language-models › article
[ARTICLE · art-170640] src=tokenstead.ai ↗ pub= topic=large-language-models verified=true sentiment=· neutral

Thinking Machines releases Inkling 2 Small weights

Thinking Machines Lab published the Inkling 2 Small weights on Hugging Face on 13 November at 21:12 UTC, uploading 24 safetensors shards totaling 394GB under Apache 2.0. The model card lists 594B total parameters with 17B active per token, routing each token to 8 of 384 experts across 48 decoder layers, with 1M tokens of context and a native NVFP4 precision plus an MXFP8 checkpoint; the NVFP4 build requires at least 500GB of aggregated VRAM, illustrated as two NVIDIA B300 GPUs or four H200s. SGLang, vLLM, TokenSpeed and Unsloth published inference recipes the same day, and the download counter had reached 3,647 with 292 likes by the following morning, though no independent benchmark reproduction has been published.

read3 min views6 publishedNov 16, 2026
Thinking Machines releases Inkling 2 Small weights
Image: Tokenstead (auto-discovered)

On 13 November at 21:12 UTC the repository for Inkling 2 Small changed status from promised to published. Thinking Machines Lab uploaded 24 safetensors shards to Hugging Face, 394GB in total, under Apache 2.0. The model card lists 594B total parameters with 17B active per token, and the download counter had reached 3,647 with 292 likes by the following morning.

Three consecutive weekly sweeps had listed this model as announced but not available. The difference now is not the benchmark table, it is that the files exist.

What it does differently #

Each token is routed to 8 of 384 experts in a sparse mixture-of-experts backbone across 48 decoder layers. That routing is why a 594B parameter model needs 17B worth of computation per token: the weights are the memory problem, not the arithmetic.

Inputs are text and images, projected into a shared space and processed by the same decoder. The card gives 1M tokens of context, which puts it in the class where a repository fits in one request.

The precision decision is the part worth noticing. The model was pretrained, midtrained and post-trained in NVFP4, and Thinking Machines calls that the native precision rather than a compression step applied afterwards. They publish a second checkpoint in MXFP8 for people who want it, and describe its benchmark gain over NVFP4 as one to two points that they do not consider meaningful.

The hardware bill #

The card is direct about this. The NVFP4 checkpoint needs at least 500GB of aggregated VRAM, which they illustrate as two NVIDIA B300 GPUs or four H200s. The MXFP8 checkpoint needs 800GB, illustrated as four B300s or eight H200s.

A machine with 512GB of unified memory is the only single box on our hardware list that comes close, and 500GB of card memory inside a 512GB machine leaves very little for the operating system and the KV cache. This is a datacenter model with an open licence, not a desktop one.

Recipes shipped the same day #

Four inference stacks published instructions on release: SGLang, vLLM, TokenSpeed and Unsloth. The vLLM recipe quotes about 1,100 output tokens per second per GPU at 100 tokens per second per user, measured on a TP8 configuration with DSpark, which is a vendor benchmark rather than a number we reproduced.

The Unsloth guide is the more useful document for anyone with less than a cluster, because it is written around llama.cpp and GGUF rather than tensor parallelism. It repeats the 594B and 17B figures and points at quantised builds.

API access is available through NVIDIA, Fireworks, Together, Baseten, Nebius, Modal and Cloudflare for people who want the model without the memory.

The numbers on the card #

Vendor-measured, with the best score in each row belonging to a different model in most cases:

Vals Index v2.2 : 56.1% against 61.3% for the best model listed #

Artificial Analysis Index v5.0 : 30.6 #

Humanity’s Last Exam, text only : 44.1% against 49.4% best #

AA-LCR v1.1 : 83.3% against 88.7% best #

DeepSWE v1.1 : 72.1% #

FrontierCode v2.0 : 25.4% #

Terminal-Bench Science v0.1 : 5.7% That last figure is the one to sit with. On terminal-based scientific work the model scores 5.7% where the best model in the table scores 4.8%, so both are close to the floor. A 594B open weight model is not automatically a strong agent.

What is not established #

No independent reproduction of any benchmark has been published. The throughput figure is the vendor’s own. The download counts move hourly and will be stale by the time you read this. Nothing here speaks to whether the mixture-of-experts routing holds up after fine-tuning, which is the claim most buyers of an Apache 2.0 model actually care about.

What is established is the part that kept changing: the weights are downloadable today, in two precisions, with four inference recipes, at a licence that permits commercial use.

Sources: [Hugging Face: Inkling 2 Small](https://huggingface.co/thinkingmachines/Inkling-2-Small) - [vLLM recipe](https://recipes.vllm.ai/thinkingmachines/Inkling-2-Small) - [Unsloth guide](https://unsloth.ai/docs/models/inkling-2) - [SGLang recipe](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling-2-Small)

Related on this site: [Open weights after Huang](https://tokenstead.ai/guides/open-weights-after-huang)
── more in #large-language-models 4 stories · sorted by recency
── more on @thinking machines lab 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/thinking-machines-re…] indexed:0 read:3min 2026-11-16 · —