# Thinking Machines releases Inkling 2 Small weights

> Source: <https://tokenstead.ai/guides/inkling-2-small-open-weights>
> Published: 2026-11-16 14:52:11+00:00

On 13 November at 21:12 UTC the repository for **Inkling 2 Small** changed status from promised to published. Thinking Machines Lab uploaded 24 safetensors shards to [Hugging Face](https://huggingface.co/thinkingmachines/Inkling-2-Small), 394GB in total, under Apache 2.0. The model card lists **594B total parameters with 17B active** per token, and the download counter had reached 3,647 with 292 likes by the following morning.

Three consecutive weekly sweeps had listed this model as announced but not available. The difference now is not the benchmark table, it is that the files exist.

## What it does differently

Each token is routed to **8 of 384 experts** in a sparse mixture-of-experts backbone across 48 decoder layers. That routing is why a 594B parameter model needs 17B worth of computation per token: the weights are the memory problem, not the arithmetic.

Inputs are text and images, projected into a shared space and processed by the same decoder. The card gives 1M tokens of context, which puts it in the class where a repository fits in one request.

The precision decision is the part worth noticing. The model was pretrained, midtrained and post-trained in **NVFP4**, and Thinking Machines calls that the native precision rather than a compression step applied afterwards. They publish a second checkpoint in MXFP8 for people who want it, and describe its benchmark gain over NVFP4 as one to two points that they do not consider meaningful.

## The hardware bill

The card is direct about this. The NVFP4 checkpoint needs **at least 500GB of aggregated VRAM**, which they illustrate as two NVIDIA B300 GPUs or four H200s. The MXFP8 checkpoint needs 800GB, illustrated as four B300s or eight H200s.

A machine with 512GB of unified memory is the only single box on our hardware list that comes close, and 500GB of card memory inside a 512GB machine leaves very little for the operating system and the KV cache. This is a datacenter model with an open licence, not a desktop one.

## Recipes shipped the same day

Four inference stacks published instructions on release: SGLang, vLLM, TokenSpeed and Unsloth. The vLLM recipe quotes **about 1,100 output tokens per second per GPU at 100 tokens per second per user**, measured on a TP8 configuration with DSpark, which is a vendor benchmark rather than a number we reproduced.

The Unsloth guide is the more useful document for anyone with less than a cluster, because it is written around llama.cpp and GGUF rather than tensor parallelism. It repeats the 594B and 17B figures and points at quantised builds.

API access is available through NVIDIA, Fireworks, Together, Baseten, Nebius, Modal and Cloudflare for people who want the model without the memory.

## The numbers on the card

Vendor-measured, with the best score in each row belonging to a different model in most cases:

- 
**Vals Index v2.2** : 56.1% against 61.3% for the best model listed
- 
**Artificial Analysis Index v5.0** : 30.6
- 
**Humanity’s Last Exam, text only** : 44.1% against 49.4% best
- 
**AA-LCR v1.1** : 83.3% against 88.7% best
- 
**DeepSWE v1.1** : 72.1%
- 
**FrontierCode v2.0** : 25.4%
- 
**Terminal-Bench Science v0.1** : 5.7%

That last figure is the one to sit with. On terminal-based scientific work the model scores 5.7% where the best model in the table scores 4.8%, so both are close to the floor. A 594B open weight model is not automatically a strong agent.

## What is not established

No independent reproduction of any benchmark has been published. The throughput figure is the vendor’s own. The download counts move hourly and will be stale by the time you read this. Nothing here speaks to whether the mixture-of-experts routing holds up after fine-tuning, which is the claim most buyers of an Apache 2.0 model actually care about.

What is established is the part that kept changing: the weights are downloadable today, in two precisions, with four inference recipes, at a licence that permits commercial use.

Sources: [Hugging Face: Inkling 2 Small](https://huggingface.co/thinkingmachines/Inkling-2-Small) - [vLLM recipe](https://recipes.vllm.ai/thinkingmachines/Inkling-2-Small) - [Unsloth guide](https://unsloth.ai/docs/models/inkling-2) - [SGLang recipe](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling-2-Small)

Related on this site: [Open weights after Huang](https://tokenstead.ai/guides/open-weights-after-huang)
