Thinking Machines' Inkling 2 Small Trains a 594B Model in 68 Hours

Thinking Machines Lab open-sourced Inkling 2 Small, a 594-billion-parameter mixture-of-experts model with 17 billion active parameters per token, under Apache 2.0, pretrained end-to-end in 4-bit NVFP4 on NVIDIA Blackwell GB300 NVL72 racks over 8.4 trillion tokens in 68 hours. The model scores 31 on the Artificial Analysis Intelligence Index, up from 13 for the original Inkling Small, supports a 1M-token context and image input, runs on 2x B300 or 4x H200 at a 500GB VRAM minimum, and is priced at $0.30 per million input tokens and $1.00 per million output tokens on Tinker. Thinking Machines' release notes present the run as evidence that low-precision FP4 training can support a large sparse model while cutting memory traffic and raising tensor-core throughput, though actual cost savings depend on cluster size, utilization and communication overhead.

Thinking Machines' Inkling 2 Small Trains a 594B Model in 68 Hours Thinking Machines open-sourced a 594B parameter Mixture-of-Experts model pretrained end-to-end in 4-bit NVFP4 on Blackwell GB300 racks in 68 hours. - Thinking Machines open-sourced Inkling 2 Small https://huggingface.co/thinkingmachines/Inkling-2-Small?utm source=alphasignal , a 594B MoE with 17B active params, under Apache 2.0. - Pretrained end-to-end natively in NVFP4 on GB300 NVL72 racks in just 68 hours over 8.4T tokens. - Scores 31 on Artificial Analysis Intelligence Index, up from 13 on Inkling Small. - Leads open-weights peers on PerceptionBench, ScreenSpot Pro, ARC-AGI-2, BrowseComp; trails on coding. - Runs on 2x B300 or 4x H200 at 500GB VRAM minimum; supports 1M context and image input. - Priced at $0.30 in / $1.00 out per million tokens on Tinker https://tinker.thinkingmachines.ai/playground?utm source=alphasignal ; works in vLLM and SGLang. Thinking Machines Lab has released the model weights https://huggingface.co/thinkingmachines/Inkling-2-Small?utm source=alphasignal for Inkling 2 Small under Apache 2.0. The 594-billion-parameter mixture-of-experts model was pretrained with native NVFP4 arithmetic on NVIDIA Blackwell hardware, processing 8.4 trillion tokens in 68 hours. It activates 17 billion parameters per token and scores 31 on the Artificial Analysis Intelligence Index, up from 13 for the original Inkling Small. The technical significance comes from using four-bit floating-point operations during pretraining instead of converting a higher-precision checkpoint after training. The release notes https://thinkingmachines.ai/news/inkling-2-small/?utm source=alphasignal provide evidence that low-precision training can support a large sparse model while reducing memory traffic and increasing tensor-core throughput on compatible hardware. Actual cost savings depend on cluster size, utilization, communication overhead and the higher-precision states retained by the training recipe. 594B weights, 17B at work Inkling 2 Small combines sparse expert routing, hybrid attention, image input and an advertised one-million-token context window. Its DSpark speculator accelerates generation by drafting tokens for the main model to verify. | Inkling 2 Small specifications | | |---|---| | Component | Specification | |---|---| | Architecture | 48-layer, decoder-only transformer with sparse mixture-of-experts feed-forward blocks | | Parameters | 594B total, 17B active per token | | Routing | Each token uses 8 of 384 experts | | Attention | Hybrid local and global attention | | Modalities | Text and images in, text out; a vision transformer projects images into the shared hidden space | | Context | Up to 1M tokens for inference; reinforcement learning used a 256K training context with compaction for longer tasks | | Checkpoints | NVFP4 and MXFP8, with NVFP4 recommended by the lab | | License | Apache 2.0 | A mixture-of-experts model loads its full checkpoint across the available devices while routing each token through a small subset of experts. Inkling therefore has the storage and communication demands of a 594B checkpoint, even though its per-token expert computation is closer to a 17B model. FP4 enters pretraining Many low-bit releases begin as BF16 checkpoints and are quantized after training. Thinking Machines instead used NVIDIA’s NVFP4 format during pretraining, including the model’s principal forward and backward matrix operations. NVFP4 represents values with four-bit floating-point numbers and shared scales for small groups of values. Those scales preserve more useful range than plain four-bit integers. Compared with FP8, four-bit operands can halve storage for quantized tensors and double peak tensor-core throughput for supported Blackwell operations. This story is for Pro members You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.