Meet Needle 2: An Open 45M-Parameter Tool-Calling Model That Ships as a 14MB Binary and Runs a Full Session in 28MB of RAM

Cactus Compute released Needle 2, an open 45M-parameter tool-calling model that ships as a 14MB binary and runs a full session in about 28MB of RAM, targeting devices with no GPU or NPU. The model achieves 500 tokens/sec on a Raspberry Pi 5 and 400–1,500 tokens/sec on Meta Quest 3S and Apple Vision Pro, using a Simple Attention Network with 27 layers and 512 width, pretrained on a proprietary 115B-token corpus. Cactus says Pebble already runs Needle locally in the Index 01 app for offline voice actions.

Cactus Compute https://cactuscompute.com/ has released Needle 2 https://github.com/cactus-compute/needle , an open 45M-parameter model for tool calling, device use, and structured extraction. The entire model ships as a single 14MB binary that runs a full session in about 28MB of RAM. Weights are trained and deployed at CQ2-bit using Cactus Quants, and the model is sealed inside the company’s own C++ engine, so there is no runtime to install and no download at inference time. Reported decode throughput is 500 tokens/sec on a Raspberry Pi 5, 400–1,500 tokens/sec on Meta Quest 3S and Apple Vision Pro, and 300–700 tokens/sec on sub-$200 phones. The design premise is narrow and stated plainly by the team: mapping a messy sentence onto a typed function signature needs no world knowledge and no open-ended prose. That framing is why 45M parameters are enough here, and why the model targets hardware with no GPU and no NPU. Is it deployable? Yes, Needle 2 https://cactuscompute.com/needle ships as prebuilt binaries and a static library for macOS, Linux x86-64, ARM64, ARMv7, RISC-V, MIPS32el , Windows, Android, iOS/watchOS/tvOS, and WebAssembly. Cactus says Pebble already runs Needle locally in the Index 01 app for offline voice actions. Which companies: Any team shipping firmware or apps on constrained hardware. Seed-stage wearable and IoT startups, mid-market consumer-electronics OEMs, robotics teams, and large device makers needing an offline fallback. Cloud-first SaaS teams gain less. Industries: smart home, wearables, low-end mobile, automotive in-cabin control, service robotics, retail kiosks and POS, routers and IP cameras, and regulated settings where audio cannot leave the device. Applications: voice-to-action on screenless devices, offline appliance control, receipt and invoice field extraction, enum tagging, and local routing that escalates to the cloud only on low confidence. Architecture: Simple Attention Network Needle 2 uses what Cactus team calls a Simple Attention Network. The recipe replaces the FFN with a Hadamard MLP, keeps GQA attention, adds engram key-value memory from hashed n-gram tables, and uses multi-lane hyper-connections. The network is 27 layers and 512 wide. The underlying study is on arXiv as A Controlled Study of Attention-Only Transformers https://arxiv.org/abs/2607.18363 . Pretraining used a proprietary 115B-token corpus, with 38B tokens of post-training. The research team notes LFM2.5-230M was pretrained on 19 trillion tokens. Needle 2 spends 70 MFLOPs per token, with 35M of 45M parameters matmul-active. LFM2.5 230M spends 460, FunctionGemma 270M spends 540, and Apple FM sits near 6,000. Engine, grammar, retrieval, and confidence Weights never decompress into RAM. The 2-bit codes expand inside vector registers and fuse into integer dot products, so the arithmetic path stays int8. One binary probes the CPU at startup and selects a kernel tier: SDOT, NEON, AVX2, RISC-V vectors, wasm SIMD, or scalar. A byte-level grammar compiled from your JSON schemas constrains every emitted token. Because the matcher knows which tokens are legal before logits exist, the engine skips up to 98% of the vocabulary projection on structural tokens. Attention uses a 256-token sliding window, and the system turn plus tool declarations are pinned as KV sinks. Memory stays near 28MB regardless of conversation length. Declare five or fewer tools and they render directly. Above five, a contrastive retrieval head embeds each schema once, scores the query per turn, and admits only the top five. Unselected tools are unreachable, not merely unlikely. Every response carries a confidence value, the minimum of a calibrated post-hoc head and the decoding probability of the call tokens. Off-topic requests return the empty call . The contract is a threshold: act above it, re-ask or escalate below it. Evaluation Cactus team evaluates on five public function-calling benchmarks using ordered strict exact match, where names, call order, and every argument must match. Needle 2 runs end-to-end through the shipped engine at CQ2-bit with retrieval on; baselines run f16 under vLLM. | Benchmark | Needle 2 CQ2 | LFM2.5 230M | FunctionGemma 270M | Apple FM | |---|---|---|---|---| | Mobile Actions 961 | 63.7 | 69.1 | 64.0 | 57.6 | | DroidCall 200 | 17.0 | 11.0 | 17.5 | — | | Seal-Tools in-domain 700 | 32.6 | 26.9 | 16.3 | — | | Seal-Tools OOD 654 | 28.7 | 17.0 | 15.6 | — | | BFCL v4 single-turn 3,641 , overall | 42.6 | 60.8 | 46.1 | 61.7 | Needle 2 leads both Seal-Tools splits and posts 98.3 function-name accuracy on Mobile Actions. It trails on BFCL v4, which Cactus attributes to distribution: its corpus is consumer device actions, not general or enterprise APIs. Well-formed output rate across the 3,641 BFCL rows is 93.4. The team states two asymmetries upfront: f16 baselines favor them, and task specialization favors Needle. Key Takeaways - Needle 2 is a 45M-parameter tool-calling model in a 14MB binary, running a session in ~28MB RAM. - CQ2-bit is trained in from pretraining, not applied post-hoc, so the deployed model is the trained model. - It leads Seal-Tools in-domain 32.6 and out-of-domain 28.7 , and trails on BFCL v4 overall 42.6 . - A 256-token sliding window with pinned tool sinks makes RAM a fixed ceiling, not a growing curve. - Confidence scores and empty-call refusals turn edge-to-cloud escalation into an explicit product decision. Check out the GitHub Repo Sources: GitHub · Hugging Face · Cactus Needle page · arXiv:2607.18363 , , https://huggingface.co/Cactus-Compute/needle2 Hugging Face Model Card and https://cactuscompute.com/needle Cactus Needle page . https://arxiv.org/abs/2607.18363 Paper Also, feel free to follow us on and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://magic.beehiiv.com/v1/f5e63dd4-5653-4f09-83e2-321a8b1ba526?email={{email}} now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.