A developer built react-native-needle, a React Native binding for the Needle 2 on-device LLM, and found that while it excels at tool-calling tasks with sub-300ms response times, it can fabricate numbe…
Cactus released Needle 2, a 14MB agentic LLM for tool call, device use, and structured extraction, running 45 million parameters at 2-bit compression in 28MB of RAM. It achieves 500 tokens/sec on a Ra…
Cactus shipped Cactus Hybrid, a post-trained build of Google's Gemma 4 E2B that returns a calibrated confidence score with every answer, routing queries to a larger model when confidence falls below 0…
Cactus released Gemma 4 E2B Hybrid, a small on-device model that outputs a confidence score (0-1) for each answer, allowing developers to route low-confidence queries to a larger model. The model matc…
Cactus released version 2 of its on-device AI inference platform, featuring model confidence-based routing to cloud fallback, lossless 4-bit quantization, and GPU acceleration on Apple Metal. The plat…
Google added native support for Gemma 4 Multi-Token Prediction (MTP) to LiteRT-LM, its on-device LLM runtime built on LiteRT (formerly TensorFlow Lite). Google reports the integration yields MTP decod…