A developer outlines a tiered architecture for deploying large language models on mobile devices with low power consumption, recommending quantized models between 1B and 4B parameters and native runti…
Cactus Needle, a 26-million-parameter open-weight language model developed by the Cactus project, is designed exclusively for function calling (tool use), outperforming much larger general-purpose mod…
ModelBest and Tsinghua University released MiniCPM-5, a 1-billion-parameter on-device AI model with a 128K context window and native tool-use capabilities, designed for agentic workflows on phones and…
A technical report proposes using a consumer RTX 3090's integrated GPU (iGPU) to run a small language model as a 'Sentinel' for monitoring and validating outputs from the primary GPU-bound model. The …