Cactus Compute released Whistle, an Apache 2.0 speech-to-text model that ships as a single 16.9 MB file and runs on CPU with no GPU or network calls, on October 2, 2026. A hands-on test on a plain Lin…
A new benchmarking framework, GreenBench, measures the energy efficiency and carbon footprint of open-source LLM inference on Apple Silicon, finding that an Apple M4 Pro with 48 GB unified memory draw…
A developer built Vellum, a self-hosted MCP server for markdown notes, and claims its search function answers in under a millisecond without a search engine or index. The server holds notes in RAM and…
The Kira Project introduces a tamper-evident, two-layer evidence architecture for local AI pipelines that records not only model actions but also the negative space of failures, using a hash-bound can…
A local AI assistant called Kira discovered that nearly half of its model time was spent re-reading boilerplate prompt text—a 'prefill tax'—after migrating from Ollama to Apple's MLX framework, which …
BaseCompute released BaseRT, a fast inference runtime for local AI on Apple Silicon, claiming up to 35% faster decode and 78% faster prefill on an Apple M4 Pro with 4-bit quantization. The runtime all…
A developer has created a guide for running coding agents like Codex CLI, Claude Code, and Cursor entirely on local hardware using Ollama, eliminating the need to send proprietary code to third-party …
A new open-source project called WAVE has introduced a vendor-neutral GPU instruction set architecture that allows developers to write GPU code once and run identical binaries on NVIDIA, AMD, Apple, a…