A developer built an AI agent with a memory architecture that runs preflight instructions, retrieving relevant skills from a library of ~90 workflow files before executing tasks. The system uses a loc…
Mistral.rs released version 0.9.0 on 7 July, claiming up to 1.8× faster CPU decoding than llama.cpp on both x86 and ARM hardware, challenging the de-facto standard for local LLM inference. The speedup…
A developer released TurboQuant, a pip-installable adapter for mlx-lm on Apple Silicon that uses a randomized Hadamard transform for data-oblivious, calibration-free quantization of weights and KV cac…
A developer added opt-in GPU backends (Metal and CUDA) to a pure-C TTS inference engine for Qwen3-TTS, achieving faster-than-real-time performance on Apple Silicon and NVIDIA GPUs. The key optimizatio…
A developer released mlx-serve, a native Zig server for MLX-format language models on Apple Silicon, enabling local, free, and private use of AI coding assistants like Claude Code. The server exposes …
Google launched Gemini Spark on Mac on July 1, enabling local file access, MCP support, and real-time topic tracking. The AI agent can now read, organize, and edit files on a user's Mac, but processes…
BaseCompute released BaseRT, a fast inference runtime for local AI on Apple Silicon, claiming up to 35% faster decode and 78% faster prefill on an Apple M4 Pro with 4-bit quantization. The runtime all…
A developer built LokalBot, an open-source macOS app that records, transcribes, and summarizes meetings entirely on-device using Apple Silicon, with features like inline autocomplete and a private tim…
Gemma 4 in Ollama 0.31 generates tokens nearly 90% faster on Apple Silicon using multi-token prediction (MTP), which employs a small draft model to propose multiple tokens that the main model verifies…
A bug in OpenAI's Codex desktop app for macOS causes a code_sign_clone directory to grow unbounded, consuming over 62 GB of disk space. The directory accumulates due to frequent auto-updates and is ne…
A collection of open-source sandbox tools for AI coding agents has been released, offering hardware-isolated microVMs, containers, and isolation harnesses that boot in under 200ms and prevent secret l…
A new PDF document titled 'Fail Fast, Run Faster: Shape Safe Deep Learning in Rust on Apple Silicon' has been published, detailing techniques for building safe deep learning models using Rust on Apple…
The Mano-P team released Mano-CUA 2.0, a 4B-parameter GUI agent that runs on a MacBook, and found that the primary bottleneck was not model size but insufficient Chinese GUI training data. After addin…
Squish, a new local AI agent runtime for Apple Silicon, claims to load models 54× faster than standard paths and serve them faster than Ollama, with full offline capability and no cloud dependencies. …
Samuel Yuan, a developer from MIT, released wavecat, a fully local personal AI agent that continuously watches a user's screen to understand their activity and goals, with all processing and data stor…
Modular announced that MAX models can now run on Apple Silicon GPUs with the 26.4 release, supporting M1 through M5 chips for text LLMs, vision models, and image diffusion models. The company is worki…
William, a tiny poetry language model trained by a developer, runs entirely in the browser using ONNX Runtime Web. The 6-layer transformer was trained on the Gutenberg Poetry Corpus and fine-tuned on …
An architect breaks down how to size a Mac mini M4 for local AI workloads, arguing that memory configuration is the critical decision, not the CPU. The analysis maps tasks to memory tiers: 16GB for ch…
DropItDown, a free macOS app for Apple Silicon, converts any file dropped onto its menu-bar icon into clean Markdown locally, with on-device OCR for images. The Markdown is filed into a user-specified…
A developer built Squish, a local LLM inference server for Apple Silicon, after finding existing tools like Ollama too slow for generating git commit messages. Squish achieves up to 9.8× faster perfor…