A user reports that threadfork, a local AI notetaker for Apple Silicon, enables secure meeting transcription without sending data to third-party servers, addressing privacy concerns for legal and cons…
A new LLM inference runtime called BaseRT achieves up to 6.4x faster local inference on Apple Silicon than llama.cpp by writing directly to Apple's Metal GPU API, skipping intermediate frameworks like…
ExecuTorch's experimental MLX delegate, released in May 2026, enables PyTorch models to run on Apple Silicon GPUs. Testing Qwen3-0.6B on an M1 Max, MLX INT4 achieved 188.9 tokens/s, 4.52x faster than …
Threadfork, a new AI meeting notetaker that runs entirely locally on Apple Silicon Macs, has been launched by developer Navneet Nair. The tool captures microphone and system audio without a meeting bo…
PyTorch 2.13, released July 8, brings FlexAttention to Apple Silicon with up to 12x speedup on sparse attention patterns, such as a 32,768-token sequence with a 256-token sliding window (35ms vs 431ms…
Narrato, an on-device Mac app for dictation and read-aloud, launches a beta with a one-time license fee and no subscription. The app processes all audio and text locally on Apple Silicon Macs, never s…
A developer benchmarked PyTorch 2.13's FlexAttention on an M1 Max Mac, finding up to 7.83x speedup over standard SDPA for sparse attention with 32,768 tokens and a 256-token local window. FlexAttentio…
A Flutter developer built vector_kit, a Dart library that accelerates semantic search by storing embeddings as a packed Float32List and using SIMD dot products, achieving 11x faster top-k cosine simil…
Nativ has released an open-source desktop app that lets users run frontier open models from Google, Cohere, and Liquid AI locally on Apple Silicon Macs with no accounts, subscriptions, or cloud depend…
Mlx-serve, an open-source native LLM inference server for Apple Silicon written in Zig, shipped an update that adds compatibility with the Ollama API alongside existing OpenAI and Anthropic API suppor…
A fork of the DwarfStar inference engine, andreaborio/ds4, aims to run large Mixture-of-Experts models like Qwen3.6-35B-A3B on 16–64 GB Apple Silicon Macs by using adaptive SSD streaming and Metal res…
Velora, an open-source macOS dictation tool that runs entirely on-device using Apple Silicon and MLX, has been released under an MIT license. The tool uses Whisper for speech-to-text and a local LLM f…
Cogni-ML, a Crystal machine learning library, now supports native Apple Silicon GPU acceleration with Metal inference for GGUF models, including nomic-embed-text-v2-moe embeddings and Qwen 3.5 9B text…
Lattice is an open-source, pure Rust inference and training engine optimized for Apple Silicon, enabling developers to run, quantize, and fine-tune LLMs locally with zero external dependencies. It lev…
PyTorch 2.13, released this week, brings FlexAttention support for Apple Silicon with up to 12x speedup on sparse patterns and a new fused LinearCrossEntropyLoss operator that cuts peak GPU memory by …
Samosa Chat, an independent Apache-2.0 project, enables local inference of the Qwen3.6-35B-A3B model on a 16 GB Apple Silicon Mac entirely on CPU with no cloud account or telemetry. The tool fits the …
A developer running a custom fork of rapid-mlx called qMLX on a 96GB Mac Studio has removed hot caching to enable multiplexing for agentic workflows, allowing concurrent sessions by relying on SSD-bas…
Pullrun, a new single-binary OCI runtime, lets users run the same OCI image as a container, Firecracker microVM, Apple Silicon VM, Kubernetes workload, or AI agent task without a daemon or overlayfs. …
Apple is accelerating development of its M7 Ultra chip, which will dramatically upgrade AI performance and bring it closer to dedicated AI accelerators like Nvidia Corp.'s Blackwell, according to Bloo…
Token Time, a new macOS app, monitors AI agent token usage and displays full-screen reminders after every 100 million tokens. The $6 one-time purchase app runs locally with no cloud or telemetry, targ…