PyTorch 2.13, released July 8, brings FlexAttention to Apple Silicon with up to 12x speedup on sparse attention patterns, such as a 32,768-token sequence with a 256-token sliding window (35ms vs 431ms…
Narrato, an on-device Mac app for dictation and read-aloud, launches a beta with a one-time license fee and no subscription. The app processes all audio and text locally on Apple Silicon Macs, never s…
A developer benchmarked PyTorch 2.13's FlexAttention on an M1 Max Mac, finding up to 7.83x speedup over standard SDPA for sparse attention with 32,768 tokens and a 256-token local window. FlexAttentio…
A Flutter developer built vector_kit, a Dart library that accelerates semantic search by storing embeddings as a packed Float32List and using SIMD dot products, achieving 11x faster top-k cosine simil…
Nativ has released an open-source desktop app that lets users run frontier open models from Google, Cohere, and Liquid AI locally on Apple Silicon Macs with no accounts, subscriptions, or cloud depend…
Mlx-serve, an open-source native LLM inference server for Apple Silicon written in Zig, shipped an update that adds compatibility with the Ollama API alongside existing OpenAI and Anthropic API suppor…
A fork of the DwarfStar inference engine, andreaborio/ds4, aims to run large Mixture-of-Experts models like Qwen3.6-35B-A3B on 16–64 GB Apple Silicon Macs by using adaptive SSD streaming and Metal res…
Velora, an open-source macOS dictation tool that runs entirely on-device using Apple Silicon and MLX, has been released under an MIT license. The tool uses Whisper for speech-to-text and a local LLM f…
Cogni-ML, a Crystal machine learning library, now supports native Apple Silicon GPU acceleration with Metal inference for GGUF models, including nomic-embed-text-v2-moe embeddings and Qwen 3.5 9B text…
Lattice is an open-source, pure Rust inference and training engine optimized for Apple Silicon, enabling developers to run, quantize, and fine-tune LLMs locally with zero external dependencies. It lev…
PyTorch 2.13, released this week, brings FlexAttention support for Apple Silicon with up to 12x speedup on sparse patterns and a new fused LinearCrossEntropyLoss operator that cuts peak GPU memory by …
Samosa Chat, an independent Apache-2.0 project, enables local inference of the Qwen3.6-35B-A3B model on a 16 GB Apple Silicon Mac entirely on CPU with no cloud account or telemetry. The tool fits the …
A developer running a custom fork of rapid-mlx called qMLX on a 96GB Mac Studio has removed hot caching to enable multiplexing for agentic workflows, allowing concurrent sessions by relying on SSD-bas…
Pullrun, a new single-binary OCI runtime, lets users run the same OCI image as a container, Firecracker microVM, Apple Silicon VM, Kubernetes workload, or AI agent task without a daemon or overlayfs. …
Apple is accelerating development of its M7 Ultra chip, which will dramatically upgrade AI performance and bring it closer to dedicated AI accelerators like Nvidia Corp.'s Blackwell, according to Bloo…
Token Time, a new macOS app, monitors AI agent token usage and displays full-screen reminders after every 100 million tokens. The $6 one-time purchase app runs locally with no cloud or telemetry, targ…
Spiel, a free and open-source Chrome extension, reads articles and PDFs aloud using a local neural voice model on Macs with Apple Silicon, ensuring privacy by processing everything on-device without c…
Ito CTO Evan released Moo, an open-source tool that versions machine states alongside Git repos, enabling isolated microVM environments for each branch or agent attempt. The tool uses copy-on-write di…
PyTorch 2.13 has been released with performance improvements including FlexAttention on Apple Silicon with up to 12x speedups, a CuTeDSL backend for Inductor, and fused nn.LinearCrossEntropyLoss reduc…
Ollama 0.31 introduces multi-token prediction for Gemma 4 on Apple Silicon, cutting token generation time by 90% on Aider's coding benchmark. The update automatically accelerates local coding agents w…