Perplexity has open-sourced Lily, a Rust and Metal local inference engine that outperforms Apple's MLX framework by 1.35x on decode throughput for Qwen3.6-35B-A3B on Mac, with prefill speeds of 4,156 …
Open-source AI models are increasingly being stripped of their safety filters and redistributed at scale, echoing the warez scene of 1995–2010, according to an analysis on Unite.AI. The practice invol…
A new guide from Vetted Consumer identifies six bottlenecks that slow local large language models, ranked by impact, with the top cause being the model not fitting in fast memory, which can drop speed…
Agentray, a new macOS app by an unnamed developer, lets users interact with LLMs by dropping files into folders instead of using chat, with the folder name serving as the instruction. The app, current…
Perplexity has open sourced Lily, a Rust and Metal inference engine for Qwen3.6-35B-A3B on Apple Silicon, achieving a 77.4% prefill speedup via fused grouped GEMM and an 89% gain from GPU-only routing…
A developer testing four language models to improve a self-rewriting AI agent found that none produced promotable edits, despite resolving dozens of bugs. The models, ranging from Qwen3-4B to Mistral …
A developer detailed a local LLM setup on an M4 Pro Mac mini with 48 GB of RAM, running Qwen3.6-35B-A3B-OptiQ-4bit and Gemma-4-E4B-it-OptiQ-4bit via MLX, to avoid cloud API costs, data privacy risks, …
Independent developer Carlos Galarza released slotstream, a Swift binary that streams Mixture-of-Experts model tensors from NVMe SSD into Apple Silicon Metal memory, enabling local inference of Alibab…
A hardware-sizing analysis by an unnamed engineer claims that two AMD Radeon PRO R9700 GPUs ($1,300 each) and a Mac Studio M3 Ultra (256GB, $6k) could outperform NVIDIA's DGX Spark (128GB, 273GB/s) fo…
FastH3, MiniMax's video-and-audio generation model, now runs on Apple Silicon via MLX and on NVIDIA's DGX Spark desktop, with two Sparks able to generate one clip together. The release also debuts the…
Mark Murphy reports that mlx-serve, an Ollama-like inference server with a GUI, delivers a 4x speed increase for running Qwen 3.8 on his 64GB M2 Ultra Mac Studio compared to his previous Ollama setup,…
A developer's experiments with an agentic terminal and a fine-tuned model failed because they were measured with self-built instruments that lied, leading to the development of a context engine called…
A developer argues that the industry's reliance on cloud-based AI APIs compromises data privacy and introduces latency, advocating for a shift toward local-first execution and client-side privacy. The…
Bitwize released Logue, an open-source local-first workspace for macOS that combines documents, tasks, and meeting transcription with on-device AI via MLX on Apple Silicon, ensuring no data leaves the…
Whispr Freely, a free macOS dictation app that runs OpenAI's Whisper large-v3-turbo locally on Apple silicon via Apple's MLX framework, is set for public release once Apple's notarization clears, with…
Black Star Designs released Memo, a local-first scratchpad for developers that runs entirely on the user's machine, with no cloud, account, or telemetry. The tool, available as a single binary for mac…
Dark Bloom, a distributed inference network, pays Mac owners to share idle compute for AI inference, with payouts via Stripe and models served through OpenRouter at lower prices. The project, which re…
Apple's M5 chip, announced October 2025, introduces a Neural Accelerator in every GPU core, making prompt processing 3.3 to 4.1 times faster than the M4, according to Apple's MLX benchmarks. Token gen…
Apple Inc. announced the M6 Mac mini and M5 Ultra Mac Studio, aimed at local AI workloads and Mac clustering, with preorders open and most versions shipping September 22. The M6 is Apple's first 2nm c…
A developer compared Apple's Mac Studio M5 Ultra and NVIDIA's DGX Spark as local AI machines, finding the Mac Studio offers 4.4x the memory bandwidth for faster LLM inference, while the DGX Spark prov…