Kyle released mere.run, a local-first inference runtime CLI that runs text, image, video, music, and 3D models on Mac and Linux without Python, using native Swift on MLX and llama.cpp. The MIT-license…
Researchers at UC Berkeley Sky Lab extended the K-Search evolutionary kernel optimization framework with a structured CUDA-to-MLX translation layer, enabling automatic transfer of decades of CUDA kern…
Writekin, a new open-source macOS app, fine-tunes a local LLM on a user's own writing from Apple Mail, iMessage, and local documents to generate text in their personal voice, with all processing runni…
Fermion Research released Neutrino-1 8B, a 3.88 GB language model trained natively with ternary weights ({-1, 0, +1}) that achieves 763 tokens per second on an H100 with speculative decoding and 24.9 …
A developer built a Swift 6 actor topology that routes on-device inference requests between MLX's GPU compute and Core ML's ANE scheduler on Apple Silicon, achieving sub-50ms first-token latency on qu…
A new weight reordering technique for Mixture-of-Experts (MoE) models optimizes physical file layout to minimize NVMe seek overhead, achieving massive gains in explicit-read inference engines like MLX…
Apple's MLX team introduced distributed inference and training capabilities for machine learning workloads across multiple Macs at WWDC26, demonstrating how to scale large language models using RDMA o…
A developer's MoE model file reordering technique, mbolt, achieved +32.3% decode throughput and −26.3% time-to-first-token on a 235B-parameter model running on a 48 GB MacBook, as independently verifi…
Apple's M5 generation introduces a dedicated Neural Accelerator on every GPU core, and BaseRT, a native Metal inference runtime, exploits these units to deliver up to 6.4× higher prompt-processing thr…
A new LLM inference runtime called BaseRT achieves up to 6.4x faster local inference on Apple Silicon than llama.cpp by writing directly to Apple's Metal GPU API, skipping intermediate frameworks like…
Cactus released Gemma 4 E2B Hybrid, a small on-device model that outputs a confidence score (0-1) for each answer, allowing developers to route low-confidence queries to a larger model. The model matc…
ExecuTorch's experimental MLX delegate, released in May 2026, enables PyTorch models to run on Apple Silicon GPUs. Testing Qwen3-0.6B on an M1 Max, MLX INT4 achieved 188.9 tokens/s, 4.52x faster than …
The mlx-community/Laguna-S-2.1-oQ2e quantized model running in-process through mlx-vlm on a 128 GB Apple M5 Max achieved a perfect overall score of 1.000 across six tasks, with 40.85 generation tok/s …
Prince Canuma's Nativ is a new macOS desktop application that wraps Apple's MLX framework to run AI models locally, offering both a chat interface and a localhost API server. The app automatically det…
Velora, an open-source macOS dictation tool that runs entirely on-device using Apple Silicon and MLX, has been released under an MIT license. The tool uses Whisper for speech-to-text and a local LLM f…
A five-engineer team on Sonnet 4.6 saw a $4,800 monthly bill for Claude Code sessions, with only $960 attributed to the model itself and $3,840 coming from inference engineering — the layer between th…
Eaon, a free and open-source native macOS app, lets users switch between 49 AI models from a single interface, supporting both cloud keys and fully offline local models via Ollama, llama.cpp, or MLX. …
PrismML shipped Bonsai 27B, a 27B-parameter model based on Qwen3.6 27B, with a 1-bit variant packing to 3.9 GB that fits on an iPhone 17 Pro, clearing the memory gate that blocked prior builds of this…
PrismML released Bonsai 27B, claiming it is the first major AI model of its size that can run on an iPhone, iPad, or Mac, with a 1-bit version fitting in about 4 GB of memory. PrismML CEO Babak Hassib…
A solo developer running brevio.news discovered that a hosted LLM API silently broke the product's weekly brief feature for four weeks after the provider retired a model ID. The developer fixed the is…