A developer demonstrates how to run fine-tuned large language models on Apple Silicon using MLX Swift bindings, bypassing CoreML's compilation latency. The approach uses lazy evaluation, direct loadin…
A developer has reverse-engineered Apple's private APIs to train neural networks directly on the Apple Neural Engine (ANE), bypassing CoreML's inference-only restriction. The project, which uses the _…
In 2026, edge inference has advanced to the point where an 80B Qwen model runs in just 4.3GB of RAM on a MacBook Pro, and a 35B model runs on an iPhone 18 Pro. This is achieved through a combination o…
ANEForge, a new open-source Python package, compiles tensor graphs directly into programs for the Apple Neural Engine (ANE) and runs them without CoreML, enabling both inference and training on the ch…
Palmier Pro, a macOS video editor built in Swift, now integrates AI agents via MCP (Model Context Protocol), allowing LLMs like Claude or Codex to automate media management, semantic search, timeline …
A developer successfully ran Google's Gemma 4 E2B and E4B models on Apple's Neural Engine (ANE) using CoreML, achieving a memory footprint under 1 GB on an iPhone 17 Pro. The project overcame ANE's la…
Apple announced RAW 9, a major overhaul of its RAW image processing engine coming with iOS 27, using a CoreML model to improve detail and reduce noise, including for older photos. The update leverages…
A developer built rustnn, a Rust implementation of the W3C WebNN specification for Firefox, using Claude Code to accelerate development. The project implements 95 operators and validates against WPT c…
Thunder Kitty 1.9.0 introduces live topic segmentation and agenda tracking that run entirely on-device on Apple Silicon, using three models simultaneously. The developer behind the app spent seven att…
A developer built an offline skin lesion segmenter for mobile devices by optimizing the Med-SAM model with CoreML and ONNX Runtime. The approach enables real-time, privacy-preserving segmentation on i…
A new open-source benchmark, "apple-silicon-llm-bench," reveals that Google's LiteRT-LM runtime outperforms MLX-Swift on the iPhone 17 Pro for Gemma 4 E2B inference, achieving 55.4 tok/s with 4.5x les…
Building a React Native (Expo) field inspection app that uses a bundled CoreML model (under 50MB) for real-time, offline AI detection of PPE violations like missing hard hats on construction sites. It…