Squish, a new local AI agent runtime for Apple Silicon, claims to load models 54× faster than standard paths and serve them faster than Ollama, with full offline capability and no cloud dependencies. …
Apple is skipping the high-end M6 Pro and Max chips to fast-track its M7 family, aiming to deliver a massive leap in memory bandwidth and Neural Engine capacity for on-device AI. The base M7 will offe…
Mariana Souza published a tutorial on quantizing and running Meta's Llama 3.2 3B model on Apple Silicon using llama.cpp with Metal GPU acceleration, achieving local inference with Q4_K_M quantization.…
Navatala Systems released Navatala GPU, an open-source cross-platform GPU compute runtime and kernel corpus supporting CUDA, HIP, Vulkan, OpenCL, and Metal backends. The alpha release includes Python …
Off Grid AI Desktop is a free, open-source app that records and transcribes meetings locally on Mac or PC using OpenAI's Whisper model, ensuring audio never leaves the machine and eliminating per-minu…
Off Grid AI Desktop is a free, open-source app that runs chat, image generation, and voice directly on Apple Silicon Macs, using unified memory and Metal GPU acceleration. It supports models like llam…
Off Grid AI Desktop is a free, open-source app that runs large language models locally on Apple Silicon Macs, leveraging unified memory for GPU inference. The app supports models like Gemma and Qwen, …
Sipp, an open-source AI inference library, enables running small local LLMs in browsers with up to 3x faster decode speeds than alternatives. Built by HCI and graphics programmers, it uses a unified A…
A developer built a GPU display backend for Emacs, creating Metal and OpenGL drivers that offload text rendering from the CPU to the GPU, enabling video playback and animated cursor effects without mo…
Alloy, a new PyTorch backend and inference engine for Apple Silicon, has been released as a technical preview. The open-source project compiles Python GPU kernels to Metal and supports LLM serving wit…
Apple announced Core AI, a new framework for running generative AI models on-device, at WWDC 26. The framework supports large language models up to 70B parameters and runs exclusively on Apple Silicon…
Build b9437 of llama.cpp fixes two default-value bugs in llama-bench that caused flash attention to be skipped on capable GPUs and GPU-layer count to use a legacy sentinel. The flash attention flag no…
Embershard, a macOS chat app with its own LLM inference engine, has been released in beta v0.1.1 for Apple Silicon devices running macOS 14 or newer. The app bypasses llama.cpp for inference, instead …
Gooey, a GPU-accelerated UI framework for the Zig programming language, has been released as an open-source project targeting macOS, Linux, and browser platforms. The framework provides declarative UI…
A developer successfully quantized Google's Gemma 4 model to 4-bit precision using llama.cpp on a Mac with Metal acceleration. The process involved converting the model to GGUF format and applying the…
This article describes a method to run OpenGL 4.6 applications, specifically Minecraft, on macOS by translating OpenGL calls through a chain of graphics APIs: OpenGL → Vulkan → Metal. The process uses…