Tracarbon, a Python library that tracks device energy consumption and calculates carbon emissions, now supports monitoring GPU power and carbon output while running local large language models. It det…
An Apple-commissioned report by Omdia, based on 1,500 conversations with enterprise tech leaders, finds that on-device AI infrastructure offers near-zero marginal cost after initial investment, and th…
Eris, a local-first AI agent built in Rust by developer Jan Paul Dahlke, runs entirely on-device with a single binary, using a Markdown vault as memory and llama.cpp with GBNF grammar-enforced tool ca…
Perplexity AI published a blog post detailing techniques for optimizing on-device inference on Apple Silicon, focusing on running large language models efficiently on Macs. The post covers quantizatio…
A developer has released Ferrum, an MIT-licensed Rust-based local LLM inference server that provides a private OpenAI-compatible endpoint on Apple Silicon. The tool, installable via Homebrew, supports…
AgentSelfEdit, an open-source sidecar that rewrites its own system prompt from execution feedback, A/B tests prompt edits and promotes only statistically-proven winners. In a recent test, an LLM-propo…
A developer tested RamaLama, an open-source CLI that runs large language models as OCI containers, on an Apple Silicon Mac with Docker. The tool simplifies local LLM deployment by pulling container im…
Bitwize released Logue, an open-source local-first workspace for macOS that combines documents, tasks, and meeting transcription with on-device AI via MLX on Apple Silicon, ensuring no data leaves the…
A developer built a system that uses hand gestures to control an AI coding agent, replacing keyboard prompts with webcam-tracked movements. The project uses MediaPipe for hand tracking and maps pinch,…
BLAKE3, a secure hash function, now runs more than twice as fast on Apple Silicon after the discovery of unannounced wide-vector SIMD capabilities in Apple M4 and M5 chips, accessed via the SMSTART in…
Eigen Labs, a Seattle-based tech company, estimates that owners of Apple Silicon hardware can earn $120 to $200 per month by selling idle compute power for AI inference through its Darkbloom network, …
RunAnywhere, Inc. shipped an update to RCLI, its on-device voice AI for macOS that runs a complete STT + LLM + TTS + VLM pipeline natively on Apple Silicon, featuring 40 macOS actions via voice, local…
A developer behind the Mac voice assistant Pace argues that privacy must be a property of the entire data path, not just a model label. The assistant keeps audio capture, transcription, planning, and …
A developer has released a CUDA-optimized fork of antirez's h3.c for NVIDIA DGX Spark, achieving a ~15.5x speedup, while also providing a native Apple Silicon implementation for MiniMax-H3 inference w…
Local-first workspace software can eliminate the need for cloud indexing by leveraging on-device AI and search capabilities, according to a guide from Neutron Tech. Modern hardware such as Apple Silic…
TT-AMX, a bare-metal C++ engine for Apple Silicon, runs Tensor-Train compressed linear layers on the undocumented AMX coprocessor, achieving a 2x speedup over dense FP16 GEMV under cold-cache conditio…
VPIPE, a new macOS app from developer tgo-app-dev, runs MiniMax H3, a 33B multimodal model generating video and audio, on Apple Silicon Macs with as little as 16 GB RAM, achieving a 5-second 0.5 MP 24…
A developer's blog post describes the 2026 AI agent stack as a layered architecture moving from local execution to a governance layer. The stack emphasizes local-first inference, structured output par…
A developer detailed a setup for running Qwen3.8 27B, an agentic coding model, locally on a 24 GB VRAM machine using llama.cpp and the pi coding agent. The configuration achieves about 8 tokens per se…
A developer demonstrates how to run fine-tuned large language models on Apple Silicon using MLX Swift bindings, bypassing CoreML's compilation latency. The approach uses lazy evaluation, direct loadin…