Prince Canuma, the maintainer of mlx-vlm, released Nativ, an MIT-licensed macOS app for local AI inference, on the day of its v0.0.1 launch. The app bundles an mlx-vlm server, pulls models from Huggin…
LM Studio enables users to run large language models entirely offline on a laptop, processing sensitive documents without internet or cloud dependency. The tool supports open-weight models like Llama,…
Local open-weight models and PII detection pipelines can process sensitive documents without sending data to the cloud, eliminating risks such as data retention, breach surface, and regulatory liabili…
Mlx-serve, an open-source native LLM inference server for Apple Silicon written in Zig, shipped an update that adds compatibility with the Ollama API alongside existing OpenAI and Anthropic API suppor…
Moonshot released Kimi K3, a 2.8 trillion-parameter open-weight model, the largest ever, with native vision and a 1M-token context window, claiming ~2.5× better scaling efficiency than K2 and pricing …
ExploitHunter.app, an open-source, local-first AI security platform for authorized security research, has been launched by developer justsml. The platform features a deep evaluation suite, supports mu…
LM Studio launched LM Studio Bionic, an AI agent for open models that supports coding, research, and document work with local or cloud-based open-source models, committing to zero data retention and n…
LM Studio launched LM Studio Bionic, a new Mac app that uses open models for coding, research, and document work, with support for local and cloud-based open-source models under a Zero Data Retention …
Cogni-ML, a Crystal machine learning library, now supports native Apple Silicon GPU acceleration with Metal inference for GGUF models, including nomic-embed-text-v2-moe embeddings and Qwen 3.5 9B text…
A local LLM runs entirely on a user's own device instead of a vendor's cloud server, ensuring data never leaves the machine. Cisco's 2024 Data Privacy Benchmark Study found that 48% of organizations a…
A developer released AI-CLI, a tiny C terminal assistant that connects user requests to a local LLM and executes returned shell actions directly, supporting over 20 platforms and most LLM engines. The…
Atuin has open-sourced the Atuin AI server, allowing users to self-host the terminal-focused AI agent that provides agentic tools directly in the shell. The server supports any OpenAI-compatible endpo…
A hybrid routing strategy that sends simple agent tasks to local AI models and complex ones to cloud APIs can cut costs by more than half for teams running high-volume workflows, according to a practi…
Silica, an open-source agent for notes and codebases, builds a live model of a folder's contents and rolls back any edit that breaks coherence, according to its developer. The local-first tool, compat…
Developer SunnyLich released Wisp, an open-source desktop AI overlay that lets users interact with AI assistants via keyboard shortcuts, voice, and context-aware prompts without leaving their current …
Developer Youssef Mejdi released Grinta v1.0.0-rc1, a local-first, provider-agnostic coding agent for long autonomous runs that integrates with LSP and DAP. The tool runs entirely on the user's machin…
Picchio, a new open-source Python tool, diagnoses whether local large language models (LLMs) are actually using the GPU or falling back to CPU by running three inference passes and reporting token spe…
Zed 1.10, released July 8, integrates llama.cpp as a native language model provider, enabling its AI agent and inline assistant to run entirely on local hardware without sending code to external serve…
A developer explains how to redirect any app using OpenAI's Chat Completions API to a local LLM on a Mac by changing the base URL to Ollama's or LM Studio's local endpoint. This works because most AI …
A developer explains quantization techniques for large language models, including GPTQ, AWQ, GGUF, and bitsandbytes, which can reduce model size by up to 75% while preserving most accuracy. The post d…