Relay, a self-hosted LLM gateway and model router released as a single static Go binary under Apache-2.0, supports OpenAI and Anthropic API dialects inbound and routes across any provider outbound wit…
Embeddings are fixed-length vectors of floating-point numbers that encode semantic meaning, enabling similar sentences to be mathematically close and powering RAG, semantic search, and recommendation …
LLMrPro, a new self-hosted LLM router, pools users' own machines as inference workers with automatic per-tier cloud fallback, exposing a single OpenAI-compatible API. The open-source project, released…
Cue, a voice-activated AI agent, cut latency by 44%, increased dictation usage by 30%, and eliminated marginal inference costs by running Google's Gemma 4 E4B model locally for text polishing, with me…
Stoke, a pre-release MIT-licensed Rust binary (~5.5 MB), acts as a kill switch for runaway AI agents by refusing over-budget or looping requests before they reach the provider, with per-key spend caps…
Developer Cole Hellman built a personal MCP server called Codicil that indexes documentation for AI coding assistants. When the embedding service went down for two weeks, Hellman didn't notice because…
N8n, a visual automation tool with 1,200-plus prebuilt nodes and agent nodes for AI workflows, has become the default engine of the AI automation economy due to its pricing quirk: one execution per wo…
The founders of Kitematic raised $65 million USD to develop Ollama, an open-source tool that lets developers run AI models locally. The platform has grown to nearly nine million users.…
Real-time voice agents using local LLMs face a hard latency ceiling of ~3 seconds imposed by Retell AI's WebSocket timeout, which forces reconnections and call termination if the first token is not pr…
A full-stack developer who knew nothing about AI security learned prompt injection by playing Lakera's free game Gandalf, which has logged nearly 9 million interactions from over 200,000 people since …
In the tenth installment of a series on building AI agents in Rust, the author explores driving a local model directly via stdin/stdout instead of HTTP, using Salvatore Sanfilippo's DwarfStar inferenc…
A community developer, GnLOLot, has published MiniCPM5-1B-Claude-Opus-Fable5-Thinking, a 1.08B-parameter model fine-tuned on Claude Fable 5 traces that runs fully on local hardware with a 657MB Q4_K_M…
Ollama, the leading platform for accessing open AI models, announced it has raised $88 million from Benchmark, Theory Ventures, 8VC, and other investors. Founded by Docker veterans, Ollama now serves …
A developer who spent three months and over $420 trying to self-host large language models concluded that using an API is far more practical for most developers. After struggling with GPU rental, debu…
EnviousWispr, a free open-source AI dictation app for macOS, runs entirely on-device using Whisper and Parakeet speech-to-text models on Apple Silicon, with no cloud, account, or subscription required…
An engineer built SafeDevTools, a collection of privacy-first developer utilities, using only a local LLM (Gemma 4:12B via Ollama) on a MacBook M4 Pro. The developer argues that AI's real value is not…
Waylou ACE CLI, an open-source AI coding assistant forked from Google Gemini CLI, adds multi-provider support for Gemini, OpenAI, Anthropic, Ollama, and DeepSeek models through a single terminal inter…
Ollama and vLLM are both open-source inference stacks for running large language models, but they serve fundamentally different use cases: Ollama is a model manager optimized for a single developer on…
LLM Inspector, a new open-source CLI tool from developer Helal Saoudi, analyzes live LLM inference processes to show exactly how GPU memory is used by weights, KV cache, and workspace, then projects o…
Glamorous Toolkit is a moldable development environment designed to help humans and AI make sense of large systems and AI-generated code by replacing code reading with deterministic contextual tools. …