A developer built llmproxy, an open-source Python proxy that solves 502 read timeouts on slow reasoning LLM APIs using SSE stream aggregation. The tool provides response caching, automatic failover to…
Axon, a TypeScript runtime and toolkit for AI agents, provides a consistent way to define agents with capabilities, sessions, and local or cloud deployment. The framework handles context, tool calls, …
Developer lordraw77 released llmproxy, a lightweight Flask server that emulates Ollama, OpenAI, and llama.cpp APIs and forwards every request to NVIDIA's OpenAI-compatible API, enabling any existing t…
A developer built Skim, a free open-source email client for Windows that uses BYOK (bring your own key) AI. The client features agentic AI that can search and read emails, treats HTML email as hostile…
Alibaba's Qwen2.5-Coder 32B outperforms Meta's Llama 3.1 70B in code generation tasks, delivering 35 tokens per second versus 12 tokens per second and a 2.1-second time-to-first-token compared to 4.8 …
Oats, a free meeting-notes app for Mac, processes all audio and transcription locally using Whisper and Ollama, ensuring no data ever leaves the user's machine. The app, which supports optional cloud …
Jharu v0.2.0, a free open-source disk cleaner and storage analyzer for macOS and Windows, targets gigabytes of junk left by development tools and machine learning frameworks. The tool identifies and s…
Running Qwen locally via Ollama or vLLM with a local Python environment avoids cloud data exposure and token limits, enabling iterative work on large datasets. Qwen2.5-Coder (7B) on an RTX 3090 genera…
A developer built a fully offline AI development stack integrating LM Studio, Ollama, and TormentNexus to eliminate cloud dependency, achieving sub-100ms response times for code completion and zero da…
Skim, a new BYOK (Bring Your Own Key) AI email client for Windows, achieves an installer under 5 MB by optimizing dependencies and compiler flags, using Tauri 2 to leverage the system WebView2. The ap…
Kdeps, a CLI coding agent that runs on llamafile and/or GGUF, enables building and deploying AI agents in YAML with workflow (DAG pipelines) and agent (autonomous LLM loop) modes. The open-source tool…
A developer built AICostTracker, a local-first CLI tool that logs API calls to OpenAI, Anthropic, Google, and Ollama models and provides a cost breakdown by project and model. The tool stores data in …
The Pi SDK v0.79.2 does not contain the ModelRuntime.create() API referenced in issue #947, but a spike test by the engineering team confirmed that local models via LM Studio and Ollama are fully cove…
Narya.ai launched Mumble Dictation, a local-first dictation app for Mac that keeps audio on-device and learns users' vocabulary, priced at $50 one-time. The app runs its full pipeline locally, includi…
A practical tutorial outlines building a real-time AI tracker that uses RSS feeds and GitHub notifications as trigger sources, filters updates through a lightweight LLM to reduce noise, and pushes rel…
A developer built Cypher, an open-source Rust CLI that scans Reddit subreddits and uses AI to score buying intent, ranking leads for freelancers. The tool supports local Ollama models and cloud fallba…
Andrew Ng released OpenWorker, an open-source desktop AI agent that produces finished deliverables instead of chat, running entirely on the user's machine. The four-layer architecture includes a Tauri…
A new CPU-only inference runtime called cpubrrr achieves up to 5× faster token generation than llama.cpp's CPU path on frontier-class mixture-of-experts models, running on an Apple M4 Max without GPU …
Gate.cat, an open-source tool from BGMLAI, intercepts shell commands from AI agents before execution to block dangerous operations like rm -rf, using a fail-closed parser with no LLM call in the veto …
ChatPanel, an open-source browser side panel, lets users orchestrate multiple LLM agents locally with privacy features like local PII redaction and voice-to-text processing. The tool supports model-ag…