{"slug": "local-privacy-focus-on-cache-ai-cli-and-llama-cpp-branch-qwen3-x", "title": "Local+privacy+focus on cache AI CLI and llama.cpp branch + Qwen3.x", "summary": "Developer alainnothere released privibe, a fork of Mistral Vibe's CLI coding agent reworked for private, local-first development with no callbacks, running against local llama.cpp servers via an OpenAI-compatible API. The fork removes cloud/account features, telemetry, and remote auth, adds KV cache persistence for session resume, and requires Python 3.12+ and uv. It is developed and tested against a local llama.cpp server, with Anthropic and Mistral backends inherited but unverified.", "body_md": "CLI coding agent for private, local-first development.\n\nprivibe is a fork of [Mistral Vibe](https://github.com/mistralai/mistral-vibe)\nreworked to not do any call back home of any kind and then run against **local**\nmodels first, I want to be able to use it and know it will not be sending data\nof any kind anywhere.\n\nThe cloud/account machinery is gone, the Mistral SDK is now an optional extra,\nand a lot of work has gone into making it fast and pleasant to use against a local\n[llama.cpp](https://github.com/ggml-org/llama.cpp) server.\n\nIf you want the original, hosted, Mistral-centric experience, use upstream Mistral Vibe. If you want a small coding agent you can point at your own local llama.cpp server, this is that.\n\nStep-by-step version of this demo: [docs/getting-started.md](/alainnothere/privibe/blob/main/docs/getting-started.md).\n\nResuming and switching between sessions, plus context files in action: the lets-document skill compiles what one session learned into a context file, and a second, unrelated session picks up from that knowledge without redoing the research (played at 4x):\n\nResuming a session with its KV cache restored from disk: with the companion\n[llama-server build](https://github.com/alainnothere/llama.cpp/tree/disk-cache-eviction),\nevicted conversations are saved to disk and reloaded on resume, so picking an\nold session back up does not reprocess the whole conversation:\n\nNote on backends:this fork is developed and tested against a localllama.cppserver (via its OpenAI-compatible API). The Anthropic and Mistral backend adapters inherited from upstream are still in the code but arenot actively tested here— treat them as unverified.\n\nRequires Python ≥ 3.12 and [uv](https://docs.astral.sh/uv/).\n\n```\ngit clone https://github.com/alainnothere/privibe.git\ncd privibe\nuv sync\n```\n\nI set up an alias on .bashrc\n\n```\nalias privibe='uv run --project /yourPathToPrivibeHere/privibe privibe'\n```\n\nso you can later just use it with privibe from whenever you are, or inside privibe folder with\n\n```\nuv run privibe\n```\n\nOn first run it writes a config to `~/.privibe/config.toml`\n\n. Point the model\nentries at your server (e.g. a local llama.cpp instance) and you're going.\n\nThere's also an ACP entrypoint for editor integrations:\n\n```\nuv run privibe-acp\n```\n\nFor the most part... exactly the same, clone the source and uv run privibe... I use it daily on git bash and works correctly, I have also tried powershell and I know it works.\n\n- Built around a local llama.cpp server (OpenAI-compatible API). The Mistral SDK\nis an\n**optional** extra (`uv sync --extra mistral`\n\n), not a requirement, and there's no API-key onboarding gate. - The upstream Anthropic and Mistral backend adapters are still present but are not tested in this fork (see the note at the top).\n- Removed the cloud and account features (nuage/teleport), telemetry, tracing, the update notifier, plan offers, data-retention, and remote auth — along with their dead code and tests. De-branded from Mistral Vibe throughout. If you find anything calling home is not intentional, I miss it, please create an issue and we'll nuke that code into outer space....\n\n- Conversation state is held in a structure (\n`ConversationList`\n\n) that keeps the prefix immutable, so the server's prompt/KV cache stays valid across turns. - KV cache is preserved on\n`--resume`\n\n/`--continue`\n\nby restoring the original system prompt instead of regenerating it (which would invalidate the cache). - Context-size auto-detection with a\n`/detect-context-size`\n\ntoggle (re-runs on model swap), and an opt-in model warmup. - Per-message reasoning effort:\n`/effort`\n\ncycles off/low/medium/xhigh for new messages without invalidating the KV cache. Needs the companion[llama-server build](https://github.com/alainnothere/llama.cpp/tree/disk-cache-eviction); stock servers silently ignore the stamps. - Long sessions stay responsive: the transcript is windowed behind a \"Load more\" button, and streaming output is render-throttled so drawing keeps up even when the model generates faster than the terminal can paint.\n- Tolerates SSE keep-alive pings from newer llama.cpp servers.\n\n- Hashed-line file tools split into explicit single-line, block, and delete\noperations, plus\n`find_symbol`\n\n. - Per-agent file\n**undo stack** with a`restore_file`\n\ntool; writes are serialized. - Tuned for smaller local models: best-effort indent correction on edits, hashed line addresses re-pointed when earlier edits shift them, and naive reads of huge files return a head preview plus guidance instead of flooding the context.\n- Cross-dialect path translation (Windows / WSL / Git Bash / Cygwin) with a\n`[paths]`\n\nconfig section. `@`\n\n-mention file completion backed by a stateless git enumeration.\n\n**Console mode**:`--console`\n\nruns the same agent as a plain-text REPL (no colors, no TUI), including tool output and session resume.**Rewind**: Alt+Up browses your previous messages; fork the conversation from any of them, and optionally restore files to that point (the file restore is gated behind a typed confirmation code).**Agent steering**: queue messages mid-turn instead of cancelling the agent.- Double-press\n`Ctrl+C`\n\nto exit (no more accidental one-key quits). - Clipboard fixes for Linux/X11 terminals, a\n`/autocopy`\n\ntoggle, and a warning when clipboard tools are missing. `/resume`\n\nsession picker showing the folder and a short preview of each session, with search: type to filter,`-word`\n\nto exclude, results ranked by match. Conversation history is re-rendered on resume.- File edits show a diff in the TUI.\n- Model name + tokens/sec in the context footer.\n- Configurable tool-result preview length (\n`/preview-lines`\n\n), scrollback (`/scrollback`\n\n), and an`/llm-debug`\n\ndump toggle. - Subagent work is preserved when you cancel mid-execution.\n\n- Bundled sample skill (lets-document) with auto-discovery and load-error surfacing.\n- Model selection skips entries with a missing API-key env var and falls back to a valid model; config upgrades append commented stubs for new keys.\n- Datetime-based version stamping;\n`.deb`\n\nand Windows-zip build scripts.\n\nConfig lives in `~/.privibe/config.toml`\n\n(set `PRIVIBE_HOME`\n\nto relocate it).\nThe `[paths]`\n\nsection (documented in\n`privibe/core/config/default_config.toml`\n\n) controls cross-dialect path\ntranslation. Models, providers, and feature toggles are set there too; several\nhave in-app `/commands`\n\n(`/model`\n\n, `/config`\n\n, `/autocopy`\n\n, … — see `/help`\n\n).\nThe active model can also be picked for a single run with `--model`\n\n.\n\nApache-2.0. privibe is a fork of Mistral Vibe (© Mistral AI); see `LICENSE`\n\n.", "url": "https://wpnews.pro/news/local-privacy-focus-on-cache-ai-cli-and-llama-cpp-branch-qwen3-x", "canonical_source": "https://github.com/alainnothere/privibe/tree/main", "published_at": "2026-08-16 00:53:38+00:00", "updated_at": "2026-08-16 01:10:30.323546+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "large-language-models", "ai-infrastructure"], "entities": ["alainnothere", "privibe", "Mistral Vibe", "llama.cpp", "Mistral SDK", "Anthropic", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/local-privacy-focus-on-cache-ai-cli-and-llama-cpp-branch-qwen3-x", "markdown": "https://wpnews.pro/news/local-privacy-focus-on-cache-ai-cli-and-llama-cpp-branch-qwen3-x.md", "text": "https://wpnews.pro/news/local-privacy-focus-on-cache-ai-cli-and-llama-cpp-branch-qwen3-x.txt", "jsonld": "https://wpnews.pro/news/local-privacy-focus-on-cache-ai-cli-and-llama-cpp-branch-qwen3-x.jsonld"}}