Show HN: Loft – offline Q&A over your local files, with citations (llama.cpp) Developer nwaoga released Loft, an open-source offline retrieval-augmented-generation app that answers questions about local folders using two local llama-server processes on a pinned llama.cpp build (b12110), with no cloud API or API key required. Loft indexes text, code, PDF, Word, PowerPoint, spreadsheets and pictures, caps each file at 2,000 chunks (about 3 MB of text), and retrieves the 6 nearest chunks (up to about 6 KB) per question, showing an "N sources" drawer with the excerpts behind each answer. The builds are unsigned, OCR of pictures and scanned PDF pages works only through the OCR built into Windows, and the project requires Node.js 18+ and Rust 1.85 or newer. Official channels: github.com/nwaoga/loft https://github.com/nwaoga/loft A local RAG notch that answers questions about the files in folders you choose — fully offline, on llama.cpp. Quick start quick-start · On your machine on-your-machine · Building https://github.com/nwaoga/loft/blob/main/docs/building.md · Known gaps Node.js 18+ and Rust 1.85 or newer the lockfile is edition 2024 . Windows also needs the MSVC C++ build tools and the Windows 10/11 SDK. WebView2 is the webview, and it is already on Windows 11. Linux needs the packages in Building https://github.com/nwaoga/loft/blob/main/docs/building.md , including WebKitGTK 4.1 and the tray library. macOS needs the Xcode command-line tools. npm install npm run fetch-llama fetch-llama downloads the llama.cpp server builds for this OS into src-tauri/binaries/ gitignored . Models are downloaded later, in the app. Windows. scripts\win-dev.bat selects the MSVC toolset and runs the app. macOS and Linux: npm run tauri dev The notch UI on its own, with mock data and no Rust: npm run dev → http://localhost:1420 http://localhost:1420 . In the app: Settings the gear → download and select a chat model and an embedding model → Folders → add a folder → Ask . Ctrl/Cmd+Shift+Space summons the notch. sample-docs/ https://github.com/nwaoga/loft/blob/main/sample-docs is a small folder for trying this. Its scans/ hold a picture and a scanned PDF, to try OCR on Windows. These builds are unsigned. macOS and Windows may warn you the first time. The app is not damaged. See First open https://github.com/nwaoga/loft/blob/main/docs/first-open.md . Installers and the per-platform steps are in Building https://github.com/nwaoga/loft/blob/main/docs/building.md . An answer, with its sources open. Linux. Loft runs llama.cpp on this computer. A question does not call a cloud API and does not need an API key. Two local llama-server processes do the work: one for chat, one for embeddings embedding models don't reliably share a process with chat . - The window is a notch: a pill at the top centre that expands. Right stands it on the right edge. Floating is a window you drag, and it stays where you leave it. - The servers are official builds of one pinned llama.cpp release b12110 . Loft refuses an archive that does not match the size and SHA-256 in src-tauri/engines.json https://github.com/nwaoga/loft/blob/main/src-tauri/engines.json . - A question the files don't cover is refused, and the refusal names your folders. Small talk "hi", "thanks", "who are you?" gets a short reply instead, with no search. Add a folder, or remove one, and ask about what is in it: text and code, PDF, Word, PowerPoint, spreadsheets, and pictures. Loft chunks them and keeps the index up to date as files change. Rebuild or delete that index from the same tab. - While a folder indexes it shows the file count and, for a large file, how far through that file it is. - A file is capped at 2,000 chunks about 3 MB of text . The folder says how many files were cut short. Rebuild to apply the cap to an older index. - Pictures and scanned PDF pages are read with the OCR built into Windows. On macOS and Linux they yield no text. A picture with no readable text is not indexed, unless picture descriptions are on. The reply leads. Under it, an "N sources" drawer holds the excerpts it was written from open for the newest answer by default . Click one to open a document or picture in its default app. Code and scripts are only revealed in the file manager. They are never run. - The question is embedded, the 6 nearest chunks are fetched, and chunks well below the best match are dropped. What remains up to about 6 KB goes into the prompt. - A PDF opens at the cited page when the default PDF app is a browser Edge, Chrome, Firefox… , Acrobat, or SumatraPDF on Windows. Otherwise it just opens. - Small models are not asked for citation numbers in the prose. They get them wrong. The drawer is the citation. You choose the engine and the device. CPU and Vulkan ship with fetch-llama . Metal does, on a Mac. CUDA and OpenVINO are optional, installed from Settings or with npm run fetch-llama cuda / openvino . Mains and battery can use different choices. - What the status shows is what the server's own log says is running. When that differs from the request, both are shown, with the reason. - Automatic prefers a discrete GPU when plugged in, and built-in graphics or the CPU on battery, until you have measured them. - OpenVINO is experimental and is not chosen automatically unless it has been measured. An Intel NPU has been run on one Windows laptop, with one model. Treat any other as unknown until Try NPU now reports a verified answer. The figures are in Engines https://github.com/nwaoga/loft/blob/main/docs/engines.md openvino-and-the-npu-what-experimental-means-here . Conversations are kept in chats.db in the app-data folder, and nowhere else. It is a different database from the index: deleting an index never touches a chat, and deleting a chat never touches your files. - The question is stored when you ask it. The answer is stored when it ends, with the model, the usage figures, and the excerpts. - Reopening a conversation shows it as it was saved. A file that is no longer indexed is marked no longer available . One whose contents changed is marked changed since . - The saved history and the prompt are separate. The next question is still sent with the last 4 turns at most. New chat starts over. Loft reopens the last conversation at launch. The catalog is a built-in list. In Settings, download a chat model and an embedding model from it. A download that misses the SHA-256 in src-tauri/catalog.json https://github.com/nwaoga/loft/blob/main/src-tauri/catalog.json is discarded. Only a different embedding model rebuilds the index. A different device keeps it. - Chat: Qwen2.5 3B Instruct the recommended one , Llama 3.2 3B, Gemma 3 4B, Phi 3.5 Mini. All Q4 K M. - Embeddings: Nomic Embed Text v1.5 recommended , EmbeddingGemma 300M, and BGE-M3. The relevance floor is measured for the first two only. BGE-M3 still uses the default, which may be wrong for it. - Picture descriptions are optional and off by default: SmolVLM 500M, a separate download. Descriptions are approximate and stored as such. Pictures inside PDFs and Office files are not described. Length, tone, citations, temperature, and whether answers may leave your files are a sheet of options beside Send. They stay in the webview's local storage. The full list is in How it works https://github.com/nwaoga/loft/blob/main/docs/how-it-works.md . The planned milestones M0–M6 are implemented. What was actually run is narrower than that, and HANDOVER.md https://github.com/nwaoga/loft/blob/main/HANDOVER.md is the record. | Platform | Where it stands | |---|---| | Windows | Tested on Windows 11, on the CPU and on a Vulkan GPU. | | macOS | Initially validated. Built and run once on a GitHub-hosted macOS 15 Apple Silicon runner 14 November 2026 : Metal, an answer from sample-docs , window resizing, a tray icon in the menu bar, and an ad-hoc-signed .app and .dmg . The hotkey registered and was not pressed. The tray menu was not opened. Not a Retina display. | | Linux | Being validated. x64 has been built and run the debug app, a .deb , and an AppImage . Still open: Wayland, a machine with a GPU, arm64, and installing those packages with a package manager. | The rest of the holes — unsigned installers, OCR only on Windows, the NPU on one machine, power changes that were simulated — are in Known gaps https://github.com/nwaoga/loft/blob/main/docs/known-gaps.md . | Doc | What it holds | |---|---| | How it works https://github.com/nwaoga/loft/blob/main/docs/how-it-works.md | Indexing, how an answer is built, OCR, options, follow-ups, chat history, and a map of the processes. | | Building https://github.com/nwaoga/loft/blob/main/docs/building.md | Windows, macOS, and Linux: prerequisites, debug runs, installers, signing. | | First open https://github.com/nwaoga/loft/blob/main/docs/first-open.md | Gatekeeper and SmartScreen, for these unsigned builds. | | Running Loft https://github.com/nwaoga/loft/blob/main/docs/using.md | Themes, notch or floating, tray, launch at login, icons. | | Engines, performance and battery https://github.com/nwaoga/loft/blob/main/docs/engines.md | Devices, fallback, OpenVINO, the NPU run, and the performance test. | | Testing https://github.com/nwaoga/loft/blob/main/docs/testing.md | Unit tests and the smoke scripts. | | Known gaps https://github.com/nwaoga/loft/blob/main/docs/known-gaps.md | What has not been exercised. | | ARCHITECTURE.md https://github.com/nwaoga/loft/blob/main/ARCHITECTURE.md | The code, the data, and why it is shaped this way. | src/ notch UI Svelte 5 + Vite App.svelte notch shell: pill ⇄ panel, tabs lib/components/ Ask, Folders, Settings themes, Models, general lib/theme.svelte.ts theme list + remembered choice colours are in app.css lib/api.ts Tauri commands + browser mocks lib/types.ts DTOs shared with Rust src-tauri/ Rust backend Tauri 2 src/lib.rs commands, window sizing, tray, hotkey src/inference.rs engines and devices: the decision rules no processes; unit-tested src/engine.rs the coordinator: every start, switch and restart of a server src/runtime.rs finding the engines, starting and stopping llama-server src/hardware.rs, power.rs what the OS says: GPUs, NPUs, drivers; mains or battery src/installer.rs optional engines: pinned download, checksum, safe unpacking src/bench.rs Test performance, Try … now, and what the measurements suggest src/history.rs chat history: conversations, messages and their sources chats.db src/models.rs model downloads src/store.rs sqlite-vec store src/parse.rs, chunk.rs, indexer.rs, jobs.rs indexing + file watching src/ocr.rs text recognition for pictures and scanned PDF pages Windows src/vision.rs optional picture descriptions from a small vision model src/rag.rs, opener.rs answers + opening sources catalog.json curated model catalog GGUF URLs engines.json the llama.cpp builds of one pinned release, with checksums tauri.conf.json transparent, top-center, always-on-top window; bundle config scripts/ fetch-llama.mjs download + verify the llama.cpp builds for this OS make-engines-manifest.py re-pin engines.json to another llama.cpp release smoke-ui.js drives the real UI in a debug build see docs/testing.md smoke-engines.js the same for engines, power, fallback and the performance test smoke-restart.js after smoke-ui.js: the chat history survived closing Loft set-version.mjs the version number package.json and Cargo.toml icons.mjs src-tauri/icons from app-icon.svg npm run icons windows/release.bat a Windows release: version, checks, installer - releases/windows/