{"slug": "i-built-a-local-llm-that-runs-entirely-in-your-browser-no-install-no-gpu-no", "title": "I built a local LLM that runs entirely in your browser. No install, no GPU, no server", "summary": "Developer Zayd Mulani built 'ghost', a single HTML file that runs a quantized LLM entirely in a browser tab with no backend, GPU, or install. The project uses WebAssembly and the wllama binding for llama.cpp to download and cache models from HuggingFace, enabling fully offline inference. Features include RAG, voice input, and multi-turn conversation, with speeds of 3-8 tokens per second on CPU.", "body_md": "A few months ago I got obsessed with a question: can you run a real LLM entirely inside a browser tab, with zero backend, zero GPU, and zero install?\n\nThe answer is yes. Here's what I built.\n\nghost is a single HTML file that downloads a quantized language model into your browser's cache on first visit, then runs inference locally in WebAssembly forever after. Fully offline after that first download. No API key. No npm. No build step. Open the file, pick a model, chat.\n\nHow it works\n\nThe inference engine is wllama — a WebAssembly binding for llama.cpp. It runs GGUF quantized models directly in the browser using WASM SIMD. I pin it to a specific version so the JS and WASM files always match (learned this the hard way after a fun debugging session involving mismatched memory imports).\n\nModels are downloaded from HuggingFace on first load and cached via the browser's Cache API. On every subsequent visit they load instantly from cache, no network needed.\n\nFeatures\n\nThree models: Qwen2.5 1.5B (smart), Qwen2 0.5B (fast), TinyLlama (lightweight)\n\nMarkdown rendering from scratch — no library, just regex transforms\n\nRAG: drag a .txt or .pdf onto the chat window. It chunks the text, embeds each chunk using wllama's embedding API, stores vectors in memory, and retrieves the top-3 relevant chunks on each message. Fully local, fully offline\n\nVoice input via the Web Speech API — mic button auto-sends on silence\n\nMulti-turn conversation memory capped at 10 turns\n\nPWA installable — works on mobile home screen too\n\nThe hard parts\n\nGetting wllama to load from a cached model was genuinely tricky. Blob URLs created in the main thread aren't accessible from wllama's internal Web Worker. IndexedDB chunk reconstruction hit a 2GB ArrayBuffer limit on Windows Chrome. The final solution was using wllama's built-in loadModelFromHF with useCache: true which handles everything internally.\n\nThe embeddings API requires toggling a flag (embeddings: true) that conflicts with normal chat completion — so I toggle it on before each embedding call and back off in a finally block.\n\nTry it\n\nLive: [https://zaydmulani09.github.io/ghost/ghost.html](https://zaydmulani09.github.io/ghost/ghost.html)\n\nGitHub: [https://github.com/zaydmulani09/ghost](https://github.com/zaydmulani09/ghost)\n\nSpeed on CPU is ~3-8 tokens/sec depending on model. Not fast, but it works, and every token stays on your machine.", "url": "https://wpnews.pro/news/i-built-a-local-llm-that-runs-entirely-in-your-browser-no-install-no-gpu-no", "canonical_source": "https://dev.to/zaydmulani09/i-built-a-local-llm-that-runs-entirely-in-your-browser-no-install-no-gpu-no-server-3c3c", "published_at": "2026-07-27 18:34:05+00:00", "updated_at": "2026-07-27 19:00:11.536693+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "developer-tools"], "entities": ["Zayd Mulani", "ghost", "wllama", "llama.cpp", "HuggingFace", "Qwen2.5", "Qwen2", "TinyLlama"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-local-llm-that-runs-entirely-in-your-browser-no-install-no-gpu-no", "markdown": "https://wpnews.pro/news/i-built-a-local-llm-that-runs-entirely-in-your-browser-no-install-no-gpu-no.md", "text": "https://wpnews.pro/news/i-built-a-local-llm-that-runs-entirely-in-your-browser-no-install-no-gpu-no.txt", "jsonld": "https://wpnews.pro/news/i-built-a-local-llm-that-runs-entirely-in-your-browser-no-install-no-gpu-no.jsonld"}}