{"slug": "privacy-first-run-your-own-health-assistant-llm-entirely-in-the-browser-no", "title": "Privacy First: Run Your Own Health Assistant LLM Entirely in the Browser (No Backend Required!)", "summary": "A developer built a private health consultation bot that runs entirely in the browser using WebLLM and WebGPU, eliminating the need for backend servers and ensuring zero data leakage. The system executes a quantized Llama-3-8B model locally via TVM.js and Web Workers, with model weights cached in IndexedDB.", "body_md": "Have you ever wondered why your most personal health queries need to travel across the globe to a centralized server just to get a simple answer? In an era where **privacy-preserving AI** is becoming a necessity rather than a luxury, the paradigm of **Edge AI** is shifting the landscape.\n\nBy leveraging **WebLLM** and the raw power of **WebGPU**, we can now execute high-performance **Large Language Models (LLMs)** directly within the browser sandbox. No API keys, no server costs, and most importantly—zero data leakage. Today, we are building a private health consultation bot that runs 100% client-side.\n\nBefore we dive into the code, let’s talk about why this matters. Traditional AI architectures rely on heavy GPU clusters. However, with the advent of the WebGPU API, we can tap into the user's local hardware. This approach offers:\n\nIf you are interested in more **production-ready examples** and advanced architectural patterns for decentralized AI, I highly recommend checking out the deep dives over at [WellAlly Tech Blog](https://www.wellally.tech/blog).\n\nTo make this work, we use **TVM (Apache TVM)** as the compilation stack, which allows models to run on different backends, and **WebLLM** as the high-level interface for the browser.\n\n``` php\ngraph TD\n A[User Input] --> B[React Frontend]\n B --> C[WebLLM Worker]\n C --> D{WebGPU Support?}\n D -- Yes --> E[TVM.js Runtime]\n D -- No --> F[Fallback/Error]\n E --> G[IndexedDB Model Cache]\n G --> H[Local GPU Inference]\n H --> I[Streamed Response]\n I --> B\n```\n\nTo follow this tutorial, ensure you have:\n\n`tech_stack`\n\n: First, we need to initialize the `MLCEngine`\n\n. Since LLMs are heavy, we should run the inference engine inside a **Web Worker** to keep the UI thread buttery smooth. 🚀\n\n``` js\n// engine.ts\nimport { CreateMLCEngine, MLCEngine } from \"@mlc-ai/web-llm\";\n\nconst modelId = \"Llama-3-8B-Instruct-q4f16_1-MLC\"; // Quantized for browser use\n\nexport async function initializeEngine(onProgress: (p: number) => void) {\n const engine = await CreateMLCEngine(modelId, {\n initProgressCallback: (report) => {\n onProgress(Math.round(report.progress * 100));\n },\n });\n return engine;\n}\n```\n\nWe want a clean way to interact with our local model. Let's wrap the logic into a custom hook.\n\n``` js\n// useHealthAI.ts\nimport { useState } from 'react';\nimport { initializeEngine } from './engine';\n\nexport const useHealthAI = () => {\n const [engine, setEngine] = useState(null);\n const [loading, setLoading] = useState(false);\n const [progress, setProgress] = useState(0);\n\n const boot = async () => {\n setLoading(true);\n const inst = await initializeEngine((p) => setProgress(p));\n setEngine(inst);\n setLoading(false);\n };\n\n const askHealthQuestion = async (prompt: string) => {\n const messages = [\n { role: \"system\", content: \"You are a private health assistant. Provide concise, empathetic advice. Always suggest seeing a doctor for serious issues.\" },\n { role: \"user\", content: prompt }\n ];\n\n const reply = await engine.chat.completions.create({ messages });\n return reply.choices[0].message.content;\n };\n\n return { boot, askHealthQuestion, loading, progress, ready: !!engine };\n};\n```\n\nNow, we integrate this into our React component. Notice how we handle the \"Loading Weights\" phase—the model is about 4GB-5GB, so clear feedback is key!\n\n``` python\n// App.tsx\nimport React, { useState } from 'react';\nimport { useHealthAI } from './useHealthAI';\n\nfunction App() {\n const { boot, askHealthQuestion, loading, progress, ready } = useHealthAI();\n const [input, setInput] = useState(\"\");\n const [answer, setAnswer] = useState(\"\");\n\n const handleConsult = async () => {\n const res = await askHealthQuestion(input);\n setAnswer(res);\n };\n\n return (\n
\n

🩺 LocalHealth AI

\n {!ready && !loading && (\n \n )}\n\n {loading &&

Downloading Model Weights: {progress}%

}\n\n {ready && (\n
\n