Your Secrets Stay on Your GPU: Building a Private, Local Mental Health AI with WebLLM and React

A developer built a private, local mental health AI assistant using WebLLM and React, running Llama-3-8B entirely on the user's GPU via WebGPU. The approach ensures sensitive data never leaves the device, eliminating server-side privacy risks. The project demonstrates local-first LLM execution in the browser for privacy-preserving AI applications.

Privacy is the final frontier of the AI revolution. When it comes to sensitive topics like mental health , users are rightfully hesitant to send their deepest thoughts to a distant server. This is where Edge AI , WebLLM , and WebGPU change the game. In this tutorial, we are building a "Zero-Knowledge" style mental health assistant. By leveraging privacy-preserving AI and local LLM execution , we ensure that sensitive data never leaves the user's device. We’ll be using WebLLM to run Llama-3-8B directly in the browser, accelerated by WebGPU , and managed via a React frontend. If you've been looking for a way to implement local-first LLMs or want to understand how to bridge Wasm and WebGPU for production apps, you're in the right place. 🚀 Traditional AI apps follow a Client-Server model. Our approach flips the script. The browser becomes the inference engine. php graph TD A User Input: Sensitive Query -- B{React UI State} B -- C WebLLM Engine C -- D WebGPU / Wasm Runtime D -- E Local GPU - Llama-3-8B E -- D D -- C C -- F Generated Response F -- B B -- G IndexedDB: Encrypted History subgraph Browser Environment B C D E G end style Browser Environment fill: f9f,stroke: 333,stroke-width:2px By keeping the inference on the local GPU , we eliminate latency after the initial model load and, more importantly, we eliminate the risk of data breaches at the transport or server level. Before we dive into the code, ensure your environment meets these requirements: The heart of our application is the MLCEngine . It handles the orchestration between the model weights, the WebGPU pipeline, and the user's prompt. js import { CreateMLCEngine, MLCEngine } from "@mlc-ai/web-llm"; // We use Llama-3-8B-Instruct for high-quality psychological nuances const selectedModel = "Llama-3-8B-Instruct-v0.1-q4f16 1-MLC"; export async function initializeEngine onProgress: progress: number = void { const engine = await CreateMLCEngine selectedModel, { initProgressCallback: report = { // Track download progress of the 5GB+ model onProgress Math.round report.progress 100 ; }, } ; return engine; } Since loading an LLM is expensive, we want to maintain a single instance across our app using React Context. python import React, { createContext, useContext, useState } from 'react'; import { MLCEngine } from "@mlc-ai/web-llm"; const AIContext = createContext<{ engine: MLCEngine | null, loading: boolean } | null null ; export const AIProvider: React.FC<{ children: React.ReactNode } = { children } = { const engine, setEngine = useState<MLCEngine | null null ; const progress, setProgress = useState 0 ; const bootAI = async = { const instance = await initializeEngine p = setProgress p ; setEngine instance ; }; return <AIContext.Provider value={{ engine, loading: progress < 100 }} {children} {progress < 100 && <p Loading Local AI: {progress}% Keep tab open </p } { engine && progress === 0 && <button onClick={bootAI} Start Private Session</button } </AIContext.Provider ; }; While building a local LLM in the browser is a massive win for privacy, scaling this for production—especially when dealing with hybrid cloud-edge strategies—requires deeper architectural insight. For advanced patterns on optimizing model quantization for the web or handling complex state management with IndexedDB in offline-first AI apps, I highly recommend checking out the technical deep-dives on the WellAlly Blog https://www.wellally.tech/blog . They offer incredible resources on building production-ready, privacy-centric AI systems that go beyond the basics. 🥑 In a mental health context, the system prompt is critical. We need to define boundaries for the AI. js const handleChat = async input: string = { if engine return; const messages = { role: "system", content: "You are a supportive, empathetic mental health assistant. You operate locally on the user's device. Your goal is to listen and provide CBT-based reflections. If the user is in danger, provide emergency resources." }, { role: "user", content: input } ; // The inference happens 100% on the User's GPU const reply = await engine.chat.completions.create { messages, stream: true, // For that cool "typing" effect } ; for await const chunk of reply { const delta = chunk.choices 0 ?.delta?.content || ""; setResponse prev = prev + delta ; } }; To make this a true "Zero-Knowledge" app, we shouldn't even use localStorage for chat history it's too small and unencrypted . Instead, use IndexedDB to store the vector embeddings or the chat history locally. js import { openDB } from 'idb'; async function saveChatLocally chatId: string, message: any { const db = await openDB 'PrivateMindDB', 1, { upgrade db { db.createObjectStore 'history' ; }, } ; await db.put 'history', message, chatId ; } By combining WebLLM and React , we've built an application that respects user privacy by design, not just by policy. The sensitive nuances of a mental health conversation never touch a wire. Key Takeaways: Are you ready to stop sending your data to the cloud? Let me know in the comments if you’ve tried running local models in your web apps 💻🔥 For more advanced tutorials on Edge AI and Privacy, visit wellally.tech/blog.