Beyond APIs: Building a Privacy-First Drug Interaction Tool with WebGPU and WebLLM

In the era of cloud-hosted AI, we’ve become comfortable sending our most sensitive data to remote servers. But when it comes to medical queries—like checking for drug-to-drug interactions—privacy isn't just a feature; it's a human right. 🛡️

With the recent explosion of WebGPU AI and the maturation of local LLMs, we can finally move the "brain" of our applications directly into the user's browser. In this tutorial, we are building a high-performance, browser-based AI tool that uses WebLLM and WebGPU to perform millisecond-level drug compatibility checks. No data ever leaves the device, ensuring 100% data residency and lightning-fast edge computing performance.

Traditionally, running a Large Language Model (LLM) required a massive Python backend with expensive GPUs. WebGPU changes the game by providing low-level access to the local graphics card directly from the browser. WebLLM leverages this to run models like Llama-3 or Mistral in the browser sandbox.

graph TD
    UserInput[User Inputs Medications] -->|React State| Engine[WebLLM Engine Instance]
    Engine -->|Compute Shaders| WebGPU[WebGPU API]
    WebGPU -->|Parallel Processing| LocalGPU[Device VRAM/GPU]
    LocalGPU -->|Token Generation| Engine
    Engine -->|Streamed Response| UI[React Frontend Display]
    subgraph Browser_Sandbox
    Engine
    WebGPU
    UI
    end
    subgraph Privacy_Boundary
    Browser_Sandbox
    end
    ExternalServer((Cloud / Internet)) -.->|Data Never Sent| Privacy_Boundary

To follow this advanced guide, you'll need:

@mlc-ai/web-llm. First, we need to create a singleton or a hook to manage our AI engine. Since a model (~2GB-5GB) takes time, we need to handle the progress state effectively.

// useWebLLM.ts
import { useState, useEffect } from "react";
import * as webllm from "@mlc-ai/web-llm";

export function useWebLLM(modelId: string) {
  const [engine, setEngine] = useState<webllm.MLCEngine | null>(null);
  const [Progress, setProgress] = useState(0);

  const initEngine = async () => {
    const engineInstance = new webllm.MLCEngine();

    // Callback to track model download/ progress
    engineInstance.setInitProgressCallback((report) => {
      setProgress(Math.round(report.progress * 100));
    });

    await engineInstance.reload(modelId);
    setEngine(engineInstance);
  };

  return { engine, Progress, initEngine };
}

Drug interaction retrieval requires a high degree of accuracy. We will use a structured system prompt to ensure the LLM acts as a clinical pharmacist.

const SYSTEM_PROMPT = `
You are a clinical pharmacy expert. 
Your task is to analyze two or more medications and identify potential drug-to-drug interactions.
Provide the output in the following format:
1. Interaction Level (Mild, Moderate, Severe)
2. Mechanism of Action
3. Recommendation
Be concise and stick to clinical facts. If no interaction is found, state so.
`;

const checkInteractions = async (engine: webllm.MLCEngine, drugs: string[]) => {
  const userPrompt = `Check interactions for: ${drugs.join(", ")}`;

  const messages: webllm.ChatCompletionMessageParam[] = [
    { role: "system", content: SYSTEM_PROMPT },
    { role: "user", content: userPrompt }
  ];

  const chunks = await engine.chat.completions.create({
    messages,
    stream: true, // We want that sweet typewriter effect!
  });

  return chunks;
};

We want a clean, professional interface that handles the streaming response and gives users confidence.

import React, { useState } from 'react';
import { useWebLLM } from './hooks/useWebLLM';

const DrugChecker = () => {
  const { engine, Progress, initEngine } = useWebLLM("Llama-3-8B-Instruct-v0.1-q4f16_1-MLC");
  const [drugs, setDrugs] = useState("");
  const [result, setResult] = useState("");

  const handleConsult = async () => {
    if (!engine) return;
    setResult(""); // Clear previous

    const stream = await checkInteractions(engine, drugs.split(","));
    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content || "";
      setResult((prev) => prev + content);
    }
  };

  if (!engine) {
    return (
      <div className="p-10 text-center">
        <button onClick={initEngine} className="bg-blue-600 text-white px-6 py-2 rounded">
          Initialize Secure Local AI
        </button>
        <p className="mt-4"> Model: {Progress}%</p>
      </div>
    );
  }

  return (
    <div className="max-w-2xl mx-auto p-6">
      <h2 className="text-2xl font-bold mb-4">🛡️ Local Drug Interaction Checker</h2>
      <textarea 
        className="w-full border p-3 rounded mb-4"
        placeholder="Enter medications (e.g., Warfarin, Aspirin)..."
        onChange={(e) => setDrugs(e.target.value)}
      />
      <button 
        onClick={handleConsult}
        className="bg-green-600 text-white px-8 py-3 rounded-lg font-semibold"
      >
        Analyze Privately
      </button>
      <div className="mt-8 p-4 bg-gray-50 rounded border whitespace-pre-wrap">
        {result || "Analysis will appear here..."}
      </div>
    </div>
  );
};

Building a proof-of-concept is easy, but making Edge AI production-ready involves handling model caching, VRAM memory management, and specialized RAG (Retrieval-Augmented Generation) architectures to ensure medical data is up to date.

For more advanced patterns on optimizing WebGPU shaders, managing local vector databases, and high-performance AI deployment strategies, I highly recommend checking out the WellAlly Technology Blog. It's the source of inspiration for this build and contains deeper dives into "Local-First" software engineering.

The browser is no longer just a document viewer; it's a powerful AI execution environment. By combining WebGPU and WebLLM, we can build tools that were impossible a year ago.

Next Steps:

What are you planning to build with WebGPU? Let me know in the comments below! 👇