Running LLMs inside a Chrome extension: WebLLM, transformers.js and the MV3 traps AWFlow, a Chrome extension for browser automation workflows, runs local LLMs inside a Manifest V3 extension using three runtimes — WebLLM for chat, transformers.js for task pipelines, and TensorFlow.js for legacy models — with inference split across two Web Workers to keep the Svelte side-panel UI responsive. The build required a WebGPU probe with a WASM fallback and surfaced a Content Security Policy rule that silently blocks the runtime, according to the developer's account. The extension defaults to 1–4B parameter local models so text never leaves the machine, while letting users switch individual nodes to OpenAI, Anthropic, Google or a local Ollama for deeper reasoning. Running LLMs inside a Chrome extension: WebLLM, transformers.js and the MV3 traps What it took to run AI models inside a Manifest V3 extension: three runtimes, two Web Workers, a WebGPU probe with a WASM fallback, and the CSP rule that silently blocks your runtime. I build AWFlow, a Chrome extension that runs automation workflows in your browser. One of its node types is “summarize, classify or extract this text with AI”. I wanted that step to work without an API key and without sending the page anywhere, which meant running the model inside the extension itself. This post covers what that took: three runtimes, two Web Workers, one CSP surprise, and a fallback path for machines without WebGPU. If you want the gentler tour of what WebLLM and Chrome’s built-in AI are, read Run AI nodes directly in Chrome with WebLLM and Chrome AI API first. This one is about the plumbing. Why run the model in the extension Look at what most “AI automation” steps actually do: summarize a page, tag a sentiment, pull five fields out of some HTML. You don’t need a frontier model in a data center for that. A 1–4B parameter model running locally does it well enough, for free, and the text never leaves the machine. When a step really does need deep reasoning, the user can switch that one node to OpenAI, Anthropic, Google or a local Ollama. The point is to make local the default, not the only option. Three runtimes, one job each There’s no single “run a model in the browser” library that covers everything, so the extension ships three: Task pipelines: summarization, classification, embeddings, speech, small chat models TensorFlow.js A few classic models for example legacy embeddings There’s also a node for Chrome’s built-in AI when the browser exposes it. A Local Models page lists what’s installed for every runtime and lets you download or delete models, so users never have to care which engine a model runs on. If you are choosing a model rather than building one in, this guide is the practical side. Keep inference off the main thread The extension’s UI is a Svelte app in the side panel and in a tab. If a 2 GB model compiles on that thread, the canvas freezes. So the first rule is simple: every runtime runs in a dedicated worker. Here is the whole layout. Two workers do the heavy work, and the UI only sends requests and draws results. Five parts The UI draws the canvas. Two workers do the heavy work, on the graphics card when they can and on WASM when they can’t. Chat goes to the WebLLM worker Downloading, compiling and generating all happen in the worker, so the side panel stays responsive. Tasks go to a second worker transformers.js and TensorFlow.js share one worker. It caches one pipeline per model id, so the second run of a workflow doesn’t reload anything, and it streams download progress back to the UI. WebLLM runs on the graphics card Its runtime is WebAssembly, and the heavy math runs as WebGPU kernels. Tasks use the GPU when it is safe A task pipeline goes to WebGPU only if the adapter exists and the task is on the list of tasks known to work there. Otherwise they use WASM Slower, but it runs on any machine. The WebLLM worker is three lines WebLLM makes the worker side fairly clean: js // worker.ts import { WebWorkerMLCEngineHandler } from '@mlc-ai/web-llm'; const handler = new WebWorkerMLCEngineHandler ; self.onmessage = msg: MessageEvent = handler.onmessage msg ; The catch: I use LangChain’s ChatWebLLM wrapper so the local model plugs into the same agent code as the cloud models the same agent that builds your workflows . ChatWebLLM builds its own main-thread engine in its constructor and has no way to accept one from outside. But WebWorkerMLCEngine implements the same MLCEngineInterface, so you can build the worker engine yourself and swap it in. In code, with the fallback that matters most: js const worker = new Worker new URL './worker.ts', import.meta.url , { type: 'module' } ; try { const engine = await CreateWebWorkerMLCEngine worker, modelId, { appConfig: model.appConfig, initProgressCallback: onProgress, } ; // engine is protected and typed as the concrete MLCEngine, // but the runtime contract is the interface, which the worker engine satisfies. model as unknown as { engine: MLCEngineInterface } .engine = engine; } catch error { worker.terminate ; // don't orphan the thread await model.initialize onProgress ; // fall back to the main-thread engine } If the worker path fails no WebGPU, a download error , it falls back to the old behavior instead of breaking the node. Probe WebGPU once, keep a WASM path Not every machine has a usable WebGPU adapter. The inference worker probes it once and remembers the answer: js let webgpu: Promise