Yes, Your Browser Can Run Its Own Tiny AI. Here’s How

A browser extension can now run a small language model locally, with no server or API key, using either Chrome's built-in Gemini Nano or a custom model via Transformers.js on WebGPU. The approach enables on-device tasks like summarization and question-answering, with trade-offs between simplicity and control.

A browser extension that reads the page you are on, answers questions about it, and summarizes it, with no server, no API key, and no data ever leaving your machine, isn’t a fantasy anymore. In 2026 it’s a weekend project. The trick is running a very small language model directly inside the browser, and there are now two solid ways to do it. This walks through both, the architecture that makes it work, the honest limits of tiny models, and a complete working extension you can load and use today. Here’s a fun question that turns out to have a genuinely interesting answer. Can your browser run its own AI, locally, with no cloud call behind it, and can you build that into a small extension. A year or two ago the honest answer was “not really, not usefully.” Today the answer is yes, and it’s not even that hard. The browser has quietly become a real place to run small language models, and a browser extension is one of the most natural homes for one, because it already has permission to see your tabs, read the page, and live in a side panel next to your reading. The key word is small. You’re not going to run a frontier model in a browser tab, and you shouldn’t try. But a small model, in the range of half a billion to a few billion parameters, quantized down to four bits, fits comfortably in browser memory and runs at usable speed on ordinary hardware. And for a lot of genuinely useful tasks, summarizing, rewriting, extracting, answering simple questions about the page in front of you, a small model is entirely enough. This piece explains the two ways to do it, shows the architecture, and gives you a complete working extension to build on. There are two distinct ways to put a small AI inside your browser in 2026, and they suit different goals. The first path is Chrome’s own built-in model. Chrome now ships a small model, Gemini Nano, baked directly into the browser, and extensions can call it through a built-in Prompt API with almost no code. You don’t ship a model, you don’t manage a download, you just ask the browser to run a prompt for you. Calling it looks about as simple as it gets, you create a session and prompt it. js const session = await self.ai.languageModel.create { systemPrompt: "You are a concise assistant.",} ;const result = await session.prompt "Summarize this in one sentence." ; That’s the whole thing. The model is already there, shared across every extension and page on the machine, running fully on-device and offline once available. The tradeoffs are that it’s English-only, it is not tuned for factual accuracy, its context window is small, and you can’t pin or choose the model version, it’s tied to Chrome’s update cycle and you get whatever ships. For a lot of extensions, none of that matters and the near-zero setup is worth everything. The second path is bringing your own model with a library called Transformers.js, from Hugging Face, running on WebGPU. Here you choose a specific small model, bundle the logic to load it, and it runs on the user’s GPU through the browser’s modern compute layer. This is more work and it means a one-time model download for the user, but it buys you control, you pick the exact model, you can choose a multilingual one, a code-focused one, or whatever fits, and the behavior doesn’t change out from under you when Chrome updates. This is the path to choose when you want a specific capability or predictable behavior, and it’s the one the working example below uses, because it teaches you more about how the whole thing fits together. The quick rule is this. If you want the simplest possible on-device AI and Chrome-only, English-only is fine, use the built-in Prompt API. If you want a specific model, cross-version stability, or capabilities the built-in model does not have, bring your own with Transformers.js. Both run entirely on the user’s machine, which is the whole point. Before the code, the shape of the thing, because there’s one design decision that matters more than any other and it trips up everyone who skips it. A small model is still hundreds of megabytes and its inference is heavy. If you load and run it in the same place as your user interface, the interface freezes solid every time the model thinks. So the golden rule of an in-browser-AI extension is that the model lives in the background service worker, not the UI. The background worker loads the model once, keeps it resident, and does all the generation. The side panel, the visible chat, holds no model at all. It just sends the worker a message like “generate an answer to this,” and the worker streams tokens back which the panel renders as they arrive. That gives you a clean three-part structure. The background service worker is the engine, it hosts the model and runs inference. The side panel is the face, it takes input and displays streamed output. And a content script is the reach into the page, it runs in the actual web page and extracts the readable text when asked, so the model has something to summarize or answer questions about. Messages flow between them, panel to worker to generate, content script to panel to fetch page text. Get that separation right and everything else is detail. Here is a complete, minimal extension that does something genuinely useful, it summarizes the page you’re on and answers questions about it, entirely locally. I’ll show the important pieces. The full set of files is packaged for download at the end. It starts with the manifest, which declares a Manifest V3 extension with the three parts, a background worker, a side panel, and a content script, plus permission to read the active tab. { "manifest version": 3, "name": "Pagewise, a tiny on-device AI", "version": "1.0.0", "permissions": "sidePanel", "activeTab", "scripting", "storage" , "host permissions": "<all urls " , "background": { "service worker": "background.js", "type": "module" }, "side panel": { "default path": "sidepanel.html" }, "content scripts": { "matches": "<all urls " , "js": "content.js" , "run at": "document idle" } , "content security policy": { "extension pages": "script-src 'self' 'wasm-unsafe-eval'; object-src 'self'" }} That last line, the content security policy allowing wasm-unsafe-eval, matters, because the model runtime needs WebAssembly, and without that permission the extension will refuse to run it. The heart is the background worker. It imports Transformers.js, builds the model pipeline exactly once as a lazy singleton, and streams generated tokens back to the panel. The model chosen here is a 1-billion-parameter instruction model in four-bit form, small enough to run on a laptop’s integrated GPU. js import { pipeline, TextStreamer } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0"; js const MODEL ID = "onnx-community/Llama-3.2-1B-Instruct-q4f16";let generatorPromise = null; function getGenerator onProgress { if generatorPromise { generatorPromise = pipeline "text-generation", MODEL ID, { dtype: "q4f16", device: "webgpu", progress callback: onProgress, } ; } return generatorPromise;} js chrome.runtime.onMessage.addListener message = { if message.type == "GENERATE" return; async = { const generator = await getGenerator progress = chrome.runtime.sendMessage { type: "PROGRESS", progress } ; const streamer = new TextStreamer generator.tokenizer, { skip prompt: true, skip special tokens: true, callback function: text = chrome.runtime.sendMessage { type: "TOKEN", text } , } ; await generator message.messages, { max new tokens: 512, do sample: false, streamer, } ; chrome.runtime.sendMessage { type: "DONE" } ; } ;} ; Notice the singleton pattern, the pipeline is built on first use and reused forever after, so the expensive load happens once. Notice too that generation streams, each token is posted back to the panel as it is produced, so the user sees the answer appear live rather than waiting for the whole thing. The content script is the simplest piece. It sits in the page and, when asked, hands back a cleaned, length-capped slice of the readable text. The cap matters, a tiny model has a small context window, so you trim the page down rather than drown it. js function extractReadableText { const root = document.querySelector "article" || document.querySelector "main" || document.body; const clone = root.cloneNode true ; clone.querySelectorAll "script, style, nav, header, footer, aside" .forEach el = el.remove ; return clone.innerText.replace /\n{3,}/g, "\n\n" .trim .slice 0, 6000 ;} js chrome.runtime.onMessage.addListener message, sender, sendResponse = { if message.type === "GET PAGE TEXT" { sendResponse { title: document.title, url: location.href, text: extractReadableText } ; }} ; And the side panel ties it together. When you click summarize, the panel asks the content script for the page text, wraps it in a prompt, and sends that to the background worker, then renders the streamed answer. The panel holds no model, it’s pure interface, which is exactly why it stays responsive while the model works. js summarizeBtn.addEventListener "click", async = { const page = await getPageText ; run { role: "system", content: "Summarize the provided page in 3 to 5 short bullet points. Be factual." }, { role: "user", content: Page title: ${page.title}\n\nPage content:\n${page.text} }, ;} ; That’s the entire idea. Page text comes in from the content script, a prompt goes to the model in the background worker, tokens stream back to the panel. Everything happens on the user’s machine. This is the part to be honest about, because a small model oversold is a disappointing extension, and a small model used well is a delightful one. A model in the half-billion to three-billion parameter range is genuinely good at a specific class of tasks. Summarizing a page, rewriting a paragraph in a different tone, extracting the key points or the named entities, classifying text, and answering simple questions where the answer is present in the text you gave it. These are the tasks where small models shine, and they are also the highest-frequency, most privacy-sensitive tasks, exactly the ones where sending every page you read to a server is both expensive and uncomfortable. A local model doing these is a real win. What a small model isn’t good at is hard reasoning, precise factual recall, long documents that exceed its small context window, and anything requiring deep world knowledge. It’ll confidently get difficult questions wrong. It isn’t a frontier assistant, and if you build it expecting one, you’ll be let down. The right mental model is a fast, private, capable helper for light tasks, not a genius. Build within that and the result feels great, push past it and it feels broken. There’s also an honest hardware note. The bring-your-own-model path needs WebGPU, which modern Chrome has, and it runs best with a GPU, though even an integrated laptop GPU handles a 1B model fine. On a machine without WebGPU the library can fall back to running on the processor, which works but is much slower. And the first run downloads the model, a few hundred megabytes, which you should tell the user about rather than silently consuming their bandwidth. Step back from the mechanics and the appeal is clear. An AI feature that runs entirely in the browser has properties a cloud-based one simply cannot match. It’s private in the strongest possible sense, the page you are reading never leaves your device, which for anything sensitive isn’t a nice-to-have but the whole game. It costs nothing to run, once the model is downloaded, every summary and every answer is free, forever, with no API meter ticking. It works offline. And it has zero latency from network round-trips, the model is right there. For a developer, that combination is a genuine unlock. You can ship an AI-powered extension with no backend, no per-user inference cost that a viral launch would turn into a ruinous bill, and no privacy policy headache about where user data goes, because it goes nowhere. The economics and the trust model are both fundamentally better for the class of tasks small models handle well. The broader point is that “AI in the browser” has crossed from demo to genuinely practical, and a browser extension is the most natural place to put it, sitting right next to the pages you read, seeing what you see, and keeping all of it on your machine. The model is small, and that isn’t a limitation to apologize for, it’s the entire reason the thing is private, free, and instant. Start with the working example, keep the tasks within what a small model does well, and you have a category of tool that simply wasn’t buildable a couple of years ago. Download the complete extension below, load it in developer mode, and you’ll have a private, on-device reading assistant running in your browser in about two minutes. From there, change the model, change the prompts, add the tools you want. The hard part, making a language model run locally in a browser, is already solved for you. The example extension is provided as a complete, loadable project. It runs a small open model locally through WebGPU and sends no data anywhere. Model behavior on small models is limited by design, so treat it as a capable helper for summarizing and light questions rather than a source of precise facts. Yes, Your Browser Can Run Its Own Tiny AI. Here’s How https://pub.towardsai.net/yes-your-browser-can-run-its-own-tiny-ai-heres-how-9799eec16df1 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.