{"slug": "how-can-llms-run-in-the-browser", "title": "(How) Can LLMs run in the browser?", "summary": "NobodyWho has spent several weeks mapping how to run language models in the browser via WebAssembly and concluded that Emscripten plus WebGPU with a CPU fallback is the practical path, since WASI in the browser and WebNN remain experimental. The company, which uses llama.cpp and the ONNX Runtime as inference engines, ruled out the wasm32-unknown-unknown target with wasm-bindgen because both engines assume a system library. NobodyWho is targeting websites rather than browser extensions, noting that even small models are typically a few hundred MB and that WebGPU runs slower on Firefox, while multi-threading requires COOP/COEP cross-origin headers.", "body_md": "[← Back to blog](https://www.nobodywho.ai/posts/)\n\n# (How) Can LLMs run in the browser?\n\nA big push here at NobodyWho is to get language models running in the browser using WebAssembly.\n\nAfter working on it for a few weeks we’ve gotten a good idea of the landscape. One thing that’s very clear, and this will come as a surprise, is that browsers are actually kinda complicated.\n\nFirst, we use Rust at NobodyWho, and there are (at least) three different compilation targets for Rust for WASM: Emscripten, WASI, and `wasm32-unknown-unknown` with `wasm-bindgen`.\nAll with different levels of support in various dependencies and within Rust itself.\n\nSecond, there are (at least) three different hardware backends: CPU, WebGPU, and WebNN, with the last two being the only reasonably fast ones and both being very new or experimental.\n\nThird, there’s the question of whether the LLM will be running in a website or a browser extension.\n\nFourth, and importantly, there are many different browsers and many different browser versions. Many of the features needed for running LLMs in a browser are relatively new, and browser support can be spotty, especially if your users haven’t updated for a while.\n\n## Spoiled for choice\n\nFortunately, many of these dimensions can get flattened by technical constraints or a specific use case.\n\nAt NobodyWho we use both llama.cpp and the ONNX Runtime as inference engines, and they both assume a system library.\nThat’s an issue for `wasm-bindgen`, since the second `unknown` in `wasm32-unknown-unknown` means “unknown OS”, i.e. no system library.\nThis leaves WASI and Emscripten, with Emscripten being the obvious choice given that WASI in the browser is… experimental<sup>[\\[1\\]](#fn1)</sup>.\n\nThe story for hardware backends is just as clear: run with WebGPU when available and fall back to CPU when WebGPU isn’t available (or when on Firefox, since apparently GPU is [slower](https://github.com/ngxson/wllama/issues/296) there).\nWhy not WebNN? Again because it’s… experimental<sup>[\\[2\\]](#fn2)</sup>.\n\nAt NobodyWho we're focusing on running LLMs in a website rather than a browser extension, since that's what we have a use case for, but there are some interesting tradeoffs between the two.\nA big issue with running an LLM in the browser is that useful models are *big*.\nEven the small models are typically a few hundred MB, which is much more than a typical website.\nGiven how important load times are, that can be a huge blocker.\nFor an extension, you download the model once, deliberately, when you install the extension, so it taking some time is less of an issue<sup>[\\[3\\]](#fn3)</sup>, while website use is generally more ephemeral.\nThis is less true for websites that users typically return to often, since the site can then cache the model so the download cost is only paid once.\n\nLastly we have browsers, and it's a mess.\nThe most relevant features are WebGPU, Memory64 (see my [previous post](https://www.nobodywho.ai/posts/letting-rust-support-models-bigger-than-4gb-in-the-browser/)), and SharedArrayBuffer (for multi-threading).\nAll of these have patchy support across browsers and platforms.\nOften, support just isn't implemented yet.\nOther times it is, but requires a feature flag to be turned on in the browser (good luck getting your non-technical users to do that).\nSometimes, it is officially supported, but just works really poorly.\nThe best example of this is Firefox's WebGPU support mentioned above.\n\nMulti-threading and SharedArrayBuffer is it's own can of worms because it requires strict cross-origin headers (COOP/COEP) which can be a real problem for some sites.\nAnd of course the easiest way to meet these requirements are different for each browser.\nOn top of that, browsers vary wildly in how much storage they allow for websites, what their eviction policies are, how aggressive they are in killing workers, etc.\nThe only good news is that the large number of different browsers, in practice, [most people](https://gs.statcounter.com/browser-market-share) use Chromium-based browsers like Chrome and Edge<sup>[\\[4\\]](#fn4)</sup>, and that is where support is by far the best.\n\n## LLMs do run in the browser\n\nBut not cleanly.\nThere are a lot of options and target platforms/browsers/use cases all requiring you to handle stuff differently.\nUntil browser support for the necessary features becomes much more mature, a library for browser LLM inference that just works in all browsers, on all platforms, for most use cases, will require a *lot* of work on top of the basic browser APIs.\nA bit ironic given that part of the dream of WASM is to *not* depend on specific platforms so heavily.\n\nI think incorporating such a general web LLM library into NobodyWho as part of our mission to support all platforms would be super cool, but for now we're exploring a few specific use cases where Emscripten and WebGPU with CPU fallback are the right choice. We haven't figured out how to handle the spotty browser support yet, but we'll get to that in time. Hopefully you'll see a release announcement right here within the next few weeks!\n\n1. [Everything is shimmed ATM](https://www.npmjs.com/package/@bytecodealliance/preview2-shim) , which makes it pretty damn slow.[↩︎](#fnref1)\n2. It's supported [nowhere](https://web-platform-dx.github.io/web-features-explorer/features/webnn/) .\nBut at least there's a[bug report](https://issues.chromium.org/issues/40206287) on that for Chromium![↩︎](#fnref2)\n3. Extensions have other issues though. Browsers generally [aggressively kill](https://developer.chrome.com/docs/extensions/develop/concepts/service-workers/lifecycle#:~:text=Chrome%20terminates%20a%20service%20worker,30%20seconds%20of%20inactivity) background processes like those an extension would hold an LLM in, so without some hacky solutions (e.g.[offscreen documents](https://developer.chrome.com/docs/extensions/reference/api/offscreen) ) to keep them alive you'd need to constantly reload the multi-GB model from disk.[↩︎](#fnref3)\n4. I'm not saying this is a *good* thing, just that it is the case, unless you're targeting some specific demographics.\nE.g. iOS browsers are all WebKit based (like Safari) while the proportion of Firefox users is higher among Google~~hating~~ skeptical tech-nerds.[↩︎](#fnref4)\n\nEverything NobodyWho does is open-source, please leave [a star on GitHub](https://github.com/nobodywho-ooo/nobodywho) to support us ❤️\n\nGet new posts in your feed reader: [subscribe via RSS](https://www.nobodywho.ai/feed.xml).", "url": "https://wpnews.pro/news/how-can-llms-run-in-the-browser", "canonical_source": "https://www.nobodywho.ai/posts/how-do-llms-run-in-the-browser/", "published_at": "2026-10-28 00:00:00+00:00", "updated_at": "2026-10-28 09:11:37.141303+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-tools"], "entities": ["NobodyWho", "WebAssembly", "Rust", "Emscripten", "WASI", "wasm-bindgen", "WebGPU", "WebNN"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-can-llms-run-in-the-browser", "markdown": "https://wpnews.pro/news/how-can-llms-run-in-the-browser.md", "text": "https://wpnews.pro/news/how-can-llms-run-in-the-browser.txt", "jsonld": "https://wpnews.pro/news/how-can-llms-run-in-the-browser.jsonld"}}